Getting automatic values from clients into hdfs. Implemented machine learning algorithms. (recommendation engine, association rule and search relevance) Visualization of data. Monitoring ongoing spark jobs to detect bottlenecks. Technical Environment: Python, Hadoop, Spark, Spark ML, Sqoop