MLlib edit  

Spark library for running Machine Learning algorithms. Supports a range of algorithms (including classifications, regressions, decision trees, recommendations, clustering and topic modelling), including iterative algorithms. As of Spark 2.0 utilises a DataFrame (Spark SQL) based API, with the original RDD based API now in maintenance only. First introduced in Spark 0.8 after being collaboratively developed with the UC Berkeley MLbase project, and still under active development.

Technology Information

Type	Sub-Project
Parent Project	Apache Spark
Last Updated	August 2017

Links

• http://spark.apache.org/mllib// - home page
• http://spark.apache.org/docs/latest/ml-guide.html - documentation
• http://www.mlbase.org/ - the UC Berkeley MLbase project

Blog Posts

• Core Hadoop Technologies (pt2) 2017-01-13 Technologies Apache Solr Sqoop Spark GraphX MLlib Spark SQL Spark Streaming Peter