Amazon EMR edit
Service for dynamically provisioning Hadoop clusters on Amazon EC2 infrastructure, with the ability to select one of more Hadoop based services to be pre-installed and configured. Supports selection of EC2 instance types, EC2 spot and reserved instances, programmatic execution of service jobs (steps), persistent or transient (terminate after pre-defined steps have been executed) clusters, automatic or manual scaling of live clusters, cloning of clusters, HDFS on local (EBS) node storage, an HDFS compatible filesystem (EMR File System - EMRFS) for accessing Amazon S3 storage (that supports consistency using DynamoDB for metadata), automatic configuration of Hadoop clusters and firewalls, integration with AWS CloudWatch and AWS Identity and Access Management, Hadoop encryption and Kerberos authentication, persistent storage of Hive metadata in AWS Glue Data Catalog, and bootstrap actions for custom configuration or installation of other services (with a GitHub repo of open source bootstrap action extensions). Manageable via the AWS Management Console, the AWS CLI, a REST API and a range of SDKs. Priced at an hourly rate (charged per second) based on the EC2 instance types being used, which is in addition to any EC2 or EBS charges. See release notes for further detailsTechnology Information
Other Names EMR; Elastic Map Reduce Type Commercial Last Updated September 2019 - v5.26 Related Technologies
Packages Apache Flink, Apache Hadoop, Apache HBase, Apache Hive, HCatalog, Apache Livy, Apache Mahout, Apache Oozie, Apache Phoenix, Apache Pig, Apache Spark, Apache Sqoop, Apache Tez, Apache Zeppelin, Apache ZooKeeper, Hue, Presto, Ganglia, JupyterHub, MXNet, TensorFlow Release History
version release date release links release comment 5.18 2018-10-25 announcement Flink 1.6.0, Zeppelin 0.8.0, and S3 Select with Hive and Presto 5.19 2018-11-06 5.20 2018-12-14 announcement Spark 2.4, Hue 4.3 5.21 2019-02 announcement Flink 1.7; Recondiguring Apps on Running Clusters 5.22 2019-03 announcement Oozie 5.1 5.23 2019-04 Multiple Master Nodes 5.24 2019-06 announcement New versions of Flink, Presto, and Hue and Spark performance improvements 5.25 2019-08 announcement 5.26 2019-09 announcement 6.0 (beta) 2019-09 announcement Links
News
Blog Posts