2018 edit

2018-12-12 Peter

Apologies for the radio silence - I’ve been ill. But to make up for it, it’s a monster bumper news this week…

Technology updates (details are on the relevant technology pages):

Apache BigTop has hit 1.3
Apache Flink has hit 1.7
Apache Gobblin has hit 0.14
Apache Ignite has hit 2.7
Apache Impala is up to 3.1
Apache Kafka has hit 2.1
Apache Parquet has hit a 1.11 release of it’s Map Reduce implementation
CDH and Cloudera Manager is up to 5.16; Cloudera Navigator to 2.15
Elasticsearch has hit 6.5, along with Elasticsearch Hadoop
Greenplum has hit 5.14
Hortonworks Data Flow is up to 3.3
Qubole has hit R54
Streamsets Data Collector has hit 3.6
Apache Hadoop Ozone is up to Ozone 0.3 alpha

Other technology news:

Some Apache Incubator updates
- Quickstep (a high performance database engine) has been retired from the incubator due to lack of activity
- Iceberg (the file based table store) from Netflix and IoTDB (a time series db) have both been accepted into the Incubator
- Griffen (the Data Quality Service platform built on Apache Hadoop and Apache Spark) has graduated
Hortonworks have a blog post on Ambari 2.7 and why it’s great (months after it’s release) - https://hortonworks.com/blog/whats-great-apache-ambari-2-7/
Elasticsearch now supports Kubernetes deployments, with Helm charts available from Elastic - link
Hortonworks have more on their cloud journey for Hadoop and HDP - link
Microsoft have more features announced for Azure Data Lake Store Gen2 - link
Also from Microsoft, a comprehensive guide for the Information they’ve published on HDInsight - link
Amazon have a monster pile of AWS announcements:
- For Amazon S3 we have full Glacier integration; object lock; Glacier Deep Archive; batch operations; blocking public access and Intelligence Tiering
- Amazon Textract is a new OCR service - link
- Amazon Lake Formation is a new service for setting up an S3 data lake, populating it with data from a range of source systems, and then securing the data - link
- Amazon Timestream is a new Time Series database - link
- You have now pause and resume EC2 instances backed by EBS - link
- There’s a new Kafka managed service (Managed Streaming for Kafka - MSK) - link
- And you can now get Lustre filesystems as a service on AWS - link
And from Azure this week, you can now get MariaDB as a manged service - link
RedHat have acquired NooBaa - an object storage solution - link
More from Hortonworks on Ozone - link
From Datanami - an article on Pachydern, an interesting alternative to Hadoop - link
Samza 1.0 is out - link; ZDNet
From Datanami - looks like Cloudera have a new ML platform coming - link