2019 edit

2019-08-07 Peter

Another big dump of new releases and product news this week. Remember, you can get daily news updates from our twitter feed (@OnDataEng)…

Technology updates (details are on the relevant technology pages):

Elasticsearch and Elasticsearch-Hadoop 7.3 are now out
Alpha 2 of Elastic Cloud on Kubernetes is also out - link
Cloudera Enterprise 6.3 is out, with upgrades to Kafka, HBase, Impala and Kudu
Cloudera Altus Director is also up to 6.3
Confluent Enterprise and Confluent Open Source are up to 5.3, with support for Kubernetes and Role Based Access Control
Apache Beam 2.14 is out
Apache Accumulo 2.0 is out, with a new fluent API, Bulk Import API, custom table stats, prioritisable scan executors, and of course, official Docker support
Amazon EMR 5.25 is out, with a couple of Spark performance improvements
StreamSets Data Collector 3.10 is out, with support for ingesting data directly from NiFi
Delta Lake, the open sourced Databricks Delta, is up to 0.3 - link

Other technology news:

From ZDNet, DGraph - an open source graph database written in Go - has just received a funding round - link
If you’re interested in Brooklin, the open source tool from LinkedIn for moving streaming data around, InfoQ have a presentation for you - link
MapRs long running will they/won’t they saga has come to an end with their purchase by HPE - Datanami ask why - link
From Cloudera, best practice for deploying Apache Hadoop, including your Linux configuration - link
An intro to Prefect, an evolution of Apache Airflow to support modern data applications - open source with commercial backing - link
Looks like Microsoft has purchased BlueTalon, which could mean better fine grained access control over your data ecosystem in Azure - link
Amazon have announced PartiQL, an evolution of SQL and SQL++ for querying relational, NoSQL and other structured data - link