The Week That Was - 10/03/2017 edit
And so we come to the end of the Hortonworks Data Platform technologies. On Monday we’ll start looking at the remainder of the Hortonworks technology offerings (yes - I know we’re meant to be working our way through the Hadoop distributions - it’ll only be a short detour), but for now let’s summarise what we’ve looked at this week.
First up was the second add-on to HDP based on a partnership with another company - Hortonworks HBD (aka Pivotal HDB, aka Apache Hawq). We then looked at the management components of the HDP stack - Apache Ambari, Cloudbreak and Hortonworks SmartSense.
So - Hortonworks HBD. It’s Pivotal HDB - you download it from Pivotal and the Hortonworks documentation links through to the Pivotal documentation, but with Hortonworks (according to the press release) providing customer support and professional implementation services. The press release is worth a look - as part of the deal Pivotal agreed to drop Pivotal HD (their Hadoop distribution) and resell HDP instead, and Hortonworks agreed to distribute HDB. But given their investment in Hive through the Stinger initiative, you have to wonder how interested Hortonworks are in pushing it. Which is possibly a shame, because Apache Hawq is probably the most mature SQL engine available on Hadoop today - whether that’s a good or bad thing, and how much traction it’s going to get I don’t know.
And on to Apache Ambari - Hortonworks competitor to Cloudera Manager. What I found most interesting about Ambari was the list of contributors - 50 from Hortonworks, 6 from IBM, 6 from Pivotal, 4 from RedHat plus some more - 82 in total. That’s a pretty significant development capacity, and probably goes to show how valuable having an easy way to provision and manage Hadoop clusters is to its adoption. What also struck me about Ambari was how much it didn’t feel like an open source technology - it only (realistically) supports the installation of HDP, the committers are all employees of large companies and the Apache documentation is pretty poor.
I split Ambari Views out from Ambari for a couple of reasons. Firstly, I ran out of time on Tuesday to include it in the Ambari technology summary, and there was probably too much to put in a single summary anyway, but also because I think Ambari Views is targeting a slightly different use case and group of users than Ambari. It feels like Hortonworks is lining this up as a competitor to Hue (they have a Hue to Ambari migration tool for starters), however it feels lightweight (in features) and heavyweight (in terms of hardware requirements) compared to Hue, and I don’t see it gaining the same traction. Perhaps if Cloudera had donated Hue to the Apache Foundation Ambari Views wouldn’t even exist.
And Cloudbreak - an extremely interesting technology where cloud infrastructure meets Docker meets Hadoop. It feels like early days for Cloudbreak, however Hadoop in the Cloud (on or off premesis) is seeing massive investment from both Cloudera and Hortonworks at the moment, and it’s a really interesting area that I’d love to come back to at some point.
Lastly to Hortonworks SmartSense. Hortonworks’ business model is interesting - their commitment to open source means you can use their entire technology stack for free, you only pay for support and professional services, but this means their support and services offering has to deliver value and be worth the money (which also means that their stack can’t be too reliable or easy to manage without their help). SmartSense is their only technology that isn’t open source, and is the key piece in the value of their support offering. And the fact it appears to contain a bunch of cluster analytics that aren’t available through Ambari is an interesting facet of that.