Apache Storm edit  

Specialised distributed stream processing technology based on a single record (not micro batch) model with at least once processing semantics. Processing flows are called topologies based on a directed acyclic graph of spouts (which produce unbounded streams of tuples) and bolts (which process streams and optionally produce output streams). Supports high throughput and low latency use cases, horizontal scalability, fault tolerance (failed workers are automatically restarted and failed over to new nodes if required), back pressure, windowing (with support for sliding and tumbling windows based on time or event counts), stateful bolts and a shared bolt storage cache (that's updatable from the command line). Also includes a higher level micro batch API (Trident) that supports exactly-once processing semantics, fault-tolerant state management and higher level operations including joins, aggregations and groupings, support for SQL (StormSQL) and frameworks and utilities to make defining and deploying topologies easier (Flux). Has both a graphical web based and command line interface, plus a REST API. Primarily written in Clojure, JVM based, but supports multiple languages through the use of Thrift for defining and submitting topologies, and the use of spouts that can interface to other languages using JSON over stdin/stdout. Originally created at BackType, before being open sourced in September 2011 after the acquisition of BackType by Twitter. Donated to the Apache Foundation in September 2013, graduating in September 2014, with a 1.0 release in April 2016. Has multiple reference cases for being deployed at scale, including Twitter, and is still under active development.

Technology Information

Other NamesStorm
VendorsThe Apache Software Foundation
TypeCommercial Open Source
Last UpdatedJune 2019 - v2.0

Related Technologies

Is packaged byHortonworks Data Platform, Hortonworks DataFlow, MapR Expansion Pack

Release History

versionrelease daterelease linksrelease comment
1.12017-03-29release notes 
1.22018-02-16release notes 
2.02019-05-30release notes 

News

Blog Posts