Microsoft Azure Data Lake Store edit  

Massively scalable HDFS compatible filesystem as a service, based on Microsoft's Cosmos technology. Claims support for up to trillions of files and single files larger than one petabyte, with no limits on account sizes, file sizes or the amount of data that can be stored, and optimisation of parallel analytics workloads, with high throughput and IOPS performance. Supports user authentication via Azure Active Directory (AAD) (combined with OAuth and OpenID), role based access control for account management, POSIX ACLs for controlling access to data, encryption for both stored data and data in transit over the network,and built in auditing (of both data access and account management activities). Supports a standard WebHDFS API, an HDFS compatible interface (adl://) that's bundled with Apache Hadoop, a blob storage API, a web UI (Data Explorer) and SDKs for a range of languages. Does not natively support geo-replication with filesystems limited to a region, but data can be manually replicated via a number of routes if required. First announced in April 2015, with a general availability release in November 2016.

Technology Information

Other Names	Azure Data Lake Store
Vendors	Microsoft
Type	Commercial
Last Updated	June 2017

Links

• https://azure.microsoft.com/en-us/services/data-lake-store/ - home page
• https://docs.microsoft.com/en-us/azure/data-lake-store/data-lake-store-overview - documentation
• https://blogs.msdn.microsoft.com/azuredatalake/2016/11/17/azure-data-lake-store-is-now-generally-available/ - announcement post
• https://docs.microsoft.com/en-us/azure/data-lake-store/data-lake-store-comparison-with-blob-storage - comparison of Blob Storage and Data Lake Store

Blog Posts

• The Week That Was - 16/06/2017 2017-06-16 Azure Data Lake Store Druid Superset HDF Schema Registory SAM Hadoop Distributions Peter