Google Cloud Composer edit  

Managed workflow orchestration service built on Apache Airflow that's designed for running data integration tasks on a repeated schedule. Implemented on a micro-service architecture, the Airflow database and web server are implemented on App Engine and access protected using Identify-Aware Proxy (an enterprise security model that enables employees to work from untrusted networks without the use of a VPN), while the scheduler, executor and worker nodes are implemented on Kubernetes Engine. Integrated with Cloud Storage for staging DAGs, plugins, data dependencies and Stackdriver for real-time logging and monitoring of Airflow service and workflow logs. Manageable via a web (Cloud Platform Console and Airflow web interfaces), command line interface (Cloud SDK) or an RPC and REST API. Allows custom Airflow plugins and Python dependencies from the Python Package Index to be installed. Priced an an hourly rate (charged per minute) based on the size of a Cloud Composer environment, which is in addition to any Kubernetes Engine, Compute Engine or Persistent Disk and network egress charges.

Technology Information

Other Names	Cloud Composer
Type	Commercial
Last Updated	May 2019 - v1.3

Related Technologies

Packages

Apache Airflow

Links

• https://cloud.google.com/composer/ - homepage
• https://cloud.google.com/composer/docs/concepts/versioning/composer-versions - bundle services version list
• https://cloud.google.com/composer/docs/release-notes - release notes
• https://cloud.google.com/composer/docs/ - documentation
• https://github.com/GoogleCloudPlatform/python-docs-samples/ - sample Composer workflows

News

See Google Cloud Platform updates