Skip to content

airflowser ๐Ÿ’  : Cloud Composer #airflow #dag

Notifications You must be signed in to change notification settings

miozilla/airflowser

Folders and files

NameName
Last commit message
Last commit date

Latest commit

ย 

History

5 Commits
ย 
ย 
ย 
ย 
ย 
ย 

Repository files navigation

airflowser ๐Ÿ’ 

airflowser : #airflow #composer #dag

Objective

  • To create and run Apache Airflow workflows in Cloud Composer to move data between Cloud Storage and BigQuery
  • Reads from a config file that provides a list of tables to copy
  • Exports the list of tables from a BigQuery dataset located in US to Cloud Storage
  • Copies the exported tables from the US to EU Cloud Storage bucket
  • Imports the list of tables into the target BigQuery Dataset in EU

Apache Airflow in Cloud Composer

screenshot_airflowser001

screenshot_airflowser002

screenshot_airflowser003

screenshot_airflowser004

screenshot_airflowser005

screenshot_airflowser006

screenshot_airflowser007

screenshot_airflowser008

screenshot_airflowser009

screenshot_airflowser010

About

airflowser ๐Ÿ’  : Cloud Composer #airflow #dag

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages