Airflow Xcom Example

We will demonstrate both the method and the Automatic method.

# Define default arguments default_args = 'owner': 'airflow', 'depends_on_past': False, 'start_date': datetime(2023, 1, 1), 'retries': 1, 'retry_delay': timedelta(minutes=5),

You have a DAG that:

from airflow import DAG from airflow.operators.python import PythonOperator from pendulum import datetime def push_function(ti): ti.xcom_push(key='model_accuracy', value=0.95) def pull_function(ti): # Pulling a specific key from a specific task accuracy = ti.xcom_pull(key='model_accuracy', task_ids='push_task') print(f"Model accuracy is: accuracy") with DAG('traditional_xcom_dag', start_date=datetime(2023, 1, 1), schedule=None) as dag: push_task = PythonOperator( task_id='push_task', python_callable=push_function ) pull_task = PythonOperator( task_id='pull_task', python_callable=pull_function ) push_task >> pull_task Use code with caution. Important Limitations to Remember

Mastering Airflow XComs: A Practical Guide with Examples In Apache Airflow, tasks are designed to be atomic and isolated. This is great for reliability, but it poses a challenge: airflow xcom example

# Task 3: Pull Automatic XCom using Jinja Templating # Note: BashOperator supports Jinja templating in bash_command t3 = BashOperator( task_id='log_result', bash_command='echo " ti.xcom_pull(task_ids="process_data") "', )

extract >> process

Inside generate_data , we use kwargs['ti'] . This gives us access to the Task Instance. We specifically call xcom_push with a custom key 'random_number' . This is useful when a single task needs to push multiple different values.