Setup Test Environment using MySQLΒΆ

By default, Airflow uses SQLite as database backend and SequentialExecutor to execute tasks as SQLite does not support multiple connections. Since SequentialExecutor runs one instance at a time, some parallel execution logic will not be exercised in this default setup.

To test out the parallel execution setup, we can use MySQL as database backend and LocalExecutor as the executor. Checkout the following setups to launch a MySQL database container:

# Launch MySQL docker container
docker-compose -f scripts/ci/docker-compose.yml run -p3306:3306 mysql

# Open airflow.cfg and add the following:
# executor = LocalExecutor
vim $AIRFLOW_HOME/airflow.cfg

export AIRFLOW__CORE__SQL_ALCHEMY_CONN=mysql://root@127.0.0.1:3306/AIRFLOW_HOME

airflow db init