2021年5月5日星期三

Airflow Docker - No module named 'pyodbc'

I have a Python script which uses Pyodbc to work with SQL Server, I'm trying to run this script with an Airflow DAG:

auction_task = PythonOperator(    task_id='auction_etl',    python_callable=get_auction_data,    dag=dag  )  

get_auction_data function in another file:

import json  import requests  import datetime  import pyodbc  import time    def get_auction_data(config):      return retrieve_from_api(config)  

But I keep getting this error in the UI:

Broken DAG: [/opt/airflow/dags/auctionds/etl/auction_etl.py] Traceback (most recent call last):    File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed    File "/opt/airflow/dags/auctionds/etl/auction_etl.py", line 9, in <module>      import pyodbc  ModuleNotFoundError: No module named 'pyodbc'  

Airflow is running inside a Docker container, I tried to manually install pyodbc in the worker container with no success. Also, if I run python command inside the worker container and run 'import pyodbc' it works with no problem. I don't know what I'm missing. Thanks!

https://stackoverflow.com/questions/67410920/airflow-docker-no-module-named-pyodbc May 06, 2021 at 10:05AM

没有评论:

发表评论