首页 文章

在Conda环境中安装Apache-Airflow

提问于
浏览
0

情况

我正在尝试使用pip将apache-airflow安装到conda环境中 . 我无法,因为setup.py egg_info一直失败:

命令“python setup.py egg_info”失败,错误代码1在C:\ Users \ joshu \ AppData \ Local \ Temp \ pip-install-3efyslfh \ apache-airflow \

我跑的时候留言

pip install "apache-airflow[s3, postgres]"

我在Windows cmd中运行它,然后在cmd中再次执行所有步骤(在打开cmd时切换输入) .

(myVenv) C:\Users\joshu\Documents>pip install "apache-airflow[s3, postgres]"
Collecting apache-airflow[postgres,s3]
  Using cached https://files.pythonhosted.org/packages/9e/12/6c70f9ef852b3061a3a6c9af03bd9dcdcaecb7d75c8898f82e3a54ad5f87/apache-airflow-1.9.0.tar.gz
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "C:\Users\joshu\AppData\Local\Temp\pip-install-3efyslfh\apache-airflow\setup.py", line 102
        async = [
              ^
    SyntaxError: invalid syntax

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in C:\Users\joshu\AppData\Local\Temp\pip-install-3efyslfh\apache-airflow\

我采取的步骤

1)我首先创建了一个conda环境,并将pip和setuptools安装到环境中:

C:\Users\joshu\Documents>conda create -n airflow pip setuptools
Solving environment: done


==> WARNING: A newer version of conda exists. <==
  current version: 4.5.4
  latest version: 4.5.10

Please update conda by running

    $ conda update -n base conda



## Package Plan ##

  environment location: C:\Users\joshu\Anaconda3\envs\airflow

  added / updated specs:
    - pip
    - setuptools


The following NEW packages will be INSTALLED:

    certifi:        2018.8.13-py37_0
    pip:            10.0.1-py37_0
    python:         3.7.0-hea74fb7_0
    setuptools:     40.0.0-py37_0
    vc:             14.1-h0510ff6_3
    vs2015_runtime: 15.5.2-3
    wheel:          0.31.1-py37_0
    wincertstore:   0.2-py37_0

C:\Users\joshu\Documents>activate airflow

(airflow) C:\Users\joshu\Documents> pip freeze
certifi==2018.8.13
wincertstore==0.2

(airflow) C:\Users\joshu\Documents>conda list
# packages in environment at C:\Users\joshu\Anaconda3\envs\airflow:
#
# Name                    Version                   Build  Channel
certifi                   2018.8.13                py37_0
pip                       10.0.1                   py37_0
python                    3.7.0                hea74fb7_0
setuptools                40.0.0                   py37_0
vc                        14.1                 h0510ff6_3
vs2015_runtime            15.5.2                        3
wheel                     0.31.1                   py37_0
wincertstore              0.2                      py37_0

(airflow) C:\Users\joshu\Documents>where pip
C:\Users\joshu\Anaconda3\envs\airflow\Scripts\pip.exe
C:\Users\joshu\Anaconda3\Scripts\pip.exe

(airflow) C:\Users\joshu\Documents>which pip
/c/Users/joshu/Anaconda3/envs/airflow/Scripts/pip

After this setup, running pip install apache-airflow gives me the error at the top of this page

我采取的其他步骤

  • 在我的系统上安装apache-airflow(没有conda环境) . 这有效,但我希望它包含在一个环境中 .

  • 升级setuptools:

代码:

(airflow) C:\Users\joshu\Documents>pip install --upgrade setuptools
Collecting setuptools
  Using cached https://files.pythonhosted.org/packages/66/e8/570bb5ca88a8bcd2a1db9c6246bb66615750663ffaaeada95b04ffe74e12/setuptools-40.2.0-py2.py3-none-any.whl
Installing collected packages: setuptools
  Found existing installation: setuptools 40.0.0
    Uninstalling setuptools-40.0.0:
      Successfully uninstalled setuptools-40.0.0
Successfully installed setuptools-40.2.0
You are using pip version 10.0.1, however version 18.0 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.

(airflow) C:\Users\joshu\Documents>pip freeze
certifi==2018.8.13
wincertstore==0.2
You are using pip version 10.0.1, however version 18.0 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.

(airflow) C:\Users\joshu\Documents>conda list
# packages in environment at C:\Users\joshu\Anaconda3\envs\airflow:
#
# Name                    Version                   Build  Channel
certifi                   2018.8.13                py37_0
pip                       10.0.1                   py37_0
python                    3.7.0                hea74fb7_0
setuptools                40.2.0                    <pip>
setuptools                40.0.0                   py37_0
vc                        14.1                 h0510ff6_3
vs2015_runtime            15.5.2                        3
wheel                     0.31.1                   py37_0
wincertstore              0.2                      py37_0

(airflow) C:\Users\joshu\Documents>pip install "apache-airflow[s3, postgres]"
Collecting apache-airflow[postgres,s3]
  Using cached https://files.pythonhosted.org/packages/9e/12/6c70f9ef852b3061a3a6c9af03bd9dcdcaecb7d75c8898f82e3a54ad5f87/apache-airflow-1.9.0.tar.gz
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "C:\Users\joshu\AppData\Local\Temp\pip-install-n1v4sa6d\apache-airflow\setup.py", line 102
        async = [
              ^
    SyntaxError: invalid syntax

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in C:\Users\joshu\AppData\Local\Temp\pip-install-n1v4sa6d\apache-airflow\

我也试过升级pip然后运行安装 - 没有运气

(airflow) C:\Users\joshu\Documents>python -m pip install --upgrade pip
Collecting pip
  Using cached https://files.pythonhosted.org/packages/5f/25/e52d3f31441505a5f3af41213346e5b6c221c9e086a166f3703d2ddaf940/pip-18.0-py2.py3-none-any.whl
Installing collected packages: pip
  Found existing installation: pip 10.0.1
    Uninstalling pip-10.0.1:
      Successfully uninstalled pip-10.0.1
Successfully installed pip-18.0

(airflow) C:\Users\joshu\Documents>which pip
/c/Users/joshu/Anaconda3/envs/airflow/Scripts/pip

(airflow) C:\Users\joshu\Documents>pip --version
pip 18.0 from c:\users\joshu\anaconda3\envs\airflow\lib\site-packages\pip (python 3.7)

(airflow) C:\Users\joshu\Documents>pip install "apache-airflow[s3, postgres]"

最终,我想在conda环境中为我的项目提供所需的气流和相关软件包 .

2 回答

  • 1

    该错误似乎是因为async is a reserved keyword in Python 3.7并且此程序包尚未更新其 setup.py 脚本 . 您可以使用Python 3.6直到更新包

    conda create -n airflow pip setuptools python=3.6
    pip install "apache-airflow[s3, postgres]"
    
  • 0

    我遇到了同样的问题 . Apache Airflow 1.10不兼容 .

    我的解决方案是构建源代码 . 我通过git获得了最新的源代码 . 此命令在dist目录中创建一个wheel文件 .

    python3 setup.py bdist_wheel
    

    然后你安装

    pip3 install apache_airflow-2.0.0.dev0+incubating-py3-none-any.whl
    

相关问题