使用pyenv和virtualenv在apache后面部署flask app时找不到numpy,pandas模块

时间:2017-07-03 20:07:51

标签: python flask virtualenv mod-wsgi pyenv

我为一个相当详细的应用程序构建了一个烧瓶前端,并且经过大量编码后它在本地运行完美。所以现在我想让它对公众可见,所以读过它需要一个apache或nginx前端。我计算出了我需要的实例大小,并在AWS上进行了切换,并运行了these两个tutorials,证明基本情况正常。 Apache设置完成后,Apache欢迎页面会显示我导航到AWS实例的IP的时间。然后,当我暂时替换我的烧瓶应用程序的“Hello World”代码并使用IP刷新浏览器时,将显示hello world。此外,如果我替换here中的flasskapp.py代码,它完全服务于apache到输入浏览器的IP,包括Numpy和Pandas在内的pip找到的模块列表,否则它会抱怨找不到!但是,可以通过在顶部添加import numpy来轻松杀死此脚本,但它会给我以下错误消息:

ImportError: cannot import name 'multiarray'

Importing the multiarray numpy extension module failed.  Most
likely you are trying to import a failed build of numpy.
If you're working with a numpy git repo, try `git clean -xdf` 
(removes all files not under version control).  
Otherwise reinstall numpy.

都在apache日志中。

当我把我的应用程序放回而不是hello world时,就会出现问题。它只是拒绝导入pandas这是Apache日志的最后一部分:

Mon Jul 03 19:42:14.691081 2017] [wsgi:error] [pid 12300:tid 140131159766784] [client 93.21.05.132:52560] Traceback (most recent call last):
[Mon Jul 03 19:42:14.691126 2017] [wsgi:error] [pid 12300:tid 140131159766784] [client 93.21.05.132:52560]   File "/var/www/html/flaskapp/flaskapp.wsgi", line 30, in <module>
[Mon Jul 03 19:42:14.691130 2017] [wsgi:error] [pid 12300:tid 140131159766784] [client 93.21.05.132:52560]     from similar_items_frontend import app as application
[Mon Jul 03 19:42:14.691137 2017] [wsgi:error] [pid 12300:tid 140131159766784] [client 93.21.05.132:52560]   File "/var/www/html/flaskapp/similar_items_frontend.py", line 4, in <module>
[Mon Jul 03 19:42:14.691140 2017] [wsgi:error] [pid 12300:tid 140131159766784] [client 93.21.05.132:52560]     from item_similar_to_doc import similar_to_doc
[Mon Jul 03 19:42:14.691145 2017] [wsgi:error] [pid 12300:tid 140131159766784] [client 93.21.05.132:52560]   File "/home/ubuntu/find-similar/frontend/../item_similar_to_doc.py", line 1, in <module>
[Mon Jul 03 19:42:14.691148 2017] [wsgi:error] [pid 12300:tid 140131159766784] [client 93.21.05.132:52560]     from PullVectors import p
[Mon Jul 03 19:42:14.691153 2017] [wsgi:error] [pid 12300:tid 140131159766784] [client 93.21.05.132:52560]   File "/home/ubuntu/find-similar/frontend/../RunSimilarity.py", line 5, in <module>
[Mon Jul 03 19:42:14.691156 2017] [wsgi:error] [pid 12300:tid 140131159766784] [client 93.21.05.132:52560]     import pandas as pd
[Mon Jul 03 19:42:14.691161 2017] [wsgi:error] [pid 12300:tid 140131159766784] [client 93.21.05.132:52560]   File "/home/ubuntu/.pyenv/versions/miniconda3-latest/envs/find_similarProject/lib/python3.6/site-packages/pandas/__init__.py", line 19, in <module>
[Mon Jul 03 19:42:14.691164 2017] [wsgi:error] [pid 12300:tid 140131159766784] [client 93.21.05.132:52560]     "Missing required dependencies {0}".format(missing_dependencies))
[Mon Jul 03 19:42:14.691180 2017] [wsgi:error] [pid 12300:tid 140131159766784] [client 93.21.05.132:52560] ImportError: Missing required dependencies ['numpy']

我觉得我已经尝试了很多痛苦的时间,头发拉动时间......

以下是000-default.conf

WSGIPythonPath /home/ubuntu/.pyenv/versions/miniconda3-latest/envs/vtenv4YTproject/lib/python3.6/site-packages/

<VirtualHost *:80>
        # The ServerName directive sets the request scheme, hostname and port that
        # the server uses to identify itself. This is used when creating
        # redirection URLs. In the context of virtual hosts, the ServerName
        # specifies what hostname must appear in the request's Host: header to
        # match this virtual host. For the default virtual host (this file) this
        # value is not decisive as it is used as a last resort host regardless.
        # However, you must set it for any further virtual host explicitly.
        #ServerName www.example.com

        ServerAdmin webmaster@localhost
        DocumentRoot /var/www/html

        WSGIDaemonProcess flaskapp threads=5
        WSGIScriptAlias / /var/www/html/flaskapp/flaskapp.wsgi

        <Directory flaskapp>
              WSGIScriptReloading On
              WSGIProcessGroup %{GLOBAL}
              WSGIApplicationGroup %{GLOBAL}
              Order allow,deny
              Allow from all
        </Directory>



        # Available loglevels: trace8, ..., trace1, debug, info, notice, warn,
        # error, crit, alert, emerg.
        # It is also possible to configure the loglevel for particular
        # modules, e.g.
        #LogLevel info ssl:warn

        ErrorLog ${APACHE_LOG_DIR}/error.log
        CustomLog ${APACHE_LOG_DIR}/access.log combined

        # For most configuration files from conf-available/, which are
        # enabled or disabled at a global level, it is possible to
        # include a line for only one particular virtual host. For example the
        # following line enables the CGI configuration for this host only
        # after it has been globally disabled with "a2disconf".
        #Include conf-available/serve-cgi-bin.conf
</VirtualHost>

# vim: syntax=apache ts=4 sw=4 sts=4 sr noet

我应该提一下,当我在ssh中进入AWS实例时,我的烧瓶应用程序在其virtualenv中完美运行(到了服务页面的烧瓶点),当然有点修修补上班,但它找到它的所有依赖项,而不是在pandas导入时崩溃。

要尝试修复它,我查看herehere,添加了here的一些代码,浏览了大量文档here,我的flaskapp.wsgi文件看起来像这样:

import sys
import logging
import os.path
sys.path.append(os.path.join(os.path.dirname(os.path.realpath(__file__)), os.pardir))
import config
import site

prev_sys_path = list(sys.path) 

site.addsitedir('/home/ubuntu/.pyenv/versions/miniconda3-latest/envs/find_similarProject/lib/python3.6/site-packages')

new_sys_path = []

for item in list(sys.path): 
    if item not in prev_sys_path:
        new_sys_path.append(item) 
        sys.path.remove(item)
sys.path[:0] = new_sys_path 

logging.basicConfig(stream=sys.stderr)
sys.path.insert(0, "/var/www/html/flaskapp")

from similar_vids_frontend import app as application
application.secret_key = config.WSGI_KEY

没有理由我可以找到为什么pandas / numpy导入使用apache失败但是在virtualenv中运行时工作正常。我想知道是否与权限和组有关,根据文档:

  

请注意,Apache运行代码的用户需要能够访问Python虚拟环境。在某些Linux发行版上,其他用户无法访问用户帐户的主目录。您可以更好地考虑在主目录之外找到WSGI应用程序代码和任何Python虚拟环境,而不是更改主目录的权限。

但是Ubuntu是那些发行版之一,然后将虚拟环境和项目安装到/文件夹中吗?

感谢任何提示或解决方案。

1 个答案:

答案 0 :(得分:1)

这现在有效,主要得益于Graham的配方herethis answer关于将模块加载到apache中。它的工作原理是hello world app的以下变体:

from flask import Flask
import numpy
app = Flask(__name__)

@app.route('/')
def hello_from_np():
  a = numpy.array([4,5,6])
  return str(a)

if __name__ == '__main__':
  app.run()
当你去IP时,

在浏览器中给你这个:

enter image description here

...而不是关于numpy的apache日志中的一堆错误。 这一切都是为Python 3.6设置的:

AH00489: Apache/2.4.7 (Ubuntu) mod_wsgi/4.5.15 Python/3.6 configured

想要记录一些细节作为自己的笔记,如果有人遇到麻烦并试图重现这一点,也许可以让你免于咆哮错误的树木。

所以我从Ubuntu Server 16.04切换到14.04.5 LTS,以防这是问题的一部分,但也因为pyenv表现不佳,在我测试的许多旧版本Python的安装过程中崩溃了一半。还选择省略pyenv-virtualenv,因为没有为此实例计划多个项目,并且出现问题要少一点。

从Ubuntu的干净安装,除了安装apache2,pyenv,libapache2-mod-wsgi,用a2enmod激活它之外,我故意在使用pip安装需求之前运行以下行,其中包括numpy,所以它没有'抱怨:

sudo apt-get install gcc
sudo apt-get install g++

这也是必需的:

sudo apt-get install apache2-dev

然后在环境中,在要求之后,

pip install mod_wsgi
mod_wsgi-express module-config

第二行返回:

LoadModule wsgi_module "/home/ubuntu/.pyenv/versions/miniconda3-latest/lib/python3.6/site-packages/mod_wsgi/server/mod_wsgi-py36.cpython-36m-x86_64-linux-gnu.so"
WSGIPythonHome "/home/ubuntu/.pyenv/versions/miniconda3-latest"

然后加载模块,docs鼓励你粘贴到httpd.conf文件中,至少在我的apache和ubuntu版本中不存在。经过大量阅读后,我避免创建一个,并避免将其粘贴到似乎是related的apache2.conf中。相反,遵循链接到顶部的ubuntu的说明,

编辑此文件:

sudo vi /etc/apache2/mods-available/wsgi.load

将此行粘贴到其中并保存:

LoadModule wsgi_module /path/to/mod_wsgi-py36.cpython-36m-x86_64-linux-gnu.so

然后运行此命令并在必要时重新启动Apache服务器:

sudo a2enmod wsgi

000-default.conf如下所示:

WSGIPythonHome "/home/ubuntu/.pyenv/versions/miniconda3-latest"

<VirtualHost *:80>
    # The ServerName directive sets the request scheme, hostname and port that
    # the server uses to identify itself. This is used when creating
    # redirection URLs. In the context of virtual hosts, the ServerName
    # specifies what hostname must appear in the request's Host: header to
    # match this virtual host. For the default virtual host (this file) this
    # value is not decisive as it is used as a last resort host regardless.
    # However, you must set it for any further virtual host explicitly.
    #ServerName www.example.com

    ServerAdmin webmaster@localhost
    DocumentRoot /var/www/html
        WSGIDaemonProcess helloapp threads=5 python-path=/var/www/html/frontend/  
        WSGIScriptAlias / /var/www/html/frontend/helloapp.wsgi

        <Directory flaskapp>
        WSGIProcessGroup helloapp
        WSGIApplicationGroup %{GLOBAL}
        Order allow,deny
        Allow from all
        </Directory>

    # Available loglevels: trace8, ..., trace1, debug, info, notice, warn,
    # error, crit, alert, emerg.
    # It is also possible to configure the loglevel for particular
    # modules, e.g.
    #LogLevel info ssl:warn

    ErrorLog ${APACHE_LOG_DIR}/error.log
    CustomLog ${APACHE_LOG_DIR}/access.log combined

    # For most configuration files from conf-available/, which are
    # enabled or disabled at a global level, it is possible to
    # include a line for only one particular virtual host. For example the
    # following line enables the CGI configuration for this host only
    # after it has been globally disabled with "a2disconf".
    #Include conf-available/serve-cgi-bin.conf
</VirtualHost>