背景：尝试在标准应用程序引擎上使用带有Django的新Google Cloud Build自动化我的构建过程。我从这里的好Google员工提供的Django民意调查示例开始： https://github.com/GoogleCloudPlatform/python-docs-samples/tree/master/appengine/standard_python37/django
环境：带有Django 3.0.1的Python3.7
对上述内容进行了一些更改，现在我的requirements.txt如下：
requirements.txt
coverage==5.0.1
Django==3.0.1
entrypoints==0.3
flake8==3.7.9
mccabe==0.6.1
mysqlclient==1.4.6
pycodestyle==2.5.0
pyflakes==2.1.1
pytz==2019.3
sqlparse==0.3.0
app.yaml
runtime: python37
handlers:
# This configures Google App Engine to serve the files in the app's static
# directory.
- url: /static
static_dir: static/
# This handler routes all requests not caught above to your main app. It is
# required when static routes are defined, but can be omitted (along with
# the entire handlers section) when there are no static files defined.
- url: /.*
script: auto
env_variables:
# the secret key used for the Django app (from PROJECT-DIRECTORY/settings.py)
SECRET_KEY: 'DJANGO-SECRET-KEY'
DEBUG: 'False' # always False for deployment
# everything after /cloudsql/ can be found by entering >> gcloud sql instances describe DATABASE-NAME << in your Terminal
# the DATABASE-NAME is the name you gave your project's PostgreSQL database
# the second line from the describe output called connectionName can be copied and pasted after /cloudsql/
DB_HOST: '/cloudsql/annular-will-XXXX:asia-east2:XXXX'
DB_PORT: '5432' # PostgreSQL port
DB_NAME: 'XXX'
DB_USER: 'XXX'
DB_PASSWORD: 'XXXXX'
# [END django_app]
使用命令行进行部署的时间
gcloud app deploy app.yaml行，是的！
是时候自动构建了，并按照这里的步骤进行操作：
我将cloudbuild.yaml的名称进行了一些小的更改，将其更改为cloud-build.yaml
结果：在推送到分支机构时触发了构建！到目前为止，一切看起来都很好！
让我们看看cloud_build.yaml： 请注意，这是[cloud build] [1]
快速入门指南中提到的步骤的一部分。
steps:
- name: "gcr.io/cloud-builders/gcloud"
args: ["app", "deploy"]
timeout: "1600s"
结果：应用已部署，但请等待，静态资产未加载:( 让我们只在我们的cloud-build.yaml中添加步骤，现在看起来像这样：
- name: 'python:3.7'
entrypoint: python3
args: ['-m', 'pip', 'install', '-t', '.', '-r', 'requirements.txt']
- name: 'python:3.7'
entrypoint: python3
args: ['./manage.py', 'collectstatic', '--noinput']
- name: "gcr.io/cloud-builders/gcloud"
args: ["app", "deploy"]
timeout: "1600s"
我们需要安装Requirements.txt，因为collectstatic需要安装Django和其他依赖项。 注意传递给pip install的-t参数。之所以通过，是因为google cloud-build.yaml的每个步骤都在单独的docker映像中运行，并且唯一常见的是/ workspace目录。因此，将所有内容安装在工作空间中是有意义的。在此步骤之后，静电消除器起作用！
查看Cloud Build的历史看起来都是绿色的： [！[在此处输入图片描述] [2]] [2]
但是该应用拒绝启动，该实例死于以下错误：
2019-12-27 01:10:49 default[20191227t010033] [2019-12-27 01:10:49 +0000] [7] [INFO] Starting gunicorn 20.0.4
2019-12-27 01:10:49 default[20191227t010033] [2019-12-27 01:10:49 +0000] [7] [INFO] Listening at: http://0.0.0.0:8081 (7)
2019-12-27 01:10:49 default[20191227t010033] [2019-12-27 01:10:49 +0000] [7] [INFO] Using worker: threads
2019-12-27 01:10:49 default[20191227t010033] [2019-12-27 01:10:49 +0000] [18] [INFO] Booting worker with pid: 18
2019-12-27 01:10:49 default[20191227t010033] [2019-12-27 01:10:49 +0000] [22] [INFO] Booting worker with pid: 22
2019-12-27 01:10:51 default[20191227t010033] [2019-12-27 01:10:51 +0000] [22] [ERROR] Exception in worker process
2019-12-27 01:10:51 default[20191227t010033] Traceback (most recent call last): File "/srv/django/db/backends/mysql/base.py", line 16, in <module> import MySQLdb as Database File "/srv/MySQLdb/__init__.py", line 18, in <module> from . import _mysql ImportError: libmariadb.so.3: cannot open shared object file: No such file or directory
2019-12-27 01:10:51 default[20191227t010033]
2019-12-27 01:10:51 default[20191227t010033] The above exception was the direct cause of the following exception:
2019-12-27 01:10:51 default[20191227t010033]
2019-12-27 01:10:51 default[20191227t010033] Traceback (most recent call last): File "/env/lib/python3.7/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker worker.init_process() File "/env/lib/python3.7/site-packages/gunicorn/workers/gthread.py", line 92, in init_process super().init_process() File "/env/lib/python3.7/site-packages/gunicorn/workers/base.py", line 119, in init_process self.load_wsgi() File "/env/lib/python3.7/site-packages/gunicorn/workers/base.py", line 144, in load_wsgi self.wsgi = self.app.wsgi() File "/env/lib/python3.7/site-packages/gunicorn/app/base.py", line 67, in wsgi self.callable = self.load() File "/env/lib/python3.7/site-packages/gunicorn/app/wsgiapp.py", line 49, in load return self.load_wsgiapp() File "/env/lib/python3.7/site-packages/gunicorn/app/wsgiapp.py", line 39, in load_wsgiapp return util.import_app(self.app_uri) File "/env/lib/python3.7/site-packages/gunicorn/util.py", line 358, in import_app mod = importlib.import_module(module) File "/opt/python3.7/lib/python3.7/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 1006, in _gcd_import File "<frozen importlib._bootstrap>", line 983, in _find_and_load File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 677, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 728, in exec_module File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed File "/srv/main.py", line 1, in <module> from mysite.wsgi import application File "/srv/mysite/wsgi.py", line 16, in <module> application = get_wsgi_application() File "/srv/django/core/wsgi.py", line 12, in get_wsgi_application django.setup(set_prefix=False) File "/srv/django/__init__.py", line 24, in setup apps.populate(settings.INSTALLED_APPS) File "/srv/django/apps/registry.py", line 114, in populate app_config.import_models() File "/srv/django/apps/config.py", line 211, in import_models self.models_module = import_module(models_module_name) File "/opt/python3.7/lib/python3.7/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "/srv/polls/models.py", line 7, in <module> class Question(models.Model): File "/srv/django/db/models/base.py", line 121, in __new__ new_class.add_to_class('_meta', Options(meta, app_label)) File "/srv/django/db/models/base.py", line 325, in add_to_class value.contribute_to_class(cls, name) File "/srv/django/db/models/options.py", line 208, in contribute_to_class self.db_table = truncate_name(self.db_table, connection.ops.max_name_length()) File "/srv/django/db/__init__.py", line 28, in __getattr__ return getattr(connections[DEFAULT_DB_ALIAS], item) File "/srv/django/db/utils.py", line 207, in __getitem__ backend = load_backend(db['ENGINE']) File "/srv/django/db/utils.py", line 111, in load_backend return import_module('%s.base' % backend_name) File "/opt/python3.7/lib/python3.7/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "/srv/django/db/backends/mysql/base.py", line 21, in <module> ) from err django.core.exceptions.ImproperlyConfigured: Error loading MySQLdb module.
2019-12-27 01:10:51 default[20191227t010033] Did you install mysqlclient?
2019-12-27 01:10:51 default[20191227t010033] [2019-12-27 01:10:51 +0000] [22] [INFO] Worker exiting (pid: 22)
2019-12-27 01:10:51 default[20191227t010033] [2019-12-27 01:10:51 +0000] [18] [ERROR] Exception in worker process
2019-12-27 01:10:51 default[20191227t010033] Traceback (most recent call last): File "/srv/django/db/backends/mysql/base.py", line 16, in <module> import MySQLdb as Database File "/srv/MySQLdb/__init__.py", line 18, in <module> from . import _mysql ImportError: libmariadb.so.3: cannot open shared object file: No such file or directory
2019-12-27 01:10:51 default[20191227t010033]
2019-12-27 01:10:51 default[20191227t010033] The above exception was the direct cause of the following exception:
2019-12-27 01:10:51 default[20191227t010033]
2019-12-27 01:10:51 default[20191227t010033] Traceback (most recent call last): File "/env/lib/python3.7/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker worker.init_process() File "/env/lib/python3.7/site-packages/gunicorn/workers/gthread.py", line 92, in init_process super().init_process() File "/env/lib/python3.7/site-packages/gunicorn/workers/base.py", line 119, in init_process self.load_wsgi() File "/env/lib/python3.7/site-packages/gunicorn/workers/base.py", line 144, in load_wsgi self.wsgi = self.app.wsgi() File "/env/lib/python3.7/site-packages/gunicorn/app/base.py", line 67, in wsgi self.callable = self.load() File "/env/lib/python3.7/site-packages/gunicorn/app/wsgiapp.py", line 49, in load return self.load_wsgiapp() File "/env/lib/python3.7/site-packages/gunicorn/app/wsgiapp.py", line 39, in load_wsgiapp return util.import_app(self.app_uri) File "/env/lib/python3.7/site-packages/gunicorn/util.py", line 358, in import_app mod = importlib.import_module(module) File "/opt/python3.7/lib/python3.7/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 1006, in _gcd_import File "<frozen importlib._bootstrap>", line 983, in _find_and_load File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 677, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 728, in exec_module File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed File "/srv/main.py", line 1, in <module> from mysite.wsgi import application File "/srv/mysite/wsgi.py", line 16, in <module> application = get_wsgi_application() File "/srv/django/core/wsgi.py", line 12, in get_wsgi_application django.setup(set_prefix=False) File "/srv/django/__init__.py", line 24, in setup apps.populate(settings.INSTALLED_APPS) File "/srv/django/apps/registry.py", line 114, in populate app_config.import_models() File "/srv/django/apps/config.py", line 211, in import_models self.models_module = import_module(models_module_name) File "/opt/python3.7/lib/python3.7/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "/srv/polls/models.py", line 7, in <module> class Question(models.Model): File "/srv/django/db/models/base.py", line 121, in __new__ new_class.add_to_class('_meta', Options(meta, app_label)) File "/srv/django/db/models/base.py", line 325, in add_to_class value.contribute_to_class(cls, name) File "/srv/django/db/models/options.py", line 208, in contribute_to_class self.db_table = truncate_name(self.db_table, connection.ops.max_name_length()) File "/srv/django/db/__init__.py", line 28, in __getattr__ return getattr(connections[DEFAULT_DB_ALIAS], item) File "/srv/django/db/utils.py", line 207, in __getitem__ backend = load_backend(db['ENGINE']) File "/srv/django/db/utils.py", line 111, in load_backend return import_module('%s.base' % backend_name) File "/opt/python3.7/lib/python3.7/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "/srv/django/db/backends/mysql/base.py", line 21, in <module> ) from err django.core.exceptions.ImproperlyConfigured: Error loading MySQLdb module.
2019-12-27 01:10:51 default[20191227t010033] Did you install mysqlclient?
2019-12-27 01:10:51 default[20191227t010033] [2019-12-27 01:10:51 +0000] [18] [INFO] Worker exiting (pid: 18)
2019-12-27 01:10:51 default[20191227t010033] [2019-12-27 01:10:51 +0000] [7] [INFO] Shutting down: Master
2019-12-27 01:10:51 default[20191227t010033] [2019-12-27 01:10:51 +0000] [7] [INFO] Reason: Worker failed to boot.
如果有帮助，我在调试过程中看到了另一个奇怪的错误：
2019-12-27 01:31:35 default[20191227t092757] Traceback (most recent call last): File "/env/lib/python3.7/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker worker.init_process() File "/env/lib/python3.7/site-packages/gunicorn/workers/gthread.py", line 92, in init_process super().init_process() File "/env/lib/python3.7/site-packages/gunicorn/workers/base.py", line 119, in init_process self.load_wsgi() File "/env/lib/python3.7/site-packages/gunicorn/workers/base.py", line 144, in load_wsgi self.wsgi = self.app.wsgi() File "/env/lib/python3.7/site-packages/gunicorn/app/base.py", line 67, in wsgi self.callable = self.load() File "/env/lib/python3.7/site-packages/gunicorn/app/wsgiapp.py", line 49, in load return self.load_wsgiapp() File "/env/lib/python3.7/site-packages/gunicorn/app/wsgiapp.py", line 39, in load_wsgiapp return util.import_app(self.app_uri) File "/env/lib/python3.7/site-packages/gunicorn/util.py", line 358, in import_app mod = importlib.import_module(module) File "/opt/python3.7/lib/python3.7/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 1006, in _gcd_import File "<frozen importlib._bootstrap>", line 983, in _find_and_load File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 677, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 728, in exec_module File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed File "/srv/main.py", line 1, in <module> from mysite.wsgi import application File "/srv/mysite/wsgi.py", line 12, in <module> from django.core.wsgi import get_wsgi_application ModuleNotFoundError: No module named 'django'```
At this point, i have almost given up on google cloud CD. If you have any working example of how to CI/CD on google app engine with Django, pls let me know.
further debugging:
- Switched to gcloud app deploy --version 1 and it still works. Note that this happens from the command line from my project and my static directory in settings:
# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/2.1/howto/static-files/
STATIC_ROOT = 'static'
STATIC_URL = '/static/'
Before the app deploy, i run python collectstatic thus creating the static directory. Hence i assume when i do app deploy, the image already has the static directory created. However, during cloud build, we had to explicitly make it as a step. Just thought of mentioning it.
-rw-r--r-- 1 amit staff 868B Dec 26 13:53 README.md
-rw-r--r-- 1 amit staff 1.1K Dec 27 09:21 app.yaml
-rw-r--r-- 1 amit staff 88B Dec 27 09:43 cloud-build.yaml
-rw-r--r-- 1 amit staff 492B Dec 26 14:34 main.py
-rwxr-xr-x 1 amit staff 538B Dec 26 13:53 manage.py*
drwxr-xr-x 7 amit staff 224B Dec 26 14:15 mysite/
drwxr-xr-x 12 amit staff 384B Dec 27 00:27 polls/
-rwxr-xr-x 1 amit staff 108B Dec 26 13:53 proxy.sh*
-rw-r--r-- 1 amit staff 173B Dec 27 09:43 requirements.txt
drwxr-xr-x 3 amit staff 96B Dec 26 13:57 static/
drwxr-xr-x 6 amit staff 192B Dec 26 15:27 venv/
[1]: https://cloud.google.com/source-repositories/docs/quickstart-triggering-builds-with-source-repositories
[2]: https://i.stack.imgur.com/aalUR.png
答案 0 :(得分：0)
我接受了@methkal Khalawi的评论 为了避免mysqlclient的所有麻烦，建议您使用pymysql，它更稳定地包含在需求文件中。
但是，PyMySQL存在一个现有问题，因此它不适用于Django3.x。这是链接https://github.com/PyMySQL/PyMySQL/issues/790
为了避免上述情况，请在您的设置文件中执行以下操作：
import pymysql
# Hack to make PyMySql work with Django 3.x
# #https://github.com/PyMySQL/PyMySQL/issues/790
pymysql.version_info = (1, 4, 6, 'final', 0) # change mysqlclient version
pymysql.install_as_MySQLdb()
现在，从requirements.txt中删除mysqlclient并获取PyMySql：
Django==3.0.2
PyMySQL==0.9.3
现在，cloud-build.yaml看起来像这样：
steps:
- name: 'python:3.7'
entrypoint: python3
args: ['-m', 'pip', 'install', '-t', '.', '-r', 'requirements.txt']
- name: 'python:3.7'
entrypoint: python3
args: ['./manage.py', 'collectstatic', '--noinput']
- name: "gcr.io/cloud-builders/gcloud"
args: ["app", "deploy"]
timeout: "1600s"
而且有效！