在Docker Compose多容器应用程序中,如何防止Postgres运行以前发出的语句

时间:2017-06-30 12:52:44

标签: postgresql docker docker-compose

我正在使用项目管道运行Scrapy项目,该项目管道将数据写入PostgreSQL数据库。它是Docker Compose多容器应用程序,具有以下docker-compose.yml

version: '3'

services:
  privoxy:
    build: ./privoxy
    links:
      - tor

  tor:
    build:
      context: ./tor
      args:
        password: "foo"

  scraper:
    build: ./scraper
    environment:
      - http_proxy=http://privoxy:8118
      - PGPASSWORD=iperpassword
    links:
      - tor
      - privoxy
      - db
    volumes:
      - scraper-data:/scraper/apkmirror_scraper/data
      - scraper-jobdir:/scraper/apkmirror_scraper/crawls

  db:
    image: postgres
    environment:
      - POSTGRES_PASSWORD=iperpassword
    volumes:
      - db-data:/var/lib/postgresql/data

volumes:
  scraper-data:
  scraper-jobdir:
  db-data:

我正在使用官方存储库中的postgres图像(https://hub.docker.com/_/postgres/),并将/var/lib/postgresql/data目录放入https://docs.docker.com/compose/compose-file/#volume-configuration-reference之后的卷中。

问题在于,当我启动应用程序时,db服务会输出与我之前尝试过的事情有关的日志和错误,而不是我现在尝试做的事情。这就是我所看到的:

apkmirrorscrapercompose_tor_1 is up-to-date
apkmirrorscrapercompose_db_1 is up-to-date
apkmirrorscrapercompose_privoxy_1 is up-to-date
Recreating apkmirrorscrapercompose_scraper_1
Attaching to apkmirrorscrapercompose_tor_1, apkmirrorscrapercompose_db_1, apkmirrorscrapercompose_privoxy_1, apkmirrorscrapercompose_scraper_1
scraper_1  |                                  List of databases
scraper_1  |    Name    |  Owner   | Encoding |  Collate   |   Ctype    |   Access privileges   
scraper_1  | -----------+----------+----------+------------+------------+-----------------------
scraper_1  |  postgres  | postgres | UTF8     | en_US.utf8 | en_US.utf8 | 
scraper_1  |  template0 | postgres | UTF8     | en_US.utf8 | en_US.utf8 | =c/postgres          +
scraper_1  |            |          |          |            |            | postgres=CTc/postgres
scraper_1  |  template1 | postgres | UTF8     | en_US.utf8 | en_US.utf8 | =c/postgres          +
scraper_1  |            |          |          |            |            | postgres=CTc/postgres
scraper_1  | (3 rows)
scraper_1  | 
scraper_1  | Postgres is up - executing command
scraper_1  | Tor appears to be working. Proceeding with command...
scraper_1  | 2017-06-30 12:32:59 [scrapy.utils.log] INFO: Scrapy 1.4.0 started (bot: apkmirror_scraper)
scraper_1  | 2017-06-30 12:32:59 [scrapy.utils.log] INFO: Overridden settings: {'BOT_NAME': 'apkmirror_scraper', 'NEWSPIDER_MODULE': 'apkmirror_scraper.spiders', 'SPIDER_MODULES': ['apkmirror_scraper.spiders']}
scraper_1  | 2017-06-30 12:32:59 [scrapy.middleware] INFO: Enabled extensions:
scraper_1  | ['scrapy.extensions.corestats.CoreStats',
scraper_1  |  'scrapy.extensions.telnet.TelnetConsole',
scraper_1  |  'scrapy.extensions.memusage.MemoryUsage',
scraper_1  |  'scrapy.extensions.closespider.CloseSpider',
scraper_1  |  'scrapy.extensions.logstats.LogStats']
db_1       | The files belonging to this database system will be owned by user "postgres".
db_1       | This user must also own the server process.
db_1       | 
db_1       | The database cluster will be initialized with locale "en_US.utf8".
db_1       | The default database encoding has accordingly been set to "UTF8".
db_1       | The default text search configuration will be set to "english".
db_1       | 
db_1       | Data page checksums are disabled.
db_1       | 
db_1       | fixing permissions on existing directory /var/lib/postgresql/data ... ok
db_1       | creating subdirectories ... ok
db_1       | selecting default max_connections ... 100
db_1       | selecting default shared_buffers ... 128MB
db_1       | selecting dynamic shared memory implementation ... posix
db_1       | creating configuration files ... ok
db_1       | running bootstrap script ... ok
db_1       | performing post-bootstrap initialization ... ok
db_1       | 
db_1       | WARNING: enabling "trust" authentication for local connections
db_1       | You can change this by editing pg_hba.conf or using the option -A, or
db_1       | --auth-local and --auth-host, the next time you run initdb.
db_1       | syncing data to disk ... ok
db_1       | 
db_1       | Success. You can now start the database server using:
db_1       | 
db_1       |     pg_ctl -D /var/lib/postgresql/data -l logfile start
db_1       | 
db_1       | waiting for server to start....LOG:  database system was shut down at 2017-06-29 13:40:40 UTC
db_1       | LOG:  MultiXact member wraparound protections are now enabled
db_1       | LOG:  database system is ready to accept connections
db_1       | LOG:  autovacuum launcher started
db_1       |  done
db_1       | server started
db_1       | ALTER ROLE
db_1       | 
db_1       | 
db_1       | /usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
db_1       | 
db_1       | LOG:  received fast shutdown request
db_1       | waiting for server to shut down....LOG:  aborting any active transactions
db_1       | LOG:  autovacuum launcher shutting down
db_1       | LOG:  shutting down
db_1       | LOG:  database system is shut down
db_1       |  done
db_1       | server stopped
db_1       | 
db_1       | PostgreSQL init process complete; ready for start up.
db_1       | 
db_1       | LOG:  database system was shut down at 2017-06-29 13:40:41 UTC
db_1       | LOG:  MultiXact member wraparound protections are now enabled
db_1       | LOG:  database system is ready to accept connections
db_1       | LOG:  autovacuum launcher started
db_1       | LOG:  received smart shutdown request
db_1       | LOG:  autovacuum launcher shutting down
db_1       | LOG:  shutting down
db_1       | LOG:  database system is shut down
db_1       | LOG:  database system was shut down at 2017-06-29 13:41:53 UTC
db_1       | LOG:  MultiXact member wraparound protections are now enabled
db_1       | LOG:  database system is ready to accept connections
db_1       | LOG:  autovacuum launcher started
db_1       | LOG:  received smart shutdown request
db_1       | LOG:  autovacuum launcher shutting down
db_1       | LOG:  shutting down
db_1       | LOG:  database system is shut down
db_1       | LOG:  database system was shut down at 2017-06-29 13:43:22 UTC
db_1       | LOG:  MultiXact member wraparound protections are now enabled
db_1       | LOG:  database system is ready to accept connections
db_1       | LOG:  autovacuum launcher started
db_1       | LOG:  received smart shutdown request
db_1       | LOG:  autovacuum launcher shutting down
db_1       | LOG:  shutting down
db_1       | LOG:  database system is shut down
db_1       | LOG:  database system was shut down at 2017-06-29 13:52:02 UTC
db_1       | LOG:  MultiXact member wraparound protections are now enabled
db_1       | LOG:  database system is ready to accept connections
db_1       | LOG:  autovacuum launcher started
db_1       | ERROR:  column "developer" of relation "apks" does not exist at character 31
db_1       | STATEMENT:  INSERT INTO apks (url, title, developer, app, version_name, version_code, architectures, package, apk_file_size, android_min_version, android_target_version, supported_dpis, md5_signature, time_uploaded, time_scraped, image_urls, images, file_urls, files, created_on, updated_on) VALUES ('http://www.apkmirror.com/apk/sony-mobile-communications/backup-restore-unknown-developer/backup-restore-unknown-developer-1-2-a-1-23-release/backup-restore-1-2-a-1-23-android-apk-download/', 'Backup & restore 1.2.A.1.23', 'Sony Mobile Communications', 'Backup & restore', '1.2.A.1.23', '2360343', ARRAY['arm'], 'com.sonymobile.synchub', 9541452, '6.0', '7.1', ARRAY['nodpi'], 'eda39fedfc33ea939a5bb4926b3980c1', '2017-04-25T16:30:00+00:00'::timestamptz, '2017-06-29T14:10:39+00:00'::timestamptz, ARRAY['http://www.apkmirror.com/wp-content/themes/APKMirror/ap_resize/ap_resize.php?src=http%3A%2F%2Fwww.apkmirror.com%2Fwp-content%2Fuploads%2F2017%2F04%2F58ff36aea8e0c.png&w=96&h=96&q=100'], '{}', ARRAY['http://www.apkmirror.com/wp-content/themes/APKMirror/download.php?id=203068'], '{}', '2017-06-29T14:19:21.661193'::timestamp, '2017-06-29T14:19:21.661207'::timestamp) RETURNING apks.id
db_1       | LOG:  received smart shutdown request
db_1       | LOG:  autovacuum launcher shutting down
db_1       | LOG:  shutting down
db_1       | LOG:  database system is shut down
db_1       | LOG:  database system was shut down at 2017-06-29 16:28:54 UTC
db_1       | LOG:  MultiXact member wraparound protections are now enabled

虽然我没有显示有关应用程序的详细信息,但我的主要观点是,我肯定错误消息中显示的数据来自之前的运行,而不是来自我下载的APK下载页面#&# 39; m目前正试图刮伤。

简而言之,似乎PostgreSQL正在缓存'以前的陈述,即使在我停止申请后也尝试重新运行它们,请docker-compose builddocker-compose up。在我看来,这种行为必须以某种方式由/var/lib/postgresql/data中的数据驱动,因为这个卷是重新启动容器时唯一的持久性。

确实如此吗?如果是这样,我怎么能禁用这个语句缓存'行为?

1 个答案:

答案 0 :(得分:3)

这只是意味着docker-compose正在重新启动先前停止的容器(或重新连接到它,如果它已经在运行),并且因为没有必要重新构建映像:

apkmirrorscrapercompose_db_1 is up-to-date

因此,您在日志中“再次”看到的是该容器的docker logs,其中包含postgres的所有生命生成日志。

只需删除容器并重新开始(数据库中的数据将保留在卷中,因此它将继续存在):

docker-compose rm db