Question

我正在尝试使用Follow this tutorial instead并行运行一些数据库操作。我在线程连接时挂起SQLAlchemy会话时遇到问题

下面是我试图并行运行的功能：

server {

    # SSL configuration
    #
    # listen 443 ssl default_server;
    # listen [::]:443 ssl default_server;
    #
    # Note: You should disable gzip for SSL traffic.
    # See: https://bugs.debian.org/773332
    #
    # Read up on ssl_ciphers to ensure a secure configuration.
    # See: https://bugs.debian.org/765782
    #
    # Self signed certs generated by the ssl-cert package
    # Don't use them in a production server!
    #
    # include snippets/snakeoil.conf;

    root /var/www/html/marketplacekit/public;

    # Add index.php to the list if you are using PHP
    index index.php index.html index.htm index.nginx-debian.html;

    server_name example.com www.example.com;


    location / {
            # First attempt to serve request as file, then
            # as directory, then fall back to displaying a 404.
            try_files $uri $uri/ /index.php?$query_string;
    }


    location ~ \.php$ {
            include snippets/fastcgi-php.conf;
            fastcgi_pass unix:/run/php/php7.2-fpm.sock;
    }

    location ~ /\.ht {
            deny all;
    }



    # pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
    #
    #location ~ \.php$ {
    #       include snippets/fastcgi-php.conf;
    #
    #       # With php7.0-cgi alone:
    #       fastcgi_pass 127.0.0.1:9000;
    #       # With php7.0-fpm:
    #       fastcgi_pass unix:/run/php/php7.0-fpm.sock;
    #}

    # deny access to .htaccess files, if Apache's document root
    # concurs with nginx's one
    #
    #location ~ /\.ht {
    #       deny all;
    #}

listen [::]:443 ssl ipv6only=on; # managed by Certbot
listen 443 ssl; # managed by Certbot
ssl_certificate /etc/letsencrypt/live/example.com-0001/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/example.com-0001/privkey.pem; # managed by Certbot
include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

如果我设置了from joblib import Parallel, delayed from sqlservice import SQLClient db = SQLClient( {'SQL_DATABASE_URI': DATABASE_URI}, model_class=Model ) def my_parallel_function(row, my_db): i, d = row foo = Foo(parent_id=None) my_db.session.add(foo) my_db.session.flush() foo_id = foo.id bar = Bar(foo_id=foo.id) my_db.session.add(bar) print(f'basis version added. Committing on session {hex(id(my_db.session))}..') foo_id = foo.id bar_id = bar.id ### ANY ONE OF THE FOLLOWING LINES HANGS IFF N_JOBS > 1 # my_db.session.flush() # my_db.session.remove() # my_db.session.close() # my_db.close() my_db.session.commit() return {'row_id': d['row_id'], 'foo_id': foo_id, 'bar_id': bar_id} # df is a Pandas DataFrame results = Parallel(n_jobs=N_JOBS, verbose=1, backend='threading') \ (delayed(cloner_versioner)(row=row, my_db=db) for row in df.T.iteritems()) db.session.commit()，则该print语句中的会话ID对于每个循环都是相同的，并且可以成功完成。

但是，如果将N_JOBS=1设置为1以外的任何值，则迭代将运行等于N_JOBS的次数，并且print语句将显示每个会话具有不同的ID。它将挂在处理会话的任何指定尝试上。

关于如何处理SQLAlchemy N_JOBS，我缺少什么？

使用线程处理SQLAlchemy会话

0 个答案: