遇到重载时,Rails应用程序不断崩溃

时间:2018-06-23 07:48:58

标签: ruby-on-rails nginx amazon-ec2 capistrano puma

我正在测试我的应用程序,因为我预计下周会很重。我目前正在使用BlazeMeter来模拟负载。我的服务器使用m5.large EC2实例托管在AWS Elasticbeanstalk和RDS上。我已经使用Puma,Capistrano和Nginx设置了我的应用程序。我的配置如下:

//nginx.conf

upstream app {
  server unix: ///home/deploy/apps/appname/shared/tmp/sockets/appname-puma.sock;
}

server {
  listen 80;
  server_name example.com;

  root / home / deploy / apps / appname / current / public;

  if ($http_x_forwarded_proto != "https") {
    return 301 https: //$server_name$request_uri;
  }
}

server {
  listen 443 default_server http2;

  root / home / deploy / apps / appname / current / public;

  try_files / system / maintenance.html $uri / index.html $uri $uri.html @app;

  access_log /
    var / log / nginx / access.log main;
  access_log /
    var / log / nginx / healthd / application.log.$year - $month - $day - $hour healthd;
  error_log /
    var / log / nginx / error.log debug;

  location / assets {
    alias /
      var / app / current / public / assets;
    gzip_static on;
    gzip on;
    expires max;
    add_header Cache - Control public;
  }

  location~ * \.(jpg | jpeg | gif | css | png | js | ico | svg | svgz) $ {
    gzip_static on;
    gzip on;
    expires max;
    add_header Cache - Control public;
  }

  location @app {
    proxy_pass http: //app; # match the name of upstream directive which is defined above
      proxy_set_header Host $http_host;
    proxy_set_header X - Real - IP $remote_addr;
    proxy_set_header X - Forwarded - For $proxy_add_x_forwarded_for;
    proxy_set_header X - Forwarded - Proto $scheme;
    proxy_set_header HTTP_CLIENT_IP $remote_addr;
  }

  error_page 500 502 503 504 / 500. html;
  location / 500. html {}
  error_page 404 / 404. html;
  location / 404. html {}
  error_page 422 / 422. html;
  location / 422. html {}
}
##config/deploy.rb

# config valid only for current version of Capistrano
lock "3.8.2"

set :application, "appname"
set :repo_url, #"censored"
set :user,            'deploy'
set :puma_threads,    [16, 64]
set :puma_workers,    2

set :pty, true
set :use_sudo,        false
set :stage,           :production
set :deploy_via,      :remote_cache
set :deploy_to,       "/home/#{fetch(:user)}/apps/#{fetch(:application)}"#{}"/var/app/current"
set :puma_bind,       "unix://#{shared_path}/tmp/sockets/#{fetch(:application)}-puma.sock"
set :puma_state,      "#{shared_path}/tmp/pids/puma.state"
set :puma_pid,        "#{shared_path}/tmp/pids/puma.pid"
set :puma_access_log, "#{release_path}/log/puma.error.log"
set :puma_error_log,  "#{release_path}/log/puma.access.log"
set :ssh_options,     { forward_agent: true, user: fetch(:user), keys: %w(~/.ssh/id_rsa.pub) }
set :puma_preload_app, true
set :puma_worker_timeout, nil
set :puma_init_active_record, true  # Change to false when not using ActiveRecord

append :linked_files, %w{config/database.yml}
append :linked_dirs, ".bundle", "tmp", "public/system"#,  %w{bin log tmp/pids tmp/cache tmp/store vendor/bundle public/system}

namespace :puma do
  desc 'Create Directories for Puma Pids and Socket'
  task :make_dirs do
    on roles(:app) do
      execute "mkdir #{shared_path}/tmp/sockets -p"
      execute "mkdir #{shared_path}/tmp/pids -p"
    end
  end

  before :start, :make_dirs
end

namespace :deploy do
  desc "Make sure local git is in sync with remote."
  task :check_revision do
    on roles(:app) do
      unless `git rev-parse HEAD` == `git rev-parse origin/master`
        puts "WARNING: HEAD is not the same as origin/master"
        puts "Run `git push` to sync changes."
        exit
      end
    end
  end

  desc 'Initial Deploy'
  task :initial do
    on roles(:app) do
      before 'deploy:restart', 'puma:start'
      invoke 'deploy'
    end
  end

  desc 'Restart application'
  task :restart do
    on roles(:app), in: :sequence, wait: 5 do
      invoke 'puma:restart'
    end
  end

  before :starting,     :check_revision
  after  :finishing,    :compile_assets
end

有许多测试失败,错误率很高。根据我来自elasticbeanstalk的日志,

  

[错误] 2465#0:* 2262测试“ / var / app / current / public / assets”的存在失败(2:没有此类文件或目录)

     

[暴击] 2465#0:* 2216连接到unix:///var/run/puma/my_app.sock失败(2:没有此类文件或目录),同时连接到上游

我是这个的新手,我不知道为什么会这样!任何帮助表示赞赏!谢谢!

更新1:在达到一定数量的模拟用户后,我的网站也显示502 Bad Gateway Nginx

更新2: 就像Myst指出的那样,我也在使用数据库。

default: &default
    adapter: sqlite3
    pool: 5
    timeout: 5000

development:
    <<: *default
    database: db/development.sqlite3

test:
    <<: *default
    adapter: mysql2
    encoding: utf8
    database: <%= ENV['RDS_DB_NAME'] %>
    username: <%= ENV['RDS_USERNAME'] %>  
    password: <%= ENV['RDS_PASSWORD'] %>
    host: <%= ENV['RDS_HOSTNAME'] %> 
    port: <%= ENV['RDS_PORT'] %>

production:
    <<: *default
    adapter: mysql2
    encoding: utf8
    database: <%= ENV['RDS_DB_NAME'] %>  
    username: <%= ENV['RDS_USERNAME'] %>
    password: <%= ENV['RDS_PASSWORD'] %>
    host: <%= ENV['RDS_HOSTNAME'] %> 
    port: <%= ENV['RDS_PORT'] %>
    pool: 50 #20
    timeout: 10000

版本: capistrano3-puma(3.1.0) 红宝石(2.3.0) 导轨(4.2.8) mysql2(0.4.9)

1 个答案:

答案 0 :(得分:0)

通常-首先处理高负载时,最好衡量计算机指标-在稳定的同时,就RPS /同时客户端而言,它可以承受多少负载。在此测试中-LA,内存使用情况,IO确定当前瓶颈是什么资源。

在nginx(server unix:///... max_conns=20;)中设置上游连接限制,默认情况下它是不受限制的,并且在压力测试下,这可能会导致工作线程膨胀并出现内存不足错误。 一旦工作人员死亡,nginx无法连接到套接字并报告错误502。

查看您的puma_error_log(看起来与配置中的访问日志混淆)是否有异常。

此外,由于您只有2个cpu内核-我怀疑是否需要64个线程乘以2个工作线程,除非您的大多数请求导致等待对外部api的调用。