我在Elastic Beanstalk上运行了一个休息api,效果很好。应用方面的一切都运行良好,并按预期工作。
该应用程序是一个rest api,用于查找不同的用户。
example url: http://service.com/user?uid=xxxx&anotherid=xxxx
如果找到具有任何ID的用户,则api会以200 OK
回复,否则,会根据404 Not Found
回复。 HTTP/1.1
状态码保护。
我们的api在很多请求上回答404 Not Found
并且弹性beanstalk将我们的环境从OK
转移到Warning
甚至转移到Degraded
的情况并不少见因为这。由于这种降级状态,nginx
似乎拒绝了与应用程序的连接。 (看起来它有{30}的阈值进入warning
和50%进入degraded
状态。这是一个问题,因为应用程序实际上按预期工作,但Elastic Beanstalks默认设置认为这是一个问题,当它真的没有。
有没有人知道在EB中编辑4xx警告和状态转换阈值的方法,还是完全禁用它们?
或者我应该真的做一个症状治疗并停止在这样的电话上使用404 Not Found
吗? (我真的不喜欢这个选项)
答案 0 :(得分:56)
更新:AWS EB最终包含一个内置设置: https://stackoverflow.com/a/51556599/1123355
旧解决方案:潜入EB实例并花费几个小时寻找EB的健康检查守护程序实际上将状态代码报告给EB进行评估的地方,我终于找到了,并提出了一个补丁,可以作为一个完美的解决方法,防止4xx
响应代码将环境转变为Degraded
环境健康状态,并毫无意义地通过此电子邮件通知您:
Environment health has transitioned from Ok to Degraded. 59.2 % of the requests are erroring with HTTP 4xx.
状态代码报告逻辑位于healthd-appstat
内,这是由EB团队开发的Ruby脚本,它不断监视/var/log/nginx/access.log
并将状态代码报告给EB,具体在以下路径中:
/opt/elasticbeanstalk/lib/ruby/lib/ruby/gems/2.2.0/gems/healthd-appstat-1.0.1/lib/healthd-appstat/plugin.rb
以下.ebextensions
文件将修补此Ruby脚本,以避免将4xx
响应代码报告回EB。这意味着EB不会因4xx
错误而降低环境健康状况,因为它只是不知道它们正在发生。这也意味着"健康"您的EB环境中的页面将始终显示0
响应代码计数{。}}。
4xx
是的,它有点难看,但它完成了工作,至少在EB团队通过某些配置参数提供忽略container_commands:
01-patch-healthd:
command: "sudo /bin/sed -i 's/\\# normalize units to seconds with millisecond resolution/if status \\&\\& status.index(\"4\") == 0 then next end/g' /opt/elasticbeanstalk/lib/ruby/lib/ruby/gems/2.2.0/gems/healthd-appstat-1.0.1/lib/healthd-appstat/plugin.rb"
02-restart-healthd:
command: "sudo /usr/bin/kill $(/bin/ps aux | /bin/grep -e '/bin/bash -c healthd' | /usr/bin/awk '{ print $2 }')"
ignoreErrors: true
错误的方法之前。在部署时,将其与您的应用程序一起包含在相对于项目根目录的以下路径中:
4xx
答案 1 :(得分:21)
答案 2 :(得分:14)
感谢您的回答Elad Nava,我遇到了同样的问题,您的解决方案对我来说非常合适!
但是,在AWS Support Center中打开故障单后,他们建议我修改nginx
配置以忽略健康检查上的4xx,而不是修改ruby脚本。为此,我还必须将配置文件添加到.ebextensions
目录,以覆盖默认的nginx.conf
文件:
files:
"/tmp/nginx.conf":
content: |
# Elastic Beanstalk Managed
# Elastic Beanstalk managed configuration file
# Some configuration of nginx can be by placing files in /etc/nginx/conf.d
# using Configuration Files.
# http://docs.amazonwebservices.com/elasticbeanstalk/latest/dg/customize-containers.html
#
# Modifications of nginx.conf can be performed using container_commands to modify the staged version
# located in /tmp/deployment/config/etc#nginx#nginx.conf
# Elastic_Beanstalk
# For more information on configuration, see:
# * Official English Documentation: http://nginx.org/en/docs/
# * Official Russian Documentation: http://nginx.org/ru/docs/
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /var/run/nginx.pid;
worker_rlimit_nofile 1024;
events {
worker_connections 1024;
}
http {
###############################
# CUSTOM CONFIG TO IGNORE 4xx #
###############################
map $status $loggable {
~^[4] 0;
default 1;
}
map $status $modstatus {
~^[4] 200;
default $status;
}
#####################
# END CUSTOM CONFIG #
#####################
port_in_redirect off;
include /etc/nginx/mime.types;
default_type application/octet-stream;
# This log format was modified to ignore 4xx status codes!
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
log_format healthd '$msec"$uri"'
'$modstatus"$request_time"$upstream_response_time"'
'$http_x_forwarded_for' if=$loggable;
sendfile on;
include /etc/nginx/conf.d/*.conf;
keepalive_timeout 1200;
}
container_commands:
01_modify_nginx:
command: cp /tmp/nginx.conf /tmp/deployment/config/#etc#nginx#nginx.conf
虽然这个解决方案非常详细,但我个人认为实施更安全,只要它不依赖于任何AWS专有脚本。我的意思是,如果出于某种原因,AWS决定删除或修改他们的ruby脚本(不管我信不信,他们喜欢在没有事先通知的情况下更改脚本),sed
的解决方法很可能会不再工作了。
答案 3 :(得分:2)
以下是基于Adriano Valente's answer的解决方案。我无法使$loggable
位工作,虽然跳过404的日志记录似乎是一个很好的解决方案。我只创建了一个定义.conf
变量的新$modstatus
文件,然后覆盖healthd
日志格式以使用$modstatus
代替$status
。此更改还需要重新启动nginx。这适用于运行Ruby 2.3(Puma)的Elastic Beanstalk的64位Amazon Linux 2016.09 v2.3.1。
# .ebextensions/nginx.conf
files:
"/tmp/nginx.conf":
content: |
# Custom config to ignore 4xx in the health file only
map $status $modstatus {
~^[4] 200;
default $status;
}
container_commands:
modify_nginx_1:
command: "cp /tmp/nginx.conf /etc/nginx/conf.d/custom_status.conf"
modify_nginx_2:
command: sudo sed -r -i 's@\$status@$modstatus@' /opt/elasticbeanstalk/support/conf/webapp_healthd.conf
modify_nginx_3:
command: sudo /etc/init.d/nginx restart
答案 4 :(得分:1)
基于Elad Nava's Answer,我认为最好直接使用elasticbeanstalk healthd的控制脚本而不是kill:
container_commands:
01-patch-healthd:
command: "sudo /bin/sed -i 's/\\# normalize units to seconds with millisecond resolution/if status \\&\\& status.index(\"4\") == 0 then next end/g' /opt/elasticbeanstalk/lib/ruby/lib/ruby/gems/2.2.0/gems/healthd-appstat-1.0.1/lib/healthd-appstat/plugin.rb"
02-restart-healthd:
command: "sudo /opt/elasticbeanstalk/bin/healthd-restart"
最后,在调查此问题时,我注意到健康和apache日志状态代码与使用%s的前者不同,而后者%> s导致它们之间存在差异。我也用这个补丁:
03-healthd-logs:
command: sed -i 's/^LogFormat.*/LogFormat "%{%s}t\\"%U\\"%>s\\"%D\\"%D\\"%{X-Forwarded-For}i" healthd/g' /etc/httpd/conf.d/healthd.conf
答案 5 :(得分:1)
我最近遇到了同样被4xx错误轰炸的问题。我尝试了上面列出的建议,但没有任何对我有用。我联系了AWS Support,这是他们的建议,它解决了我的问题。我有一个运行2个实例的Elastic Beanstalk应用程序。
以下是nginix.config文件的全部内容:
files:
"/etc/nginx/nginx.conf":
content: |
# Elastic Beanstalk Nginx Configuration File
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
# Custom config
# HTTP 4xx ignored.
map $status $loggable {
~^[4] 0;
default 1;
}
# Custom config
# HTTP 4xx ignored.
map $status $modstatus {
~^[4] 200;
default $status;
}
include /etc/nginx/mime.types;
default_type application/octet-stream;
access_log /var/log/nginx/access.log;
log_format healthd '$msec"$uri"$modstatus"$request_time"$upstream_response_time"$http_x_forwarded_for';
include /etc/nginx/conf.d/*.conf;
include /etc/nginx/sites-enabled/*;
}
答案 6 :(得分:1)
AWS支持自2018年4月起提供的解决方案:
files:
"/tmp/custom-site-nginx.conf":
mode: "000664"
owner: root
group: root
content: |
map $http_upgrade $connection_upgrade {
default "upgrade";
"" "";
}
# Elastic Beanstalk Modification(EB_INCLUDE)
# Custom config
# HTTP 4xx ignored.
map $status $loggable {
~^[4] 0;
default 1;
}
server {
listen 80;
gzip on;
gzip_comp_level 4;
gzip_types text/html text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;
if ($time_iso8601 ~ "^(\d{4})-(\d{2})-(\d{2})T(\d{2})") {
set $year $1;
set $month $2;
set $day $3;
set $hour $4;
}
access_log /var/log/nginx/healthd/application.log.$year-$month-$day-$hour healthd if=$loggable;
access_log /var/log/nginx/access.log;
location / {
proxy_pass http://docker;
proxy_http_version 1.1;
proxy_set_header Connection $connection_upgrade;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
container_commands:
override_beanstalk_nginx:
command: "mv -f /tmp/custom-site-nginx.conf /etc/nginx/sites-available/elasticbeanstalk-nginx-docker-proxy.conf"