我们有静态nginx缓存,集群中的每个节点都像负载均衡器和数据节点一样。
我们正在使用一致性哈希进行负载平衡。这是简单的设置
HTTP客户端->负载均衡器-> [如果结果在缓存中可用,则返回]->使用一致哈希散列路由-> [如果缓存,则返回]-> [从s3读取]
代理缓存路径:
proxy_cache_path /var/cache/lb-nginx
levels=1:2 keys_zone=lb_cache:10m max_size=200g
loader_files=10000 loader_threshold=10m loader_sleep=50ms
inactive=365d use_temp_path=off;
proxy_cache_path /var/cache/nginx
levels=1:2 keys_zone=s3_cache:10m max_size=600g
loader_files=10000 loader_threshold=10m loader_sleep=50ms
inactive=365d use_temp_path=off;
上游配置
# load balancer redirects autogen
upstream backend {
hash $request_uri consistent;
# This helps to not to create connection to backend everytime.
keepalive 34 ;
}
# load balancer end point
server {
error_log /var/log/nginx/xxx_debug_lb.log debug;
listen 80;
server_name hostname.abc.yyz 1.2.3.4; # hostname ipaddress
location / {
resolver 1.2.3.4;
# refer : http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_intercept_errors
# Proxy backend settings.
proxy_intercept_errors on;
proxy_pass http://backend;
error_log /var/log/nginx/_debug_lb_2.log debug;
# Cache results in LB as well to save bandwidth
proxy_cache lb_cache;
proxy_cache_valid 200 30d; # 30 days
proxy_cache_valid 404 1d; # 1 day
proxy_cache_use_stale error timeout http_500 http_502 http_503 http_504;
proxy_cache_lock on;
proxy_cache_bypass $http_cache_purge;
# disable convert get to head, otherwise, HEAD request will be denied
proxy_cache_convert_head off;
# disable cache convert requires setting cache key, $request_method is not part of proxy_cache_key by default
proxy_cache_methods GET HEAD;
proxy_cache_key $scheme$host$request_method$request_uri;
add_header Cache-Control max-age=2851000;
add_header X-Lb-Cache-Status $upstream_cache_status;
# http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_valid
# make sure the response will be cached.
proxy_ignore_headers Set-Cookie;
}
}
S3终点
server {
error_log /var/log/nginx/_debug_s3.log debug;
listen 80 default_server;
location /lua_debug {
return 200 "Hello world!";
}
location / {
# Calling aws_auth, to set HTTP Headers
set $s3_host s3.amazonaws.com;
set $bucket "abcd"
set $s3_uri "/$bucket$uri";
access_by_lua "local aws = require 'resty.aws_auth'; aws.s3_set_headers(ngx.var.s3_host, ngx.var.s3_uri)";
# Somehow nginx is looking for resolver ¯\_(ツ)_/¯
resolver 1.2.3.4;
# refer : http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_intercept_errors
# Proxy backend settings.
proxy_intercept_errors on;
proxy_pass https://$s3_host$s3_uri;
error_log /var/log/nginx/_debug_s3_2.log debug;
# Cache related settings
proxy_cache s3_cache;
proxy_cache_valid 200 365d; # 1year
proxy_cache_valid 404 1d; # 1day
proxy_cache_use_stale error timeout http_500 http_502 http_503 http_504;
proxy_cache_lock on;
proxy_cache_bypass $http_cache_purge;
# disable convert get to head, otherwise, HEAD request will be denied
proxy_cache_convert_head off;
# disable cache convert requires setting cache key, $request_method is not part of proxy_cache_key by default
proxy_cache_methods GET HEAD;
proxy_cache_key $scheme$request_method$proxy_host$request_uri;
add_header Cache-Control max-age=31536000;
add_header X-Cache-Status $upstream_cache_status;
# http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_valid
# make sure the response will be cached.
proxy_ignore_headers Set-Cookie;
}
}
但是,当我尝试使用IP地址(由于server_name支持ip和主机名)来达到负载平衡器时, 1.对于初始请求,它显示缓存-HIT / MISS
X缓存状态:HIT X-Lb-Cache-Status:MISS
用于HIT / HIT的后续请求
X缓存状态:HIT X-Lb-Cache-Status:命中率
大约15分钟后,再次出现缓存未命中
X缓存状态:HIT X-Lb-Cache-Status:MISS
有人可以帮助我,我想念什么?