使用Varnish 3.0.4，我们看到很多backend_busy，但我们没有在任何后端设置max_connection设置。
我正在试图找出原因，因为似乎有些客户请求被中止了。
什么可能导致backend_busy？
感谢任何提示
这是我的varnishstat -1
client_conn 0 0.00 Client connections accepted
client_drop 0 0.00 Connection dropped, no sess/wrk
client_req 0 0.00 Client requests received
cache_hit 0 0.00 Cache hits
cache_hitpass 0 0.00 Cache hits for pass
cache_miss 0 0.00 Cache misses
backend_conn 0 0.00 Backend conn. success
backend_unhealthy 0 0.00 Backend conn. not attempted
backend_busy 386013 8.38 Backend conn. too many
backend_fail 0 0.00 Backend conn. failures
backend_reuse 1140751 24.77 Backend conn. reuses
backend_toolate 292966 6.36 Backend conn. was closed
backend_recycle 31329 0.68 Backend conn. recycles
backend_retry 364845 7.92 Backend conn. retry
fetch_head 93857 2.04 Fetch head
fetch_length 0 0.00 Fetch with Length
fetch_chunked 0 0.00 Fetch chunked
fetch_eof 1007 0.02 Fetch EOF
fetch_bad 766163 16.63 Fetch had bad headers
fetch_close 16152 0.35 Fetch wanted close
fetch_oldhttp 783263 17.00 Fetch pre HTTP/1.1 closed
fetch_zero 14372 0.31 Fetch zero len
fetch_failed 83 0.00 Fetch failed
fetch_1xx 792868 17.21 Fetch no body (1xx)
fetch_204 28600 0.62 Fetch no body (204)
fetch_304 0 0.00 Fetch no body (304)
n_sess_mem 0 . N struct sess_mem
n_sess 849 . N struct sess
n_object 0 . N struct object
n_vampireobject 0 . N unresurrected objects
n_objectcore 0 . N struct objectcore
n_objecthead 0 . N struct objecthead
n_waitinglist 0 . N struct waitinglist
n_vbc 674 . N struct vbc
n_wrk 1819 . N worker threads
n_wrk_create 155 0.00 N worker threads created
n_wrk_failed 237296 5.15 N worker threads not created
n_wrk_max 0 0.00 N worker threads limited
n_wrk_lqueue 237394 5.15 work request queue length
n_wrk_queued 237937 5.17 N queued work requests
n_wrk_drop 128 0.00 N dropped work requests
n_backend 96 . N backends
n_expired 135 . N expired objects
n_lru_nuked 7710 . N LRU nuked objects
n_lru_moved 0 . N LRU moved objects
losthdr 0 0.00 HTTP header overflows
n_objsendfile 0 0.00 Objects sent with sendfile
n_objwrite 49182 1.07 Objects sent with write
n_objoverflow 0 0.00 Objects overflowing workspace
s_sess 25 0.00 Total Sessions
s_req 104479 2.27 Total Requests
s_pipe 0 0.00 Total pipe
s_pass 205233 4.46 Total pass
s_fetch 0 0.00 Total fetch
s_hdrbytes 0 0.00 Total header bytes
s_bodybytes 1093243 23.73 Total body bytes
sess_closed 0 0.00 Session Closed
sess_pipeline 385979 8.38 Session Pipeline
sess_readahead 1140751 24.77 Session Read Ahead
sess_linger 758 0.02 Session Linger
sess_herd 482049 10.46 Session herd
shm_records 823074 17.87 SHM records
shm_writes 470696378 10218.54 SHM writes
shm_flushes 10834209993 235204.18 SHM flushes due to overflow
shm_cont 167868 3.64 SHM MTX contention
shm_cycles 19 0.00 SHM cycles through buffer
sms_nreq 18 0.00 SMS allocator requests
sms_nobj 1092005 . SMS outstanding allocations
sms_nbytes 635568 . SMS outstanding bytes
sms_balloc 85592445 . SMS bytes allocated
sms_bfree 5323477 . SMS bytes freed
backend_req 42 0.00 Backend requests made
n_vcl 44231 0.96 N vcl total
n_vcl_avail 37 0.00 N vcl available
n_vcl_discard 23820 0.52 N vcl discarded
n_ban 0 . N total active bans
n_ban_add 0 0.00 N new bans added
n_ban_retire 2136286212 46377.49 N old bans deleted
n_ban_obj_test 2136286212 46377.49 N objects tested
n_ban_re_test 857240 18.61 N regexps tested against
n_ban_dups 1 0.00 N duplicate bans removed
hcb_nolock 1 0.00 HCB Lookups without lock
hcb_lock 0 0.00 HCB Lookups with lock
hcb_insert 28435 0.62 HCB Inserts
esi_errors 14884 0.32 ESI parse errors (unlock)
esi_warnings 246931 5.36 ESI parse warnings (unlock)
accept_fail 218496 4.74 Accept failures
client_drop_late 5603745 121.65 Connection dropped late
uptime 29071691616 631128.92 Client uptime
dir_dns_lookups 46063 1.00 DNS director lookups
dir_dns_failed 685875 14.89 DNS director failed lookups
dir_dns_hit 356122 7.73 DNS director cached lookups hit
dir_dns_cache_full 355962 7.73 DNS director full dnscache
vmods 0 . Loaded VMODs
n_gzip 0 0.00 Gzip operations
n_gunzip 0 0.00 Gunzip operations
LCK.sms.creat 0 0.00 Created locks
LCK.sms.destroy 0 0.00 Destroyed locks
LCK.sms.locks 0 0.00 Lock Operations
LCK.sms.colls 0 0.00 Collisions
LCK.smp.creat 0 0.00 Created locks
LCK.smp.destroy 0 0.00 Destroyed locks
LCK.smp.locks 0 0.00 Lock Operations
LCK.smp.colls 0 0.00 Collisions
LCK.sma.creat 0 0.00 Created locks
LCK.sma.destroy 0 0.00 Destroyed locks
LCK.sma.locks 0 0.00 Lock Operations
LCK.sma.colls 0 0.00 Collisions
LCK.smf.creat 0 0.00 Created locks
LCK.smf.destroy 0 0.00 Destroyed locks
LCK.smf.locks 0 0.00 Lock Operations
LCK.smf.colls 0 0.00 Collisions
LCK.hsl.creat 0 0.00 Created locks
LCK.hsl.destroy 0 0.00 Destroyed locks
LCK.hsl.locks 0 0.00 Lock Operations
LCK.hsl.colls 0 0.00 Collisions
LCK.hcb.creat 0 0.00 Created locks
LCK.hcb.destroy 0 0.00 Destroyed locks
LCK.hcb.locks 0 0.00 Lock Operations
LCK.hcb.colls 0 0.00 Collisions
LCK.hcl.creat 0 0.00 Created locks
LCK.hcl.destroy 0 0.00 Destroyed locks
LCK.hcl.locks 0 0.00 Lock Operations
LCK.hcl.colls 0 0.00 Collisions
LCK.vcl.creat 0 0.00 Created locks
LCK.vcl.destroy 0 0.00 Destroyed locks
LCK.vcl.locks 0 0.00 Lock Operations
LCK.vcl.colls 0 0.00 Collisions
LCK.stat.creat 0 0.00 Created locks
LCK.stat.destroy 0 0.00 Destroyed locks
LCK.stat.locks 0 0.00 Lock Operations
LCK.stat.colls 0 0.00 Collisions
LCK.sessmem.creat 0 0.00 Created locks
LCK.sessmem.destroy 0 0.00 Destroyed locks
LCK.sessmem.locks 0 0.00 Lock Operations
LCK.sessmem.colls 0 0.00 Collisions
LCK.wstat.creat 0 0.00 Created locks
LCK.wstat.destroy 0 0.00 Destroyed locks
LCK.wstat.locks 0 0.00 Lock Operations
LCK.wstat.colls 0 0.00 Collisions
LCK.herder.creat 0 0.00 Created locks
LCK.herder.destroy 0 0.00 Destroyed locks
LCK.herder.locks 0 0.00 Lock Operations
LCK.herder.colls 0 0.00 Collisions
LCK.wq.creat 0 0.00 Created locks
LCK.wq.destroy 0 0.00 Destroyed locks
LCK.wq.locks 0 0.00 Lock Operations
LCK.wq.colls 0 0.00 Collisions
LCK.objhdr.creat 0 0.00 Created locks
LCK.objhdr.destroy 0 0.00 Destroyed locks
LCK.objhdr.locks 0 0.00 Lock Operations
LCK.objhdr.colls 0 0.00 Collisions
LCK.exp.creat 0 0.00 Created locks
LCK.exp.destroy 0 0.00 Destroyed locks
LCK.exp.locks 0 0.00 Lock Operations
LCK.exp.colls 0 0.00 Collisions
LCK.lru.creat 0 0.00 Created locks
LCK.lru.destroy 0 0.00 Destroyed locks
LCK.lru.locks 0 0.00 Lock Operations
LCK.lru.colls 0 0.00 Collisions
LCK.cli.creat 0 0.00 Created locks
LCK.cli.destroy 0 0.00 Destroyed locks
LCK.cli.locks 0 0.00 Lock Operations
LCK.cli.colls 0 0.00 Collisions
LCK.ban.creat 0 0.00 Created locks
LCK.ban.destroy 0 0.00 Destroyed locks
LCK.ban.locks 0 0.00 Lock Operations
LCK.ban.colls 0 0.00 Collisions
LCK.vbp.creat 0 0.00 Created locks
LCK.vbp.destroy 0 0.00 Destroyed locks
LCK.vbp.locks 0 0.00 Lock Operations
LCK.vbp.colls 0 0.00 Collisions
LCK.vbe.creat 0 0.00 Created locks
LCK.vbe.destroy 0 0.00 Destroyed locks
LCK.vbe.locks 0 0.00 Lock Operations
LCK.vbe.colls 0 0.00 Collisions
LCK.backend.creat 0 0.00 Created locks
LCK.backend.destroy 0 0.00 Destroyed locks
LCK.backend.locks 0 0.00 Lock Operations
LCK.backend.colls 0 0.00 Collisions
SMA.s0.c_req 0 0.00 Allocator requests
SMA.s0.c_fail 0 0.00 Allocator failures
SMA.s0.c_bytes 0 0.00 Bytes allocated
SMA.s0.c_freed 0 0.00 Bytes freed
SMA.s0.g_alloc 0 . Allocations outstanding
SMA.s0.g_bytes 0 . Bytes outstanding
SMA.s0.g_space 0 . Bytes available
SMA.Transient.c_req 0 0.00 Allocator requests
SMA.Transient.c_fail 0 0.00 Allocator failures
SMA.Transient.c_bytes 0 0.00 Bytes allocated
SMA.Transient.c_freed 0 0.00 Bytes freed
SMA.Transient.g_alloc 0 . Allocations outstanding
SMA.Transient.g_bytes 0 . Bytes outstanding
SMA.Transient.g_space 0 . Bytes available
VBE.default(127.0.0.1,,8080).vcls 0 . VCL references
VBE.default(127.0.0.1,,8080).happy 0 . Happy health probes
VBE.wiki(127.0.0.1,,8081).vcls 0 . VCL references
VBE.wiki(127.0.0.1,,8081).happy 0 . Happy health probes
VBE.tiles(127.0.0.1,,9090).vcls 0 . VCL references
VBE.tiles(127.0.0.1,,9090).happy 0 . Happy health probes
VBE.geoserver(127.0.0.1,,43074).vcls 0 . VCL references
VBE.geoserver(127.0.0.1,,43074).happy 0 . Happy health probes
VBE.iconserver(127.0.0.1,,43077).vcls 0 . VCL references
VBE.iconserver(127.0.0.1,,43077).happy 0 . Happy health probes
VBE.uploadserver(127.0.0.1,,43079).vcls 0 . VCL references
VBE.uploadserver(127.0.0.1,,43079).happy 0 . Happy health probes
VBE.graphserver(127.0.0.1,,43080).vcls 0 . VCL references
VBE.graphserver(127.0.0.1,,43080).happy 0 . Happy health probes
VBE.kmlserver(127.0.0.1,,43082).vcls 0 . VCL references
VBE.kmlserver(127.0.0.1,,43082).happy 0 . Happy health probes
VBE.feedbackserver(127.0.0.1,,43085).vcls 0 . VCL references
VBE.feedbackserver(127.0.0.1,,43085).happy 0 . Happy health probes
VBE.weatherserver(127.0.0.1,,43086).vcls 0 . VCL references
VBE.weatherserver(127.0.0.1,,43086).happy 0 . Happy health probes
VBE.analyticsserver(127.0.0.1,,43087).vcls 0 . VCL references
VBE.analyticsserver(127.0.0.1,,43087).happy 0 . Happy health probes
VBE.chartserver(127.0.0.1,,43088).vcls 0 . VCL references
VBE.chartserver(127.0.0.1,,43088).happy 0 . Happy health probes
VBE.watcher(127.0.0.1,,8090).vcls 0 . VCL references
VBE.watcher(127.0.0.1,,8090).happy 0 . Happy health probes
VBE.render(127.0.0.1,,8040).vcls 0 . VCL references
VBE.render(127.0.0.1,,8040).happy 0 . Happy health probes
VBE.route(127.0.0.1,,43073).vcls 0 . VCL references
VBE.route(127.0.0.1,,43073).happy 0 . Happy health probes
VBE.osmsync(127.0.0.1,,9010).vcls 0 . VCL references
VBE.osmsync(127.0.0.1,,9010).happy 0 . Happy health probes
VBE.search(127.0.0.1,,8100).vcls 0 . VCL references
VBE.search(127.0.0.1,,8100).happy 0 . Happy health probes
VBE.indexserver(127.0.0.1,,8101).vcls 0 . VCL references
VBE.indexserver(127.0.0.1,,8101).happy 0 . Happy health probes
VBE.rewriteproxy(127.0.0.1,,9030).vcls 0 . VCL references
VBE.rewriteproxy(127.0.0.1,,9030).happy 0 . Happy health probes
VBE.pomplo(127.0.0.1,,9081).vcls 0 . VCL references
VBE.pomplo(127.0.0.1,,9081).happy 0 . Happy health probes
VBE.admin(127.0.0.1,,9000).vcls 0 . VCL references
VBE.admin(127.0.0.1,,9000).happy 0 . Happy health probes
VBE.graphite(127.0.0.1,,7070).vcls 0 . VCL references
VBE.graphite(127.0.0.1,,7070).happy 0 . Happy health probes
VBE.notificationserver(127.0.0.1,,8070).vcls 0 . VCL references
VBE.notificationserver(127.0.0.1,,8070).happy 0 . Happy health probes
VBE.orient(127.0.0.1,,2480).vcls 0 . VCL references
VBE.orient(127.0.0.1,,2480).happy 0 . Happy health probes
VBE.geoip(127.0.0.1,,43930).vcls 0 . VCL references
VBE.geoip(127.0.0.1,,43930).happy 0 . Happy health probes
这是我的vcl（包含在default.vcl中） import std;
acl internal {
"localhost";
"192.168.0.0"/16;
"10.0.0.0"/8;
}
probe status {
.url = "/status.json";
.timeout = 100 ms;
.interval = 1s;
.threshold = 1;
}
probe rootCheck {
.url = "/";
.timeout = 100 ms;
.interval = 1s;
.threshold = 1;
}
backend tiles {
.host = "127.0.0.1";
.port = "9090";
.probe = {
.url = "/serverInfo.json";
.timeout = 100 ms;
.interval = 1s;
.threshold = 1;
}
}
backend geoserver {
.host = "127.0.0.1";
.port = "43074";
.probe = rootCheck;
}
backend iconserver {
.host = "127.0.0.1";
.port = "43077";
.probe = rootCheck;
}
backend uploadserver {
.host = "127.0.0.1";
.port = "43079";
.probe = rootCheck;
}
backend graphserver {
.host = "127.0.0.1";
.port = "43080";
.probe = rootCheck;
}
backend kmlserver {
.host = "127.0.0.1";
.port = "43082";
.probe = rootCheck;
}
backend weatherserver {
.host = "127.0.0.1";
.port = "43086";
.probe = {
.url = "/status";
.timeout = 100 ms;
.interval = 1s;
.threshold = 1;
}
}
backend analyticsserver {
.host = "127.0.0.1";
.port = "43087";
.probe = status;
}
backend chartserver {
.host = "127.0.0.1";
.port = "43088";
.probe = status;
}
backend watcher {
.host = "127.0.0.1";
.port = "8090";
.probe = {
.url = "/test";
.timeout = 100 ms;
.interval = 1s;
.threshold = 1;
}
}
backend route {
.host = "127.0.0.1";
.port = "43073";
.probe = status;
}
backend osmsync {
.host = "127.0.0.1";
.port = "9010";
.probe = {
.url = "/state.json";
.timeout = 100 ms;
.interval = 1s;
.threshold = 1;
}
}
backend search {
.host = "127.0.0.1";
.port = "8100";
.probe = status;
}
backend indexserver {
.host = "127.0.0.1";
.port = "8101";
.probe = rootCheck;
}
backend rewriteproxy {
.host = "127.0.0.1";
.port = "9030";
.probe = rootCheck;
}
backend webserver {
.host = "127.0.0.1";
.port = "9081";
.probe = rootCheck;
}
backend admin {
.host = "127.0.0.1";
.port = "9000";
.probe = rootCheck;
}
backend graphite {
.host = "127.0.0.1";
.port = "7070";
.probe = rootCheck;
}
backend notificationserver {
.host = "127.0.0.1";
.port = "8070";
.probe = status;
}
backend orient {
.host = "127.0.0.1";
.port = "2480";
}
backend geoip {
.host = "127.0.0.1";
.port = "43930";
.probe = status;
}
sub allow_only_internal {
if (!client.ip ~ internal) {
error 405 "Not allowed.";
}
}
sub vcl_recv {
if (req.url ~ "\.(png|gif|jpg|swf|css|js|bis|apk|plist|ipa|woff|svg|eot|ttf|json)(\?.*|)$") {
unset req.http.cookie;
}
if (req.url ~ "^/admin") {
call allow_only_internal;
}
if( req.http.host ~ "^tile" ) {
unset req.http.cookie;
set req.http.host = "tiles";
set req.backend = tiles;
} else if( req.http.host ~ "^data" ) {
set req.backend = geoserver;
} else if( req.http.host ~ "^icon" ) {
unset req.http.cookie;
set req.backend = iconserver;
} else if( req.http.host ~ "^upload" ) {
set req.backend = uploadserver;
} else if( req.http.host ~ "^graphite" ) { // must be before graph
if( req.url !~ "^/render" ) { //allow render API anywhere
call allow_only_internal;
}
set req.backend = graphite;
} else if( req.http.host ~ "^graph" ) {
set req.backend = graphserver;
} else if( req.http.host ~ "^kml" ) {
set req.backend = kmlserver;
} else if( req.http.host ~ "^notification" ) {
set req.backend = notificationserver;
} else if( req.http.host ~ "^osmsync" ) {
set req.backend = osmsync;
} else if( req.http.host ~ "^watcher" ) {
set req.backend = watcher;
} else if( req.http.host ~ "^route" ) {
set req.backend = route;
} else if( req.http.host ~ "^search" ) {
set req.backend = search;
} else if( req.http.host ~ "^index" ) {
call allow_only_internal;
set req.backend = indexserver;
} else if( req.http.host ~ "^weather" ) {
set req.backend = weatherserver;
} else if( req.http.host ~ "^analytics" ) {
set req.backend = analyticsserver;
} else if( req.http.host ~ "^chart" ) {
set req.backend = chartserver;
} else if( req.http.host ~ "^geoip" ) {
set req.backend = geoip;
} else if( req.http.host ~ "^rewriteproxy" || req.http.host ~ "^showroom" ) {
set req.backend = rewriteproxy;
} else if( req.http.host ~ "^admin" ) {
call allow_only_internal;
set req.backend = admin;
} else if( req.http.host ~ "^orient" ) {
call allow_only_internal;
set req.backend = orient;
}
//url matching is at the bottom to avoid confusion
else if( req.url ~ "^/query" ) {
set req.backend = geoserver;
} else if( req.url ~ "^/data/" ) {
set req.url = regsub( req.url, "^/data(/.*)", "\1" );
set req.backend = geoserver;
} else if( req.url ~ "^/icon/" ) {
set req.url = regsub( req.url, "^/icon(/.*)", "\1" );
set req.backend = iconserver;
} else if( req.url ~ "^/graph/" ) {
set req.url = regsub( req.url, "^/graph(/.*)", "\1" );
set req.backend = graphserver;
} else if( req.http.host ~ "^(www\.|static|map)" ) {
set req.backend = webserver;
} else {
// todo: handle HTTPS with req.http.X-Forwarded-Proto
// redirect dev and demo to avoid dead links for old urls
error 750 regsub(req.http.host, "^(w+\.|dev\.|demo\.)?", "http://www.") + req.url;
}
}
sub vcl_deliver {
if (resp.http.Location ~ ".*:8080.*") {
set resp.http.Location = regsub(resp.http.Location, ":[0-9]+", "");
}
if (req.backend == graphite) {
set resp.http.Access-Control-Allow-Origin = "*";
set resp.http.Access-Control-Allow-Methods = "GET, OPTIONS";
set resp.http.Access-Control-Allow-Headers = "origin, authorization, accept";
}
unset resp.http.x-url;
unset resp.http.x-host;
return (deliver);
}
sub vcl_fetch {
if (req.url ~ "\.(png|gif|jpg|swf|css|js|bis|apk|plist|ipa|woff|svg|eot|ttf|json)(\?.*|)$") {
unset beresp.http.set-cookie;
}
if( beresp.http.content-type ~ "application/json"
|| beresp.http.content-type ~ "application/javascript"
|| (beresp.http.content-type ~ "application/octet-stream" && req.url ~ "\.bis$")
|| beresp.http.content-type ~ "text/javascript"
|| beresp.http.content-type ~ "text/plain"
|| beresp.http.content-type ~ "text/html"
|| beresp.http.content-type ~ "text/css" ) {
set beresp.do_gzip = true;
}
//allow ban lurker to work. https://www.varnish-software.com/static/book/Cache_invalidation.html#smart-bans
set beresp.http.x-url = req.url;
set beresp.http.x-host = req.http.host;
}
sub vcl_pipe {
if (req.http.upgrade) {
set bereq.http.upgrade = req.http.upgrade;
}
}
sub vcl_recv {
if (req.http.Upgrade ~ "(?i)websocket") {
return (pipe);
}
}
sub vcl_error {
if (obj.status == 750) {
set obj.http.Location = obj.response;
set obj.status = 302;
return (deliver);
}
set obj.http.Content-Type = "text/html; charset=utf-8";
set obj.http.Retry-After = "3";
if( req.http.host ~ "^map" ) {
synthetic regsuball(regsuball(std.fileread("/etc/varnish/error_map.html"), "\$obj\.status\$", obj.status), "\$obj\.response\$", obj.response);
} else {
synthetic regsuball(regsuball(std.fileread("/etc/varnish/error.html"), "\$obj\.status\$", obj.status), "\$obj\.response\$", obj.response);
}
return (deliver);
}
答案 0 :(得分：1)
这通常是因为当时请求的导演中没有一个后端是健康的。
您的VCL中似乎没有使用任何服务器，也没有任何健康探测器。所以很难知道为什么会发生这种情况。您还应该使用您定义的后端并对它们进行运行状况检查。
如果您看到所有后端更类似于以下内容，则varnishstat最不能为您提供一些有用的信息：
VBE.wiki(127.0.0.1,,8081).vcls 1
VBE.wiki(127.0.0.1,,8081).happy 18446744073709551615
您还可以在varnishadm控制台中通过“debug.health”查看当前状态和简历。
答案 1 :(得分：1)
尝试更新 libvarnishapi lib - 这对我有帮助（ubuntu）：
sudo apt-get install libvarnishapi1
并重启。
如果你使用的是错误的版本（我的 libvarnishapi-dev ），你就不能让varnishstat正常工作。
答案 2 :(得分：0)
默认情况下，backend_busy递增时，表示客户端收到5xx错误响应。但是，通过使用VCL（Varnish Configuration Language），您似乎可以将Varnish配置为从忙碌的后端恢复＆＃34;通过使用不同的后端，或通过提供过时或合成的响应。
（Source）
注意：我认为backend_busy与backend_unhealthy不同。
实际上，Varnish定期对后端进行ping操作，以确保它仍然正常并且响应迅速。如果它没有足够快地收到200响应，则后端标记为运行状况不佳，并且每个新请求都会递增backend_unhealthy，直到后端恢复并及时发送200响应。