为缓存django站点设置清漆的正确方法

时间:2013-11-15 08:51:46

标签: django varnish varnish-vcl django-cache

我刚刚在我的后端服务器前安装了一个只有清漆的服务器,我有两个不同的django站点,通过nginx + gunicorn服务

它似乎有用,但我得到Header Age = 0,并且查看文档,这不是很好。

我想为匿名用户缓存页面,但不是为经过身份验证的用户缓存,或者如果用户有一个名为“AUTHENTICATION”的Cookie

这是我的default.vcl

backend django {
    .host = "backend1";
    .port = "8080";
}


sub vcl_recv {  

  # unless sessionid/csrftoken is in the request, don't pass ANY cookies (referral_source, utm, etc)  
  if (req.request == "GET" && (req.url ~ "^/static" || (req.http.cookie !~ "sessionid" && req.http.cookie !~ "csrftoken" && req.http.cookie !~ "AUTHENTICATION"))) {  
    remove req.http.Cookie;  
  }  


    #normalize accept-encoding to account for different browsers  
    if (req.http.Accept-Encoding) {
        if (req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg)$") {
            # No point in compressing these
            remove req.http.Accept-Encoding;
        } elsif (req.http.Accept-Encoding ~ "gzip") {
            set req.http.Accept-Encoding = "gzip";
        } elsif (req.http.Accept-Encoding ~ "deflate") {
            set req.http.Accept-Encoding = "deflate";
        } else {
            # unknown algorithm
            remove req.http.Accept-Encoding;
        }
    }  



}  

sub vcl_fetch {  

  # /static and /media files always cached  
  if (req.url ~ "^/static" || req.url ~ "^/media") {  
       unset beresp.http.set-cookie;  
       return (deliver);  
  }  

  # pass through for anything with a session/csrftoken set  
  if (beresp.http.set-cookie ~ "sessionid" || beresp.http.set-cookie ~ "csrftoken" || beresp.http.set-cookie ~ "AUTHENTICATION") {  
    return (hit_for_pass);  
  } else {  
    return (deliver);  
  }  

} 

是否为每个用户设置了sessionid,即使他们没有登录,也阻止了Varnish有效地为匿名用户缓存页面?

编辑:

使用isvarnishworking.com这是输出:

HTTP/1.1 200 OK
Server: cloudflare-nginx
Date:   Fri, 15 Nov 2013 09:30:20 GMT
Content-Type:   text/html; charset=utf-8
Connection: keep-alive
Set-Cookie: __cfduid=d281023a84b2e5351d109c1848eeca1601384507820317; expires=Mon, 23-Dec-2019 23:50:00 GMT; path=/; domain=.mydomain.com; HttpOnly
Vary:   Cookie
X-Frame-Options:    SAMEORIGIN
X-Varnish:  1602772074
Age:    0
Via:    1.1 varnish
CF-RAY: cdaec14fab00412
Content-Encoding:   gzip

编辑2:

我的新default.vcl:

backend django {
    .host = "backend1";
    .port = "8080";
}

sub vcl_recv {  

    #normalize accept-encoding to account for different browsers  
    if (req.http.Accept-Encoding) {
        if (req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg)$") {
            # No point in compressing these
            remove req.http.Accept-Encoding;
        } elsif (req.http.Accept-Encoding ~ "gzip") {
            set req.http.Accept-Encoding = "gzip";
        } elsif (req.http.Accept-Encoding ~ "deflate") {
            set req.http.Accept-Encoding = "deflate";
        } else {
            # unknown algorithm
            remove req.http.Accept-Encoding;
        }
    }  
}



sub vcl_fetch {  
  if (req.url ~ "^/static" || req.url ~ "^/media") {  
    unset beresp.http.set-cookie;  
  }  

  if (beresp.http.set-cookie !~ "sessionid" && beresp.http.set-cookie !~ "csrftoken" && beresp.http.set-cookie !~ "AUTHENTICATION") {  
    unset beresp.http.set-cookie; 
  }
} 

来自isvarnishworking.com

的结果
HTTP/1.1 200 OK
Server: cloudflare-nginx
Date:   Fri, 15 Nov 2013 12:08:42 GMT
Content-Type:   text/html; charset=utf-8
Connection: keep-alive
Set-Cookie: __cfduid=d55ea1b56e978cbbf3384d0fa2f21571e1384517322491; expires=Mon, 23-Dec-2019 23:50:00 GMT; path=/; domain=.mydomain.com; HttpOnly
Vary:   Cookie
X-Frame-Options:    SAMEORIGIN
X-Varnish:  1240916568
Age:    0
Via:    1.1 varnish
CF-RAY: cdbd4119f3b0412
Content-Encoding:   gzip

编辑3:

backend default {
    .host = "backend1";
    .port = "8080";
}

sub vcl_recv {  

  # unless sessionid/csrftoken is in the request, don't pass ANY cookies (referral_source, utm, etc)  
  if (req.request == "GET" && (req.url ~ "^/static" || (req.http.cookie !~ "sessionid" && req.http.cookie !~ "csrftoken" && req.http.cookie !~ "AUTHENTICATION"))) {  
    remove req.http.Cookie;  
  }  

    #normalize accept-encoding to account for different browsers  
    if (req.http.Accept-Encoding) {
        if (req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg)$") {
            # No point in compressing these
            remove req.http.Accept-Encoding;
        } elsif (req.http.Accept-Encoding ~ "gzip") {
            set req.http.Accept-Encoding = "gzip";
        } elsif (req.http.Accept-Encoding ~ "deflate") {
            set req.http.Accept-Encoding = "deflate";
        } else {
            # unknown algorithm
            remove req.http.Accept-Encoding;
        }
    }  

}  

sub vcl_fetch {  

  # /static and /media files always cached  
  if (req.url ~ "^/static" || req.url ~ "^/media") {  
       unset beresp.http.set-cookie; 
  }

  if (beresp.http.set-cookie !~ "sessionid" && beresp.http.set-cookie !~ "csrftoken" && beresp.http.set-cookie !~ "AUTHENTICATION") {  
    unset beresp.http.set-cookie;
  }

} 

我的后端响应(前面没有清漆)是:

GET / HTTP/1.1
Host: www.mydomain.com
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:25.0) Gecko/20100101 Firefox/25.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: nb-no,nb;q=0.9,no-no;q=0.8,no;q=0.6,nn-no;q=0.5,nn;q=0.4,en-us;q=0.3,en;q=0.1
Accept-Encoding: gzip, deflate
Cookie: __cfduid=d8f496aef561efd7a30c3d9f909a02cf31384507505064; sessionid=twoq45r21gn341545ohubilyp739r42ee; _ga=GA1.2.382479980.1384507508
Connection: keep-alive

HTTP/1.1 200 OK
Server: cloudflare-nginx
Date: Fri, 15 Nov 2013 14:37:53 GMT
Content-Type: text/html; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
Vary: Accept-Language, Cookie
X-Frame-Options: SAMEORIGIN
Content-Language: nb
CF-RAY: cdcae94f68105af
Content-Encoding: gzip

2 个答案:

答案 0 :(得分:3)

  

可能是为每个用户设置了sessionid,即使它们不是   登录,这阻止了Varnish有效地缓存页面   匿名用户?

你是对的。注销后立即启动新会话,并在用户的计算机上植入新的会话cookie。为了解决这个问题,我创建了一个自定义注销视图,我将其用于与Varnish一起使用的网站:

from django.conf import settings
from django.contrib.auth.views import logout

def logout_user(request):
    """After logging out some of the cookies should be deleted,
    allowing upstream cache to work effectively."""
    response = logout(request)
    request.session.modified = False  # forces session middleware not to set its own cookie
    response.delete_cookie(settings.CSRF_COOKIE_NAME)
    response.delete_cookie(settings.SESSION_COOKIE_NAME)
    return response

正如你所看到的,我强制会话中间件不要设置新的cookie,然后我删除旧的cookie(我也摆脱了csrf cookie)。

修改:此外,此代码似乎完全没必要,as Varnish do this automatically for any cookie being set

  # pass through for anything with a session/csrftoken set  
  if (beresp.http.set-cookie ~ "sessionid" || beresp.http.set-cookie ~ "csrftoken" || beresp.http.set-cookie ~ "AUTHENTICATION") {  
    return (hit_for_pass);  
  } else {  
    return (deliver);  
  }  

另请注意,hit_for_pass会使特定网址在几分钟内无法缓存(对于所有用户!)。尝试这三种诊断:

  1. 清除Cookie,删除上面的代码,重新启动Varnish,检查Age是否仍设置为0
  2. 检查来自后端(nginx)的标头。也许是自己设置Age值,或者使用其他缓存控制cookie强制Varnish这样做?
  3. 使用varlog检查是否正在缓存这些响应。
  4. 编辑2:您的isvarnishworking.com输出显示服务器设置了名为__cfduid的Cookie。每次设置cookie时,Varnish会自动进入命中传递模式(请参阅上面编辑中链接的代码)。这很可能是问题的原因。我想这就是我认为不必要的代码的原因。我会尝试明确删除所有未知的cookie:

    sub vcl_fetch {  
      if (req.url ~ "^/static" || req.url ~ "^/media") {  
        unset beresp.http.set-cookie;  
      }  
    
      if (beresp.http.set-cookie !~ "sessionid" && beresp.http.set-cookie !~ "csrftoken" && beresp.http.set-cookie !~ "AUTHENTICATION") {  
        unset beresp.http.set-cookie; 
      }
    } 
    

答案 1 :(得分:0)

开箱即用的清漆设置用于不发送Vary:cookie标头的损坏的应用程序服务器。 Django很聪明,如果你对会话ID做任何事情,它会发送一个Vary:cookie标题。因此,最好的办法是编辑您的vcl并删除任何与cookie有关的内容。清漆将处理它自己的变化。

这是默认的

sub vcl_recv {
    if (req.restarts == 0) {
        if (req.http.x-forwarded-for) {
            set req.http.X-Forwarded-For =
                req.http.X-Forwarded-For + ", " + client.ip;
        } else {
            set req.http.X-Forwarded-For = client.ip;
        }
    }
    if (req.request != "GET" &&
      req.request != "HEAD" &&
      req.request != "PUT" &&
      req.request != "POST" &&
      req.request != "TRACE" &&
      req.request != "OPTIONS" &&
      req.request != "DELETE") {
        /* Non-RFC2616 or CONNECT which is weird. */
        return (pipe);
    }
    if (req.request != "GET" && req.request != "HEAD") {
        /* We only deal with GET and HEAD by default */
        return (pass);
    }
    /// REMOVE THIS SECTION
    if (req.http.Authorization || req.http.Cookie) {
        /* Not cacheable by default */
        return (pass);
    }
    /// ENDREMOVE SECTION
    return (lookup);
}

同样在你的django代码中,无论是python还是模板,请确保不要检查/使用request.user或session。如果这样做,这意味着页面对于不同的用户表现不同,因此应该以不同的方式缓存(这就是为什么django将发送Vary:cookie标头,这意味着根据cookie缓存不同,即sessionid)< / p>