Question

我正在编写用于验证URL的SSL证书的微服务，基本上将这两件事结合在一起：

来自此处的SSL检查器的实现： https://github.com/alanorth/check-tls-certs/blob/master/main.go
然后通过API调用生成工作程序的方法如下： https://medium.com/smsjunk/handling-1-million-requests-per-minute-with-golang-f70ac505fcaa

因此，实际上，我正在启动小型服务，该服务接受URL列表并为它们验证SSL证书。问题是，在某些请求之后，SSL检查器开始冻结了很长时间，并且我不知道为什么（此日志中的每个有效负载每次都是同一组5000个URL ）：< / p>

DEBUG: 2018/09/29 14:33:58 dispatcher.go:34: DNS timeout set to 10 seconds
DEBUG: 2018/09/29 14:33:58 dispatcher.go:35: SSL timeout set to 10 seconds
DEBUG: 2018/09/29 14:33:58 dispatcher.go:36: HTTP timeout set to 10 seconds
DEBUG: 2018/09/29 14:33:58 dispatcher.go:37: Starting 10000 workers...
DEBUG: 2018/09/29 14:33:58 api.go:113: Starting API at :8888...
DEBUG: 2018/09/29 14:34:04 api.go:50: Received request, starting task '0af84d1e-52b3-41bc-935c-a5a22a007a2c'...
DEBUG: 2018/09/29 14:34:08 api.go:50: Received request, starting task 'e2379281-f98d-4185-8776-46c032bf6bf9'...
DEBUG: 2018/09/29 14:34:11 api.go:50: Received request, starting task 'faeb6b1d-8567-427f-81b7-63cdc2154314'...
DEBUG: 2018/09/29 14:34:15 api.go:50: Received request, starting task '702ca7b2-4b23-434c-9921-e72532766b16'...
DEBUG: 2018/09/29 14:34:15 dispatcher.go:59: Finished processing URLs for task '0af84d1e-52b3-41bc-935c-a5a22a007a2c' (took 11 seconds)!
DEBUG: 2018/09/29 14:34:20 dispatcher.go:59: Finished processing URLs for task 'e2379281-f98d-4185-8776-46c032bf6bf9' (took 12 seconds)!
DEBUG: 2018/09/29 14:34:22 api.go:50: Received request, starting task 'aa2a6bd6-f207-41a4-9dd4-a48ad72b85de'...
DEBUG: 2018/09/29 14:34:29 dispatcher.go:59: Finished processing URLs for task '702ca7b2-4b23-434c-9921-e72532766b16' (took 14 seconds)!
DEBUG: 2018/09/29 14:34:33 dispatcher.go:59: Finished processing URLs for task 'aa2a6bd6-f207-41a4-9dd4-a48ad72b85de' (took 11 seconds)!
DEBUG: 2018/09/29 14:34:55 api.go:50: Received request, starting task 'ea8c7c69-c533-4c9e-a4e4-439b41df2f52'...
DEBUG: 2018/09/29 14:34:59 api.go:50: Received request, starting task '6f2a2374-6911-4ff4-bbe2-b3aa378a2938'...
DEBUG: 2018/09/29 14:35:01 api.go:50: Received request, starting task '73cae838-9971-403f-bdfd-6e4790624fe8'...
DEBUG: 2018/09/29 14:35:04 api.go:50: Received request, starting task 'ee04997d-efd2-47df-9359-b46c90859224'...
DEBUG: 2018/09/29 14:35:06 dispatcher.go:59: Finished processing URLs for task 'ea8c7c69-c533-4c9e-a4e4-439b41df2f52' (took 11 seconds)!
DEBUG: 2018/09/29 14:35:07 api.go:50: Received request, starting task '5918b20d-ab52-484a-888e-2651344e8c5e'...
DEBUG: 2018/09/29 14:35:09 dispatcher.go:59: Finished processing URLs for task '6f2a2374-6911-4ff4-bbe2-b3aa378a2938' (took 10 seconds)!
DEBUG: 2018/09/29 14:35:19 dispatcher.go:59: Finished processing URLs for task 'ee04997d-efd2-47df-9359-b46c90859224' (took 15 seconds)!
DEBUG: 2018/09/29 14:35:20 dispatcher.go:59: Finished processing URLs for task '5918b20d-ab52-484a-888e-2651344e8c5e' (took 13 seconds)!
DEBUG: 2018/09/29 14:50:06 dispatcher.go:59: Finished processing URLs for task 'faeb6b1d-8567-427f-81b7-63cdc2154314' (took 955 seconds)!
DEBUG: 2018/09/29 14:50:57 dispatcher.go:59: Finished processing URLs for task '73cae838-9971-403f-bdfd-6e4790624fe8' (took 956 seconds)!

我将超时设置为time.Second * 10，我将其传递给net.DialTimeout()，但这并不能节省一天的时间。那意味着，而不是这行：https://github.com/alanorth/check-tls-certs/blob/master/main.go#L232我有这样的东西：

ipConn,err := net.DialTimeout("tcp", result.Domain + ":443", time.Second * 10)
if err != nil {
    result.Errors["ssl"] = append(result.Errors["ssl"], err.Error())
    if s.FailIfInvalid {
        result.Success = false
    }
    return
}

tc := &tls.Config{ServerName: result.Domain}
if s.CheckInsecure {
    tc.InsecureSkipVerify = true
}

conn:= tls.Client(ipConn, tc)

err = conn.Handshake()
if err != nil {
    switch e := err.(type) {
    case x509.CertificateInvalidError:
        result.Errors["ssl"] = append(result.Errors["ssl"], e.Error())
        if s.FailIfInvalid {
            result.Success = false
        }
        return
}
defer conn.Close()

此外，正在运行的线程上的strace表示它挂在epoll_pwait()的系统调用上，pprof没有给我任何信息，因为在这段时间内应用程序本身正在用户空间中静静地等待。

下一步该怎么做才能正确解决此问题？它似乎与打开的文件描述符没有任何关系（数字是合理的），内存/ CPU消耗也很低。非常感谢您的帮助。谢谢！

SSL证书的批量扫描在Golang中挂起

0 个答案: