声纳分析失败,出现SocketTimeoutException

时间:2014-01-01 17:16:27

标签: sonarqube

尝试从ant运行Sonar时,由于SocketTimeoutException而导致零星失败。

设置:我已经在配置为与Postgres 9.2一起运行的RHEL 6盒子上运行Sonar 4.0。我使用Jenkins 1.544每晚构建175个项目。我已经让Jenkins在从机上使用单个执行器串行构建项目但是最近我通过swarm添加了第二个slave,所以现在我有两个节点(每个都有一个执行器),并行构建。我使用声纳蚂蚁任务从蚂蚁运行声纳,这在很大程度上运作良好。

当我只有一个奴隶时,我偶尔会在尝试加载引导程序属性时因为SocketTimeoutException而失败。现在我已经添加了第二个节点,这似乎更频繁地发生了。有趣的是,当我使用ant在命令行上构建项目时,我也看到了同样的失败。这似乎不是服务器资源问题。我是声纳服务器的唯一用户,我能够在没有任何实际负载的情况下得到此错误。

这是今天早上从命令行运行时得到的堆栈跟踪:

 [exec] BUILD FAILED
 [exec] /var/lib/jenkins/sonar.buildfile:113: org.sonar.runner.impl.RunnerException: Unable to execute Sonar
 [exec]     at org.sonar.runner.impl.BatchLauncher$1.delegateExecution(BatchLauncher.java:79)
 [exec]     at org.sonar.runner.impl.BatchLauncher$1.run(BatchLauncher.java:63)
 [exec]     at java.security.AccessController.doPrivileged(Native Method)
 [exec]     at org.sonar.runner.impl.BatchLauncher.doExecute(BatchLauncher.java:57)
 [exec]     at org.sonar.runner.impl.BatchLauncher.execute(BatchLauncher.java:50)
 [exec]     at org.sonar.runner.api.EmbeddedRunner.doExecute(EmbeddedRunner.java:71)
 [exec]     at org.sonar.runner.api.Runner.execute(Runner.java:89)
 [exec]     at org.sonar.ant.SonarTask.launchAnalysis(SonarTask.java:53)
 [exec]     at org.sonar.ant.SonarTask.execute(SonarTask.java:48)
 [exec]     at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
 [exec]     at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
 [exec]     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 [exec]     at java.lang.reflect.Method.invoke(Method.java:601)
 [exec]     at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
 [exec]     at org.apache.tools.ant.Task.perform(Task.java:348)
 [exec]     at org.apache.tools.ant.Target.execute(Target.java:392)
 [exec]     at org.apache.tools.ant.Target.performTasks(Target.java:413)
 [exec]     at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1399)
 [exec]     at org.apache.tools.ant.Project.executeTarget(Project.java:1368)
 [exec]     at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41)
 [exec]     at org.apache.tools.ant.Project.executeTargets(Project.java:1251)
 [exec]     at org.apache.tools.ant.Main.runBuild(Main.java:811)
 [exec]     at org.apache.tools.ant.Main.startAnt(Main.java:217)
 [exec]     at org.apache.tools.ant.launch.Launcher.run(Launcher.java:280)
 [exec]     at org.apache.tools.ant.launch.Launcher.main(Launcher.java:109)
 [exec] Caused by: org.sonar.api.utils.SonarException: Unable to request: /batch_bootstrap/properties?dryRun=false
 [exec]     at org.sonar.batch.bootstrap.ServerClient.request(ServerClient.java:92)
 [exec]     at org.sonar.batch.bootstrap.ServerClient.request(ServerClient.java:82)
 [exec]     at org.sonar.batch.bootstrap.ServerClient.request(ServerClient.java:78)
 [exec]     at org.sonar.batch.bootstrap.BatchSettings.downloadSettings(BatchSettings.java:97)
 [exec]     at org.sonar.batch.bootstrap.BatchSettings.init(BatchSettings.java:72)
 [exec]     at org.sonar.batch.bootstrap.BatchSettings.<init>(BatchSettings.java:55)
 [exec]     at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
 [exec]     at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
 [exec]     at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
 [exec]     at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
 [exec]     at org.picocontainer.injectors.AbstractInjector.newInstance(AbstractInjector.java:145)
 [exec]     at org.picocontainer.injectors.ConstructorInjector$1.run(ConstructorInjector.java:342)
 [exec]     at org.picocontainer.injectors.AbstractInjector$ThreadLocalCyclicDependencyGuard.observe(AbstractInjector.java:270)
 [exec]     at org.picocontainer.injectors.ConstructorInjector.getComponentInstance(ConstructorInjector.java:364)
 [exec]     at org.picocontainer.injectors.AbstractInjectionFactory$LifecycleAdapter.getComponentInstance(AbstractInjectionFactory.java:56)
 [exec]     at org.picocontainer.behaviors.AbstractBehavior.getComponentInstance(AbstractBehavior.java:64)
 [exec]     at org.picocontainer.behaviors.Stored.getComponentInstance(Stored.java:91)
 [exec]     at org.picocontainer.DefaultPicoContainer.getInstance(DefaultPicoContainer.java:698)
 [exec]     at org.picocontainer.DefaultPicoContainer.getComponent(DefaultPicoContainer.java:646)
 [exec]     at org.picocontainer.DefaultPicoContainer.getComponent(DefaultPicoContainer.java:631)
 [exec]     at org.picocontainer.parameters.BasicComponentParameter$1.resolveInstance(BasicComponentParameter.java:118)
 [exec]     at org.picocontainer.parameters.ComponentParameter$1.resolveInstance(ComponentParameter.java:136)
 [exec]     at org.picocontainer.injectors.SingleMemberInjector.getParameter(SingleMemberInjector.java:78)
 [exec]     at org.picocontainer.injectors.SingleMemberInjector.getMemberArguments(SingleMemberInjector.java:61)
 [exec]     at org.picocontainer.injectors.MethodInjector.getMemberArguments(MethodInjector.java:100)
 [exec]     at org.picocontainer.injectors.MethodInjector$2.run(MethodInjector.java:112)
 [exec]     at org.picocontainer.injectors.AbstractInjector$ThreadLocalCyclicDependencyGuard.observe(AbstractInjector.java:270)
 [exec]     at org.picocontainer.injectors.MethodInjector.decorateComponentInstance(MethodInjector.java:120)
 [exec]     at org.picocontainer.injectors.CompositeInjector.decorateComponentInstance(CompositeInjector.java:58)
 [exec]     at org.picocontainer.injectors.Reinjector.reinject(Reinjector.java:142)
 [exec]     at org.picocontainer.injectors.ProviderAdapter.getComponentInstance(ProviderAdapter.java:96)
 [exec]     at org.picocontainer.DefaultPicoContainer.getInstance(DefaultPicoContainer.java:698)
 [exec]     at org.picocontainer.DefaultPicoContainer.getComponent(DefaultPicoContainer.java:646)
 [exec]     at org.picocontainer.DefaultPicoContainer.getComponent(DefaultPicoContainer.java:631)
 [exec]     at org.picocontainer.parameters.BasicComponentParameter$1.resolveInstance(BasicComponentParameter.java:118)
 [exec]     at org.picocontainer.parameters.ComponentParameter$1.resolveInstance(ComponentParameter.java:136)
 [exec]     at org.picocontainer.injectors.SingleMemberInjector.getParameter(SingleMemberInjector.java:78)
 [exec]     at org.picocontainer.injectors.ConstructorInjector$CtorAndAdapters.getParameterArguments(ConstructorInjector.java:309)
 [exec]     at org.picocontainer.injectors.ConstructorInjector$1.run(ConstructorInjector.java:335)
 [exec]     at org.picocontainer.injectors.AbstractInjector$ThreadLocalCyclicDependencyGuard.observe(AbstractInjector.java:270)
 [exec]     at org.picocontainer.injectors.ConstructorInjector.getComponentInstance(ConstructorInjector.java:364)
 [exec]     at org.picocontainer.injectors.AbstractInjectionFactory$LifecycleAdapter.getComponentInstance(AbstractInjectionFactory.java:56)
 [exec]     at org.picocontainer.behaviors.AbstractBehavior.getComponentInstance(AbstractBehavior.java:64)
 [exec]     at org.picocontainer.behaviors.Stored.getComponentInstance(Stored.java:91)
 [exec]     at org.picocontainer.DefaultPicoContainer.getInstance(DefaultPicoContainer.java:698)
 [exec]     at org.picocontainer.DefaultPicoContainer.getComponent(DefaultPicoContainer.java:646)
 [exec]     at org.picocontainer.DefaultPicoContainer.getComponent(DefaultPicoContainer.java:631)
 [exec]     at org.picocontainer.parameters.BasicComponentParameter$1.resolveInstance(BasicComponentParameter.java:118)
 [exec]     at org.picocontainer.parameters.ComponentParameter$1.resolveInstance(ComponentParameter.java:136)
 [exec]     at org.picocontainer.injectors.SingleMemberInjector.getParameter(SingleMemberInjector.java:78)
 [exec]     at org.picocontainer.injectors.ConstructorInjector$CtorAndAdapters.getParameterArguments(ConstructorInjector.java:309)
 [exec]     at org.picocontainer.injectors.ConstructorInjector$1.run(ConstructorInjector.java:335)
 [exec]     at org.picocontainer.injectors.AbstractInjector$ThreadLocalCyclicDependencyGuard.observe(AbstractInjector.java:270)
 [exec]     at org.picocontainer.injectors.ConstructorInjector.getComponentInstance(ConstructorInjector.java:364)
 [exec]     at org.picocontainer.injectors.AbstractInjectionFactory$LifecycleAdapter.getComponentInstance(AbstractInjectionFactory.java:56)
 [exec]     at org.picocontainer.behaviors.AbstractBehavior.getComponentInstance(AbstractBehavior.java:64)
 [exec]     at org.picocontainer.behaviors.Stored.getComponentInstance(Stored.java:91)
 [exec]     at org.picocontainer.DefaultPicoContainer.instantiateComponentAsIsStartable(DefaultPicoContainer.java:1033)
 [exec]     at org.picocontainer.DefaultPicoContainer.addAdapterIfStartable(DefaultPicoContainer.java:1025)
 [exec]     at org.picocontainer.DefaultPicoContainer.startAdapters(DefaultPicoContainer.java:1002)
 [exec]     at org.picocontainer.DefaultPicoContainer.start(DefaultPicoContainer.java:766)
 [exec]     at org.sonar.api.platform.ComponentContainer.startComponents(ComponentContainer.java:91)
 [exec]     at org.sonar.api.platform.ComponentContainer.execute(ComponentContainer.java:77)
 [exec]     at org.sonar.batch.bootstrapper.Batch.startBatch(Batch.java:92)
 [exec]     at org.sonar.batch.bootstrapper.Batch.execute(Batch.java:74)
 [exec]     at org.sonar.runner.batch.IsolatedLauncher.execute(IsolatedLauncher.java:45)
 [exec]     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 [exec]     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
 [exec]     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 [exec]     at java.lang.reflect.Method.invoke(Method.java:601)
 [exec]     at org.sonar.runner.impl.BatchLauncher$1.delegateExecution(BatchLauncher.java:75)
 [exec]     ... 24 more
 [exec] Caused by: java.net.SocketTimeoutException: Read timed out
 [exec]     at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
 [exec]     at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
 [exec]     at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
 [exec]     at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
 [exec]     at sun.net.www.protocol.http.HttpURLConnection$6.run(HttpURLConnection.java:1674)
 [exec]     at sun.net.www.protocol.http.HttpURLConnection$6.run(HttpURLConnection.java:1672)
 [exec]     at java.security.AccessController.doPrivileged(Native Method)
 [exec]     at sun.net.www.protocol.http.HttpURLConnection.getChainedException(HttpURLConnection.java:1670)
 [exec]     at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1243)
 [exec]     at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:468)
 [exec]     at org.sonar.api.utils.HttpDownloader$BaseHttpDownloader$HttpInputSupplier.getInput(HttpDownloader.java:274)
 [exec]     at org.sonar.api.utils.HttpDownloader$BaseHttpDownloader$HttpInputSupplier.getInput(HttpDownloader.java:235)
 [exec]     at org.sonar.batch.bootstrap.ServerClient.request(ServerClient.java:88)
 [exec]     ... 94 more
 [exec] Caused by: java.net.SocketTimeoutException: Read timed out
 [exec]     at java.net.SocketInputStream.socketRead0(Native Method)
 [exec]     at java.net.SocketInputStream.read(SocketInputStream.java:150)
 [exec]     at java.net.SocketInputStream.read(SocketInputStream.java:121)
 [exec]     at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
 [exec]     at java.io.BufferedInputStream.read1(BufferedInputStream.java:275)
 [exec]     at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
 [exec]     at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:633)
 [exec]     at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:579)
 [exec]     at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1322)
 [exec]     at sun.net.www.protocol.http.HttpURLConnection.getHeaderField(HttpURLConnection.java:2677)
 [exec]     at java.net.URLConnection.getContentEncoding(URLConnection.java:533)
 [exec]     at org.sonar.api.utils.HttpDownloader$BaseHttpDownloader$HttpInputSupplier.getInput(HttpDownloader.java:272)
 [exec]     ... 96 more
 [exec] 
 [exec] Total time: 23 seconds

我查看了sonar access.log,可以看到初始连接并从上面列出的故障中重定向:

  

172.20.2.172 - - [01/01/2014:09:45:25 -0500]“GET / api / server / version HTTP / 1.1”200 3“ - ”“Ant / 2.1”       172.20.2.172 - - [01/01/2014:09:45:26 -0500]“GET / batch_bootstrap / index HTTP / 1.1”302 125“ - ”“Ant / 2.1”       172.20.2.172 - - [01/01/2014:09:45:26 -0500]“GET /deploy/bootstrap/index.txt HTTP / 1.1”200 3899“ - ”“Ant / 2.1”

有趣的是我稍后看了一下日志,似乎在我开始构建后30分钟显示连接失败:

  

172.20.2.172 - - [01/01/2014:09:45:25 -0500]“GET / api / server / version HTTP / 1.1”200 3“ - ”“Ant / 2.1”       172.20.2.172 - - [01/01/2014:09:45:26 -0500]“GET / batch_bootstrap / index HTTP / 1.1”302 125“ - ”“Ant / 2.1”       172.20.2.172 - - [01/01/2014:09:45:26 -0500]“GET /deploy/bootstrap/index.txt HTTP / 1.1”200 3899“ - ”“Ant / 2.1”       172.20.2.172 - - [01/01/2014:10:16:20 -0500]“GET / batch_bootstrap / properties?dryRun = false HTTP / 1.1”401 41“ - ”“声纳   空/空“

现在我正在使用LDAP插件来针对活动目录对声纳用户进行身份验证,并且我已经读过这可能会导致一些缓慢,可能导致超时。我添加了ncsd,但没有注意到任何改进。我也升级到使用sonar-ant-task-2.1.jar,但没有看到变化。我可以从我的浏览器访问/ batch_bootstrap / properties?dryRun = false,虽然有时它需要比其他人更长的时间。如果我重新运行失败的工作,它几乎总是在第二次尝试时成功。

我不确定下一步该尝试什么。我想扩大群集节点的数量,但我担心这只会导致更多的失败。我不想告诉开发人员忽略构建失败的电子邮件。由于工作的数量,我能够定期遇到这个问题。如果有人有他们认为有助于避免或隔离问题的日志记录或配置更改,我可以尝试使用它们。

谢谢 - 山姆

更新:我创建了一个短脚本来获取/ batch_bootstrap / properties?dryRun = false并使用curl将结果发送到/ dev / null。我将卷曲 - 最大时间设置为2秒并将其放在cron上,每2分钟运行一次。几乎每个请求都在2秒超时之前完成(48小时内只有26次失败,所有这些都是从00:30 - 01:00开始,当时我正在运行一些维护任务)。因为我开始以这种方式ping服务器,所以我没有因为SocketTimeoutException而导致更多的构建失败。我不是真的喜欢这个解决方案,但它似乎现在正在工作。如果有人尝试,我仍然对替代品感兴趣。

更新:我能够退出cron的频率而不是每5分钟运行一次而不会发现问题。当我尝试仅运行它10分钟时,我再次看到SocketTimeoutException失败。我也进行了一般的yum更新以获得良好的衡量标准,但显然没有改进。

谢谢 - 山姆

更新:当弹性搜索作为一个单独的进程被分解时,Sonar 4.5会逐渐变得更糟。在声纳日志文件中,我遇到了其中几个故障:

2014.12.24 03:42:28 ERROR web[o.s.s.s.SearchClient]  could not execute request: org.elasticsearch.action.get.GetRequestBuilder@48c9ec56
org.elasticsearch.client.transport.NoNodeAvailableException: No node available

我随着时间的推移记录了负载,发现它在夜间构建周期中比我预期的要高。除了声纳失败之外,我还没有看到任何其他奇怪的行为。最近有一些新的硬件可用,所以我将声纳全部移动到一个新的盒子上。由于这样做,我没有看到任何与超时相关的构建失败。

因此,如果有人看到类似的错误,此类错误似乎表明您需要投入更多资源。

谢谢 - 山姆

0 个答案:

没有答案