我正在运行每日自动化测试(使用JClouds)许多AWS EC2计算机。 问题是,由于“超出请求限制”错误,有时机器无法启动。
从下面的堆栈跟踪中我认为我的操作失败了5次,5次是失败限制。也许这个动作是jclouds的方法describeInstancesInRegion?为什么会失败?
我知道有一个jclouds错误,有时会启动一个空/空名称的机器。跟随describeInstancesInRegion调用会破坏吗?
堆栈跟踪:
12/20/13 2:00:26 PM - Cannot retry after server error, **command has exceeded retry limit 5**: [method=public abstract com.google.common.util.concurrent.ListenableFuture org.jclouds.aws.ec2.services.AWSInstanceAsyncClient.**describeInstancesInRegion**(java.lang.String,java.lang.String[])[eu-west-1, [Ljava.lang.String;@5987955b], request=POST https://ec2.eu-west-1.amazonaws.com/ HTTP/1.1]
12/20/13 2:00:26 PM - << problem customizing node(eu-west-1/i-38cb4677): : Operation failed. org.jclouds.aws.AWSResponseException: request POST https://ec2.eu-west-1.amazonaws.com/ HTTP/1.1 failed with code 503, error: AWSError{requestId='8937a5d2-f17f-4a42-af32-ed02125a73a4', requestToken='null', code='**RequestLimitExceeded**', message='Request limit exceeded.', context='{Response=, Errors=}'}
12/20/13 2:00:26 PM - at org.jclouds.aws.handlers.ParseAWSErrorFromXmlContent.handleError(ParseAWSErrorFromXmlContent.java:76)
12/20/13 2:00:26 PM - at org.jclouds.http.handlers.DelegatingErrorHandler.handleError(DelegatingErrorHandler.java:69)
12/20/13 2:00:26 PM - at org.jclouds.http.internal.BaseHttpCommandExecutorService.shouldContinue(BaseHttpCommandExecutorService.java:180)
12/20/13 2:00:26 PM - at org.jclouds.http.internal.BaseHttpCommandExecutorService.invoke(BaseHttpCommandExecutorService.java:150)
12/20/13 2:00:26 PM - at org.jclouds.rest.internal.InvokeSyncToAsyncHttpMethod.invoke(InvokeSyncToAsyncHttpMethod.java:131)
12/20/13 2:00:26 PM - at org.jclouds.rest.internal.InvokeSyncToAsyncHttpMethod.apply(InvokeSyncToAsyncHttpMethod.java:97)
12/20/13 2:00:26 PM - at org.jclouds.rest.internal.InvokeSyncToAsyncHttpMethod.apply(InvokeSyncToAsyncHttpMethod.java:58)
12/20/13 2:00:26 PM - at org.jclouds.reflect.FunctionalReflection$FunctionalInvocationHandler.handleInvocation(FunctionalReflection.java:117)
12/20/13 2:00:26 PM - at com.google.common.reflect.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:70)
12/20/13 2:00:26 PM - at $Proxy67.describeInstancesInRegion(Unknown Source)
12/20/13 2:00:26 PM - at org.jclouds.ec2.compute.strategy.EC2GetNodeMetadataStrategy.getRunningInstanceInRegion(EC2GetNodeMetadataStrategy.java:68)
12/20/13 2:00:26 PM - at org.jclouds.aws.ec2.compute.strategy.AWSEC2GetNodeMetadataStrategy.getRunningInstanceInRegion(AWSEC2GetNodeMetadataStrategy.java:56)
12/20/13 2:00:26 PM - at org.jclouds.ec2.compute.strategy.EC2GetNodeMetadataStrategy.getNode(EC2GetNodeMetadataStrategy.java:60)
12/20/13 2:00:26 PM - at org.jclouds.compute.predicates.internal.RefreshNodeAndDoubleCheckOnFailUnlessStatusInvalid.refreshOrNull(RefreshNodeAndDoubleCheckOnFailUnlessStatusInvalid.java:60)
12/20/13 2:00:26 PM - at org.jclouds.compute.predicates.internal.RefreshNodeAndDoubleCheckOnFailUnlessStatusInvalid.refreshOrNull(RefreshNodeAndDoubleCheckOnFailUnlessStatusInvalid.java:40)
12/20/13 2:00:26 PM - at org.jclouds.compute.predicates.internal.RefreshAndDoubleCheckOnFailUnlessStatusInvalid.apply(RefreshAndDoubleCheckOnFailUnlessStatusInvalid.java:59)
12/20/13 2:00:26 PM - at org.jclouds.compute.predicates.internal.RefreshAndDoubleCheckOnFailUnlessStatusInvalid.apply(RefreshAndDoubleCheckOnFailUnlessStatusInvalid.java:42)
12/20/13 2:00:26 PM - at org.jclouds.compute.config.ComputeServiceTimeoutsModule$RetryablePredicateGuardingNull$1.apply(ComputeServiceTimeoutsModule.java:148)
12/20/13 2:00:26 PM - at org.jclouds.compute.config.ComputeServiceTimeoutsModule$RetryablePredicateGuardingNull$1.apply(ComputeServiceTimeoutsModule.java:143)
12/20/13 2:00:26 PM - at org.jclouds.compute.config.ComputeServiceTimeoutsModule$RetryablePredicateGuardingNull.apply(ComputeServiceTimeoutsModule.java:160)
12/20/13 2:00:26 PM - at org.jclouds.util.Predicates2$RetryablePredicate.apply(Predicates2.java:111)
12/20/13 2:00:26 PM - at org.jclouds.compute.config.ComputeServiceTimeoutsModule$RetryablePredicateGuardingNull.apply(ComputeServiceTimeoutsModule.java:124)
12/20/13 2:00:26 PM - at org.jclouds.compute.functions.PollNodeRunning.apply(PollNodeRunning.java:65)
12/20/13 2:00:26 PM - at org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:121)
12/20/13 2:00:26 PM - at org.jclouds.compute.functions.PollNodeRunning.apply(PollNodeRunning.java:44)
12/20/13 2:00:26 PM - at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
12/20/13 2:00:26 PM - at org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:53)
12/20/13 2:00:26 PM - at java.util.concurrent.FutureTask.run(FutureTask.java:138)
12/20/13 2:00:26 PM - at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
12/20/13 2:00:26 PM - at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
12/20/13 2:00:26 PM - at java.lang.Thread.run(Thread.java:662)
12/20/13 2:00:26 PM -
12/20/13 2:01:02 PM - Failed to create Cloudify Management VM: Error org.cloudifysource.esc.driver.provisioning.CloudProvisioningException: Failed to start machine.
12/20/13 2:00:26 PM - JClouds Deployer is shutting down a server that failed to start properly: eu-west-1/i-38cb4677
12/20/13 2:01:02 PM - Failed to start a management machine: Operation failed. java.util.concurrent.ExecutionException: org.cloudifysource.esc.driver.provisioning.CloudProvisioningException: Failed to start machine
12/20/13 2:01:02 PM - at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:232)
12/20/13 2:01:02 PM - at org.cloudifysource.esc.driver.provisioning.BaseProvisioningDriver.doStartManagementMachines(BaseProvisioningDriver.java:327)
12/20/13 2:01:02 PM - at java.util.concurrent.FutureTask.get(FutureTask.java:91)
12/20/13 2:01:02 PM - at org.cloudifysource.esc.driver.provisioning.jclouds.DefaultProvisioningDriver.startManagementMachines(DefaultProvisioningDriver.java:427)
12/20/13 2:01:02 PM - at org.cloudifysource.esc.shell.installer.CloudGridAgentBootstrapper.createManagementServers(CloudGridAgentBootstrapper.java:331)
12/20/13 2:01:02 PM - at org.cloudifysource.esc.shell.installer.CloudGridAgentBootstrapper.getOrCreateManagementServers(CloudGridAgentBootstrapper.java:278)
12/20/13 2:01:02 PM - at org.cloudifysource.esc.shell.commands.BootstrapCloud.doExecute(BootstrapCloud.java:224)
12/20/13 2:01:02 PM - at org.cloudifysource.esc.shell.installer.CloudGridAgentBootstrapper.bootstrapCloudAndWait(CloudGridAgentBootstrapper.java:203)
12/20/13 2:01:02 PM - at org.cloudifysource.shell.commands.AbstractGSCommand.execute(AbstractGSCommand.java:103)
12/20/13 2:01:02 PM - at org.apache.felix.gogo.commands.basic.AbstractCommand.execute(AbstractCommand.java:35)
12/20/13 2:01:02 PM - at org.apache.felix.gogo.runtime.Closure.executeCmd(Closure.java:474)
12/20/13 2:01:02 PM - at org.apache.felix.gogo.runtime.Closure.executeStatement(Closure.java:400)
12/20/13 2:01:02 PM - at org.apache.felix.gogo.runtime.Pipe.run(Pipe.java:108)
12/20/13 2:01:02 PM - at org.apache.felix.gogo.runtime.Closure.execute(Closure.java:183)
12/20/13 2:01:02 PM - at org.apache.felix.gogo.runtime.Closure.execute(Closure.java:120)
12/20/13 2:01:02 PM - at org.apache.felix.gogo.runtime.CommandSessionImpl.execute(CommandSessionImpl.java:89)
12/20/13 2:01:02 PM - at org.apache.karaf.shell.console.jline.Console.run(Console.java:172)
12/20/13 2:01:02 PM - at org.apache.karaf.shell.console.Main.run(Main.java:191)
12/20/13 2:01:02 PM - at org.apache.karaf.shell.console.Main.run(Main.java:89)
12/20/13 2:01:02 PM - at org.cloudifysource.shell.GigaShellMain.main(GigaShellMain.java:126)
12/20/13 2:01:02 PM - Caused by: org.cloudifysource.esc.driver.provisioning.CloudProvisioningException: Failed to start machine
12/20/13 2:01:02 PM - at org.cloudifysource.esc.jclouds.JCloudsDeployer.createServer(JCloudsDeployer.java:178)
12/20/13 2:01:02 PM - at org.cloudifysource.esc.driver.provisioning.jclouds.DefaultProvisioningDriver.createServer(DefaultProvisioningDriver.java:260)
12/20/13 2:01:02 PM - at org.cloudifysource.esc.driver.provisioning.jclouds.DefaultProvisioningDriver.createServer(DefaultProvisioningDriver.java:236)
12/20/13 2:01:02 PM - at org.cloudifysource.esc.driver.provisioning.BaseProvisioningDriver$1.call(BaseProvisioningDriver.java:315)
12/20/13 2:01:02 PM - at org.cloudifysource.esc.driver.provisioning.BaseProvisioningDriver$1.call(BaseProvisioningDriver.java:310)
12/20/13 2:01:02 PM - at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
12/20/13 2:01:02 PM - at java.util.concurrent.FutureTask.run(FutureTask.java:138)
12/20/13 2:01:02 PM - at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
12/20/13 2:01:02 PM - at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
12/20/13 2:01:02 PM - at java.lang.Thread.run(Thread.java:662)
12/20/13 2:01:02 PM - Caused by: org.jclouds.compute.RunNodesException: error running 1 node group(ec2-user-master-ec2gitservicestest-cloudify-manager1) location(eu-west-1) image(ami-c37474b7) size(m1.small) options({groupNames=[default], keyPair=ec2-sgtest-eu, userDataCksum=2f4a740b})
12/20/13 2:01:02 PM - Execution failures:
12/20/13 2:01:02 PM -
12/20/13 2:01:02 PM - 0 error[s]
12/20/13 2:01:02 PM - Node failures:
12/20/13 2:01:02 PM -
12/20/13 2:01:02 PM - 1) AWSResponseException on node eu-west-1/i-38cb4677:
12/20/13 2:01:02 PM - org.jclouds.aws.AWSResponseException: request POST https://ec2.eu-west-1.amazonaws.com/ HTTP/1.1 failed with code 503, error: AWSError{requestId='8937a5d2-f17f-4a42-af32-ed02125a73a4', requestToken='null', code='RequestLimitExceeded', message='Request limit exceeded.', context='{Response=, Errors=}'}
jclouds version is: 1.6.2-incubating
答案 0 :(得分:3)
你需要告诉jclouds在轮询实例状态更新时不经常发出请求。您可以使用以下替换来执行此操作。
Properties overrides = new Properties();
overrides.setProperty(POLL_INITIAL_PERIOD, POLL_PERIOD_TWENTY_SECONDS);
overrides.setProperty(POLL_MAX_PERIOD, POLL_PERIOD_TWENTY_SECONDS);
ComputeServiceContext context = ContextBuilder.newBuilder(PROVIDER)
.credentials(username, apiKey)
.overrides(overrides)
.buildView(ComputeServiceContext.class);
computeService = context.getComputeService();
有关完整示例,请查看CloudServersPublish。虽然它是Rackspace Cloud的一个例子,但代码几乎相同。