Question

在向hdinsight群集提交pig作业时，我收到502错误。有关如何修复群集的任何想法？上次我收到此错误我删除了群集并再次创建它，想找到一个更好的方法来修复错误。这是我得到的错误：

Microsoft.WindowsAzure.Management.HDInsight.Framework.Core.Library.WebRequest.HttpLayerException: Request failed with code:BadGateway
Content:<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"/>
<title>502 - Web server received an invalid response while acting as a gateway or proxy server.</title>
<style type="text/css">
<!--
body{margin:0;font-size:.7em;font-family:Verdana, Arial, Helvetica, sans-serif;background:#EEEEEE;}
fieldset{padding:0 15px 10px 15px;} 
h1{font-size:2.4em;margin:0;color:#FFF;}
h2{font-size:1.7em;margin:0;color:#CC0000;} 
h3{font-size:1.2em;margin:10px 0 0 0;color:#000000;} 
#header{width:96%;margin:0 0 0 0;padding:6px 2% 6px 2%;font-family:"trebuchet MS", Verdana, sans-serif;color:#FFF;
background-color:#555555;}
#content{margin:0 0 0 2%;position:relative;}
.content-container{background:#FFF;width:96%;margin-top:8px;padding:10px;position:relative;}
-->
</style>
</head>
<body>
<div id="header"><h1>Server Error</h1></div>
<div id="content">
 <div class="content-container"><fieldset>
  <h2>502 - Web server received an invalid response while acting as a gateway or proxy server.</h2>
  <h3>There is a problem with the page you are looking for, and it cannot be displayed. When the Web server (while acting as a gateway or proxy) contacted the upstream content server, it received an invalid response from the content server.</h3>
 </fieldset></div>
</div>
</body>
</html>


Server stack trace: 
   at Microsoft.Hadoop.Client.WebHCatRest.HadoopRemoteJobSubmissionRestClient.<SendRequestWithErrorChecking>d__2f.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at Microsoft.Hadoop.Client.WebHCatRest.HadoopRemoteJobSubmissionRestClient.<MakeAsyncJobSubmissionRequest>d__25.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at Microsoft.Hadoop.Client.WebHCatRest.HadoopRemoteJobSubmissionRestClient.<SubmitPigJob>d__14.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at Microsoft.Hadoop.Client.HadoopJobSubmissionPocoClient.RemoteHadoop.RemoteHadoopJobSubmissionPocoClient.<SubmitPigJob>d__1d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at Microsoft.WindowsAzure.Management.HDInsight.JobSubmission.PocoClient.HDInsightJobSubmissionPocoClient.<SubmitPigJob>d__10.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at Microsoft.WindowsAzure.Management.HDInsight.JobSubmission.HDInsightHadoopClient.<CreatePigJobAsync>d__16.MoveNext()

Exception rethrown at [0]: 
   at Microsoft.WindowsAzure.Management.HDInsight.Framework.Core.Library.TaskExtensions.WaitForResult[T](Task`1 task, TimeSpan timeout)
   at Microsoft.WindowsAzure.Management.HDInsight.Framework.Core.Library.TaskExtensions.WaitForResult[T](Task`1 task)
   at Microsoft.WindowsAzure.Management.HDInsight.JobSubmission.HDInsightHadoopClient.CreatePigJob(PigJobCreateParameters pigJobCreateParameters)

Answer 1

我很抱歉这里有麻烦，不应该发生。你还在经历这种行为吗？如果可能，您可以通过电子邮件跟进我（微软的mwinkle），或者通过the support portal

打开支持票

Answer 2

我有同样的问题，但关于Hive。为解决我的问题，我重新启动了头节点上的apache服务。

通过Azure启用远程连接
登录服务器
打开任务管理器
打开“服务”标签
按说明排序
使用Apache Hadoop ...

完成这些步骤后，请尝试提交作业。

Answer 3

MSDN上已经提出过这个问题。您似乎遇到了轻负载集群上HDInsight和Azure存储之间的已知错误。有关修复的文档，请参阅此链接：https://social.msdn.microsoft.com/Forums/en-US/c6bd04e9-e69b-4e96-b2ef-fa8abcf3bbc3/502-error-in-hbase-cluster?forum=hdinsight

hdinsight pig job提交502错误

3 个答案: