如何在EMR上设置减速机数量

时间:2019-02-27 03:34:00

标签: java hadoop mapreduce amazon-emr

我通过aws上的自定义jar步骤来运行一系列Mapper和Reducer任务。我只想通过一个reducer运行最后一个reducer任务。我该怎么办?

2 个答案:

答案 0 :(得分:0)

也许:

 <input type="button" class="btn btn-success" name="lineitems" id="lineitems" value="Add Line Items" onclick="window.open('add_lineitems.php?order_id=<?php echo $order_id; ?>',  'LineItems','height=500,width=1300,left=100,top=100,resizable=yes,scrollbars=yes,toolbar=yes,menubar=no,location=no,directories=no, status=yes');">

来自:How do you force the number of reducers in a map reduce job to be higher? - Hortonworks

或者也许:

if($l1)
{

echo "<script>window.close();</script>";
}

?>
<script>
    window.onunload = refreshParent;
    function refreshParent() {
        window.opener.location.reload();
    }
</script>

发件人:hadoop - How does Hive choose the number of reducers for a job? - Stack Overflow

答案 1 :(得分:0)

我已经通过将job.setNumReduceTasks(1)放入包含映射器和reducer的java文件的main方法中来解决了这个问题。