我的Reducer上出现了内存错误：
mapreduce.reduce.memory.mb
所以是的，有一个物理内存问题可以通过增加<Response xmlns="https://www.googleapis.com/books/v1/volumes">
<items>
<e>
<accessInfo>
<accessViewStatus>NONE</accessViewStatus>
<country>IN</country>
<embeddable>false</embeddable>
<epub>
<isAvailable>false</isAvailable>
</epub>
<pdf>
<isAvailable>false</isAvailable>
</pdf>
<publicDomain>false</publicDomain>
<quoteSharingAllowed>false</quoteSharingAllowed>
<textToSpeechPermission>ALLOWED</textToSpeechPermission>
<viewability>NO_PAGES</viewability>
<webReaderLink>http://books.google.co.in/books/reader?id=8bkxV-WcLFoC&hl=&printsec=frontcover&output=reader&source=gbs_api</webReaderLink>
</accessInfo>
<etag>iKsLXIuXjx8</etag>
<id>8bkxV-WcLFoC</id>
<kind>books#volume</kind>
<saleInfo>
<country>IN</country>
<isEbook>false</isEbook>
<saleability>NOT_FOR_SALE</saleability>
</saleInfo>
<searchInfo>
<textSnippet>Beginning with an &quot;Ant on an Apple&quot;, illustrations, simple text, and a pesky fly who will not stay on his page introduce the letters of the alphabet.</textSnippet>
</searchInfo>
<selfLink>https://www.googleapis.com/books/v1/volumes/8bkxV-WcLFoC</selfLink>
<volumeInfo>
<allowAnonLogging>false</allowAnonLogging>
<authors>
<e>Bruno Munari</e>
</authors>
<canonicalVolumeLink>https://books.google.com/books/about/ABC.html?hl=&id=8bkxV-WcLFoC</canonicalVolumeLink>
<contentVersion>0.0.1.0.preview.0</contentVersion>
<imageLinks>
<smallThumbnail>http://books.google.com/books/content?id=8bkxV-WcLFoC&printsec=frontcover&img=1&zoom=5&source=gbs_api</smallThumbnail>
<thumbnail>http://books.google.com/books/content?id=8bkxV-WcLFoC&printsec=frontcover&img=1&zoom=1&source=gbs_api</thumbnail>
</imageLinks>
<industryIdentifiers>
<e>
<identifier>MINN:319510009151144</identifier>
<type>OTHER</type>
</e>
</industryIdentifiers>
<infoLink>http://books.google.co.in/books?id=8bkxV-WcLFoC&dq=abc&hl=&source=gbs_api</infoLink>
<language>en</language>
<maturityRating>NOT_MATURE</maturityRating>
<panelizationSummary>
<containsEpubBubbles>false</containsEpubBubbles>
<containsImageBubbles>false</containsImageBubbles>
</panelizationSummary>
<previewLink>http://books.google.co.in/books?id=8bkxV-WcLFoC&q=abc&dq=abc&hl=&cd=1&source=gbs_api</previewLink>
<printType>BOOK</printType>
<publishedDate>1960</publishedDate>
<readingModes>
<image>false</image>
<text>false</text>
</readingModes>
<title>ABC.</title>
</volumeInfo>
</e>
<e>
<accessInfo>
<accessViewStatus>NONE</accessViewStatus>
<country>IN</country>
<embeddable>false</embeddable>
<epub>
<isAvailable>false</isAvailable>
</epub>
<pdf>
<isAvailable>false</isAvailable>
</pdf>
<publicDomain>false</publicDomain>
<quoteSharingAllowed>false</quoteSharingAllowed>
<textToSpeechPermission>ALLOWED</textToSpeechPermission>
<viewability>NO_PAGES</viewability>
<webReaderLink>http://books.google.co.in/books/reader?id=7s4WAQAAMAAJ&hl=&printsec=frontcover&output=reader&source=gbs_api</webReaderLink>
</accessInfo>
<etag>miIPSeJ57M8</etag>
<id>7s4WAQAAMAAJ</id>
<kind>books#volume</kind>
<saleInfo>
<country>IN</country>
<isEbook>false</isEbook>
<saleability>NOT_FOR_SALE</saleability>
</saleInfo>
<selfLink>https://www.googleapis.com/books/v1/volumes/7s4WAQAAMAAJ</selfLink>
<volumeInfo>
<allowAnonLogging>false</allowAnonLogging>
<authors>
<e>Bruno Munari</e>
</authors>
<canonicalVolumeLink>https://books.google.com/books/about/ABC.html?hl=&id=7s4WAQAAMAAJ</canonicalVolumeLink>
<categories>
<e>Juvenile Nonfiction</e>
</categories>
<contentVersion>preview-1.0.0</contentVersion>
<imageLinks>
<smallThumbnail>http://books.google.com/books/content?id=7s4WAQAAMAAJ&printsec=frontcover&img=1&zoom=5&source=gbs_api</smallThumbnail>
<thumbnail>http://books.google.com/books/content?id=7s4WAQAAMAAJ&printsec=frontcover&img=1&zoom=1&source=gbs_api</thumbnail>
</imageLinks>
<industryIdentifiers>
<e>
<identifier>UOM:49015000911801</identifier>
<type>OTHER</type>
</e>
</industryIdentifiers>
<infoLink>http://books.google.co.in/books?id=7s4WAQAAMAAJ&dq=abc&hl=&source=gbs_api</infoLink>
<language>en</language>
<maturityRating>NOT_MATURE</maturityRating>
<previewLink>http://books.google.co.in/books?id=7s4WAQAAMAAJ&q=abc&dq=abc&hl=&cd=2&source=gbs_api</previewLink>
<printType>BOOK</printType>
<publishedDate>1960</publishedDate>
<readingModes>
<image>false</image>
<text>false</text>
</readingModes>
<title>ABC.</title>
</volumeInfo>
</e>
<e>
<accessInfo>
<accessViewStatus>NONE</accessViewStatus>
<country>IN</country>
<embeddable>false</embeddable>
<epub>
<isAvailable>false</isAvailable>
</epub>
<pdf>
<isAvailable>false</isAvailable>
</pdf>
<publicDomain>false</publicDomain>
<quoteSharingAllowed>false</quoteSharingAllowed>
<textToSpeechPermission>ALLOWED</textToSpeechPermission>
<viewability>NO_PAGES</viewability>
<webReaderLink>http://books.google.co.in/books/reader?id=Q6LsAAAAMAAJ&hl=&printsec=frontcover&output=reader&source=gbs_api</webReaderLink>
</accessInfo>
<etag>wHyvfXqhLU8</etag>
<id>Q6LsAAAAMAAJ</id>
<kind>books#volume</kind>
<saleInfo>
<country>IN</country>
<isEbook>false</isEbook>
<saleability>NOT_FOR_SALE</saleability>
</saleInfo>
<searchInfo>
<textSnippet>An intense examination of the effects of technology on literacy and language.</textSnippet>
</searchInfo>
<selfLink>https://www.googleapis.com/books/v1/volumes/Q6LsAAAAMAAJ</selfLink>
<volumeInfo>
<allowAnonLogging>false</allowAnonLogging>
<authors>
<e>Ivan Illich</e>
<e>Barry Sanders</e>
</authors>
<canonicalVolumeLink>https://books.google.com/books/about/ABC.html?hl=&id=Q6LsAAAAMAAJ</canonicalVolumeLink>
<categories>
<e>Information storage and retrieval systems</e>
</categories>
<contentVersion>1.1.1.0.preview.0</contentVersion>
<description>An intense examination of the effects of technology on literacy and language. The authors argue that there is a phenomenon transforming modern culture--language is becoming part of a technology of "information systems" with an emphasis on control, rather than human exchange. As a result, all language is becoming debased.</description>
<imageLinks>
<smallThumbnail>http://books.google.com/books/content?id=Q6LsAAAAMAAJ&printsec=frontcover&img=1&zoom=5&source=gbs_api</smallThumbnail>
<thumbnail>http://books.google.com/books/content?id=Q6LsAAAAMAAJ&printsec=frontcover&img=1&zoom=1&source=gbs_api</thumbnail>
</imageLinks>
<industryIdentifiers>
<e>
<identifier>IND:30000036960999</identifier>
<type>OTHER</type>
</e>
</industryIdentifiers>
<infoLink>http://books.google.co.in/books?id=Q6LsAAAAMAAJ&dq=abc&hl=&source=gbs_api</infoLink>
<language>en</language>
<maturityRating>NOT_MATURE</maturityRating>
<pageCount>166</pageCount>
<previewLink>http://books.google.co.in/books?id=Q6LsAAAAMAAJ&q=abc&dq=abc&hl=&cd=3&source=gbs_api</previewLink>
<printType>BOOK</printType>
<publishedDate>1988</publishedDate>
<publisher>Marion Boyars Publishers</publisher>
<readingModes>
<image>false</image>
<text>false</text>
</readingModes>
<subtitle>The Alphabetization of the Popular Mind</subtitle>
<title>ABC</title>
</volumeInfo>
</e>
<e>
<accessInfo>
<accessViewStatus>SAMPLE</accessViewStatus>
<country>IN</country>
<embeddable>true</embeddable>
<epub>
<isAvailable>false</isAvailable>
</epub>
<pdf>
<isAvailable>false</isAvailable>
</pdf>
<publicDomain>false</publicDomain>
<quoteSharingAllowed>false</quoteSharingAllowed>
<textToSpeechPermission>ALLOWED</textToSpeechPermission>
<viewability>PARTIAL</viewability>
<webReaderLink>http://books.google.co.in/books/reader?id=f1vdiFRI52gC&hl=&printsec=frontcover&output=reader&source=gbs_api</webReaderLink>
</accessInfo>
<etag>R8E127WkV8s</etag>
<id>f1vdiFRI52gC</id>
<kind>books#volume</kind>
<saleInfo>
<country>IN</country>
<isEbook>false</isEbook>
<saleability>NOT_FOR_SALE</saleability>
</saleInfo>
<searchInfo>
<textSnippet>Includes 26 scenes which depict a Letterland character surrounded by objects beginning with the target sound. This children&#39;s alphabet book helps children in promoting phonemic awareness and vocabulary development.</textSnippet>
</searchInfo>
<selfLink>https://www.googleapis.com/books/v1/volumes/f1vdiFRI52gC</selfLink>
<volumeInfo>
<allowAnonLogging>false</allowAnonLogging>
<authors>
<e>Lyn Wendon</e>
</authors>
<canonicalVolumeLink>https://books.google.com/books/about/ABC.html?hl=&id=f1vdiFRI52gC</canonicalVolumeLink>
<categories>
<e>Alphabet</e>
</categories>
<contentVersion>0.2.0.0.preview.1</contentVersion>
<description>Includes 26 scenes which depict a Letterland character surrounded by objects beginning with the target sound. This children's alphabet book helps children in promoting phonemic awareness and vocabulary development.</description>
<imageLinks>
<smallThumbnail>http://books.google.com/books/content?id=f1vdiFRI52gC&printsec=frontcover&img=1&zoom=5&edge=curl&source=gbs_api</smallThumbnail>
<thumbnail>http://books.google.com/books/content?id=f1vdiFRI52gC&printsec=frontcover&img=1&zoom=1&edge=curl&source=gbs_api</thumbnail>
</imageLinks>
<industryIdentifiers>
<e>
<identifier>9781862092211</identifier>
<type>ISBN_13</type>
</e>
<e>
<identifier>1862092214</identifier>
<type>ISBN_10</type>
</e>
</industryIdentifiers>
<infoLink>http://books.google.co.in/books?id=f1vdiFRI52gC&dq=abc&hl=&source=gbs_api</infoLink>
<language>en</language>
<maturityRating>NOT_MATURE</maturityRating>
<pageCount>58</pageCount>
<previewLink>http://books.google.co.in/books?id=f1vdiFRI52gC&printsec=frontcover&dq=abc&hl=&cd=4&source=gbs_api</previewLink>
<printType>BOOK</printType>
<publishedDate>2003-04-01</publishedDate>
<publisher>Letterland</publisher>
<readingModes>
<image>true</image>
<text>false</text>
</readingModes>
<title>ABC</title>
</volumeInfo>
</e>
</items>
<kind>books#volumes</kind>
<totalItems>2279</totalItems>
</Response>
来解决，但我不明白它为什么会发生。
进入管道的数据越多，发生此内存问题的可能性就越大。问题是我的大多数减速器（大约90％）通过，并且因为数据应该已经被写入磁盘，所以应该释放内存，因为数据应该已经被写入磁盘。
我在这里缺少什么？
答案 0 :(得分：0)
默认情况下，在YARN中，容器预先分配有内存。在这种情况下，无论群集中的可用内存如何，所有reducer容器都将具有6 GB内存。这些容器的JVM不会在容器之间重复使用或共享（至少在Hadoop2中），这意味着如果一个reducer上的内存超出其内存6 GB限制，它将不会从其他空闲容器中获取资源（如果这是你的担忧）
现在，为什么只有少数减速器总是高于其内存（给定90％通过）提示数据中可能存在SKEW，这意味着此减速器可能正在处理更多输入组或更多键。