Question

我正在尝试在Apache Pig中编写代码以添加矩阵。

matrixM = LOAD 'Mmatrix.txt' USING PigStorage (',') AS (i,j,v);
matrixN = LOAD 'Nmatrix.txt' USING PigStorage (',') AS (i,j,v);
unionres = UNION matrixM, matrixN;
DUMP unionres;
res = GROUP unionres BY (i,j);
DUMP res;
ILLUSTRATE res;
final_res = FOREACH res GENERATE group.$0 AS i, group.$1 AS j, SUM(unionres.v) AS v;
DUMP final_res;

我在本地模式下运行代码时遇到异常。

org.apache.hadoop.io.ReadaheadPool - Failed readahead on ifile
EBADF: Bad file descriptor
    at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posix_fadvise(Native Method)
    at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posixFadviseIfPossible(NativeIO.java:267)
    at org.apache.hadoop.io.nativeio.NativeIO$POSIX$CacheManipulator.posixFadviseIfPossible(NativeIO.java:146)
    at org.apache.hadoop.io.ReadaheadPool$ReadaheadRequestImpl.run(ReadaheadPool.java:208)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

有人可以解释一下我哪里出错了吗？

Answer 1

通常上面的消息是WARN，之后会显示实际的错误消息。当磁盘坏或满时，我只看到过这种类型的WARN消息。您的本地/ tmp是否已满？

ifile EBADF上的预读失败：执行矩阵添加时文件描述符错误

1 个答案: