我写一个hive-UDTF来通过在HDFS上加载.dat文件来解析ip地址,但遇到错误:
java.io.FileNotFoundException: hdfs:/THE_IP_ADDRESS:9000/tmp/ip_20170204.dat (Permission denied)
但实际上,dfs目录/ tmp和.data文件都具有完全访问权限:777
,我无法修改配置以禁用dfs权限。
我的UDTF中读取文件的行:
IP.load("hdfs://THE_IP_ADDRESS:9000/tmp/ip_20170204.dat");
和静态方法.load()
:
public static void load(String filename) {
ipFile = new File(filename);
load();
if (enableFileWatch) {
watch();
}
}
private static void load() {
lastModifyTime = ipFile.lastModified();
FileInputStream fin = null;
lock.lock();
try {
dataBuffer = ByteBuffer.allocate(Long.valueOf(ipFile.length()).intValue());
fin = new FileInputStream(ipFile);
int readBytesLength;
byte[] chunk = new byte[4096];
while (fin.available() > 0) {
readBytesLength = fin.read(chunk);
dataBuffer.put(chunk, 0, readBytesLength);
}
dataBuffer.position(0);
int indexLength = dataBuffer.getInt();
byte[] indexBytes = new byte[indexLength];
dataBuffer.get(indexBytes, 0, indexLength - 4);
indexBuffer = ByteBuffer.wrap(indexBytes);
indexBuffer.order(ByteOrder.LITTLE_ENDIAN);
offset = indexLength;
int loop = 0;
while (loop++ < 256) {
index[loop - 1] = indexBuffer.getInt();
}
indexBuffer.order(ByteOrder.BIG_ENDIAN);
} catch (IOException ioe) {
ioe.printStackTrace();
} finally {
try {
if (fin != null) {
fin.close();
}
} catch (IOException e){
e.printStackTrace();
}
lock.unlock();
}
}