我正在编译hadoop-yarn-nodemanager。
编译环境:#!/home/XXX/PycharmProjects/XX/venv/bin/python
# EASY-INSTALL-ENTRY-SCRIPT: 'pip==10.0.1','console_scripts','pip3'
__requires__ = 'pip==10.0.1'
import re
import sys
from pkg_resources import load_entry_point
if __name__ == '__main__':
sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
sys.exit(
load_entry_point('pip==10.0.1', 'console_scripts', 'pip3')()
)
,MacOS-10.14
,java-1.7.0_80
与cmake3.13.0-rc3
,clang-1000.10.44.4
,Maven 3.6.0
。
我正在尝试在MacOS上安装protocbuf 2.5.0
,但正如其文档所示,
仅* nix平台支持本机hadoop库。该库不不能与Cygwin或Mac OS X平台一起使用。
因此,我必须重新编译Hadoop的源代码。在下载的Hadoop-2.2.0
文件夹中,运行hadoop-2.2.0-src
,经过几分钟的编译,新的本机库应该位于mvn package -Pdist,native -DskipTests -Dtar
处。但是,我一直在找回错误消息。有些已经通过修改源代码而修复,但是现在我陷入了编译hadoop-2.2.0-src/hadoop-dist/target/hadoop-2.2.0/lib/native
的陷阱。
这是错误消息:
hadoop-yarn-server-nodemanager
我试图将cmake的编译器从 [exec] [ 57%] Linking C executable target/usr/local/bin/test-container-executor
[exec] /Applications/CMake.app/Contents/bin/cmake -E cmake_link_script CMakeFiles/test-container-executor.dir/link.txt --verbose=1
[exec] /Library/Developer/CommandLineTools/usr/bin/cc -g -Wall -O2 -D_GNU_SOURCE -D_REENTRANT -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk -Wl,-search_paths_first -Wl,-headerpad_max_install_names CMakeFiles/test-container-executor.dir/main/native/container-executor/test/test-container-executor.c.o -o target/usr/local/bin/test-container-executor libcontainer.a
[exec] Undefined symbols for architecture x86_64:
[exec] "_fcloseall", referenced from:
[exec] _launch_container_as_user in libcontainer.a(container-executor.c.o)
[exec] ld: symbol(s) not found for architecture x86_64
[exec] clang: error: linker command failed with exit code 1 (use -v to see invocation)
[exec] make[2]: *** [target/usr/local/bin/test-container-executor] Error 1
[exec] make[1]: *** [CMakeFiles/test-container-executor.dir/all] Error 2
[exec] make: *** [all] Error 2
切换到clang
,但是没有用。
相对于错误消息,我发现以下代码。
在gcc
中:
hadoop-2.2.0-src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/CMakeLists.txt
在add_executable(test-container-executor
main/native/container-executor/test/test-container-executor.c
)
target_link_libraries(test-container-executor
container
)
output_directory(test-container-executor target/usr/local/bin)
中:
hadoop-2.2.0-src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/configuration.c
在int launch_container_as_user(const char *user, const char *app_id,
const char *container_id, const char *work_dir,
const char *script_name, const char *cred_file,
const char* pid_file, char* const* local_dirs,
char* const* log_dirs, const char *resources_key,
char* const* resources_values) {...}
中:
hadoop-2.2.0-src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/native/CMakeFiles/test-container-executor.dir/link.txt
对于压缩文件/Library/Developer/CommandLineTools/usr/bin/cc -g -Wall -O2 -D_GNU_SOURCE -D_REENTRANT -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk -Wl,-search_paths_first -Wl,-headerpad_max_install_names CMakeFiles/test-container-executor.dir/main/native/container-executor/test/test-container-executor.c.o -o target/usr/local/bin/test-container-executor libcontainer.a
,我在解压缩后发现hadoop-2.2.0-src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/native/libcontainer.a
,但是由于编码问题而无法打开它。
此外,以前在编译此项目时出现错误:
container-executor.c.o
要修复此问题,我将 [exec] /Users/markdana/Downloads/hadoop-2.2.0-src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/container-executor.c:1252:48: error: too many arguments to function call, expected 4, have 5
[exec] if (mount("none", mount_path, "cgroup", 0, controller) == 0) {
[exec] ~~~~~ ^~~~~~~~~~
[exec] /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include/sys/mount.h:399:1: note: 'mount' declared here
[exec] int mount(const char *, const char *, int, void *);
中的函数mount()
的声明修改为:
mount.h
这有点愚蠢,我知道,但是至少可以用。然后遇到问题中显示的新问题。我想知道他们是否担心,或者链接库存在一些错误。
已经调试了整整一天,对完成的工作一无所知。如果您可以指出密钥,或者分享一些类似的处理cmake链接问题的经验,将不胜感激。
答案 0 :(得分:1)
在OS X上似乎不存在功能fcloseall
。从Porting UNIX/Linux Applications to OS X:
fcloseall
此功能是
fclose
的扩展。尽管OS X支持fclose
,但不支持fcloseall
。通过将文件指针存储在数组中并遍历数组,可以使用fclose
来实现fcloseall
。
您需要重新设计应用程序并存储应该用fcloseall
关闭的每个文件。之后,您可以对每个此类文件使用简单的close
,如引用中所述。