在运行时避免在HDP上修改mapred.child.env，以便R可以使用RHive建立与hiveserver2的连接

Question

我试图让R＆R的RHive软件包与hiveserver2进行良好的通信。

尝试使用以下方法连接到hiveserver2时收到错误：

>rhive.connect(host="localhost",port=10000, hiveServer2=TRUE, user="root", password="hadoop")

初次运行时的输出：

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/hdp/2.2.0.0-2041/hadoop/client/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/2.2.0.0-2041/hadoop/client/slf4j-log4j12.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/2.2.0.0-2041/hadoop/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/2.2.0.0-2041/hive/lib/hive-jdbc-0.14.0.2.2.0.0-2041-standalone.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/2.2.0.0-2041/hive/lib/hive-jdbc.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
15/03/19 07:08:22 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
15/03/19 07:08:23 WARN shortcircuit.DomainSocketFactory: The short-circuit local reads feature cannot be used because libhadoop cannot be loaded.
15/03/19 07:08:24 INFO jdbc.Utils: Supplied authorities: localhost:10000
15/03/19 07:08:24 INFO jdbc.Utils: Resolved authority: localhost:10000
15/03/19 07:08:24 INFO jdbc.HiveConnection: Will try to open client transport with JDBC Uri: jdbc:hive2://localhost:10000/default

这会导致错误：

错误：org.apache.hive.service.cli.HiveSQLException：处理语句时出错：无法在运行时修改mapred.child.env。它不在允许在运行时修改的参数列表中

在后续运行相同命令时，输出将减少为：

15/03/19 07:16:24 INFO jdbc.Utils: Supplied authorities: localhost:10000
15/03/19 07:16:24 INFO jdbc.Utils: Resolved authority: localhost:10000
15/03/19 07:16:24 INFO jdbc.HiveConnection: Will try to open client transport with JDBC Uri: jdbc:hive2://localhost:10000/default
Error: org.apache.hive.service.cli.HiveSQLException: Error while processing statement: Cannot modify mapred.child.env at runtime. It is not in list of params that are allowed to be modified at runtime

这向我表明我可能在某处没有足够的权限...但是，我使用root运行它。所以，我不确定我错过了哪些权限......

我已使用安装指南via README安装了RHive。

注意：如果我使用CRAN version of the package，则会出现同样的错误。

我目前正在使用Hortonworks Data Platform 2.2 (HDP 2.2)的虚拟框图片。因此，已经安装了hadoop和hiveserver2。我已经安装了R版本3.1.2。

以下是我安装RHive的方法：

# Set up paths for HIVE_HOME, HADOOP_HOME, and HADOOP_CONF
export HIVE_HOME=/usr/hdp/2.2.0.0-2041/hive

export HADOOP_HOME=/usr/hdp/2.2.0.0-2041/hadoop

export HADOOP_CONF_DIR=/etc/hadoop/conf

# R Location via RHOME
R_HOME=/usr/lib64/R

# Place R_HOME into hadoop config location
sudo sh -c "echo \"R_HOME='$R_HOME'\" >> $HADOOP_HOME/conf/hadoop-env.sh"

# Add remote enable to Rserve config.
sudo sh -c "echo 'remote enable' >> /etc/Rserv.conf"

# Launch the daemon
R CMD Rserve

# Confirm launch
netstat -nltp

# Install ant to build java files
sudo yum -y install ant

# Install package dependencies
sudo R --no-save << EOF
install.packages( c('rJava','Rserve','RUnit'), repos='http://cran.us.r-project.org', INSTALL_opts=c('--byte-compile') )
EOF

# Install RHive package
git clone https://github.com/nexr/RHive.git
cd RHive
ant build
sudo R CMD INSTALL RHive

要检查打开R并使用EOF之间的语句，或直接从shell运行命令：

sudo R --no-save << EOF
Sys.setenv(HIVE_HOME="/usr/hdp/2.2.0.0-2041/hive")
Sys.setenv(HADOOP_HOME="/usr/hdp/2.2.0.0-2041/hadoop")
Sys.setenv(HADOOP_CONF_DIR="/etc/hadoop/conf")
library(RHive)
rhive.connect(host="localhost",port=10000, hiveServer2=TRUE, user="root", password="hadoop")
EOF

Answer 1

link提到了答案。

基本上，你必须添加一个属性＆＃34; hive.security.authorization.sqlstd.confwhitelist.append＆＃34;有价值＆＃34; mapred.child.env＆＃34;在/etc/hive/conf/hive-site.xml

中

此解决方案对我有用，但我使用Ambari UI进行此配置更改。