Python 2.7.6 Pydoop安装在Ubuntu上失败

时间:2015-04-15 09:02:51

标签: python hadoop ubuntu-14.04 python-2.6

我在安装pydoop软件包时遇到此错误:

root@ubuntu:~# pip install pydoop
Downloading/unpacking pydoop
  Downloading pydoop-1.0.0.tar.gz (931kB): 931kB downloaded
  Running setup.py (path:/tmp/pip_build_root/pydoop/setup.py) egg_info for package pydoop
    using setuptools version 15.0
    Traceback (most recent call last):
      File "<string>", line 17, in <module>
      File "/tmp/pip_build_root/pydoop/setup.py", line 59, in <module>
        import pydoop.hdfs.core.impl as hdfsimpl
      File "pydoop/hdfs/__init__.py", line 99, in <module>
        init()
      File "pydoop/hdfs/__init__.py", line 93, in init
        pydoop.hadoop_classpath(), _ORIG_CLASSPATH, pydoop.hadoop_conf()
      File "pydoop/__init__.py", line 106, in hadoop_classpath
        return _PATH_FINDER.hadoop_classpath(hadoop_home)
      File "pydoop/hadoop_utils.py", line 540, in hadoop_classpath
        hadoop_home = self.hadoop_home()
      File "pydoop/hadoop_utils.py", line 399, in hadoop_home
        PathFinder.__error("hadoop home", "HADOOP_HOME")
      File "pydoop/hadoop_utils.py", line 385, in __error
        raise ValueError("%s not found, try setting %s" % (what, env_var))
    ValueError: hadoop home not found, try setting HADOOP_HOME
    Complete output from command python setup.py egg_info:
    using setuptools version 15.0

追踪(最近一次呼叫最后一次):

  File "<string>", line 17, in <module>

  File "/tmp/pip_build_root/pydoop/setup.py", line 59, in <module>

    import pydoop.hdfs.core.impl as hdfsimpl

  File "pydoop/hdfs/__init__.py", line 99, in <module>

    init()

  File "pydoop/hdfs/__init__.py", line 93, in init

    pydoop.hadoop_classpath(), _ORIG_CLASSPATH, pydoop.hadoop_conf()

  File "pydoop/__init__.py", line 106, in hadoop_classpath

    return _PATH_FINDER.hadoop_classpath(hadoop_home)

  File "pydoop/hadoop_utils.py", line 540, in hadoop_classpath

    hadoop_home = self.hadoop_home()

  File "pydoop/hadoop_utils.py", line 399, in hadoop_home

    PathFinder.__error("hadoop home", "HADOOP_HOME")

  File "pydoop/hadoop_utils.py", line 385, in __error

    raise ValueError("%s not found, try setting %s" % (what, env_var))

ValueError: hadoop home not found, try setting HADOOP_HOME

清理...... 命令python setup.py egg_info失败,错误代码为1 / tmp / pip_build_root / pydoop 在/home/abhimanyu/.pip/pip.log

中存储失败的调试日志

3 个答案:

答案 0 :(得分:0)

试试这个:

sudo sh -c "ls ; export HADOOP_HOME=/yourpathtohadoopinstallation/; export JAVA_HOME=/yourpathtojavainstallation/; pip install pydoop"

这对我有把戏

答案 1 :(得分:0)

更新/ etc / profile并设置HADOOP_HOME和JAVA_HOME。更新/ etc / profile后,注销并重新登录。

现在运行sudo -E pip install pydoop

-E保留环境

有关详细信息,请参阅https://github.com/crs4/pydoop/issues/170

答案 2 :(得分:0)

因为我不能评论saseaturtle发布的答案。我把它放在这里。

我首先遇到了完全相同的问题,pydoop抱怨找不到HADOOP_HOME和HADOOP_PREFIX,而这两个字段就在那里,以下命令可以正确显示它们的值。

echo $HADOOP_HOME; echpHADOOP_PREFIX

我尝试过saseaturtle发布的解决方案,但是这次pydoop抱怨丢失javac,虽然shell中的 javac 类型可以显示正确的反馈并且所有这些$ JAVA_HOME或$ JAVA_LIB环境都设置正确。

最后,我使用以下命令成功安装了pydoop:

sudo sh -c "export PATH=/usr/java/jdk1.8.0_92/bin:$PATH; export HADOOP_HOME=/usr/local/hadoop-2.6.3; export JAVA_HOME=/home/why/download/jdk1.8.0_92; pip install pydoop"

我的猜测是所有上述问题都来自环境设置,因为命令是以sudo身份运行的。 root用户可能没有将那些需要的环境变量设置为当前用户,这就是添加一些导出命令解决问题的原因。