Sqoop返回“404 Not Found”的响应状态

时间:2014-11-12 22:28:20

标签: sqoop

我在CentOS 7上安装了最新的Hadoop 2.5.1和Sqoop 1.99.3(来自tarball文件)。 根据此处的说明安装了Sqoop:http://sqoop.apache.org/docs/1.99.3/Installation.html

然后,我想根据此页面测试Sqoop服务器:http://sqoop.apache.org/docs/1.99.3/Sqoop5MinutesDemo.html

我运行了以下命令:

show version --all

并收到以下错误消息:

Exception: com.sun.jersey.api.client.UniformInterfaceException Message: GET http://localhost:12000/sqoop/version returned a response status of 404 Not Found

这是我尝试启动Sqoop服务器后立即包含的配置和日志文件(通过运行 Sqoop.sh服务器启动):

/ etc / bashrc中的Sqoop变量:

export SQOOP_HOME="/usr/local/sqoop"
export SQOOP_CONF_DIR=$SQOOP_HOME/server/conf

/usr/local/sqoop/server/conf/Catalina.properties:

/usr/local/hadoop/share/hadoop/common/*.jar,
/usr/local/hadoop/share/hadoop/common/lib/*.jar,
/usr/local/hadoop/share/hadoop/hdfs/*.jar,
/usr/local/hadoop/share/hadoop/hdfs/lib/*.jar,
/usr/local/hadoop/share/hadoop/mapreduce/*.jar,
/usr/local/hadoop/share/hadoop/mapreduce/lib/*.jar,
/usr/local/hadoop/share/hadoop/tools/*.jar,
/usr/local/hadoop/share/hadoop/tools/lib/*.jar,
/usr/local/hadoop/share/hadoop/yarn/*.jar,
/usr/local/hadoop/share/hadoop/yarn/lib/*.jar

/usr/local/sqoop/server/conf/sqoop.properties:

org.apache.sqoop.submission.engine.mapreduce.configuration.directory=/usr/local/hadoop/etc/hadoop/

/usr/local/sqoop/server/logs/localhost.2014-11-13.log:

SEVERE: Exception sending context initialized event to listener instance of class org.apache.sqoop.server.ServerInitializer
java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration

/usr/local/sqoop/log/sqoop.log:

2014-11-13 08:19:55,466 INFO  audit.AuditLoggerManager [org.apache.sqoop.audit.AuditLoggerManager.initialize(AuditLoggerManager.java:79)] Begin audit logger manager initialization
2014-11-13 08:19:55,470 WARN  audit.FileAuditLogger [org.apache.sqoop.audit.FileAuditLogger.initialize(FileAuditLogger.java:67)] appender: log4j.appender.defaultAppender
2014-11-13 08:19:55,471 INFO  audit.AuditLoggerManager [org.apache.sqoop.audit.AuditLoggerManager.initializeLoggers(AuditLoggerManager.java:126)] Audit Logger has been initialized: default
2014-11-13 08:19:55,471 INFO  audit.AuditLoggerManager [org.apache.sqoop.audit.AuditLoggerManager.initialize(AuditLoggerManager.java:85)] Audit logger manager initialized: OK
2014-11-13 08:19:55,472 INFO  repository.RepositoryManager [org.apache.sqoop.repository.RepositoryManager.initialize(RepositoryManager.java:80)] Setting system properties: {derby.stream.error.file=/usr/loc
2014-11-13 08:19:55,478 INFO  repository.JdbcRepositoryContext [org.apache.sqoop.repository.JdbcRepositoryContext.<init>(JdbcRepositoryContext.java:154)] [repo-ctx] handler=org.apache.sqoop.repository.derb
2014-11-13 08:19:55,683 INFO  derby.DerbyRepositoryHandler [org.apache.sqoop.repository.derby.DerbyRepositoryHandler.initialize(DerbyRepositoryHandler.java:207)] DerbyRepositoryHandler initialized.
2014-11-13 08:19:55,686 INFO  repository.JdbcRepositoryProvider [org.apache.sqoop.repository.JdbcRepositoryProvider.initializeRepositoryHandler(JdbcRepositoryProvider.java:160)] JdbcRepositoryProvider init
2014-11-13 08:19:55,686 INFO  repository.JdbcRepositoryProvider [org.apache.sqoop.repository.JdbcRepositoryProvider.initialize(JdbcRepositoryProvider.java:67)] JdbcRepository initialized.
2014-11-13 08:19:55,686 INFO  repository.RepositoryManager [org.apache.sqoop.repository.RepositoryManager.initialize(RepositoryManager.java:118)] Creating or upgrading on disk structures if necessary
2014-11-13 08:19:56,022 DEBUG repository.JdbcRepositoryTransaction [org.apache.sqoop.repository.JdbcRepositoryTransaction.begin(JdbcRepositoryTransaction.java:73)] Tx count-begin: 1, rollback: false
2014-11-13 08:19:56,022 INFO  repository.JdbcRepository [org.apache.sqoop.repository.JdbcRepository$1.doIt(JdbcRepository.java:130)] Creating repository schema objects
2014-11-13 08:19:56,198 DEBUG derby.DerbyRepositoryHandler [org.apache.sqoop.repository.derby.DerbyRepositoryHandler.detectVersion(DerbyRepositoryHandler.java:272)] Detecting old version of repository
2014-11-13 08:19:56,354 DEBUG repository.JdbcRepositoryTransaction [org.apache.sqoop.repository.JdbcRepositoryTransaction.commit(JdbcRepositoryTransaction.java:84)] Tx count-commit: 1, rollback: false
2014-11-13 08:19:56,354 DEBUG repository.JdbcRepositoryTransaction [org.apache.sqoop.repository.JdbcRepositoryTransaction.close(JdbcRepositoryTransaction.java:103)] Tx count-close: 0, rollback: false
2014-11-13 08:19:56,354 INFO  repository.JdbcRepositoryTransaction [org.apache.sqoop.repository.JdbcRepositoryTransaction.close(JdbcRepositoryTransaction.java:111)] Attempting transaction commit
2014-11-13 08:19:56,360 DEBUG repository.JdbcRepositoryTransaction [org.apache.sqoop.repository.JdbcRepositoryTransaction.begin(JdbcRepositoryTransaction.java:73)] Tx count-begin: 1, rollback: false
2014-11-13 08:19:56,375 DEBUG derby.DerbyRepositoryHandler [org.apache.sqoop.repository.derby.DerbyRepositoryHandler.detectVersion(DerbyRepositoryHandler.java:272)] Detecting old version of repository
2014-11-13 08:19:56,375 DEBUG repository.JdbcRepositoryTransaction [org.apache.sqoop.repository.JdbcRepositoryTransaction.commit(JdbcRepositoryTransaction.java:84)] Tx count-commit: 1, rollback: false
2014-11-13 08:19:56,376 DEBUG repository.JdbcRepositoryTransaction [org.apache.sqoop.repository.JdbcRepositoryTransaction.close(JdbcRepositoryTransaction.java:103)] Tx count-close: 0, rollback: false
2014-11-13 08:19:56,376 INFO  repository.JdbcRepositoryTransaction [org.apache.sqoop.repository.JdbcRepositoryTransaction.close(JdbcRepositoryTransaction.java:111)] Attempting transaction commit
2014-11-13 08:19:56,376 INFO  repository.RepositoryManager [org.apache.sqoop.repository.RepositoryManager.initialize(RepositoryManager.java:128)] Repository initialized: OK
2014-11-13 08:19:56,377 INFO  connector.ConnectorManager [org.apache.sqoop.connector.ConnectorManager.initialize(ConnectorManager.java:174)] Connector config urls: [jar:file:/usr/local/sqoop-1.99.3-bin-had
2014-11-13 08:19:56,378 DEBUG connector.ConnectorHandler [org.apache.sqoop.connector.ConnectorHandler.<init>(ConnectorHandler.java:58)] Connector configuration: {org.apache.sqoop.connector.class=org.apache
2014-11-13 08:19:56,415 INFO  connector.ConnectorHandler [org.apache.sqoop.connector.ConnectorHandler.<init>(ConnectorHandler.java:113)] Connector [org.apache.sqoop.connector.jdbc.GenericJdbcConnector] ini
2014-11-13 08:19:56,416 DEBUG repository.JdbcRepositoryTransaction [org.apache.sqoop.repository.JdbcRepositoryTransaction.begin(JdbcRepositoryTransaction.java:73)] Tx count-begin: 1, rollback: false
2014-11-13 08:19:56,416 DEBUG repository.JdbcRepositoryTransaction [org.apache.sqoop.repository.JdbcRepositoryTransaction.begin(JdbcRepositoryTransaction.java:73)] Tx count-begin: 2, rollback: false
2014-11-13 08:19:56,416 DEBUG derby.DerbyRepositoryHandler [org.apache.sqoop.repository.derby.DerbyRepositoryHandler.findConnector(DerbyRepositoryHandler.java:441)] Looking up connector: generic-jdbc-conne
2014-11-13 08:19:56,477 DEBUG derby.DerbyRepositoryHandler [org.apache.sqoop.repository.derby.DerbyRepositoryHandler.findConnector(DerbyRepositoryHandler.java:488)] Looking up connector: generic-jdbc-conne
2014-11-13 08:19:56,477 DEBUG repository.JdbcRepositoryTransaction [org.apache.sqoop.repository.JdbcRepositoryTransaction.commit(JdbcRepositoryTransaction.java:84)] Tx count-commit: 2, rollback: false
2014-11-13 08:19:56,477 DEBUG repository.JdbcRepositoryTransaction [org.apache.sqoop.repository.JdbcRepositoryTransaction.close(JdbcRepositoryTransaction.java:103)] Tx count-close: 1, rollback: false
2014-11-13 08:19:56,478 DEBUG connector.ConnectorManager [org.apache.sqoop.connector.ConnectorManager.registerConnectors(ConnectorManager.java:227)] Registered connector: connector-generic-jdbc-connector:1
2014-11-13 08:19:56,478 DEBUG repository.JdbcRepositoryTransaction [org.apache.sqoop.repository.JdbcRepositoryTransaction.commit(JdbcRepositoryTransaction.java:84)] Tx count-commit: 1, rollback: false
2014-11-13 08:19:56,478 DEBUG repository.JdbcRepositoryTransaction [org.apache.sqoop.repository.JdbcRepositoryTransaction.close(JdbcRepositoryTransaction.java:103)] Tx count-close: 0, rollback: false
2014-11-13 08:19:56,478 INFO  repository.JdbcRepositoryTransaction [org.apache.sqoop.repository.JdbcRepositoryTransaction.close(JdbcRepositoryTransaction.java:111)] Attempting transaction commit
2014-11-13 08:19:56,478 INFO  connector.ConnectorManager [org.apache.sqoop.connector.ConnectorManager.initialize(ConnectorManager.java:200)] Connectors loaded: {generic-jdbc-connector={generic-jdbc-connect
2014-11-13 08:19:56,486 DEBUG repository.JdbcRepositoryTransaction [org.apache.sqoop.repository.JdbcRepositoryTransaction.begin(JdbcRepositoryTransaction.java:73)] Tx count-begin: 1, rollback: false
2014-11-13 08:19:56,486 DEBUG derby.DerbyRepositoryHandler [org.apache.sqoop.repository.derby.DerbyRepositoryHandler.findFramework(DerbyRepositoryHandler.java:537)] Looking up framework metadata
2014-11-13 08:19:56,503 DEBUG derby.DerbyRepositoryHandler [org.apache.sqoop.repository.derby.DerbyRepositoryHandler.findFramework(DerbyRepositoryHandler.java:583)] Looking up framework metadta found: fram
2014-11-13 08:19:56,503 DEBUG repository.JdbcRepositoryTransaction [org.apache.sqoop.repository.JdbcRepositoryTransaction.commit(JdbcRepositoryTransaction.java:84)] Tx count-commit: 1, rollback: false
2014-11-13 08:19:56,503 DEBUG repository.JdbcRepositoryTransaction [org.apache.sqoop.repository.JdbcRepositoryTransaction.close(JdbcRepositoryTransaction.java:103)] Tx count-close: 0, rollback: false
2014-11-13 08:19:56,503 INFO  repository.JdbcRepositoryTransaction [org.apache.sqoop.repository.JdbcRepositoryTransaction.close(JdbcRepositoryTransaction.java:111)] Attempting transaction commit
2014-11-13 08:19:56,504 INFO  framework.FrameworkManager [org.apache.sqoop.framework.FrameworkManager.initialize(FrameworkManager.java:159)] Submission manager initialized: OK
2014-11-13 08:19:56,507 INFO  mapreduce.MapreduceSubmissionEngine [org.apache.sqoop.submission.mapreduce.MapreduceSubmissionEngine.initialize(MapreduceSubmissionEngine.java:75)] Initializing Map-reduce Sub
2014-11-13 08:19:56,512 INFO  core.SqoopServer [org.apache.sqoop.core.SqoopServer.destroy(SqoopServer.java:35)] Shutting down Sqoop server

根据日志文件中的最后一行,Sqoop服务器自行关闭并且无法启动。我搜索了上面所有可能的错误消息组合,但没有发现任何有用的信息。我该如何解决这个问题?

2 个答案:

答案 0 :(得分:0)

当无法找到hadoop库时,这很常见。你可以做两件事:

  1. 通过apache Big Top或通过其中一个Hadoop分销商安装Sqoop(即Cloudera
  2. 编辑tomcat配置以包含这些jar(正如您在上面尝试过的那样)。日志文件为您提供缺少哪些类的详细信息。您可以通过命令行search查找遗失的罐子。
  3. 我强烈推荐第一个选项。

答案 1 :(得分:0)

谢谢!我仔细检查了catalana.properties文件,并确保common.loader部分是一行。如果此部分由多行组成,则TomCat将不会加载Sqoop webapp。不接受换行符。