我正在使用Hadoop 2.7.1版本的sqoop 1.4.6和java 1.8。所有的守护人员都在正常运作。当我使用sqoop导入时,我得到以下错误。你能告诉我发生错误的地方以及如何解决这个错误。提前谢谢。
sqoop import --bindir ./ --connect jdbc:mysql://localhost/mydb --username root --password yashu123 --table shipper --m 1 --target-dir /file
Warning: /usr/local/sqoop/../hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
Warning: /usr/local/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
2017-02-07 10:30:00,197 INFO [main] sqoop.Sqoop: Running Sqoop version: 1.4.6
2017-02-07 10:30:00,215 WARN [main] tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
2017-02-07 10:30:00,349 INFO [main] manager.MySQLManager: Preparing to use a MySQL streaming resultset.
2017-02-07 10:30:00,349 INFO [main] tool.CodeGenTool: Beginning code generation
Tue Feb 07 10:30:00 IST 2017 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
2017-02-07 10:30:00,813 INFO [main] manager.SqlManager: Executing SQL statement: SELECT t.* FROM `shipper` AS t LIMIT 1
2017-02-07 10:30:00,855 INFO [main] manager.SqlManager: Executing SQL statement: SELECT t.* FROM `shipper` AS t LIMIT 1
2017-02-07 10:30:00,885 INFO [main] orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/local/hadoop
Note: ./shipper.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
2017-02-07 10:30:02,259 INFO [main] orm.CompilationManager: Writing jar file: ./shipper.jar
2017-02-07 10:30:02,427 WARN [main] manager.MySQLManager: It looks like you are importing from mysql.
2017-02-07 10:30:02,428 WARN [main] manager.MySQLManager: This transfer can be faster! Use the --direct
2017-02-07 10:30:02,428 WARN [main] manager.MySQLManager: option to exercise a MySQL-specific fast path.
2017-02-07 10:30:02,428 INFO [main] manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
2017-02-07 10:30:02,430 INFO [main] mapreduce.ImportJobBase: Beginning import of shipper
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/local/hbase/lib/slf4j-log4j12-1.6.4.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
2017-02-07 10:30:02,656 WARN [main] util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2017-02-07 10:30:02,660 INFO [main] Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
2017-02-07 10:30:03,295 INFO [main] Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
2017-02-07 10:30:03,561 INFO [main] client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
Tue Feb 07 10:30:09 IST 2017 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
2017-02-07 10:30:09,358 INFO [main] db.DBInputFormat: Using read commited transaction isolation
2017-02-07 10:30:09,580 INFO [main] mapreduce.JobSubmitter: number of splits:1
2017-02-07 10:30:09,649 INFO [main] Configuration.deprecation: mapred.job.name is deprecated. Instead, use mapreduce.job.name
2017-02-07 10:30:09,649 INFO [main] Configuration.deprecation: mapred.cache.files.timestamps is deprecated. Instead, use mapreduce.job.cache.files.timestamps
2017-02-07 10:30:09,649 INFO [main] Configuration.deprecation: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class
2017-02-07 10:30:09,650 INFO [main] Configuration.deprecation: mapreduce.inputformat.class is deprecated. Instead, use mapreduce.job.inputformat.class
2017-02-07 10:30:09,650 INFO [main] Configuration.deprecation: mapreduce.outputformat.class is deprecated. Instead, use mapreduce.job.outputformat.class
2017-02-07 10:30:09,650 INFO [main] Configuration.deprecation: mapred.output.value.class is deprecated. Instead, use mapreduce.job.output.value.class
2017-02-07 10:30:09,650 INFO [main] Configuration.deprecation: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
2017-02-07 10:30:09,650 INFO [main] Configuration.deprecation: mapred.cache.files is deprecated. Instead, use mapreduce.job.cache.files
2017-02-07 10:30:09,650 INFO [main] Configuration.deprecation: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir
2017-02-07 10:30:09,650 INFO [main] Configuration.deprecation: mapred.job.classpath.files is deprecated. Instead, use mapreduce.job.classpath.files
2017-02-07 10:30:09,651 INFO [main] Configuration.deprecation: user.name is deprecated. Instead, use mapreduce.job.user.name
2017-02-07 10:30:09,651 INFO [main] Configuration.deprecation: mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
2017-02-07 10:30:09,651 INFO [main] Configuration.deprecation: mapred.cache.files.filesizes is deprecated. Instead, use mapreduce.job.cache.files.filesizes
2017-02-07 10:30:09,651 INFO [main] Configuration.deprecation: mapred.output.key.class is deprecated. Instead, use mapreduce.job.output.key.class
2017-02-07 10:30:09,849 INFO [main] mapreduce.JobSubmitter: Submitting tokens for job: job_1486442978038_0001
2017-02-07 10:30:10,353 INFO [main] impl.YarnClientImpl: Submitted application application_1486442978038_0001 to ResourceManager at /0.0.0.0:8032
2017-02-07 10:30:10,403 INFO [main] mapreduce.Job: The url to track the job: http://http://yasodhara-ideacentre-300S-08IHH:8088/proxy/application_1486442978038_0001/
2017-02-07 10:30:10,404 INFO [main] mapreduce.Job: Running job: job_1486442978038_0001
2017-02-07 10:30:16,612 INFO [main] mapreduce.Job: Job job_1486442978038_0001 running in uber mode : false
2017-02-07 10:30:16,614 INFO [main] mapreduce.Job: map 0% reduce 0%
2017-02-07 10:30:21,813 INFO [main] mapreduce.Job: map 100% reduce 0%
2017-02-07 10:30:21,829 INFO [main] mapreduce.Job: Job job_1486442978038_0001 completed successfully
2017-02-07 10:30:21,936 ERROR [main] tool.ImportTool: Imported Failed: No enum constant org.apache.hadoop.mapreduce.JobCounter.MB_MILLIS_MAPS
答案 0 :(得分:0)
如果您尝试将数据从mysql导入Hive,请考虑以下情况:
**MySql Table :**
create table company
(
id int,
name varchar2(20),
location varchar2(20)
);
**Hive Table :**
create database acad;
use acad;
create table company
(
id int,
name string,
location string
);
此表将存储在HDFS默认仓库位置:
/user/hive/warehouse/acad.db/company
现在使用sqoop加载数据应该使用命令:
sqoop import --connect jdbc:mysql://localhost/b1 --username 'root' -P --
table 'company' --hive-import --hive-table 'company' -m 1 --warehouse-dir
/user/hive/warehouse/acad.db;
确保你指定--warehouse-dir到db,所以当mapreduce作业运行时,它将公司表从mysql导入/user/hive/warehouse/acad.db位置创建公司文件夹,里面你将拥有你的输出文件:_SUCCESS和part-m-00000。
通常可以通过创建表和存储值来理解这一点,它将存储在location / user / hive / warehouse中///
当你可以使用sqoop做同样的事情时,hive将能够从该文件中读取数据,并在hive shell中显示数据
通过这种方式,您将能够使用sqoop成功将数据导入配置单元。 关于该错误,它实际上不会阻止您使用来自mysql的sqoop将数据导入配置单元