我尝试使用下面的代码通过Oozie Java Action连接到Posgres Hive Metastore。
我将hive-site.xml传递给了操作，因此它应该拥有所需的所有信息。
HiveMetaStoreClient client = new HiveMetaStoreClient(conf);
log.info("Successfully created the HiveMetaStoreClient");
try {
log.info(String.format("Loading the partitions for %s.%s", database, table));
List<Partition> partitions = client.listPartitions(database, table, (short) 200);
log.info(String.format("Processing %d partitions", partitions.size()));
for (Partition partition : partitions) {
StorageDescriptor sd = partition.getSd();
String location = sd.getLocation();
String newLocation = location.replace(from, to);
log.info(String.format("Moving from %s to %s", location, newLocation));
sd.setLocation(newLocation);
}
} catch (TException e) {
logExceptionStack(e);
}
日志并不完全错误，但它表明它使用DERBY后端查看其他一些Metastore;
我难以找到问题所在并强迫HiveMetaStoreClient指向正确的Metastore。
2016-02-08 16:48:05,972 INFO [uber-SubtaskRunner] com.xxxxxxx.PartitionMigrator.Program: Attempting to create the HiveMetaStoreClient
2016-02-08 16:48:06,123 INFO [uber-SubtaskRunner] com.xxxxxxx.PartitionMigrator.Program: hiveconf metastoreURI: null
2016-02-08 16:48:06,194 INFO [uber-SubtaskRunner] org.apache.hadoop.hive.metastore.HiveMetaStore: 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
2016-02-08 16:48:06,222 INFO [uber-SubtaskRunner] org.apache.hadoop.hive.metastore.ObjectStore: ObjectStore, initialize called
2016-02-08 16:48:06,385 INFO [uber-SubtaskRunner] DataNucleus.Persistence: Property datanucleus.cache.level2 unknown - will be ignored
2016-02-08 16:48:06,385 INFO [uber-SubtaskRunner] DataNucleus.Persistence: Property hive.metastore.integral.jdo.pushdown unknown - will be ignored
2016-02-08 16:48:06,506 WARN [uber-SubtaskRunner] DataNucleus.Connection: BoneCP specified but not present in CLASSPATH (or one of dependencies)
2016-02-08 16:48:06,840 WARN [uber-SubtaskRunner] DataNucleus.Connection: BoneCP specified but not present in CLASSPATH (or one of dependencies)
2016-02-08 16:48:08,339 INFO [uber-SubtaskRunner] org.apache.hadoop.hive.metastore.ObjectStore: Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order"
2016-02-08 16:48:09,286 INFO [uber-SubtaskRunner] DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as "embedded-only" so does not have its own datastore table.
2016-02-08 16:48:09,286 INFO [uber-SubtaskRunner] DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only" so does not have its own datastore table.
2016-02-08 16:48:10,400 INFO [uber-SubtaskRunner] DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as "embedded-only" so does not have its own datastore table.
2016-02-08 16:48:10,400 INFO [uber-SubtaskRunner] DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only" so does not have its own datastore table.
2016-02-08 16:48:10,676 INFO [uber-SubtaskRunner] org.apache.hadoop.hive.metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY
2016-02-08 16:48:10,677 INFO [uber-SubtaskRunner] org.apache.hadoop.hive.metastore.ObjectStore: Initialized ObjectStore
2016-02-08 16:48:10,798 WARN [uber-SubtaskRunner] org.apache.hadoop.hive.metastore.ObjectStore: Version information not found in metastore. hive.metastore.schema.verification is not enabled so recording the schema version 1.1.0
2016-02-08 16:48:10,928 WARN [uber-SubtaskRunner] org.apache.hadoop.hive.metastore.ObjectStore: Failed to get database default, returning NoSuchObjectException
2016-02-08 16:48:11,019 INFO [uber-SubtaskRunner] org.apache.hadoop.hive.metastore.HiveMetaStore: Added admin role in metastore
2016-02-08 16:48:11,021 INFO [uber-SubtaskRunner] org.apache.hadoop.hive.metastore.HiveMetaStore: Added public role in metastore
2016-02-08 16:48:11,097 INFO [uber-SubtaskRunner] org.apache.hadoop.hive.metastore.HiveMetaStore: No user is added in admin role, since config is empty
2016-02-08 16:48:11,193 INFO [uber-SubtaskRunner] com.xxxxxxx.PartitionMigrator.Program: Successfully created the HiveMetaStoreClient