我必须编写一个批处理作业，它的来源是Teradata DB＆amp;作为目标，使用Spring批处理框架的Postgres DB。任何人都可以让我知道或推荐我如何实现这一目标的教程，如果可能的话？
如果没有，为了做到这一点，我可以探索的其他选择是什么？
更新
我有一个Spring批处理作业，试图让它与Oracle（Source）＆amp; Postgres（目标），稍后我将把源更改为Teradata。但是，我得到的错误，我认为这是因为，在Oracle DB中查找作业序列（无法创建序列，因为DB用户具有只读访问权限）。如何制作作业，在目标DB（Postgres）中创建序列？
错误：
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.executeLocalJobs(JobLauncherCommandLineRunner.java:227)
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.launchJobFromProperties(JobLauncherCommandLineRunner.java:121)
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.run(JobLauncherCommandLineRunner.java:115)
at org.springframework.boot.SpringApplication.runCommandLineRunners(SpringApplication.java:672)
... 5 common frames omitted
Caused by: java.sql.SQLException: ORA-02289: sequence does not exist
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:112)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:331)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:288)
at oracle.jdbc.driver.T4C8Oall.receive(T4C8Oall.java:745)
at oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:210)
at oracle.jdbc.driver.T4CStatement.executeForDescribe(T4CStatement.java:804)
at oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1049)
at oracle.jdbc.driver.T4CStatement.executeMaybeDescribe(T4CStatement.java:845)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1154)
at oracle.jdbc.driver.OracleStatement.executeQuery(OracleStatement.java:1313)
at org.springframework.jdbc.support.incrementer.AbstractSequenceMaxValueIncrementer.getNextKey(AbstractSequenceMaxValueIncrementer.java:69)
... 40 common frames omitted
2016-08-12 12:34:57.467 INFO 11660 --- [main] s.c.a.AnnotationConfigApplicationContext : Closing org.springframework.context.annotation.AnnotationConfigApplicationContext@1cea91 b: startup date [Fri Aug 12 12:34:22 IST 2016]; root of context hierarchy
2016-08-12 12:34:57.469 INFO 11660 --- [main] o.s.j.e.a.AnnotationMBeanExporter : Unregistering JMX-exposed beans on shutdown
** BatchConfiguration.java **
@Configuration
@EnableBatchProcessing
@ComponentScan(basePackageClasses = MyBatchConfigurer.class)
public class BatchConfiguration {
@Autowired
public JobBuilderFactory jobBuilderFactory;
@Autowired
public StepBuilderFactory stepBuilderFactory;
@Autowired
@Qualifier("oracleDBDataSource")
public DataSource dataSource1;
@Autowired
@Qualifier("postgresDataSource")
public DataSource dataSource2;
//Read an Event from oracle DB
@Bean
public ItemReader<Event> reader(){
JdbcCursorItemReader<Event> databaseReader = new JdbcCursorItemReader<>();
String sqlQ = "select * from test_event";
databaseReader.setDataSource(dataSource1);
databaseReader.setSql(sqlQ);
databaseReader.setRowMapper(new BeanPropertyRowMapper<>(Event.class));
return databaseReader;
}
//EventItemProcessor
@Bean
public EventItemProcessor processor() {
return new EventItemProcessor();
}
@Bean
public JdbcBatchItemWriter<Event> writer() {
JdbcBatchItemWriter<Event> writer = new JdbcBatchItemWriter<Event>();
writer.setItemSqlParameterSourceProvider(new BeanPropertyItemSqlParameterSourceProvider<Event>());
writer.setSql("insert into EVENTS (event_id, event_title, event_start_date, event_end_date, event_type,need_offline_sync, event_status, created_by, created_date, last_updated_by,last_updated_date) VALUES (:event_id, :event_title, :event_start_date, :event_end_date, :event_type,:need_offline_sync, :event_status, :created_by, :created_date, :last_updated_by, :last_updated_date)");
writer.setDataSource(dataSource2);
return writer;
}
//Postgres Data source for Target
@Bean(name="postgresDataSource")
public DataSource postgresDataSource() throws SQLException {
final SimpleDriverDataSource dataSource = new SimpleDriverDataSource();
dataSource.setDriver(new org.postgresql.Driver());
dataSource.setUrl("jdbc:postgresql://localhost:5432/batch");
dataSource.setUsername("batch");
dataSource.setPassword("batch");
return dataSource;
}
//Oracle Data source for source
@Primary
@Bean(name="oracleDBDataSource")
public DataSource oracleDBDataSource() throws SQLException {
final SimpleDriverDataSource dataSource = new SimpleDriverDataSource();
dataSource.setDriver(new oracle.jdbc.driver.OracleDriver()); //(new oracle.jdbc.pool.OracleDataSource());
dataSource.setUrl("jdbc:oracle:thin:@xxxxxxx.com:1521/xxxxx");
dataSource.setUsername("xxxx");
dataSource.setPassword("xxxxx");
return dataSource;
}
@Bean
public JdbcTemplate jdbcTemplate(final DataSource dataSource) {
return new JdbcTemplate(dataSource);
}
@Bean
public JobExecutionListener listener() {
return new JobCompletionNotificationListener(new JdbcTemplate(dataSource2));
}
// end::listener[]
// tag::jobstep[]
@Bean
public Job importUserJob() {
return jobBuilderFactory.get("importUserJob")
.incrementer(new RunIdIncrementer())
.listener(listener())
.flow(step1())
.end()
.build();
}
@Bean
public Step step1() {
return stepBuilderFactory.get("step1")
.<Event, Event> chunk(10)
.reader(reader())
.processor(processor())
.writer(writer())
.build();
}
// end::jobstep[]
}
答案 0 :(得分：0)
你的问题非常广泛，研究工作很少。但是，可以在spring批处理作业中配置数据库读取器和数据库写程序。
您必须配置两个数据源（源数据源是Teradata DB，目标数据源是Postgres DB）。有关如何使用Spring Batch执行此操作的信息，请参阅此问题和答案：Use of multiple DataSources in Spring Batch
然后你必须创建一个包含源数据源的读者和一个包含目标数据源的编写器。
根据您的具体情况稍微调整标准的Spring Batch Boot教程：
@Configuration
@EnableBatchProcessing
public class BatchConfiguration {
@Autowired
public JobBuilderFactory jobBuilderFactory;
@Autowired
public StepBuilderFactory stepBuilderFactory;
@Autowired
@Qualifier("source")
public DataSource source;
@Autowired
@Qualifier("target")
public DataSource target;
@Bean
public ItemReader<Person> reader() {
JdbcCursorItemReader<Person> reader = new JdbcCursorItemReader<Person>();
String sql = "select * from Person";
reader.setSql(sql);
reader.setDataSource(source);
reader.setRowMapper(rowMapper());
return reader;
}
@Bean
public JdbcBatchItemWriter<Person> writer() {
JdbcBatchItemWriter<Person> writer = new JdbcBatchItemWriter<Person>();
writer.setItemSqlParameterSourceProvider(new BeanPropertyItemSqlParameterSourceProvider<Person>());
writer.setSql("INSERT INTO people (first_name, last_name) VALUES (:firstName, :lastName)");
writer.setDataSource(target);
return writer;
}
@Bean
public Job yourJob() {
return jobBuilderFactory.get("yourJob")
.incrementer(new RunIdIncrementer())
.flow(step1())
.end()
.build();
}
@Bean
public Step step1() {
return stepBuilderFactory.get("step1")
.<Person, Person> chunk(10)
.reader(reader())
.writer(writer())
.build();
}
}
对于此示例的其余配置，请结帐https://spring.io/guides/gs/batch-processing/，有关详细说明，请查看本教程http://www.codingpedia.org/ama/spring-batch-tutorial-with-spring-boot-and-java-configuration/（您也可以在那里找到RowMapper的示例）