如何向Postgresql CopyManager

时间:2015-11-11 10:47:55

标签: java hibernate postgresql session stdin

我有一个使用hibernate管理数据库操作的java web项目。

按顺序执行了两个SQL:

  1. 执行" CREATE TEMP temp_table ......;"通过hibernate本机sql查询。
  2. 执行"从SDTIN"复制temp_table;通过org.postgresql.copy.CopyManager将csv文件传输到temp_table。
  3. 我的代码:

    public boolean loadElectricFishData(Integer fileId, File eventfile, File samplefile) {
        Session session = stagingSessionFactory.getCurrentSession();
        String elecCountTableName = "temp_eb_elec_count_" + fileId;
    
        return createTable(session, elecCountTableName , "nemostaging.eb_elec_count")
                && copyTableFromCSV(session, elecCountTableName, eventfile) ? true : false;
    }
    
    private boolean createTable(Session session, String tableName, String likeTableName) {
        SQLQuery query = session.createSQLQuery("CREATE TABLE nemostaging." + tableName + " (LIKE " + likeTableName + " INCLUDING ALL)");
        query.executeUpdate();
        session.flush();
        session.close();
        return true;
    }
    
    private boolean copyTableFromCSV(Session session, final String tableName, final File csv) {
        SessionImpl sessionImpl = (SessionImpl)session;
        Connection connection = sessionImpl.getTransactionCoordinator().getJdbcCoordinator().getLogicalConnection().getConnection();
        FileReader fileReader = null;
        try {
            CopyManager copyManager = new CopyManager((BaseConnection) connection);
            fileReader = new FileReader(csv);
            copyManager.copyIn("COPY nemostaging." + tableName + " FROM STDIN DELIMITER ',' CSV HEADER", fileReader);
        } catch (SQLException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        } catch (FileNotFoundException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        } catch (IOException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        } finally {
            if (fileReader != null) {
                try {
                    fileReader.close();
                } catch (IOException e) {
                    // TODO Auto-generated catch block
                    e.printStackTrace();
                }
            }
        }
    
        return false;
    }
    

    我的问题是我需要为这两个操作使用相同的会话,因为临时表对另一个会话是不可见的。而CopyManager不会重用相同的会话。任何人都知道如何指示org.postgresql.copy.CopyManager的当前会话,但不仅仅是当前连接?

    非常感谢!

    ------更新-二○一五年十一月一十三日----- 嗨Craig Ringer, 我尝试了你的建议,但有例外:

      

    java.lang.reflect.UndeclaredThrowableException       在com.sun.proxy。$ Proxy20.unwrap(未知来源)       at nz.co.niwa.nemo.uploadservice.core.dao.impl.StagingLoadDataDaoImpl $ 1.execute(StagingLoadDataDaoImpl.java:88)       在org.hibernate.jdbc.WorkExecutor.executeWork(WorkExecutor.java:54)       在org.hibernate.internal.SessionImpl $ 2.accept(SessionImpl.java:1933)       在org.hibernate.internal.SessionImpl $ 2.accept(SessionImpl.java:1930)       at org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.coordinateWork(JdbcCoordinatorImpl.java:211)       在org.hibernate.internal.SessionImpl.doWork(SessionImpl.java:1951)       在org.hibernate.internal.SessionImpl.doWork(SessionImpl.java:1937)       at nz.co.niwa.nemo.uploadservice.core.dao.impl.StagingLoadDataDaoImpl.loadSpotlightData(StagingLoadDataDaoImpl.java:85)       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)       在java.lang.reflect.Method.invoke(Method.java:597)       在org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:319)       在org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)       在org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)       在org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:110)       在org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)       在org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:110)       在org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)       在org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:90)       在org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)       在org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)       在com.sun.proxy。$ Proxy14.loadSpotlightData(未知来源)       at nz.co.niwa.nemo.uploadservice.core.manage.impl.StagingLoadDataManageImpl.loadSpotlightData(StagingLoadDataManageImpl.java:30)       at nz.co.niwa.nemo.uploadservice.core.service.impl.StagingLoadDataServiceImpl.loadSpotlightData(StagingLoadDataServiceImpl.java:44)       at nz.co.niwa.nemo.uploadservice.core.test.service.StagingLoadDataServiceTest.testLoadSpotlightData(StagingLoadDataServiceTest.java:39)       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)       在java.lang.reflect.Method.invoke(Method.java:597)       在org.junit.runners.model.FrameworkMethod $ 1.runReflectiveCall(FrameworkMethod.java:45)       在org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)       在org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)       在org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)       在org.springframework.test.context.junit4.statements.RunBeforeTestMethodCallbacks.evaluate(RunBeforeTestMethodCallbacks.java:74)       在org.springframework.test.context.junit4.statements.RunAfterTestMethodCallbacks.evaluate(RunAfterTestMethodCallbacks.java:83)       在org.springframework.test.context.junit4.statements.SpringRepeat.evaluate(SpringRepeat.java:72)       在org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:231)       在org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)       在org.junit.runners.ParentRunner $ 3.run(ParentRunner.java:231)       在org.junit.runners.ParentRunner $ 1.schedule(ParentRunner.java:60)       在org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)       在org.junit.runners.ParentRunner.access $ 000(ParentRunner.java:50)       在org.junit.runners.ParentRunner $ 2.evaluate(ParentRunner.java:222)       在org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61)       在org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:71)       在org.junit.runners.ParentRunner.run(ParentRunner.java:300)       在org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:174)       在org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)       在org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)       在org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)       在org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:675)       在org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)       在org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)   引起:java.lang.reflect.InvocationTargetException       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)       在java.lang.reflect.Method.invoke(Method.java:597)       在org.hibernate.engine.jdbc.internal.proxy.ConnectionProxyHandler.continueInvocation(ConnectionProxyHandler.java:130)       在org.hibernate.engine.jdbc.internal.proxy.AbstractProxyHandler.invoke(AbstractProxyHandler.java:81)       ......还有55个   引起:java.sql.SQLException:无法解包到org.postgresql.copy.CopyManager       at org.postgresql.jdbc4.AbstractJdbc4Connection.unwrap(AbstractJdbc4Connection.java:269)       ......还有61个

    我的代码:

    public boolean loadSpotlightData(final Integer fileId, final File datafile) {
        final String spotCountTableName = "tmp_eb_spot_count_" + fileId;
        Session session = stagingSessionFactory.getCurrentSession();
        SQLQuery query = session.createSQLQuery("CREATE TEMP TABLE " + spotCountTableName + " (LIKE nemostaging.eb_spot_count INCLUDING ALL)");
        query.executeUpdate();
        session.doWork(new Work() {
            @Override
            public void execute(Connection connection) throws SQLException {
                CopyManager copyManager = connection.unwrap(org.postgresql.copy.CopyManager.class);
                FileReader fileReader = null;
                try {
                    fileReader = new FileReader(datafile);
                    copyManager.copyIn("COPY " + spotCountTableName + " FROM STDIN DELIMITER ',' CSV HEADER", fileReader);
                } catch (SQLException e) {
                    LOG.error("Errors occur while COPY nemostaging." + spotCountTableName + " FROM STDIN DELIMITER ',' CSV HEADER;", e);
                } catch (FileNotFoundException e) {
                    LOG.error("The CSV file cannot be found while copying into database.", e);
                } catch (IOException e) {
                    LOG.error("Errors occur while reading the csv file.", e);
                } finally {
                    if (fileReader != null) {
                        try {
                            fileReader.close();
                        } catch (IOException e) {
                            LOG.error("Errors occur while closing file reader.", e);
                        }
                    }
                }
            }
        });
    
        Query query1 = session.getNamedQuery("staging.loadSpotlightData");
        query1.setParameter("fileId", fileId);
        query1.list();
    
        return true;
    }
    

1 个答案:

答案 0 :(得分:2)

你必须做的是运行Hibernate查询,然后要求Hibernate Session获取实现Connection 的基础org.postgresql.PGConnection对象并使用它。

您曾经能够使用Session.connection()方法解包会话,但现在已弃用。而是use the doWork API。您传递的java.sql.Connection可能也包含在您的连接池中,因此您可能需要解包它以获取实际的连接对象。如果您正在使用JDBC4驱动程序和池(您应该),Wrapper Connection实现了unwrap(org.postgresql.copy.CopyManager.class)接口,例如: session.doWork( new Work() { public void execute(Connection connection) throws SQLException { CopyManager cm = connection.unwrap(org.postgresql.copy.CopyManager.class); ... do stuff ... } } );

像(未经测试)的东西:

DriverManager

您无法直接从bazel run -c opt //tensorflow/models/rnn/translate:translate -- \ --data_dir ./data_dir --train_dir ./checkpoints_directory \ --en_vocab_size=40000 --fr_vocab_size=40000 创建新的单独连接,并将其与其他会话的临时表一起使用。你也不需要。最好从Hibernate会话中解开池连接。