Question

我正在构建一个使用Spring @EnableAsync和@Async批注的多线程Spring Boot应用程序。当我使用单线程（CorePoolSize 1，MaxPoolSize 1）运行应用程序时，一切都会按预期进行。当我在似乎是随机发生的事件上将池大小增加到大于1时，得到了 java.sql.SQLNonTransientException：[Amazon] JDBC并非所有参数都已填充。调用Amazon AWS时出错Redshift数据库。

在ServiceProcessBean.java中，我自动连接了ProcessService类（要完成的线程工作）和ShipmentDAO，该类加载了要由ProcessService.process（）方法处理的货运ID列表，如下代码。 >

@Component
public class ShipmentBatchBean {

  private Logger logger = LoggerFactory.getLogger(this.getClass());

  @Autowired
  private ShipmentDAO shipmentDAO;

  @Autowired
  private ProcessService processService;

  @Scheduled(
    initialDelayString = "${executor.delay.initial}",
    fixedDelayString = "${executor.delay.fixed}"
  )
  public void cronJob() throws InterruptedException, ExecutionException {

    List<CompletableFuture<Boolean>> asyncResponse = new ArrayList<>();

    logger.info("Starting cronJob() method");

    try {
      List<String> shipments = shipmentDAO.getAllShipmentsReadyForIeta();
      logger.info("There are {} shipments to be processed in the data store", 
          shipments.size());

      for(String shipment : shipments) {
        asyncResponse.add(processService.process(shipment));
      }
    } catch (Exception ex) {
      logger.error(ex.getMessage());
      ex.printStackTrace();
    }

    CompletableFuture.allOf(asyncResponse.toArray(
      new CompletableFuture[asyncResponse.size()]
    )).join();

    logger.info("Ending cronJob() method");
  }

}

最后，在ProcessService中，我们自动关联多个存储库和JSON服务，并使用@Async批注开始process（）方法。请参见下面的代码段。

@Service
public class ProcessServiceBean implements ProcessService {

    private Logger logger = LoggerFactory.getLogger(getClass());

    @Autowired
    private ShipmentDAO shipmentDAO;

    @Autowired
    private OssItemDAO ossItemDAO;

    @Autowired
    private OssHeaderDAO ossHeaderDAO;

    @Autowired
    private OssDataJsonServiceBean ossDataJsonServiceBean;

    @Override
    @Async
    public CompletableFuture<Boolean> process(String shipmentId) {

      Shipment shipment = null;

      logger.debug("Retrieving from ieta_input (shipment id {})",
        shipmentId);
      try {
        shipment = shipmentDAO.getOneBy(shipmentId);
      } catch (SQLException e) {
        logger.error("process of shipment {}) ended in error",
          shipmentId,
      };
      return CompletableFuture.completedFuture(false);
    }

 code snipped for brevity.

最后，在ShipmentDAO中，我们具有getOneBy（）方法，该方法返回请求的发货记录。

@Override
public Shipment getOneBy(String shipmentId) throws SQLException {
    Shipment shipment = null;
    Connection conn = null;

    String sql = "SELECT * FROM myschema.tablename WHERE shipmentid = ? LIMIT 1";

    try {
      conn = dataSource.getConnection();
      PreparedStatement ps = conn.prepareStatement(sql);
      ps.setString(1, shipmentId);
      ResultSet rs = ps.executeQuery();
      if(rs.next()) {
        shipment = new Shipment(
            rs.getLong("rowid"),
            rs.getString("shipmentid"),
            ...
            rs.getString("more_info_here")
        );
      }
      ps.close();
    } catch(SQLException e) {
      logger.error("ShipmentDAO.getOneBy() failed: {}",
        e.getMessage());
      e.printStackTrace();
      throw e;
    } finally {
      if(conn != null) {
      try {
        conn.close();
      } catch (SQLException se) { }
    }
  }
  return shipment;
}

我已经联系了AWS，他们在知识库中只有两个实例，在该实例中已报告此错误，只有“用户代码问题”是解决方案。

我看不到任何明显的线程安全问题，但它的行为类似，因为只有一个线程被发射并且每批装运都按顺序处理时，代码才能完美运行。

有人看到明显的问题或需要更多信息吗？任何帮助表示赞赏。

堆栈跟踪如下：

2018-Oct-22 15:57:30.960 ERROR- [shipment-executor-8] [Amazon][JDBC](10900) Not all parameters have been populated.
org.springframework.dao.InvalidDataAccessResourceUsageException: could not extract ResultSet; SQL [n/a]; nested exception is org.hibernate.exception.SQLGrammarException: could not extract ResultSet
at org.springframework.orm.jpa.vendor.HibernateJpaDialect.convertHibernateAccessException(HibernateJpaDialect.java:261)
at org.springframework.orm.jpa.vendor.HibernateJpaDialect.translateExceptionIfPossible(HibernateJpaDialect.java:244)
at org.springframework.orm.jpa.AbstractEntityManagerFactoryBean.translateExceptionIfPossible(AbstractEntityManagerFactoryBean.java:503)
at org.springframework.dao.support.ChainedPersistenceExceptionTranslator.translateExceptionIfPossible(ChainedPersistenceExceptionTranslator.java:59)
at org.springframework.dao.support.DataAccessUtils.translateIfNecessary(DataAccessUtils.java:209)
at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:147)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.data.jpa.repository.support.CrudMethodMetadataPostProcessor$CrudMethodMetadataPopulatingMethodInterceptor.invoke(CrudMethodMetadataPostProcessor.java:133)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.data.repository.core.support.SurroundingTransactionDetectorMethodInterceptor.invoke(SurroundingTransactionDetectorMethodInterceptor.java:57)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213)
at com.sun.proxy.$Proxy91.getFirstByFactDeliveryNumberIsOrSapDeliveryNumberIs(Unknown Source)
at com.accenture.service.ProcessServiceBean.process(ProcessServiceBean.java:91)
at com.accenture.service.ProcessServiceBean$$FastClassBySpringCGLIB$$45e1a1ec.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:282)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.interceptor.AsyncExecutionInterceptor$1.call(AsyncExecutionInterceptor.java:115)
at org.springframework.aop.interceptor.AsyncExecutionAspectSupport$CompletableFutureDelegate$1.get(AsyncExecutionAspectSupport.java:328)
at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.hibernate.exception.SQLGrammarException: could not extract ResultSet
at org.hibernate.exception.internal.SQLStateConversionDelegate.convert(SQLStateConversionDelegate.java:106)
at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:42)
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:109)
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:95)
at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.extract(ResultSetReturnImpl.java:79)
at org.hibernate.loader.Loader.getResultSet(Loader.java:2117)
at org.hibernate.loader.Loader.executeQueryStatement(Loader.java:1900)
at org.hibernate.loader.Loader.executeQueryStatement(Loader.java:1876)
at org.hibernate.loader.Loader.doQuery(Loader.java:919)
at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:336)
at org.hibernate.loader.Loader.doList(Loader.java:2617)
at org.hibernate.loader.Loader.doList(Loader.java:2600)
at org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2429)
at org.hibernate.loader.Loader.list(Loader.java:2424)
at org.hibernate.loader.hql.QueryLoader.list(QueryLoader.java:501)
at org.hibernate.hql.internal.ast.QueryTranslatorImpl.list(QueryTranslatorImpl.java:371)
at org.hibernate.engine.query.spi.HQLQueryPlan.performList(HQLQueryPlan.java:216)
at org.hibernate.internal.SessionImpl.list(SessionImpl.java:1326)
at org.hibernate.internal.QueryImpl.list(QueryImpl.java:87)
at org.hibernate.jpa.internal.QueryImpl.list(QueryImpl.java:606)
at org.hibernate.jpa.internal.QueryImpl.getSingleResult(QueryImpl.java:529)
at org.hibernate.jpa.criteria.compile.CriteriaQueryTypeQueryAdapter.getSingleResult(CriteriaQueryTypeQueryAdapter.java:54)
at org.springframework.data.jpa.repository.query.JpaQueryExecution$SingleEntityExecution.doExecute(JpaQueryExecution.java:208)
at org.springframework.data.jpa.repository.query.JpaQueryExecution.execute(JpaQueryExecution.java:87)
at org.springframework.data.jpa.repository.query.AbstractJpaQuery.doExecute(AbstractJpaQuery.java:116)
at org.springframework.data.jpa.repository.query.AbstractJpaQuery.execute(AbstractJpaQuery.java:106)
at org.springframework.data.repository.core.support.RepositoryFactorySupport$QueryExecutorMethodInterceptor.doInvoke(RepositoryFactorySupport.java:499)
at org.springframework.data.repository.core.support.RepositoryFactorySupport$QueryExecutorMethodInterceptor.invoke(RepositoryFactorySupport.java:477)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.data.projection.DefaultMethodInvokingMethodInterceptor.invoke(DefaultMethodInvokingMethodInterceptor.java:56)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:282)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:136)
... 24 more
Caused by: java.sql.SQLNonTransientException: [Amazon][JDBC](10900) Not all parameters have been populated.
at com.amazon.exceptions.ExceptionConverter.toSQLException(Unknown Source)
at com.amazon.jdbc.common.SPreparedStatement.executeQuery(Unknown Source)
at sun.reflect.GeneratedMethodAccessor79.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.tomcat.jdbc.pool.StatementFacade$StatementProxy.invoke(StatementFacade.java:114)
at com.sun.proxy.$Proxy101.executeQuery(Unknown Source)
at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.extract(ResultSetReturnImpl.java:70)
... 55 more

Answer 1

我们终于有了解决方案！经过一系列艰苦的测试后，它归结为JDBC驱动程序不是线程安全的。我们在Redshift-jdbc41和redshift-jdbc42驱动程序的多个版本中都遇到了这个问题，最后解决了一个<version>1.2.10.1009</version>不再抛出错误并同时处理多个线程的问题。如果您有使用Amazon支持的机票，可以将其引至案例ID 5466870321 ，我们在其中提供了所有必要的详细信息，使他们可以确认问题出在驱动程序本身。

正如code tachyon所指出的（感谢您的输入），问题似乎出在JDBC驱动程序的预准备语句功能上，因此我们没有看到数据库本身因这些问题而导致数据损坏因为它们在请求转发到Redshift之前一直存在。

最后一点，令我惊讶的是，亚马逊发布了相当多的JDBC驱动程序版本，而我们在1.2.10.1009之后测试的所有内容都包含此线程安全问题，但Amazon对此并不了解，并且除了这篇文章外，没有一个堆栈溢出或其他Google搜索显示这是一个问题。

我希望这可以帮助发现其多线程应用程序无法正常工作的其他人。

Answer 2

遇到类似问题，cuplrit是在使用命名参数的同时在服务器端准备语句缓存的。准备好的语句和命名的参数是分开发送的，并且redshift jdbc驱动程序无法为同一查询解析跨多个并发连接发送的命名的参数。临时解决方法是替换“？”在“字符串sql =” SELECT * FROM myschema.tablename中，哪里的transportid =？ LIMIT 1“;带有实际参数。这当然使您容易受到sql注入攻击，并且您会失去带有执行计划的已缓存准备好的语句的好处。但是，我们决定采用该选项，直到永久性修复程序YMMV。

java.sql.SQLNonTransientException：[Amazon] [JDBC]（10900）并非所有参数都已填充

2 个答案: