我有一个电子商务应用程序的Grails项目，它也有数据处理的后台工作。最近，我们收到了一份来自客户的XML文件，该订单大约有300多个订单（文件大约35000多行）。该文件需要大约9个小时才能完成，但客户希望在2-3小时内完成此操作。
最明显的解决方案是使用线程同时处理10-15个订单以减少处理时间。我使用Java执行器框架实现了相同的功能，并创建了一个说3个线程的池。 Grails项目使用hibernate作为ORM。
接下来的过程如下：
1.) Address details are verified and If not present created in DB.
2.) Item details are verified and if not present created in DB.
3.) Order against the saved item is created/updated.
现在，出于测试目的，将3个订单提交到池中的3个线程。到达ITem创建的第一个线程（步骤2）锁定项目对象，其他两个线程在该点继续等待。一旦第一个线程完成所有操作，其他两个线程就会失败并出现以下异常。
org.springframework.dao.CannotAcquireLockException: could not insert: [com.test.ABCITem]; SQL [insert into party_item_relations (version, active_flag, amazonasin, b2b_downloaded, baseuom, bom, card_size, case_size, colour, configured, cost, country_of_origin, currency, date_created, drop_ship, end_date, external_id, flex_field1, flex_field10, flex_field2, flex_field3, flex_field4, flex_field5, flex_field6, flex_field7, flex_field8, flex_field9, gender, gtin_code, height, heightuom, image_file, ingredients, internal_item, item_category, item_description, kanban_enabled, key_terms, last_updated, lead_time, length, lengthuom, long_description, manufacturer, model, moq, number_of_cards, over_ship_percent, pack_size, parent_item_category, parentitem_id, party1_id, party1item_number, party2_id, party2item_number, planner_code, price, product_family, product_group, product_type_name, sales_channel, scent, size, source_rank, start_date, status, tam_split, upc_code, variation_theme, weight, weightuom, width, widthuom) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)]; nested exception is org.hibernate.exception.LockAcquisitionException: could not insert: [com.test.ABCITem]
at org.springframework.orm.hibernate3.SessionFactoryUtils.convertHibernateAccessException(SessionFactoryUtils.java:639)
at org.springframework.orm.hibernate3.HibernateAccessor.convertHibernateAccessException(HibernateAccessor.java:412)
at org.springframework.orm.hibernate3.HibernateTemplate.doExecute(HibernateTemplate.java:411)
at org.springframework.orm.hibernate3.HibernateTemplate.execute(HibernateTemplate.java:339)
at org.codehaus.groovy.grails.orm.hibernate.metaclass.SavePersistentMethod.performSave(SavePersistentMethod.java:56)
at org.codehaus.groovy.grails.orm.hibernate.metaclass.AbstractSavePersistentMethod.doInvokeInternal(AbstractSavePersistentMethod.java:215)
at org.codehaus.groovy.grails.orm.hibernate.metaclass.AbstractDynamicPersistentMethod.invoke(AbstractDynamicPersistentMethod.java:63)
at sun.reflect.GeneratedMethodAccessor425.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1259)
at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoCachedMethodSite.invoke(PojoMetaMethodSite.java:189)
at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:53)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:124)
at org.codehaus.groovy.grails.orm.hibernate.HibernateGormInstanceApi.save(HibernateGormEnhancer.groovy:911)
at com.test.ABCITem.save(ABCITem.groovy)
at com.test.ABCITem$save$0.call(Unknown Source)
at com.test.MasterDataService.createPartyItemRelation(MasterDataService.groovy:259)
at grails.plugin.executor.PersistenceContextRunnableWrapper.run(PersistenceContextRunnableWrapper.groovy:34)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.hibernate.exception.LockAcquisitionException: could not insert: [com.test.ABCITem]
at org.hibernate.exception.SQLStateConverter.convert(SQLStateConverter.java:107)
at org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:66)
at org.hibernate.id.insert.AbstractReturningDelegate.performInsert(AbstractReturningDelegate.java:63)
at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:2346)
at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:2853)
at org.hibernate.action.EntityIdentityInsertAction.execute(EntityIdentityInsertAction.java:71)
at org.hibernate.engine.ActionQueue.execute(ActionQueue.java:273)
at org.codehaus.groovy.grails.orm.hibernate.support.ClosureEventTriggeringInterceptor.performSaveOrReplicate(ClosureEventTriggeringInterceptor.java:250)
at org.hibernate.event.def.AbstractSaveEventListener.performSave(AbstractSaveEventListener.java:203)
at org.hibernate.event.def.AbstractSaveEventListener.saveWithGeneratedId(AbstractSaveEventListener.java:129)
at org.hibernate.event.def.DefaultSaveOrUpdateEventListener.saveWithGeneratedOrRequestedId(DefaultSaveOrUpdateEventListener.java:210)
at org.hibernate.event.def.DefaultSaveOrUpdateEventListener.entityIsTransient(DefaultSaveOrUpdateEventListener.java:195)
at org.hibernate.event.def.DefaultSaveOrUpdateEventListener.performSaveOrUpdate(DefaultSaveOrUpdateEventListener.java:117)
at org.hibernate.event.def.DefaultSaveOrUpdateEventListener.onSaveOrUpdate(DefaultSaveOrUpdateEventListener.java:93)
at org.codehaus.groovy.grails.orm.hibernate.support.ClosureEventTriggeringInterceptor.onSaveOrUpdate(ClosureEventTriggeringInterceptor.java:108)
at org.hibernate.impl.SessionImpl.fireSaveOrUpdate(SessionImpl.java:685)
at org.hibernate.impl.SessionImpl.saveOrUpdate(SessionImpl.java:677)
at org.hibernate.impl.SessionImpl.saveOrUpdate(SessionImpl.java:673)
at org.codehaus.groovy.grails.orm.hibernate.metaclass.SavePersistentMethod$1.doInHibernate(SavePersistentMethod.java:58)
at org.springframework.orm.hibernate3.HibernateTemplate.doExecute(HibernateTemplate.java:406)
... 110 more
Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLTransactionRollbackException: Deadlock found when trying to get lock; try restarting transaction
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.springsource.loaded.ri.ReflectiveInterceptor.jlrConstructorNewInstance(ReflectiveInterceptor.java:991)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
at com.mysql.jdbc.Util.getInstance(Util.java:386)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1064)
我已经查看了此错误的各种解决方案，但主要建议对where子句字段进行排序。其他解决方案，如增加innodb引擎的超时时间也没有用。
我的项目结构如下所示。（这是伪代码而不是精确的类）
1。）Runnable实现将每个订单作为新线程执行。
public class BackGroundTask implements Runnable{
private List<JSONObject> requests;
public BackGroundTask(final List<JSONObject> requests){
this.requests = Lists.newArrayList(requests);
}
@Override
public void run() {
this.dataUtilsService.processData(this.requests);
}
}
2。）第三方服务，它接收XML文件并从执行程序池为每个订单启动新线程。
public class ThirdPartyService{
def executorService
public void process(File file){
Document doc = xmlParser.parse(file);
NodeList orders = ordersData.getElementsByTagName(TAG_ORDER)
for (Element order : orders) {
//order converted to List<jsonobject> and passed below.
executorService.submit(new backGroundTask(requests));
}
}
}
3.）实际处理数据的处理器服务结构。
public class DataUtilsService{
@Transactional(propagation=PropagationType.REQUIRES_NEW)
public void processData(List<JSONObject> requests){
//Loops through requests and processes data.
item.save();
}
}
请建议在此处使用除线程以外的其他内容进行并发处理，以减少处理时间。