Question

我使用多个线程将数百万条记录插入数据库。我正在使用3个班级。

的getConnection ：

import java.sql.Connection;
import java.sql.SQLException;
import org.apache.commons.dbcp2.BasicDataSource;
import LoadConfig;
public class DBCP {
    private static final org.apache.log4j.Logger log = org.apache.log4j.Logger.getLogger("DBCP");
    private Connection conn;
    public static Connection getOracleConnection() throws SQLException,
         ClassNotFoundException {
     String hostName = LoadConfig.Getconfig("hostName");
     String sid = LoadConfig.Getconfig("sid");
     String userName = LoadConfig.Getconfig("userName");
     String password = LoadConfig.Getconfig("password");

     return getOracleConnection(hostName, sid, userName, password);
 }

    private static Connection getOracleConnection(String hostName, String sid, String userName, String password) 
            throws SQLException {
        BasicDataSource dataSource = new BasicDataSource();
        try 
        {
            dataSource.setDriverClassName("oracle.jdbc.driver.OracleDriver");
            dataSource.setUrl("jdbc:oracle:thin:@" + hostName + ":1521:" + sid);
            dataSource.setUsername(userName);
            dataSource.setPassword(password);
            dataSource.setMaxTotal(300);
            return dataSource.getConnection();
        } catch (SQLException ex) {
            log.error("SQLException"+ex);
            return null;
        }
    }
}

阅读文字：

    public class Readtxt implements Runnable{
    private static org.apache.log4j.Logger log = org.apache.log4j.Logger.getLogger("ReadFile");
    File file = new File(path);
    ExecutorService executor = Executors.newFixedThreadPool(100);


    @Override
    public void run() {
        try {
          Connection conn = data.getOracleConnection() 
            try (BufferedReader br = new BufferedReader(new FileReader(file.getAbsolutePath()))) {
                String line;
                int i=0;
                log.info("Start read and update file: " + fileName);
                while ((line = br.readLine()) != null) {
                    Runnable Insert = new InsertDT(conn, line);
                    executor.execute(Insert);
                }
                log.info("Finish reading: " + fileName);
            } catch (IOException ex) {
                log.fatal("ReadFile IOException: " + ex);

        } catch (Exception ex) {
            log.fatal("Connect Error: " + ex);
        } 
    }
}

和插入课程：

public class InsertDT implements Runnable{
    private static org.apache.log4j.Logger log = org.apache.log4j.Logger.getLogger("InsertDT");
    private String line
    Connection conn = null;
    CallableStatement statement;
    public InsertDT(Connection _conn, String _line){
            this.conn = _conn;
            this.line = _line;
    }
    @Override
    public void run() {
        try {
            try {
                this.statement = conn.prepareCall("{ Call Produce  (?,?)}");
                statement.setString("Data", line);
                statement.executeUpdate();
            } catch (Exception ex) {
            } 
        } catch (SQLException | ClassNotFoundException ex) {
            log.error(ex);
        }
    }

}

但该项目需要花费大量时间将记录插入数据库。有没有人知道如何解决这个问题？

Answer 1

首先，您要构建datasource的每个读取线程。

Connection conn = data.getOracleConnection()

拥有单例数据源是件好事。您将在应用程序启动期间构建，并在每次需要时从Factory实施中使用它。

其次，您与[{1}}分享了相同的connection。

datasource

最好为每个Runnable Insert = new InsertDT(conn, line);分配connection。因此，每次您引用相同的thread并获取新的datasource并将其提供给每个线程以使用它。

更重要的是，你的线程池有多大以及你分配了多少并发线程并不重要;重要的是connection。

如何在多线程中使用连接池？

1 个答案: