Question

我有一张名为employee的表，有10 000条记录。

我必须使用多线程Java将这些数据写入多个CSV文件，每个文件有2 000条记录。

示例代码如下：

    public class PrintbyThreads {

    public static void main(String[] args) {
        ExecutorService executor = Executors.newFixedThreadPool(5);
        for (int i = 0; i < 5; i++) {
            Runnable worker = new PrintFiles("" + i);
            executor.execute(worker);
          }
        executor.shutdown();
        while (!executor.isTerminated()) {
        }
        System.out.println("Finished all threads");
    }
}

和

public class PrintFiles extends Thread implements Runnable {
    private String command;

    PrintFiles() {
        super("my extending thread");
        System.out.println("my thread created" + this);
        start();
    }

    public PrintFiles(String command) {
        this.command = command;
    }

    public void run() {
        System.out.println(Thread.currentThread().getName()
                + " Start. Command = " + command);
        dataBaseExecution();
        System.out.println(Thread.currentThread().getName() + " End. Command ="
                + command);
    }

    public synchronized void dataBaseExecution() {
        String tableName = "Employee";
        String filename = "D:/db2csv/";
        int recordsAtTime = 2000;
        try {
            Class.forName("oracle.jdbc.driver.OracleDriver");
            Connection con = DriverManager.getConnection(
                    "URL", "Uname", "Password");
            Statement stmt = con.createStatement();
            stmt.setFetchSize(recordsAtTime);
            ResultSet rs = stmt.executeQuery("select empid,empname,managerid from Employee");
            int columnCount = rs.getMetaData().getColumnCount();
            FileWriter fw = new FileWriter(filename + "" + tableName + ".csv");
            for (int i = 1; i <= columnCount; i++) {
                fw.append(rs.getMetaData().getColumnName(i));
                fw.append(",");

            }
            fw.append(System.getProperty("line.separator"));
            while (rs.next()) {
                for (int i = 1; i <= columnCount; i++) {
                    if (rs.getObject(i) != null) {
                        String data = rs.getObject(i).toString();
                        fw.append(data);
                        fw.append(",");
                    } else {
                        String data = "null";
                        fw.append(data);
                        fw.append(",");
                    }

                }
                // new line entered after each row
                fw.append(System.getProperty("line.separator"));
            }

            fw.flush();
            fw.close();
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    @Override
    public String toString() {
        return this.command;
    }
}

我的代码提供了包含所有10000条记录的单个CSV文件。但我需要5个CSV文件，每个文件有2000条记录。

Thread1必须先将2000条记录处理到employee1.csv

线程2必须处理另外2000条记录到employee2.csv

,,,,,等

通过使用我的线程代码，我该如何实现这一要求？

Answer 1

在您的代码中，您可能需要一个分区程序，它将为您提供索引并索引线程需要处理的内容。

公共类MyPartitioner {

public Map partition(int gridSize,int range) {

    Map partitionMap = new HashMap();
    int startingIndex = 0;
    int endingIndex = range;

    for(int i=0; i< gridSize; i++){
      Map threadMap = new HashMap();

        threadMap.putInt("startingIndex",startingIndex);
        threadMap.putInt("endingIndex", endingIndex);

        startingIndex = endingIndex+1;
        endingIndex += range; 

        partitionMap.put("Thread:-"+i, threadMap);
    }

    return partitionMap;
}

}

Executor框架使用index from index和index for worker thread来处理分配的行而不是所有行。在您选择的查询中使用过滤条件

where id >= :fromId and id <= :toId

将表数据写入java中的多个CSV文件

1 个答案: