Question

我正在尝试读取包含由换行符分隔的158000条记录（12MB）的文件并将该记录放入mysql数据库但是在插入大约49000条记录之后我的java程序抛出异常

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
    at java.util.jar.Manifest$FastInputStream.<init>(Manifest.java:315)
    at java.util.jar.Manifest$FastInputStream.<init>(Manifest.java:310)
    at java.util.jar.Manifest.read(Manifest.java:178)
    at java.util.jar.Manifest.<init>(Manifest.java:52)
    at java.util.jar.JarFile.getManifestFromReference(JarFile.java:167)
    at java.util.jar.JarFile.getManifest(JarFile.java:148)
    at sun.misc.URLClassPath$JarLoader$2.getManifest(URLClassPath.java:696) at sun.misc.URLClassPath$JarLoader$2.getManifest(URLClassPath.java:696)

    at java.net.URLClassLoader.defineClass(URLClassLoader.java:228)
    at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
    at com.mysql.jdbc.Util.handleNewInstance(Util.java:430) at com.mysql.jdbc.Util.handleNewInstance(Util.java:430)

    at com.mysql.jdbc.PreparedStatement.getInstance(PreparedStatement.java:553)
    at com.mysql.jdbc.ConnectionImpl.clientPrepareStatement(ConnectionImpl.java:1378)
    at com.mysql.jdbc.ConnectionImpl.prepareStatement(ConnectionImpl.java:4143)
    at com.mysql.jdbc.ConnectionImpl.prepareStatement(ConnectionImpl.java:4042)
    at loadcsvdatabase.LoadCsvDatabase.fnLoadCsvIntoMemory(LoadCsvDatabase.java:52)
    at loadcsvdatabase.LoadCsvDatabase.main(LoadCsvDatabase.java:29)

插入数据的Java代码

FileInputStream file;
        DataInputStream dataIn;
        BufferedReader bReader;
        Connection conn= openConnection();
        String strRecord=new String();
        String[]strSplittedRecord;
        file= new FileInputStream("D:\\Java\\CountryWhois.txt");
        dataIn= new DataInputStream(file);
        bReader= new BufferedReader(new InputStreamReader(dataIn));
        PreparedStatement insertStmt;
        String strQuery="insert into countrywhois values(?,?,?,?,?,?)";
        while((strRecord=bReader.readLine())!=null)
        {

            strSplittedRecord=strRecord.split(",");

         try {

             insertStmt=conn.prepareStatement(strQuery);
             insertStmt.setString(1,strSplittedRecord[0].substring(1, strSplittedRecord[0].length()-1));
             insertStmt.setString(2,strSplittedRecord[1].substring(1, strSplittedRecord[1].length()-1));
             insertStmt.setString(3,strSplittedRecord[2].substring(1, strSplittedRecord[2].length()-1));
             insertStmt.setString(4,strSplittedRecord[3].substring(1, strSplittedRecord[3].length()-1));
             insertStmt.setString(5,strSplittedRecord[4].substring(1, strSplittedRecord[4].length()-1));
             insertStmt.setString(6,strSplittedRecord[5].substring(1, strSplittedRecord[5].length()-1));
             int nResultInsert=insertStmt.executeUpdate();
             if(nResultInsert!=0)
             {
                System.out.println("Inserted 1");


             }
             else
             {
                if(conn!=null)
                {
                    conn.close();
                    System.out.println("Disconnected from database");
                }
             }
         }
         catch (Exception e) 
         {
             conn.close();
             e.printStackTrace(System.out);        
         }

        }

它在行 insertStmt = conn.prepareStatement（strQuery）;

中引发异常

请建议我为什么我的程序内存不足...

Answer 1

每次插入记录时都要创建一个新的PreparedStatement。这不是PreparedStatements应该如何使用的方式。此外，您永远不会关闭创建的PreparedStatements，这可能会导致OutOfMemoryException，但这不是根本问题。

您必须只创建一个PreparedStatement并重复使用它。然后，在插入所有记录后，关闭PreparedStatement和Connection。

Answer 2

insertStmt=conn.prepareStatement(strQuery);

应该在while循环之外。在初始化strQuery

之后准确地说出来

Answer 3

您应该按照这种方式修改代码：

...

try {
  Connection conn = openConnection();
  try {

    String strQuery="insert into countrywhois values(?,?,?,?,?,?)";
    PreparedStatement insertStmt=conn.prepareStatement(strQuery);
    while (...) {
        try {
            ....
            int nResultInsert=insertStmt.executeUpdate();

            // handle your result here (e.g. print/log)
            // DO NOT close() here, continue with loop

        } catch (Exception e) {
            // handle your exception here (e.g. print/log)
            // DO NOT close() here, continue with loop, OR break loop
        }
    }

  } finally {
      // closing SQL Connection no matter what happens to your while-loop
      conn.close();
  }
} catch (Exception e) {
    // handle your exception here (e.g. print/log)
}

Error Occuring :: java.lang.OutOfMemoryError：Java堆空间

3 个答案: