我需要分析mysql表中的条目并根据各自的结果进行更新。
为此，我编写了一个Java程序，该程序将遍历ID列表，查询数据库，分析返回的行，然后在文件中写入Update sql命令。
public static void main(String[] args) {
Path inputFile, outFile;
MysqlDataSource dataSource = new MysqlDataSource();
dataSource.setPassword(pass);
dataSource.setUser(user);
dataSource.setServerName(server);
dataSource.setDatabaseName(db);
try {
dataSource.setServerTimezone("UTC");
} catch (SQLException e) {
System.out.println("Error while setting timezone of server");
System.out.println(e.getMessage());
return;
}
try {
dataSource.setAutoReconnect(true);
dataSource.setAutoReconnectForPools(true);
} catch (SQLException e) {
System.out.println("Error while setting auoreconnect");
System.out.println(e.getMessage());
return;
}
List<String> IDList = readList(inputFile);
try {
PrintWriter pw = new PrintWriter(Files.newBufferedWriter(outFile));
for(int i = 0; i < IDList.size(); i++) {
try {
findConsequences(IDList.get(i), dataSource, pw);
} catch (IOException | SQLException e) {
System.out.println("Error while executing query (find consequences for ID " + IDList.get(i) + ")");
e.printStackTrace();
System.out.println(e.getMessage());
return;
}
}
pw.close();
}
catch(IOException e) {
System.out.println("Error while writing to file");
System.out.println(e.getMessage());
return;
}
}
private static void findConsequences(String ID, MysqlDataSource dataSource, PrintWriter pw) throws IOException, SQLException{
List<Result> resultList = new ArrayList<>();
String query = "Select * from table_bla where Nr = '" + ID + "';";
Connection conn;
Statement stmt;
ResultSet rs;
ResultSetMetaData rsmd;
conn = dataSource.getConnection();
stmt = conn.createStatement();
rs = stmt.executeQuery(query);
rsmd = rs.getMetaData();
int columnsNumber = rsmd.getColumnCount();
String[] tmpArray = new String[columnsNumber];
while (rs.next()) {
for (int i = 1; i <= columnsNumber; i++) {
tmpArray[i-1] = rs.getString(i) == null ? "" : rs.getString(i);
}
resultList.add(new Result(tmpArray));
}
rs.close();
stmt.close();
conn.close();
//Analyze ....
pw.println("Update table_bla set ...");
}
这适用于大约45000个ID。然后程序崩溃并出现以下异常
Error while executing query (find consequences for ID 1057599046)
java.sql.SQLNonTransientConnectionException: Could not create connection to database server. Attempted reconnect 3 times. Giving up.
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:110)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:89)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:63)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:73)
at com.mysql.cj.jdbc.ConnectionImpl.connectWithRetries(ConnectionImpl.java:905)
at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:830)
at com.mysql.cj.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:455)
at com.mysql.cj.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:240)
at com.mysql.cj.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:199)
at com.mysql.cj.jdbc.MysqlDataSource.getConnection(MysqlDataSource.java:402)
at com.mysql.cj.jdbc.MysqlDataSource.getConnection(MysqlDataSource.java:125)
at com.mysql.cj.jdbc.MysqlDataSource.getConnection(MysqlDataSource.java:110)
at project.AddConsequences.findConsequences(AddConsequences.java:142)
at project.AddConsequences.main(AddConsequences.java:83)
Caused by: com.mysql.cj.exceptions.CJCommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:61)
at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:105)
at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:151)
at com.mysql.cj.exceptions.ExceptionFactory.createCommunicationsException(ExceptionFactory.java:167)
at com.mysql.cj.protocol.a.NativeSocketConnection.connect(NativeSocketConnection.java:91)
at com.mysql.cj.NativeSession.connect(NativeSession.java:152)
at com.mysql.cj.jdbc.ConnectionImpl.connectWithRetries(ConnectionImpl.java:849)
... 9 more
Caused by: java.net.NoRouteToHostException: Die angeforderte Adresse kann nicht zugewiesen werden (Address not available)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at com.mysql.cj.protocol.StandardSocketFactory.connect(StandardSocketFactory.java:155)
at com.mysql.cj.protocol.a.NativeSocketConnection.connect(NativeSocketConnection.java:65)
... 11 more
Could not create connection to database server. Attempted reconnect 3 times. Giving up.
在我不明白为什么它突然无法创建连接之前，考虑到它适用于所有ID。如果我更改代码，使它以ID崩溃之前不久的列表开始，它将再次执行约40000-50000项并因相同错误而崩溃。
对于根本无法建立连接的情况，仅搜索粗略结果，而此处不是这种情况。
操作系统：Ubuntu 16.04
MySQL：5.7.26
JDBC驱动程序：8.0.16
将Connection和Statement对象移到函数外部之后，在获得大约相同数量的ID之后，发生以下错误
Error while executing query (find consequences for ID 1057626409)
com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
at com.mysql.cj.jdbc.exceptions.SQLError.createCommunicationsException(SQLError.java:174)
at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:64)
at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:835)
at com.mysql.cj.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:455)
at com.mysql.cj.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:240)
at com.mysql.cj.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:199)
at com.mysql.cj.jdbc.MysqlDataSource.getConnection(MysqlDataSource.java:402)
at com.mysql.cj.jdbc.MysqlDataSource.getConnection(MysqlDataSource.java:125)
at com.mysql.cj.jdbc.MysqlDataSource.getConnection(MysqlDataSource.java:110)
at project.AddConsequences.main(AddConsequences.java:82)
Caused by: com.mysql.cj.exceptions.CJCommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:61)
at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:105)
at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:151)
at com.mysql.cj.exceptions.ExceptionFactory.createCommunicationsException(ExceptionFactory.java:167)
at com.mysql.cj.protocol.a.NativeProtocol.readMessage(NativeProtocol.java:562)
at com.mysql.cj.protocol.a.NativeProtocol.checkErrorMessage(NativeProtocol.java:732)
at com.mysql.cj.protocol.a.NativeProtocol.checkErrorMessage(NativeProtocol.java:709)
at com.mysql.cj.protocol.a.NativeProtocol.checkErrorMessage(NativeProtocol.java:132)
at com.mysql.cj.protocol.a.NativeAuthenticationProvider.proceedHandshakeWithPluggableAuthentication(NativeAuthenticationProvider.java:540)
at com.mysql.cj.protocol.a.NativeAuthenticationProvider.connect(NativeAuthenticationProvider.java:202)
at com.mysql.cj.protocol.a.NativeProtocol.connect(NativeProtocol.java:1452)
at com.mysql.cj.NativeSession.connect(NativeSession.java:165)
at com.mysql.cj.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:955)
at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:825)
... 7 more
Caused by: java.io.EOFException: Can not read response from server. Expected to read 4 bytes, read 0 bytes before connection was unexpectedly lost.
at com.mysql.cj.protocol.FullReadInputStream.readFully(FullReadInputStream.java:67)
at com.mysql.cj.protocol.a.SimplePacketReader.readHeader(SimplePacketReader.java:63)
at com.mysql.cj.protocol.a.SimplePacketReader.readHeader(SimplePacketReader.java:45)
at com.mysql.cj.protocol.a.NativeProtocol.readMessage(NativeProtocol.java:556)
... 16 more
Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
答案 0 :(得分：0)
问题似乎与我创建的对象数量有关。据我所知，因为我为每个ID创建了一个新的Connection and Statement对象，所以我浪费了太多内存，并且GC必须使用大量CPU运行更长的时间，并停止执行程序。这似乎导致了这些连接错误。
更改我的程序，使其仅使用一个连接/语句后，就可以正常运行。另外，我现在使用c3p0作为池管理器，但这可能不是必需的。
inside fun 1
inside fun 2
inside fun 3
inside fun 4
inside fun 5
5
inside fun 6
inside fun 7
inside fun 8
inside fun 9
inside fun 10
10