我的问题是这样的：如何读取一个巨大的（数百万行）文件，即使在完成文件填充之后仍保持线程活着
问题是我有一个从javafx应用程序线程启动的线程，然后（新线程）对文本文件执行一些读/写操作，在面对要解析的巨大文件时不会自然退出通过，特别是1700万行。
我认为这是由于线程保留了我缺少的某些资源，但是，因为我使用了try-with-resource模型，所以我是不确定这是怎么回事。
这里是引发线程的javafx控制器类（使用fxml）：
MainController.java：
/**
* This method handles what happens when the calculate button is clicked.
* The main thing this does is disable/enable a few Nodes, as well as sparks
* off the background thread.
*
* @param event
*/
@FXML
private void convert_button_action(ActionEvent event) {
closing_label.setVisible(true);
convert_button.setDisable(true);
input_text = input_NCLocation_field.getText();
output_text = output_Location_Field.getText();
indicator_node.setVisible(true);
if (!toggleSwitch.isSelected()) {
(new Thread(new FileWriter(input_text, output_text, indicator_node))).start();
} else {
DateWriter temp = new DateWriter(input_text, output_text, indicator_node, yr_mn_dy.isSelected());
(new Thread(temp)).start();
}
}
没有什么太花哨的东西，只是一些使事物可见/不可见，并根据用户的输入启动适当的线程。接下来是整个Thread类，因为它不是太大。所有这一切确实是要么转换一行看起来像：yearmonthday进入年，月，日，或者如果用户点击了要求它的复选框，它将年月和日列分隔成单独的文件。只是一个用例的便利工具。
请注意run()方法末尾的println语句。我每次都会看到这个println，但在它发生之后，没有任何反应。程序没有退出，线程不会停止，没有。
package File_Conversion;
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.File;
import java.io.FileReader;
import java.io.IOException;
import java.text.DecimalFormat;
import java.util.ArrayList;
import javafx.application.Platform;
import javafx.scene.control.ProgressIndicator;
/**
* This class is the background 'worker' thread that does all of the heavy duty
* file IO for splitting up the NC file. It periodically sends reports back to
* the main application thread to update the progress indicator.
*
* @author William
*/
public class DateWriter implements Runnable {
private final ProgressIndicator myIndicator;
private static File ncFile;
private final String outputLocationFile;
private float zmax, zmin, xmax, xmin, ymax, ymin;
private ArrayList<Float> xList, yList, zList;
private final DecimalFormat numberFormat = new DecimalFormat("#.000000");
private final DecimalFormat numberFormatMinMax = new DecimalFormat("#.00000");
private final boolean yr_mon_day;
/**
* This is the main constructor, it needs a valid NC file to continue.
*
* @param inputNCFile
* @param outputLocation
* @param myIndicator
* @param yr_mon_day
*/
public DateWriter(String inputNCFile, String outputLocation, ProgressIndicator myIndicator, boolean yr_mon_day) {
this.yr_mon_day = yr_mon_day;
this.myIndicator = myIndicator;
ncFile = new File(inputNCFile);
outputLocationFile = outputLocation;
}
/**
* The primary run() method, starts the thread.
*/
@Override
public void run() {
convertDate();
Platform.runLater(new Runnable() {
@Override
public void run() {
File_Conversion.stage_returner().close();
}
});
System.out.println("I'm at the end of the run...??");
}
public boolean convertDate() {
BufferedReader br = null;
java.io.FileWriter yearWriter = null, MonthWriter = null, DayWriter = null
,fWriter = null;
BufferedWriter yearBuf = null, monthBuf = null, dayBuf = null, writer = null;
try {
br = new BufferedReader(new FileReader(ncFile));
if (yr_mon_day) {
yearWriter = new java.io.FileWriter(outputLocationFile + "\\" + ncFile.getName().substring(0, ncFile.getName().lastIndexOf(".")) + "_modified_year.csv", false);
yearBuf = new BufferedWriter(yearWriter);
MonthWriter = new java.io.FileWriter(outputLocationFile + "\\" + ncFile.getName().substring(0, ncFile.getName().lastIndexOf(".")) + "_modified_month.csv", false);
monthBuf = new BufferedWriter(MonthWriter);
DayWriter = new java.io.FileWriter(outputLocationFile + "\\" + ncFile.getName().substring(0, ncFile.getName().lastIndexOf(".")) + "_modified_day.csv", false);
dayBuf = new BufferedWriter(DayWriter);
String input;
String temp;
String temp2;
String temp3;
while ((input = br.readLine()) != null) {
temp = input.substring(0, 4);
temp2 = input.substring(4, 6);
temp3 = input.substring(6);
Platform.runLater(new Runnable() {
@Override
public void run() {
myIndicator.setProgress(-1);
}
});
yearBuf.write(temp + "\n");
monthBuf.write(temp2 + "\n");
dayBuf.write(temp3 + "\n");
}
} else {
fWriter = new java.io.FileWriter(outputLocationFile + "\\" + ncFile.getName() + "_modified.csv", false);
writer = new BufferedWriter(fWriter);
String input;
String temp;
while ((input = br.readLine()) != null) {
temp = input.substring(0, 4) + "," + input.substring(4, 6) + "," + input.substring(6);
Platform.runLater(new Runnable() {
@Override
public void run() {
myIndicator.setProgress(-1);
}
});
writer.write(temp + "\n");
}
}
} catch (IOException e) {
e.printStackTrace(System.out);
}finally{
try{
if (br!=null) br.close();
if (yearBuf !=null) yearBuf.close();
if (monthBuf != null)monthBuf.close();
if (dayBuf != null)dayBuf.close();
if (yearWriter != null)yearWriter.close();
if (MonthWriter != null)MonthWriter.close();
if (DayWriter != null)DayWriter.close();
if (fWriter != null) fWriter.close();
if (writer != null) writer.close();
}catch(IOException e){
e.printStackTrace(System.out);
}
}
return true;
}
}
再一次，没什么花哨的，一些缓冲的流和作家，这就是它！值得注意的是，这适用于小型/非大型文件。只有在面对数百万行文件时，我才会看到这种行为。
非常感谢您提供的任何帮助，谢谢！
修改1
只是为了澄清一下，if / else的部分原因在于资源尝试的疯狂，而另一个是更传统的方式，只是为了举例说明它已经尝试了两种方式，相同的症状出现在贯穿任一逻辑块的线程中，所以我非常肯定我关闭资源的方式与它无关。
答案 0 :(得分：1)
编辑：不是说我可以快速阅读代码。试试这个。我早点错过了什么。 join（）只是等待它完成工作。我们需要稍后调用stop（）或等效函数。不推荐使用stop（），这就是我仍然建议使用线程池的原因。 Executors.newCachedThreadPool()应该做到这一点。
Thread t=new Thread();
t.join();
t.stop();
旧解决方案（可能有用）： 确保线程死亡的最简单方法是使用Executors和ExecutorService
ExecutorService executorService = Executors.newFixedThreadPool(10);
executorService.execute(myRunnable);//execute right now
executorService.submit(myRunnable);//execute when <10 threads active
Future<MyType> future = executorService.submit(myCallable);//Runnable and Callable are efficient
MyType result = future.get();
executorService.submit(myThread);//more expensive to create threads and you are using a thread pool anyways
executorService.shutdown();//don't forget to do this when you are done executing or the program may hang
使用Runnable简单地在线程中执行工作。在需要结果时使用Callable。
另一种方法是致电myThread.join();
第三种方法：SwingUtils.invokeLater（myRunnable）; //最简单
编辑： 清理try-catch的解决方案： Java try/catch/finally best practices while acquiring/closing resources ..矫枉过正的解决方案，但很简单 Java io ugly try-finally block
答案 1 :(得分：0)
所以事实证明这是一个解决方案，我昨天有一个想法，但从未采取过行动。
基本上我遇到的问题的根源（我认为）是由于超出了我的写缓冲区，导致了未定义的行为。
现在，我不知道这是因为java bufferedwriter的java实现不好，还是正在发生什么，但解决方案结果相对简单：刷新流每一个单次迭代现在，我知道你在想什么，Gah！经常这样！减速必须是巨大的！是的确，减速速度非常快，它使得1700万行文件解析从大约14秒到大约4分钟。
我可以稍微提高刷新次数以提高性能，但每10次迭代冲洗一次，它仍然会破坏。
我确定这是由于如何在java中处理读/写操作和内存管理的内部结果，而我没有时间深入研究它。如果有人想花时间对这种行为做出很好的解释，我会非常乐意将我接受的答案换成他们的帖子，因为它更完整。
未来检查的固定（现在正在工作）DateWriter类的代码：
package File_Conversion;
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.File;
import java.io.FileReader;
import java.io.IOException;
import java.text.DecimalFormat;
import java.util.ArrayList;
import javafx.application.Platform;
import javafx.scene.control.ProgressIndicator;
/**
* This class is the background 'worker' thread that does all of the heavy duty
* file IO for splitting up the NC file. It periodically sends reports back to
* the main application thread to update the progress indicator.
*
* @author William
*/
public class DateWriter implements Runnable {
private final ProgressIndicator myIndicator;
private static File ncFile;
private final String outputLocationFile;
private float zmax, zmin, xmax, xmin, ymax, ymin;
private ArrayList<Float> xList, yList, zList;
private final DecimalFormat numberFormat = new DecimalFormat("#.000000");
private final DecimalFormat numberFormatMinMax = new DecimalFormat("#.00000");
private final boolean yr_mon_day;
/**
* This is the main constructor, it needs a valid NC file to continue.
*
* @param inputNCFile
* @param outputLocation
* @param myIndicator
* @param yr_mon_day
*/
public DateWriter(String inputNCFile, String outputLocation, ProgressIndicator myIndicator, boolean yr_mon_day) {
this.yr_mon_day = yr_mon_day;
this.myIndicator = myIndicator;
ncFile = new File(inputNCFile);
outputLocationFile = outputLocation;
}
/**
* The primary run() method, starts the thread.
*/
@Override
public void run() {
convertDate();
Platform.runLater(new Runnable() {
@Override
public void run() {
File_Conversion.stage_returner().close();
}
});
System.out.println("At the end of the method.");
}
public boolean convertDate() {
BufferedReader br = null;
java.io.FileWriter yearWriter = null, monthWriter = null, dayWriter = null, fWriter = null;
BufferedWriter yearBuf = null, monthBuf = null, dayBuf = null, writer = null;
try {
br = new BufferedReader(new FileReader(ncFile));
if (yr_mon_day) {
yearWriter = new java.io.FileWriter(outputLocationFile + "\\" + ncFile.getName().substring(0, ncFile.getName().lastIndexOf(".")) + "_modified_year.csv", false);
yearBuf = new BufferedWriter(yearWriter);
monthWriter = new java.io.FileWriter(outputLocationFile + "\\" + ncFile.getName().substring(0, ncFile.getName().lastIndexOf(".")) + "_modified_month.csv", false);
monthBuf = new BufferedWriter(monthWriter);
dayWriter = new java.io.FileWriter(outputLocationFile + "\\" + ncFile.getName().substring(0, ncFile.getName().lastIndexOf(".")) + "_modified_day.csv", false);
dayBuf = new BufferedWriter(dayWriter);
String input;
String temp;
String temp2;
String temp3;
while ((input = br.readLine()) != null) {
temp = input.substring(0, 4);
temp2 = input.substring(4, 6);
temp3 = input.substring(6);
Platform.runLater(new Runnable() {
@Override
public void run() {
myIndicator.setProgress(-1);
}
});
yearBuf.write(temp + "\n");
monthBuf.write(temp2 + "\n");
dayBuf.write(temp3 + "\n");
yearBuf.flush();
monthBuf.flush();
dayBuf.flush();
temp = null;
temp2 = null;
temp3 = null;
}
} else {
fWriter = new java.io.FileWriter(outputLocationFile + "\\" + ncFile.getName() + "_modified.csv", false);
writer = new BufferedWriter(fWriter);
String input;
String temp;
while ((input = br.readLine()) != null) {
temp = input.substring(0, 4) + "," + input.substring(4, 6) + "," + input.substring(6);
Platform.runLater(new Runnable() {
@Override
public void run() {
myIndicator.setProgress(-1);
}
});
writer.write(temp + "\n");
writer.flush();
}
}
} catch (IOException e) {
e.printStackTrace(System.out);
} finally {
try {
if (br != null) {
br.close();
}
if (yearBuf != null) {
yearBuf.close();
}
if (monthBuf != null) {
monthBuf.close();
}
if (dayBuf != null) {
dayBuf.close();
}
if (yearWriter != null) {
yearWriter.close();
}
if (monthWriter != null) {
monthWriter.close();
}
if (dayWriter != null) {
dayWriter.close();
}
if (fWriter != null) {
fWriter.close();
}
if (writer != null) {
writer.close();
}
} catch (IOException e) {
e.printStackTrace(System.out);
}
}
return true;
}
}