Question

我有一个方法，它逐行从文件中读取数据并在昏迷之间取值，然后将此值放入INSERT查询中。以这种方式保存的文件中的数据：

–,08:10,–,20:20,08:15,08:16,20:26,20:27,08:20,08:21,20:31,20:32,08:30,08:31,20:40,20:41,08:37,08:38,20:46
20:47,08:48,08:50,20:56,20:57,09:00,09:01,21:07,21:08
08:53,–,17:43,09:01,09:03,09:13,09:15,18:02,18:04,–,–,09:19,09:25

这是我的实际代码：

    public void insertTime(SQLiteDatabase database, String table) throws FileNotFoundException {
        BufferedReader br = null;
        String line;

        try {
            int j = 0;
            br = new BufferedReader(new InputStreamReader(context.getAssets().open("time.txt")));
            database.beginTransaction();
            while ((line = br.readLine()) != null) {
                j++;

                String query = "INSERT INTO "+table+""+j+" (arrival, departure) VALUES (?,?)";
                SQLiteStatement statement = database.compileStatement(query);
                // use comma as separator
                String[] time = line.split(",");


                for(int i = 1; i < time.length; i+=2) {

                    statement.bindString(1,time[i-1]);//arrival
                    statement.bindString(2,time[i]);//departure
                    statement.executeInsert();
                    statement.clearBindings();
                }

            }
                database.setTransactionSuccessful();
                database.endTransaction();

        } catch (IOException e) {
            e.printStackTrace();
        } finally {
            if (br != null) {
                try {
                    br.close();
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
        }
    }

问题是数据插入速度非常慢，尽管我使用SQLiteStatement和transactions。例如，当我插入69000行时，大约需要65,929秒。

我在代码中有哪些更改以提高插入速度？

更新

好的，我已经简化了我的代码，我摆脱了BufferedReader，现在它看起来像这样

public void insertTime(SQLiteDatabase database) throws FileNotFoundException {
        database.beginTransaction();
        int r = 0;
        while (r < 122) {
            r++;
            String query = "INSERT INTO table_1 (arrival, departure) VALUES (?,?)";
            SQLiteStatement statement = database.compileStatement(query);

            for(int i = 1; i < 1100; i++) {
                statement.bindString(1,i+"");//arrival
                statement.bindString(2,i+"");//departure
                statement.executeInsert();
                statement.clearBindings();
            }
        }

        database.setTransactionSuccessful();
        database.endTransaction();
}

但它仍然可以插入数据，超过2分钟。你有什么想法如何提高我的第二个例子的速度？

Answer 1

Here是关于提高SQL插入速度的每种方法的非常详细的帖子。

Answer 2

将-和beginTransaction()移到setTransactionSuccessful()循环之外，速度会更快。

Answer 3

为router.put('/:id', function(req, res, next) { db.suppliers.save( { id: +req.params.id, name: req.body.name, email: req.body.email, column1: req.body.column1, column2: req.body.column2, column3: req.body.column3, manyOtherColumns: req.body.manyOtherColumns, etc... }, function (err, result) { if (err) { return next(err); } return res.status(200).json({ status: 'SUCCESS', message: 'Supplier has been saved' }); }) });循环中的每个项启动新事务。如果您只有1个事务来完成所有插入操作，它可能会更快一些。

此外，如果您的数据已损坏并且while()没有为您提供至少2个项目，那么由于String.split被抛出，您的交易将无法正常结束。

Answer 4

Every time you insert a row in a table with indexes, the indexes have to be adjusted. That operation can be costly. Indexes are kept as b-trees and if you hit the rebalance point, you're bound to have a slowdown. One thing you can do to test this is to remove your indexes. You could also drop the indexes, insert, then re-create the indexes.

Answer 5

对于那些使用 JDBC (Java) 的人：可以肯定的是，您是否首先将 autoCommit 设置为 FALSE？

我猜是这样，因为您处理的是显式事务。

通过明确设置自动提交关闭我获得的性能提升超过 1000 次！

所以：

Class.forName("org.sqlite.JDBC");
String urlInput = "jdbc:sqlite:" + databaseFile;
databaseConnection = DriverManager.getConnection(urlInput);
databaseConnection.setAutoCommit( false);

还有：

String sql = "INSERT INTO " + TABLE_NAME + " ( type, bi, ci, fvi, tvi, content_type) VALUES ('V',?,?,?,?,'rtf')";
PreparedStatement psi = databaseConnection.prepareStatement(sql);
for( Item item :  items) {
    psi.setInt(1, item.property1);
    // ....
    count = psi.executeUpdate();
}
databaseConnection.commit();
databaseConnection.setAutoCommit( true);

所以，当有人忘记这一点时，这可能会产生巨大的影响。

SQLite - 提高插入速度

5 个答案: