SQLite3内存数据库到磁盘的纯Python备份

时间:2014-04-30 18:53:12

标签: python sqlite database-backups

如果不安装其他模块,如何使用SQLite backup API将内存数据库备份到磁盘数据库?我已成功执行磁盘到磁盘备份,但将已存在的内存中连接传递给sqlite3_backup_init函数似乎是问题所在。

我的玩具示例,改编自https://gist.github.com/achimnol/3021995并减少到最小值,如下所示:

import sqlite3
import ctypes

# Create a junk in-memory database
sourceconn = sqlite3.connect(':memory:')
cursor = sourceconn.cursor()
cursor.execute('''CREATE TABLE stocks
             (date text, trans text, symbol text, qty real, price real)''')
cursor.execute("INSERT INTO stocks VALUES ('2006-01-05','BUY','RHAT',100,35.14)")
sourceconn.commit()

target = r'C:\data\sqlite\target.db'
dllpath = u'C:\\Python27\DLLs\\sqlite3.dll'

# Constants from the SQLite 3 API defining various return codes of state.
SQLITE_OK = 0
SQLITE_ERROR = 1
SQLITE_BUSY = 5
SQLITE_LOCKED = 6
SQLITE_OPEN_READONLY = 1
SQLITE_OPEN_READWRITE = 2
SQLITE_OPEN_CREATE = 4

# Tweakable variables
pagestocopy = 20
millisecondstosleep = 100

# dllpath = ctypes.util.find_library('sqlite3') # I had trouble with this on Windows
sqlitedll = ctypes.CDLL(dllpath)
sqlitedll.sqlite3_backup_init.restype = ctypes.c_void_p

# Setup some ctypes
p_src_db = ctypes.c_void_p(None)
p_dst_db = ctypes.c_void_p(None)
null_ptr = ctypes.c_void_p(None)

# Check to see if the first argument (source database) can be opened for reading.
# ret = sqlitedll.sqlite3_open_v2(sourceconn, ctypes.byref(p_src_db), SQLITE_OPEN_READONLY, null_ptr)
#assert ret == SQLITE_OK
#assert p_src_db.value is not None

# Check to see if the second argument (target database) can be opened for writing.
ret = sqlitedll.sqlite3_open_v2(target, ctypes.byref(p_dst_db), SQLITE_OPEN_READWRITE | SQLITE_OPEN_CREATE, null_ptr)
assert ret == SQLITE_OK
assert p_dst_db.value is not None

# Start a backup.
print 'Starting backup to SQLite database "%s" to SQLite database "%s" ...' % (sourceconn, target)
p_backup = sqlitedll.sqlite3_backup_init(p_dst_db, 'main', sourceconn, 'main')
print '    Backup handler: {0:#08x}'.format(p_backup)
assert p_backup is not None

# Step through a backup.
while True:
    ret = sqlitedll.sqlite3_backup_step(p_backup, pagestocopy)
    remaining = sqlitedll.sqlite3_backup_remaining(p_backup)
    pagecount = sqlitedll.sqlite3_backup_pagecount(p_backup)
    print '    Backup in progress: {0:.2f}%'.format((pagecount - remaining) / float(pagecount) * 100)
    if remaining == 0:
        break
    if ret in (SQLITE_OK, SQLITE_BUSY, SQLITE_LOCKED):
        sqlitedll.sqlite3_sleep(millisecondstosleep)

# Finish the bakcup
sqlitedll.sqlite3_backup_finish(p_backup)

# Close database connections
sqlitedll.sqlite3_close(p_dst_db)
sqlitedll.sqlite3_close(p_src_db)

我在第49行(ctypes.ArgumentError: argument 3: <type 'exceptions.TypeError'>: Don't know how to convert parameter 3)收到错误p_backup = sqlitedll.sqlite3_backup_init(p_dst_db, 'main', sourceconn, 'main')。不知何故,我需要将对内存数据库的引用传递给sqlite3_backup_init函数。

我不太清楚C掌握API本身的具体细节。

设置:Windows 7,ActiveState Python 2.7

4 个答案:

答案 0 :(得分:2)

只能通过创建它的SQLite库(在本例中为Python的内置SQLite)访问内存数据库。

Python的sqlite3模块无法访问备份API,因此无法复制内存数据库。

您需要安装一个额外的模块,或者首先使用磁盘数据库。

答案 1 :(得分:1)

从Python 3.7开始,此功能在standard library中可用。以下是一些直接从官方文档中复制的示例:

  

示例1,将现有数据库复制到另一个数据库中:

import sqlite3

def progress(status, remaining, total):
    print(f'Copied {total-remaining} of {total} pages...')

con = sqlite3.connect('existing_db.db')
bck = sqlite3.connect('backup.db')
with bck:
    con.backup(bck, pages=1, progress=progress)
bck.close()
con.close()
  

示例2,将现有数据库复制到临时副本中:

import sqlite3

source = sqlite3.connect('existing_db.db')
dest = sqlite3.connect(':memory:')
source.backup(dest)

要回答将内存数据库备份到磁盘的特定问题,看起来像这样。这是使用标准库backup方法的快速脚本:

import sqlite3


source = sqlite3.connect(':memory:')
dest = sqlite3.connect('backup.db')

c = source.cursor()
c.execute("CREATE TABLE test(id INTEGER PRIMARY KEY, msg TEXT);")
c.execute("INSERT INTO test VALUES (?, ?);", (1, "Hello World!"))
source.commit()

source.backup(dest)

dest.close()
source.close()

backup.db数据库可以加载到sqlite3中并进行检查:

$ sqlite3 backup.db
SQLite version 3.24.0 2018-06-04 14:10:15
Enter ".help" for usage hints.
sqlite> .schema
CREATE TABLE test(id INTEGER PRIMARY KEY, msg TEXT);
sqlite> SELECT * FROM test;
1|Hello World!

答案 2 :(得分:0)

虽然这不是一个严格的问题解决方案(因为它不使用备份API),但它可以作为一种最小化的方法,适用于内存中的 small 数据库。

import os
import sqlite3

database = sqlite3.connect(':memory:')

# fill the in memory db with your data here

dbfile = 'dbcopy.db'
if os.path.exists(dbfile):
    os.remove(dbfile) # remove last db dump

new_db = sqlite3.connect(dbfile)
c = new_db.cursor() 
c.executescript("\r\n".join(database.iterdump()))
new_db.close()

答案 3 :(得分:0)

@cl是错误的。我使用 cout << student_info[i].id_num; cout << setw(10) << student_info[i].student_name << setw(10); for(int j = 0; j < LOTTERYNUMBERS; j++) cout << student_info[i].lotteryNumbers[j] << " "; cout << setw(10) << student_info[i].lotteryMatches << setw(10) << setprecision(2) << fixed << showpoint << "$" << student_info[i].prizeMoney << endl; 解决了这个问题,提取了Python连接对象的基础C指针,并创建了一个小包:https://pypi.org/project/sqlite3-backup/

源代码位于https://sissource.ethz.ch/schmittu/sqlite3_backup/blob/master/sqlite3_backup/backup.py