我正在解析一个文件,并希望将事务中的行逐行导入到我的MS SQL数据库中,因此我不必将其全部加载到RAM中。
try:
with transaction.atomic(using='mssql'):
with open(filepath, 'r', newline='\n') as clean_file:
for row in clean_file:
measurement = json.loads(row)
current_measurement = Measurementdata(
mea_datetime=measurement['imp_datetime'],
dun=measurement['dun_id'],
exp=measurement['exp_id'],
ptr=measurement['ptr_id'],
mea_value=measurement['imp_value']
)
current_measurement.save()
使用SQLite作为数据库引擎,此代码可以正常运行...
'mssql': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(BASE_DIR, 'mssql.sqlite3'),
}
...并且日志显示交易有效......
(0.000) BEGIN; args=None
(0.000) INSERT INTO "MeasurementData" ("MEA_DateTime", "DUN_ID", "EXP_ID", "PTR_ID", "MEA_Value") VALUES ('2000-01-01 07:51:24', 5, 12, 24, 44.22); args=['2000-01-01 07:51:24', 5, 12, 24, 44.22]
(0.000) INSERT INTO "MeasurementData" ("MEA_DateTime", "DUN_ID", "EXP_ID", "PTR_ID", "MEA_Value") VALUES ('2000-01-01 07:51:24', 5, 12, 24, 343.22); args=['2000-01-01 07:51:24', 5, 12, 24, 343.22]
(0.000) INSERT INTO "MeasurementData" ("MEA_DateTime", "DUN_ID", "EXP_ID", "PTR_ID", "MEA_Value") VALUES ('2000-01-01 07:51:24', 5, 12, 24, 44.22); args=['2000-01-01 07:51:24', 5, 12, 24, 44.22]
(0.000) INSERT INTO "MeasurementData" ("MEA_DateTime", "DUN_ID", "EXP_ID", "PTR_ID", "MEA_Value") VALUES ('2000-01-01 07:51:24', 5, 12, 24, 343.22); args=['2000-01-01 07:51:24', 5, 12, 24, 343.22]
(0.000) INSERT INTO "MeasurementData" ("MEA_DateTime", "DUN_ID", "EXP_ID", "PTR_ID", "MEA_Value") VALUES ('2000-01-01 07:51:24', 5, 12, 24, 44.22); args=['2000-01-01 07:51:24', 5, 12, 24, 44.22]
...但是只要我将数据库引擎切换到MS SQL ......
'mssql': {
'ENGINE': 'sql_server.pyodbc',
'NAME': '----',
'HOST': '----',
'USER': '----',
'PASSWORD': '----',
}
...并尝试导入数据,日志看起来像这样。
(0.012) QUERY = 'SET NOCOUNT ON INSERT INTO [MeasurementData] ([MEA_DateTime], [DUN_ID], [EXP_ID], [PTR_ID], [MEA_Value]) VALUES (%s, %s, %s, %s, %s); SELECT CAST(SCOPE_IDENTITY() AS bigint)' - PARAMS = (datetime.datetime(2000, 1, 1, 7, 51, 25), 5, 12, 24, 188.22); args=(datetime.datetime(2000, 1, 1, 7, 51, 25), 5, 12, 24, 188.22)
(0.012) QUERY = 'SET NOCOUNT ON INSERT INTO [MeasurementData] ([MEA_DateTime], [DUN_ID], [EXP_ID], [PTR_ID], [MEA_Value]) VALUES (%s, %s, %s, %s, %s); SELECT CAST(SCOPE_IDENTITY() AS bigint)' - PARAMS = (datetime.datetime(2000, 1, 1, 7, 51, 25), 5, 12, 24, 15.22); args=(datetime.datetime(2000, 1, 1, 7, 51, 25), 5, 12, 24, 15.22)
(0.013) QUERY = 'SET NOCOUNT ON INSERT INTO [MeasurementData] ([MEA_DateTime], [DUN_ID], [EXP_ID], [PTR_ID], [MEA_Value]) VALUES (%s, %s, %s, %s, %s); SELECT CAST(SCOPE_IDENTITY() AS bigint)' - PARAMS = (datetime.datetime(2000, 1, 1, 7, 51, 25), 5, 12, 24, 23.22); args=(datetime.datetime(2000, 1, 1, 7, 51, 25), 5, 12, 24, 23.22)
(0.013) QUERY = 'SET NOCOUNT ON INSERT INTO [MeasurementData] ([MEA_DateTime], [DUN_ID], [EXP_ID], [PTR_ID], [MEA_Value]) VALUES (%s, %s, %s, %s, %s); SELECT CAST(SCOPE_IDENTITY() AS bigint)' - PARAMS = (datetime.datetime(2000, 1, 1, 7, 51, 23), 5, 12, 24, 453.22); args=(datetime.datetime(2000, 1, 1, 7, 51, 23), 5, 12, 24, 453.22)
(0.012) QUERY = 'SET NOCOUNT ON INSERT INTO [MeasurementData] ([MEA_DateTime], [DUN_ID], [EXP_ID], [PTR_ID], [MEA_Value]) VALUES (%s, %s, %s, %s, %s); SELECT CAST(SCOPE_IDENTITY() AS bigint)' - PARAMS = (datetime.datetime(2000, 1, 1, 7, 51, 23), 5, 12, 24, 44.22); args=(datetime.datetime(2000, 1, 1, 7, 51, 23), 5, 12, 24, 44.22)
...并且没有迹象表明交易已经开始。导入需要5倍的时间,并且抛出异常时不会回滚。我使用的Django版本是1.11.8。
答案 0 :(得分:0)
如果您的MSSQL数据库名称是“气候”,那么您需要更改此行:
with transaction.atomic(using='mssql'):
到此:
with transaction.atomic(using='Climate'):