SQLite add multiple columns using for loop

Question

I am trying to create a SQL database which will have many columns (around 4030). Therefore I cannot use

const_iterator

for every column. Now I am trying to do multiple ALTER TABLE operation but I know that, there's no loop in the ADD branch so no repetition is allowed. So I tried another way to implement my table. I tried,

ALTER TABLE table_name ADD column_name

The code that I wrote in below.

c.execute('''ALTER TABLE table_name ADD (?) ''',k) #k changes in every iteration

I encounter an error which is basically a syntax error, I understood that in SQLite not allowed to do import scipy.io import sqlite3 import os # another irrelevant codes here... conn = sqlite3.connect('CANBUS.db') #connection to .db matFile = scipy.io.loadmat('folder/table.mat') #column names comes from here c = conn.cursor() #cursor function I do not know that function well c.execute('''CREATE TABLE IF NOT EXISTS table_name (ID integer PRIMARY KEY)''') for k in matFile: c.execute('''ALTER TABLE table_name ADD (?) ''',k) #k changes in every loop command.

The message in the console was this. How can I add multiple columns in a single command?

ALTER TABLE table_name ADD (?),k

Thank you for your advices.

Answer 1

Only your first mention of the ALTER TABLE command is correct: you must not use parentheses.

And it is not possible to use parameters for table/column names. You have to put the column name directly into the SQL command string (and quote it if it is not a valid identifier):

import png.*

And instead of executing one command for each column, you could just construct the CREATE TABLE command with all columns:

for k in matFile:
    c.execute("ALTER TABLE table_name ADD " + k)

And the default limit on the number of columns is 2000, so you will not be able to do this anyway. You should properly normalize your database structure.

Answer 2

SQL has the notion Data Definition Language vs. Data Management Language (ref.). All statements related to tables, indexes or other metadata elements are Data Definition Language instructions, when date insertion, modification, extraction or suppression are DML ones. Parameterized queries can only be used in DML not in DDL. That's the reason why you must build your ALTER TABLE (or CREATE TABLE) command as a string and execute that string, when you have been told many times that you shall always use parameterized queries in SQL. But that rules only applies to DML.

BTW, ALTER TABLE can be an expensive operation and you should not put it in a loop: put all columns in one single CREATE TABLE instruction as shown in CL.'s answer.

But if you need more than 2000 columns you should use a different design for your database. In fact one hundred is already too much and is a sign of a bad design.

A common pattern would be to design the table like that: KEY, TYPE, VALUE, (eventually other columns like DATE, INSERT_DATE...). Here you could have KEY, SIGNAL_ID, VALUE, ...