绑定列表到Pandas read_sql_query中的params与其他params

Question

我一直在尝试测试各种方法来运行我的代码。首先，我有这个清单：

member_list = [111,222,333,444,555,...]

我试图将其传递给此查询：

query = pd.read_sql_query(
"""
select member id
    ,yearmonth
from queried_table
where yearmonth between ? and ?
    and member_id in ?
""", db2conn, params = [201601, 201603, member_list])

但是，我收到一条错误消息：

  

＆＃39;参数类型无效。 param-index = 2 param-type = list＆＃39;，＆＃39; HY105＆＃39;

所以我环顾四周并尝试使用格式化的字符串：

query = pd.read_sql_query(
"""
select member id
    ,yearmonth
from queried_table
where yearmonth between ? and ?
    and member_id in (%s)
""" % ','.join(['?']*len(member_list), db2conn, params = [201601, 201603, tuple(member_list)])

现在，我收到错误：

  

＆＃39; SQL包含18622参数标记，但提供了3个参数＆＃39;，＆＃39; HY000＆＃39;

因为它希望在格式化字符串中填写所有?占位符。

所以，最终，有没有办法以某种方式评估列表并传递每个单独的元素以绑定到?，还是有另一种方法可以用来使它工作？

顺便说一下，我使用pyodbc作为我的连接器。

提前致谢！

Answer 1

将其分为三个部分，以帮助隔离问题并提高可读性：

1. 构建SQL字符串
2. 设置参数值
3. 执行pandas.read_sql_query

构建SQL

首先确保正确设置?个占位符。使用str.format与str.join和len根据?长度动态填写member_list。下面的示例假设有3个member_list元素。

实施例

member_list = (1,2,3)
sql = """select member_id, yearmonth
         from queried_table
         where yearmonth between {0} and {0}
         and member_id in ({1})"""
sql = sql.format('?', ','.join('?' * len(member_list)))
print(sql)

返回

select member_id, yearmonth
from queried_table
where yearmonth between ? and ?
and member_id in (?,?,?)

设置参数值

现在确保将参数值组织成 flat 元组

实施例

# generator to flatten values of irregular nested sequences,
# modified from answers http://stackoverflow.com/questions/952914/making-a-flat-list-out-of-list-of-lists-in-python
def flatten(l):
    for el in l:
        try:
            yield from flatten(el)
        except TypeError:
            yield el

params = tuple(flatten((201601, 201603, member_list)))
print(params)

返回

(201601, 201603, 1, 2, 3)

执行

最后在sql来电中将params和read_sql_query值结合在一起

query = pd.read_sql_query(sql, db2conn, params)

Answer 2

如果你正在使用python 3.6+，你也可以使用格式化的字符串litteral来查询（cf https://docs.python.org/3/whatsnew/3.6.html#whatsnew36-pep498）

start, end = 201601, 201603
selected_members = (111, 222, 333, 444, 555)  # requires to be a tuple

query = f"""
    SELECT member_id, yearmonth FROM queried_table
    WHERE yearmonth BETWEEN {start} AND {end}
      AND member_id IN {selected_members}
"""

df = pd.read_sql_query(query, db2conn)

Answer 3

query = 'Select count(*) cnt from TBL_DESK_AUDIT  where trunc(DATETIMECREATED) = trunc(sysdate) and DESK_NAME =' + "'"+dataframe_list1[0][0] + "'"
print(query)
df_TBL_DESK_AUDIT = pd.read_sql_query(query, connect);