Question

我遇到了BigQuery参数化问题。我正在将开始日期和结束日期以及数据库中存在的一系列潜在字段传递给该函数。开始日期和结束日期格式为“ yyyymmdd”。

目标是能够传递一组日期和一组字段，并收集与两个日期之间的字段数组有关的数据。

日期操作按预期进行。

像这样传递fields数组：例如：[“ user_pseudo_id”，“ event_name”，“ event_timestamp”]（可能在数组中输入其他条目）

有效地，我想进一步参数化查询，使其看起来类似于以下内容，其中@search_params替换查询的SELECT部分中的各个变量。目的是使fields数组具有更大的可伸缩性，从单个条目到多个条目。

通过搜索，我相信ArrayQueryParameter（代替ScalarQueryParameter）可以解决此问题，但是我没有找到有关此用法的大量文档。

query_job = client.query("""
    SELECT @search_params, _TABLE_SUFFIX AS suffix
    FROM `analytics_180354243.events_*` 
    WHERE REGEXP_EXTRACT(_TABLE_SUFFIX, r'[0-9]+')
    BETWEEN @start_date AND @end_date
    """, job_config=job_config)

以下完整功能

def query_awe(start_date, end_date, fields):
    credentials = service_account.Credentials.from_service_account_file('auth.json')

    project_id = 'my-project-id'

    client = bigquery.Client(credentials=credentials, project=project_id)

    search_params = ""

    for i in fields:
        search_params += i + ", "
    search_params = search_params[:-2]

    query_params = [
        bigquery.ScalarQueryParameter('start_date', 'STRING', start_date),
        bigquery.ScalarQueryParameter('end_date', 'STRING', end_date),
        bigquery.ScalarQueryParameter('search_params', 'STRING', search_params),

    ]
    bigquery.ArrayQueryParameter

    job_config = bigquery.QueryJobConfig()
    job_config.use_legacy_sql = False
    job_config.query_parameters = query_params

    query_job = client.query("""
        SELECT user_pseudo_id, event_name, _TABLE_SUFFIX AS suffix
        FROM `analytics_180354243.events_*` #Each day saved as events_yyyymmdd
        WHERE REGEXP_EXTRACT(_TABLE_SUFFIX, r'[0-9]+')
        BETWEEN @start_date AND @end_date
        ORDER BY user_pseudo_id DESC
        """, job_config=job_config)

    results = query_job.result()  # Waits for job to complete.

    for row in results:
        print(row)

Answer 1

仅使用字符串格式怎么办？

def query_awe(start_date, end_date, fields):
    credentials = service_account.Credentials.from_service_account_file('auth.json')

    project_id = 'my-project-id'

    client = bigquery.Client(credentials=credentials, project=project_id)

    job_config = bigquery.QueryJobConfig()
    job_config.use_legacy_sql = False

    my_query = """
        SELECT {0}, _TABLE_SUFFIX AS suffix
        FROM `analytics_180354243.events_*` #Each day saved as events_yyyymmdd
        WHERE REGEXP_EXTRACT(_TABLE_SUFFIX, r'[0-9]+')
        BETWEEN {1} AND {2}
        ORDER BY user_pseudo_id DESC
        """

    my_query = my_query.format(', '.join(fields), start_date, end_date)
    query_job = client.query(my_query, job_config=job_config)
    results = query_job.result()  # Waits for job to complete.

    for row in results:
        print(row)

Answer 2

我只是使用一个简单的替换功能来完成您要问的事情。

myQuery = """SELECT <var_search_params>, _TABLE_SUFFIX AS suffix
FROM `analytics_180354243.events_*` 
WHERE REGEXP_EXTRACT(_TABLE_SUFFIX, r'[0-9]+')
BETWEEN @start_date AND @end_date
"""
myQuery.replace("<var_search_params>", "Foo, Bar")

query_job = client.query(myQuery, job_config=job_config)

Python Google BigQuery参数化SELECT

2 个答案: