从熊猫数据框高效创建大表行

时间:2020-06-11 15:21:22

标签: python python-3.x pandas google-cloud-bigtable bigtable

有没有一种方法可以从熊猫数据框中有效地创建大表行?当使用提供的示例存储大量行和列时,https://cloud.google.com/bigtable/docs/writing-data#batch实际上需要遍历数据帧中的每个元素。有矢量方法可以做到这一点吗?


from google.cloud import bigtable


def write_batch(project_id, instance_id, table_id):
    client = bigtable.Client(project=project_id, admin=True)
    instance = client.instance(instance_id)
    table = instance.table(table_id)

    timestamp = datetime.datetime.utcnow()
    column_family_id = "stats_summary"

    rows = [table.direct_row("tablet#a0b81f74#20190501"),
            table.direct_row("tablet#a0b81f74#20190502")]

    rows[0].set_cell(column_family_id,
                     "connected_wifi",
                     1,
                     timestamp)
    rows[0].set_cell(column_family_id,
                     "os_build",
                     "12155.0.0-rc1",
                     timestamp)
    rows[1].set_cell(column_family_id,
                     "connected_wifi",
                     1,
                     timestamp)
    rows[1].set_cell(column_family_id,
                     "os_build",
                     "12145.0.0-rc6",
                     timestamp)

    response = table.mutate_rows(rows)
    for i, status in enumerate(response):
        if status.code != 0:
            print("Error writing row: {}".format(status.message))

    print('Successfully wrote 2 rows.')```

0 个答案:

没有答案