我想计算Google Big Query Storage的表格成本,但我不知道如何分别查看每个表的存储大小。
答案 0 :(得分:23)
或者从GUI中,您可以使用元数据内部表__TABLES__,例如,这将为您提供GB的大小:
select
sum(size_bytes)/pow(10,9) as size
from
<your_dataset>.__TABLES__
where
table_id = '<your_table>'
答案 1 :(得分:7)
有两种方法可以做到这一点,但请注意,以字节为单位的表的大小属性对于主动接收流式插入的表不可用。
一个。使用BQ命令行工具和JQ linux库来解析JSON。
bq --format=json show publicdata:samples.gsod | jq '.numBytes | tonumber'
这个结果:
17290009238
B中。使用REST api进行Tables:get调用
GET https://www.googleapis.com/bigquery/v2/projects/projectId/datasets/datasetId/tables/tableId
这将返回一个完整的JSON,您可以解析并获取numBytes
。
{
"kind": "bigquery#table",
"description": "This dataset contains weather information collected by NOAA, such a…",
"creationTime": "1335916040125",
"tableReference": {
"projectId": "publicdata",
"tableId": "gsod",
"datasetId": "samples"
},
"numRows": "114420316",
"numBytes": "17290009238",
"etag": "\"Gn3Hpo5WaKnpFuT457VBDNMgZBw/MTQxMzkzNzk4Nzg0Ng\"",
"location": "US",
"lastModifiedTime": "1413937987846",
"type": "TABLE",
"id": "publicdata:samples.gsod",
"selfLink": "https://www.googleapis.com/bigquery/v2/projects/publicdata/datasets…",
"schema": {
"fields": [
{
"description": "The World Meteorological Organization (WMO) / DATSAV3 station numbe…",
"type": "INTEGER",
"name": "station_number",
"mode": "REQUIRED"
},
{
"description": "The Weather-Bureau-Army-Navy (WBAN) station number where the data w…",
"type": "INTEGER",
"name": "wban_number",
"mode": "NULLABLE"
},
{
"description": "The year the data was collected in",
"type": "INTEGER",
"name": "year",
"mode": "REQUIRED"
},
{
"description": "The month the data was collected in",
"type": "INTEGER",
"name": "month",
"mode": "REQUIRED"
},
{
"description": "The day the data was collected in.",
"type": "INTEGER",
"name": "day",
"mode": "REQUIRED"
},
{
"description": "The mean temperature of the day in degrees Fahrenheit, accurate to …",
"type": "FLOAT",
"name": "mean_temp",
"mode": "NULLABLE"
},
{
"description": "The number of observations used to calculate mean_temp.",
"type": "INTEGER",
"name": "num_mean_temp_samples",
"mode": "NULLABLE"
},
{
"description": "The mean dew point of the day in degrees Fahrenheit, accurate to on…",
"type": "FLOAT",
"name": "mean_dew_point",
"mode": "NULLABLE"
},
{
"description": "The number of observations used to calculate mean_dew_point.",
"type": "INTEGER",
"name": "num_mean_dew_point_samples",
"mode": "NULLABLE"
},
{
"description": "The mean sea level pressure of the day in millibars, accurate to on…",
"type": "FLOAT",
"name": "mean_sealevel_pressure",
"mode": "NULLABLE"
},
{
"description": "The number of observations used to calculate mean_sealevel_pressure…",
"type": "INTEGER",
"name": "num_mean_sealevel_pressure_samples",
"mode": "NULLABLE"
},
{
"description": "The mean station pressure of the day in millibars, accurate to one …",
"type": "FLOAT",
"name": "mean_station_pressure",
"mode": "NULLABLE"
},
{
"description": "The number of observations used to calculate mean_station_pressure.…",
"type": "INTEGER",
"name": "num_mean_station_pressure_samples",
"mode": "NULLABLE"
},
{
"description": "The mean visibility of the day in miles, accurate to one tenth of a…",
"type": "FLOAT",
"name": "mean_visibility",
"mode": "NULLABLE"
},
{
"description": "The number of observations used to calculate mean_visibility.",
"type": "INTEGER",
"name": "num_mean_visibility_samples",
"mode": "NULLABLE"
},
{
"description": "The mean wind speed of the day in knots, accurate to one tenth of a…",
"type": "FLOAT",
"name": "mean_wind_speed",
"mode": "NULLABLE"
},
{
"description": "The number of observations used to calculate mean_wind_speed.",
"type": "INTEGER",
"name": "num_mean_wind_speed_samples",
"mode": "NULLABLE"
},
{
"description": "The maximum sustained wind speed reported on the day in knots, accu…",
"type": "FLOAT",
"name": "max_sustained_wind_speed",
"mode": "NULLABLE"
},
{
"description": "The maximum wind gust speed reported on the day in knots, accurate …",
"type": "FLOAT",
"name": "max_gust_wind_speed",
"mode": "NULLABLE"
},
{
"description": "The maximum temperature of the day in degrees Fahrenheit, accurate …",
"type": "FLOAT",
"name": "max_temperature",
"mode": "NULLABLE"
},
{
"description": "Indicates the source of max_temperature.",
"type": "BOOLEAN",
"name": "max_temperature_explicit",
"mode": "NULLABLE"
},
{
"description": "The minimum temperature of the day in degrees Fahrenheit, accurate …",
"type": "FLOAT",
"name": "min_temperature",
"mode": "NULLABLE"
},
{
"description": "Indicates the source of min_temperature.",
"type": "BOOLEAN",
"name": "min_temperature_explicit",
"mode": "NULLABLE"
},
{
"description": "The total precipitation of the day in inches, accurate to one hundr…",
"type": "FLOAT",
"name": "total_precipitation",
"mode": "NULLABLE"
},
{
"description": "The snow depth of the day in inches, accurate to one tenth of an in…",
"type": "FLOAT",
"name": "snow_depth",
"mode": "NULLABLE"
},
{
"description": "Indicates if fog was reported on this day.",
"type": "BOOLEAN",
"name": "fog",
"mode": "NULLABLE"
},
{
"description": "Indicates if rain was reported on this day.",
"type": "BOOLEAN",
"name": "rain",
"mode": "NULLABLE"
},
{
"description": "Indicates if snow was reported on this day.",
"type": "BOOLEAN",
"name": "snow",
"mode": "NULLABLE"
},
{
"description": "Indicates if hail was reported on this day.",
"type": "BOOLEAN",
"name": "hail",
"mode": "NULLABLE"
},
{
"description": "Indicates if thunder was reported on this day.",
"type": "BOOLEAN",
"name": "thunder",
"mode": "NULLABLE"
},
{
"description": "Indicates if a tornado was reported on this day.",
"type": "BOOLEAN",
"name": "tornado",
"mode": "NULLABLE"
}
]
}
}
℃。有一些称为__TABLES__
和__TABLES_SUMMARY__
您可以运行如下查询:
SELECT size_bytes FROM <dataset>.__TABLES__ WHERE table_id='mytablename'
该查询的__TABLES__
部分可能看起来不熟悉。 __TABLES_SUMMARY__
是一个元表,包含有关数据集中表的信息。您可以自己使用此元表。例如,查询SELECT * FROM publicdata:samples.__TABLES_SUMMARY__
将返回有关publicdata:samples
数据集中表格的元数据。您也可以SELECT * FROM publicdata:samples.__TABLES__
可用字段:
__TABLES_SUMMARY__
元表的字段(TABLE_QUERY
查询中都可用)包括:
table_id
:表格的名称。creation_time
:表格创建后的时间,自1970年1月1日UTC以来的毫秒数。这与表中的creation_time
字段相同。type
:无论是视图(2)还是常规表(1)。 TABLE_QUERY()
中的以下字段不,因为它们是__TABLES__
的成员,但不是__TABLES_SUMMARY__
的成员。他们留在这里是为了历史,并部分记录__TABLES__
metatable:
last_modified_time
:自1970年1月1日UTC以来的毫秒,表格已更新(元数据或表格内容)。请注意,如果您使用tabledata.insertAll()
将记录流式传输到您的表格,则可能会过时几分钟。row_count
:表格中的行数。size_bytes
:表格的总大小(以字节为单位)。答案 2 :(得分:5)
您可以使用命令行工具
执行此操作 bq show ds_name.table_name
它将在表格中显示一些信息,包括“总字节数”。在此参考https://cloud.google.com/bigquery/bq-command-line-tool