Question

I recently discovered I accidentally deleted a table from BigQuery, which was constructed by querying other tables (which I still have).

As table deletes in BigQuery are permanent (right?), I would like to reconstruct the lost table, preferably without rewriting the query.

The Query History in the webUI obviously only displays a limited number of queries. The one I am looking for is unfortunately not in that list.

So my question is, is it possible to somehow recover the queries which have disappeared from the query history?

(I do know the creation date of the query I am looking for)

Answer 1

在CLI中，您可以运行bq ls -j -a来检索项目中所有用户的作业。

然后，您可以为每个作业ID bq show -j <job_id>运行，并且为了获得更多详细信息，您将选择使用json响应：

bq show --format=prettyjson -j job_joQEqPwOiOoBlOhDBEgKxQAlKJQ

这将返回以下格式，其中包含您的查询，您的用户和字节处理等...

{
  "configuration": {
    "dryRun": false, 
    "query": {
      "createDisposition": "CREATE_IF_NEEDED", 
      "destinationTable": {
        "datasetId": "", 
        "projectId": "", 
        "tableId": ""
      }, 
      "query": "", 
      "writeDisposition": "WRITE_TRUNCATE"
    }
  }, 
  "etag": "", 
  "id": "", 
  "jobReference": {
    "jobId": "", 
    "projectId": ""
  }, 
  "kind": "bigquery#job", 
  "selfLink": "", 
  "statistics": {
    "creationTime": "1435006022346", 
    "endTime": "1435006144730", 
    "query": {
      "cacheHit": false, 
      "totalBytesProcessed": "105922683030"
    }, 
    "startTime": "1435006023171", 
    "totalBytesProcessed": "105922683030"
  }, 
  "status": {
    "state": "DONE"
  }, 
  "user_email": ""
}

使用API，您需要将allUsers属性传递给所有用户列出作业https://cloud.google.com/bigquery/docs/reference/v2/jobs/list#allUsers

Answer 2

Using the JobID, you can query for a specific job (documented here). This will give you a Jobs resource, which will contain your query.

If you don't know the JobID... it depends on how the query was ran I assume. It's possibly logged by the App Engine (if you ran it via code) in the Logs section of the Developer console. You could also take a look at the Jobs List (credit to the OP for that one) and look in there for your recent jobs. From the list you get Jobs Resources as well, so they will contain all you need.

Answer 3

这可以使用堆栈驱动程序审核日志来完成。 Here是更多信息。

即使您尚未设置堆栈驱动程序日志记录，您仍然可以尝试查找查询。默认情况下，BigQuery日志处于启用状态，您可以在30天内从stackdriver检索这些日志。

Is it possible to retrieve an extended or full query history in google bigquery?

3 个答案: