Question

我正在使用＆＃34; bigquery_service = build（＆＃39; bigquery＆＃39;，＆＃39; v2＆＃39;，http = http）＆＃34;并且它在我的笔记本电脑中正确执行，但是当我在服务器上执行此查询时，我收到以下错误。

File "/usr/lib64/python2.6/threading.py", line 532, in __bootstrap_inner
    self.run()
  File "/usr/lib64/python2.6/threading.py", line 484, in run
    self.__target(*self.__args, **self.__kwargs)
  File "QueryAPI.py", line 117, in localMain
    exportDataToGCS(canonicalDate);
  File "QueryAPI.py", line 177, in exportDataToGCS
    bigquery_service = build('bigquery', 'v2', http=http)
  File "/homeBigQuery/src/oauth2client/util.py", line 132, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/homeBigQuery/src/googleapiclient/discovery.py", line 198, in build
    resp, content = http.request(requested_url)
  File "/homeBigQuery/src/oauth2client/util.py", line 132, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/homeBigQuery/src/oauth2client/client.py", line 538, in new_request
    redirections, connection_type)
  File "/homeBigQuery/src/httplib2/__init__.py", line 1570, in request
    (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
  File "/homeBigQuery/src/httplib2/__init__.py", line 1317, in _request
    (response, content) = self._conn_request(conn, request_uri, method, body, headers)
  File "/homeBigQuery/src/httplib2/__init__.py", line 1286, in _conn_request
    response = conn.getresponse()
  File "/usr/lib64/python2.6/httplib.py", line 980, in getresponse
    raise ResponseNotReady()

Answer 1

您可以尝试this 3rd party library for BigQuery。它是围绕Google BigQuery API的薄包装。

登录BigQuery：

import bigquery as bq

with open('my_key.pem', 'r') as key:
    key_content = key.read()

bq_client = bq.get_client(project_id='foo',
                          service_account='bar',
                          private_key=key_content)

提交加载作业：

schema = [{'name':'foo', 'type':'STRING'}]
job = bq_client.import_data_from_uris(source_uris=['gs://foo/bar'],
                                        dataset='foo',
                                        table='bar',
                                        schema=schema,
                                        source_format=bq.client.JOB_FORMAT_CSV,
                                        write_disposition=bq.client.JOB_WRITE_TRUNCATE,
                                        field_delimiter=',',
                                        skip_leading_rows=1)


try:
    job_resource = bq_client.wait_for_job(job, timeout=60)        
except bq.errors.BigQueryTimeoutException:
    logging.critical('BigQuery loading timeout.')

Answer 2

您是否尝试删除“http = http”参数？

这是使用BQ API的一个例子：

import httplib2

from apiclient.discovery import build
from oauth2client.client import SignedJwtAssertionCredentials

# REPLACE WITH YOUR Project ID
PROJECT_NUMBER = 'XXXXXXXXXXX'
# REPLACE WITH THE SERVICE ACCOUNT EMAIL FROM GOOGLE DEV CONSOLE
SERVICE_ACCOUNT_EMAIL = 'XXXXX@developer.gserviceaccount.com'

# OBTAIN THE KEY FROM THE GOOGLE APIs CONSOLE
# More instructions here: http://goo.gl/w0YA0
f = file('key.p12', 'rb')
key = f.read()
f.close()

credentials = SignedJwtAssertionCredentials(
    SERVICE_ACCOUNT_EMAIL,
    key,
    scope='https://www.googleapis.com/auth/bigquery')

http = httplib2.Http()
http = credentials.authorize(http)

service = build('bigquery', 'v2')
datasets = service.datasets()
response = datasets.list(projectId=PROJECT_NUMBER).execute(http)

print 'Dataset list:'
for dataset in response['datasets']:
  print '%s' % dataset['datasetReference']['datasetId']

Answer 3

通过为我运行此服务器的服务器指定代理服务器主机和端口来解决此问题。在凭证的帮助下启动授权的http服务对象时，我修改了代码如下：

之前：

http = httplib2.Http()
http = credentials.authorize(http)
service = build('bigquery', 'v2', http=http)

FIX：

http = httplib2.Http(proxy_info = httplib2.ProxyInfo(httplib2.socks.PROXY_TYPE_HTTP, 'your-proxy-host-name', your-proxy-port))
http = credentials.authorize(http)
service = build('bigquery', 'v2', http=http)

Python + BigQuery + ResponseNotReady（）

3 个答案: