Question

我从批处理请求到Google App Engine上的GMail API收到以下错误：

RequestTooLargeError: The request to API call datastore_v3.Put() was too large.

从Stackoverflow上的其他问题我明白问题与memcache有关。然而，我不知道如何解决这个问题，因为即使我每批运行1个请求它也会出现，而且在我实际上可以对电子邮件的内容做一些事情之前（比如压缩它）。

我的代码目前看起来如下：

          count = 0 #start a new batch request after every 1000 requests
          batch = BatchHttpRequest(callback=get_items)
          for i in new_items:
            batch.add(service.users().messages().get(userId=email, id=i), request_id=str(count))
            count += 1
            if count % 1000 == 0:
              for n in range(0, 5): 
                try:
                  batch.execute(http=http)
                  break
                except Exception as e:
                  if n < 4:
                    time.sleep((2 ** n) + random.randint(0, 1000) / 1000)
                  else:
                    raise
              batch = BatchHttpRequest(callback=get_items) 
          if count % 1000 != 0:
            for n in range(0, 5): 
              try:
                batch.execute(http=http)
                break
              except Exception as e:
                if n < 4:
                  time.sleep((2 ** n) + random.randint(0, 1000) / 1000)
                else:
                  raise

什么是可能的解决方案？

修改

添加回调功能

def get_items(request_id, response, exception):
  if exception is not None:
      print 'An error occurred: %s' % exception
  else:
      save_messages = request_id.split('/', 2)[1]
      email = request_id.split('/', 2)[2]
      in_reply_to = ''
      m_id = ''
      for r in response['payload']['headers']:
          if r['name'].lower() == 'message-id':
              m_id = r.get('value')
          elif r['name'].lower() == 'in-reply-to':
              in_reply_to = r.get('value')
          elif r['name'].lower() == 'from':
              sender, t_t = stripEmails(r.get('value'), None, None, True, email, False, False)
      if m_id:
        incoming = Gmail(id=m_id) #skip if exists already
      else:
        logging.info(response)
        logging.exception('No message ID detect')
        return
      incoming.email = email
      incoming.response = json.dumps(response)
      incoming.put()

Answer 1

通过使用compressed = true来保存太大的文本项作为JsonProperty，问题就解决了。

Gmail API Python：RequestTooLargeError：API调用datastore_v3.Put（）的请求太大

1 个答案: