如何在python中表示复杂的POST数据以进行请求

时间：2019-05-14 13:01:21

标签： python python-3.x web-scraping python-requests

我正在尝试自动化Web表单。我实际上在Chrome中运行查询时的有效负载在检查窗口上看起来像这样... 

    data: [{"property":"TimeQuick","value":"Active"},{"property":"TimeQuickDurationOptions","value":3},{"property":"TimeQuickStartDate","value":"05/15/2019 00:00:00"},{"property":"TimeQuickEndDate","value":"05/15/2019 23:59:59"},{"property":"ProviderCode","value":["FPL"]},{"property":"SellerCode","value":[""]},{"property":"Ref","value":""},{"property":"POR","value":["SOCO"]},{"property":"POD","value":["FPC"]},{"property":"Path","value":""},{"property":"ServiceIncrement","value":["DAILY"]},{"property":"TSClass","value":[""]},{"property":"TSType","value":[""]},{"property":"TSWindow","value":""},{"property":"TSPeriod","value":""},{"property":"TSSubClass","value":""},{"property":"Time","value":"Active"},{"property":"TimeDurationOptions","value":3},{"property":"TimeStartDate","value":"05/15/2019 00:00:00"},{"property":"TimeEndDate","value":"05/15/2019 23:59:59"},{"property":"ShowActiveData","value":true},{"property":"DaylightSavings","value":false}]
    sort: [{"property":"TOL","direction":"DESC","root":"data"}]
    pagingEnabled: 1
    page: 1
    limit: 50

我试图将其表示为这样的python变量

    datareq = [
             ('data', 
              {'property':'TimeQuick', 'value':'Active'},
              {"property":"TimeQuickDurationOptions","value":'3'},
              {"property":"TimeQuickStartDate","value":"05/15/2019 00:00:00"},
              {"property":"TimeQuickEndDate","value":"05/15/2019 23:59:59"},
              {"property":"ProviderCode","value":["FPL"]},
              {"property":"SellerCode","value":[""]},
              {"property":"Ref","value":""},
              {"property":"POR","value":["SOCO"]},
              {"property":"POD","value":["FPC"]},
              {"property":"Path","value":""},
              {"property":"ServiceIncrement","value":["DAILY"]},
              {"property":"TSClass","value":[""]},
              {"property":"TSType","value":[""]},
              {"property":"TSWindow","value":""},
              {"property":"TSPeriod","value":""},
              {"property":"TSSubClass","value":""},
              {"property":"Time","value":"Active"},
              {"property":"TimeDurationOptions","value":3},
              {"property":"TimeStartDate","value":"05/15/2019 00:00:00"},
              {"property":"TimeEndDate","value":"05/15/2019 23:59:59"},
              {"property":"ShowActiveData","value":'true'},
              {"property":"DaylightSavings","value":'false'}
              ),
             ('sort',
              {"property":"TOL","direction":"DESC","root":"data"}
              ),
             ('pagingEnabled', 1),
             ('limit', 50)

             ]

我运行s.post(myurl, data=datareq, cert=mycerts)时遇到以下错误：

    Traceback (most recent call last):

      File "<ipython-input-78-37ee269ebbce>", line 1, in <module>
        logblah=s.post(myurl, data= datareq, cert=mycerts)

      File "C:\Users\me\AppData\Local\Continuum\anaconda3\Lib\python\requests\sessions.py", line 555, in post
        return self.request('POST', url, data=data, json=json, **kwargs)

      File "C:\Users\me\AppData\Local\Continuum\anaconda3\Lib\python\requests\sessions.py", line 494, in request
        prep = self.prepare_request(req)

      File "C:\Users\me\AppData\Local\Continuum\anaconda3\Lib\python\requests\sessions.py", line 437, in prepare_request
        hooks=merge_hooks(request.hooks, self.hooks),

      File "C:\Users\me\AppData\Local\Continuum\anaconda3\Lib\python\requests\models.py", line 308, in prepare
        self.prepare_body(data, files, json)

      File "C:\Users\me\AppData\Local\Continuum\anaconda3\Lib\python\requests\models.py", line 499, in prepare_body
        body = self._encode_params(data)

      File "C:\Users\me\AppData\Local\Continuum\anaconda3\Lib\python\requests\models.py", line 97, in _encode_params
        for k, vs in to_key_val_list(data):

    ValueError: too many values to unpack (expected 2)

我假设问题出在构建datareq变量的方式上。进行POST的正确方法是什么？

编辑/更新： 我已经尝试过Alex和Ivan的建议，但都没有用。他们每个人都从服务器给我一个神秘的消息success=false。如果我用json=datareq而不是data=datareq来尝试，那么即使有很多结果，我也会从服务器得到0的响应，所以看起来好像越来越近了，但还不是全部。 / p>

5 个答案:

答案 0 :(得分：0)

这是我会用的：

datareq = {
    "data": {
        "TimeQuick": 'Active',
        "TimeQuickDurationOptions":'3',
        "TimeQuickStartDate":"05/15/2019 00:00:00",
        "TimeQuickEndDate":"05/15/2019 23:59:59",
        "ProviderCode":["FPL"],
        "SellerCode":[""],
        "Ref":"",
        "POR":["SOCO"],
        "POD":["FPC"],
        "Path":"",
        "ServiceIncrement":["DAILY"],
        "TSClass":[""],
        "TSType":[""],
        "TSWindow":"",
        "TSPeriod":"",
        "TSSubClass":"",
        "Time":"Active",
        "TimeDurationOptions":3,
        "TimeStartDate":"05/15/2019 00:00:00",
        "TimeEndDate":"05/15/2019 23:59:59",
        "ShowActiveData":'true',
        "DaylightSavings":'false'
    },
    "sort": {
        "TOL": {
            "direction":"DESC",
            "root":"data"
        }
    },
    "pagingEnabled":1,
    "limit":50
}

并使用给定属性：

print(datareq['data']['TimeQuickStartDate'])
print(datareq['sort']['TOL']['root'])
print(datareq['limit'])

答案 1 :(得分：0)

尽管您的datareq数据结构通常对python有效，但不能用作data方法中的requests.post参数。

它不是显式编写的，但是如果您为requests库阅读documentation，则可能会注意到data参数应该是普通的python dict或一个list中的tuple，其中每个顶级元组恰好包含2个成员-一个键和一个值。

这正是您的traceback输出的内容：

ValueError: too many values to unpack (expected 2)

具体来说，您在tuple中的第一个datareq无效：

datareq = [
             ('data', 
              {'property'...

，因为data应该是“键”，而其他property应该是“值”。

为正确起见，datareq应该如下所示：

datareq = [
    ('data', 
        (  # note tuple here
            {'property':'TimeQuick', 'value':'Active'},
            {"property":"TimeQuickDurationOptions","value":'3'},
            {"property":"TimeQuickStartDate","value":"05/15/2019 00:00:00"},
            {"property":"TimeQuickEndDate","value":"05/15/2019 23:59:59"},
            {"property":"ProviderCode","value":["FPL"]},
            {"property":"SellerCode","value":[""]},
            {"property":"Ref","value":""},
            {"property":"POR","value":["SOCO"]},
            {"property":"POD","value":["FPC"]},
            {"property":"Path","value":""},
            {"property":"ServiceIncrement","value":["DAILY"]},
            {"property":"TSClass","value":[""]},
            {"property":"TSType","value":[""]},
            {"property":"TSWindow","value":""},
            {"property":"TSPeriod","value":""},
            {"property":"TSSubClass","value":""},
            {"property":"Time","value":"Active"},
            {"property":"TimeDurationOptions","value":3},
            {"property":"TimeStartDate","value":"05/15/2019 00:00:00"},
            {"property":"TimeEndDate","value":"05/15/2019 23:59:59"},
            {"property":"ShowActiveData","value":'true'},
            {"property":"DaylightSavings","value":'false'},
        ),
    ),
    ('sort',  # same goes for the "sort" key
        (
            {"property":"TOL","direction":"DESC","root":"data"},
        ),
    ),
    ('pagingEnabled', 1),
    ('limit', 50),
]

答案 2 :(得分：0)

您可以将常规JSON用作import json datareq = { "data": [{"property":"TimeQuick","value":"Active"},{"property":"TimeQuickDurationOptions","value":3},{"property":"TimeQuickStartDate","value":"05/15/2019 00:00:00"},{"property":"TimeQuickEndDate","value":"05/15/2019 23:59:59"},{"property":"ProviderCode","value":["FPL"]},{"property":"SellerCode","value":[""]},{"property":"Ref","value":""},{"property":"POR","value":["SOCO"]},{"property":"POD","value":["FPC"]},{"property":"Path","value":""},{"property":"ServiceIncrement","value":["DAILY"]},{"property":"TSClass","value":[""]},{"property":"TSType","value":[""]},{"property":"TSWindow","value":""},{"property":"TSPeriod","value":""},{"property":"TSSubClass","value":""},{"property":"Time","value":"Active"},{"property":"TimeDurationOptions","value":3},{"property":"TimeStartDate","value":"05/15/2019 00:00:00"},{"property":"TimeEndDate","value":"05/15/2019 23:59:59"},{"property":"ShowActiveData","value":true},{"property":"DaylightSavings","value":false}], "sort": [{"property":"TOL","direction":"DESC","root":"data"}], "pagingEnabled": 1, "page": 1, "limit": 50 } s.post(myurl, data=json.dumps(datareq), cert=mycerts) 的参数：

class App extends React.Component{
  state ={ inputValue:"" };
  
  render(){
    return(
      <div>
        <input type="text" value={this.state.inputValue} onChange={this.handleChange} />
        <button onClick={this.handleSubmit}>Submit</button>
      </div>
    );
  }
  
  handleChange=(e)=>{
   this.setState({ inputValue: e.target.value });
  }
  
  handleSubmit=()=>{
    console.log("inputValue::", this.state.inputValue);
  }
}

ReactDOM.render(<App/>, document.getElementById("root"));

答案 3 :(得分：0)

下面是该数据的python表示形式

data = {
    "data": [
        {"property": "TimeQuick", "value": "Active"}, 
        {"property": "TimeQuickDurationOptions", "value": 3},
        {"property": "TimeQuickStartDate", "value": "05/15/2019 00:00:00"},
        {"property": "TimeQuickEndDate", "value": "05/15/2019 23:59:59"},
        {"property": "ProviderCode", "value": ["FPL"]},
        {"property": "SellerCode", "value": [""]},
        {"property": "Ref", "value": ""},
        {"property": "POR", "value": ["SOCO"]},
        {"property": "POD", "value": ["FPC"]},
        {"property": "Path", "value": ""},
        {"property": "ServiceIncrement", "value": ["DAILY"]},
        {"property": "TSClass", "value": [""]},
        {"property": "TSType", "value": [""]},
        {"property": "TSWindow", "value": ""},
        {"property": "TSPeriod", "value": ""},
        {"property": "TSSubClass", "value": ""},
        {"property": "Time", "value": "Active"},
        {"property": "TimeDurationOptions", "value": 3},
        {"property": "TimeStartDate", "value": "05/15/2019 00:00:00"},
        {"property": "TimeEndDate", "value": "05/15/2019 23:59:59"},
        {"property": "ShowActiveData", "value": True},
        {"property": "DaylightSavings", "value": False}],
    "sort": [
        {"property": "TOL", "direction": "DESC", "root": "data"}
    ],
    "pagingEnabled": 1,
    "page": 1,
    "limit": 50,
}

您不需要将任何json对象放入任何有效的python字典可以执行的request.post中。您唯一需要担心的是服务器接受的json模式。当您将python字典自动放入库的post方法时，它将自动转换为json。

答案 4 :(得分：0)

服务器是否期望POST正文为application/json或application/x-www-form-urlencoded类型？如果是json，请使用语法request.post(url, json=payload_dict, ....)（请注意，使用json kwarg代替data）。如果您使用data，则假定内容类型为application/x-www-form-urlencoded，而json假定为json。如果确实期望使用json（因为数据被嵌套在几层深度之内）是有意义的，那么我将在@IşıkKaplan的答案中进行扩展：

# the dict as posted by @Işık Kaplan
payload = {
    "data": [
        {"property": "TimeQuick", "value": "Active"}, 
        {"property": "TimeQuickDurationOptions", "value": 3},
        {"property": "TimeQuickStartDate", "value": "05/15/2019 00:00:00"},
        {"property": "TimeQuickEndDate", "value": "05/15/2019 23:59:59"},
        {"property": "ProviderCode", "value": ["FPL"]},
        {"property": "SellerCode", "value": [""]},
        {"property": "Ref", "value": ""},
        {"property": "POR", "value": ["SOCO"]},
        {"property": "POD", "value": ["FPC"]},
        {"property": "Path", "value": ""},
        {"property": "ServiceIncrement", "value": ["DAILY"]},
        {"property": "TSClass", "value": [""]},
        {"property": "TSType", "value": [""]},
        {"property": "TSWindow", "value": ""},
        {"property": "TSPeriod", "value": ""},
        {"property": "TSSubClass", "value": ""},
        {"property": "Time", "value": "Active"},
        {"property": "TimeDurationOptions", "value": 3},
        {"property": "TimeStartDate", "value": "05/15/2019 00:00:00"},
        {"property": "TimeEndDate", "value": "05/15/2019 23:59:59"},
        {"property": "ShowActiveData", "value": True},
        {"property": "DaylightSavings", "value": False}],
    "sort": [
        {"property": "TOL", "direction": "DESC", "root": "data"}
    ],
    "pagingEnabled": 1,
    "page": 1,
    "limit": 50,
}

# assuming you have variables named `s`, `myurl`, and `mycerts`
# note `json=payload`, not `data=payload`
s.post(myurl, json=payload, cert=mycerts)
相关问题
最新问题