Question

我正在尝试使用python requests模块通过点击“分析”从this网站获取某些查询DNA序列的解链温度。我使用以下代码。

import requests

referer = "http://sg.idtdna.com/calc/analyzer"
data = {"settings":{"Sequence":"GGGGGGGAAAAAAGGGGGG","NaConc":50,"MgConc":0,"DNTPsConc":0,"OligoConc":0.25,"NucleotideType":"DNA"}}

r = requests.post(referer, params=data)
print r.content

然而，＆＃34; MELT TEMP＆＃34;并且响应的r.content中的所有其他字段都是空白的。这是尽管＆＃34;响应＆＃34;选项卡名称＆＃34;分析＆＃34;在Chrome浏览器中检查我所拥有的内容。

在此处查看被检查的元素。

如果我浏览上图中的“响应”选项卡，我会发现以下内容中包含我所追求的信息。

{"Sequence":"GGG GGG GAA AAA AGG GGG G","NaConc":50,"MgConc":0,"dNTPsConc":0,"Complement":"CCC CCC TTT TTT CCC CCC C","Length":19,"GCContent":68.4,"MeltTemp":61.4,"MolecularWeight":6097,"ExtCoefficient":205900,"NmoleOD":4.86,"UgOD":29.61,"NucleotideType":"DNA","OligoConc":0.25,"HasErrors":false,"HasModelErrors":false,"ModelErrors":null,"MinMeltTemp":0,"MaxMeltTemp":0,"Mods":[{"ExtensionData":{},"IntProductId":1012,"ProductName":"DNA","ProductType":0,"PublishedProductNbr":1012,"ShortName":"","TotalProductCount":19}],"Errors":[]}

Answer 1

问题：我在哪里可以找到/请求点击的javascript？

在页面中查找来自javascript function

的相关button Analyze来源

<button class="btn btn-primary btn-md btn-block"
    id="analyze-button"
    style="width:150px;"
    data-bind="click: useTool.bind($data, 'analyze') "
    onclick="TrackPageEvent('Analyze button')">
    Analyze
</button>

在页面源内找到相关功能click: useTool'＆lt; script＆gt; ...＆lt; / script＆gt;`。

from bs4 import BeautifulSoup
soup = BeautifulSoup(r.content, "lxml")
for script in soup.find_all("script"):     
    for f in ['useTool', 'TrackPageEvent']:
        if script.find(f):
            print('MATCH({}) in {}'.format(f, script))

否匹配！

查找所有'＆lt; script src = ...＆gt;＆lt; / script＆gt;`
```
for script in soup.find_all("script"):     
    src = script.get('src')
    if src:
        print('{}'.format(src))
```
<强>输出：
  /site/Scripts/knockout-2.2.1.js
  /site/Scripts/amplify/amplify.core.min.js
  /site/Scripts/amplify/amplify.store.min.js
  /site/scripts/idtdna/Common/RequireJSConfig.js?v=1
  /bootstrap/3.1.1/js/bootstrap.min.js
  /JScripts/idtdna-searchcloudV6.js?v=2
  /JScripts/knockout-3.4.0.js
  /JScripts/uAnalyticsLoad2.js?v=2
  /JScripts/IDTHeader.js?v=11
  /calc/Scripts/json2.min.js
  /calc/Areas/Analyzer/Scripts/Analyzer3.js?v=5

以 / calc 开头的脚本之一可能是相关脚本。

下载/calc/Areas/Analyzer/Scripts/Analyzer3.js?v=5并搜索相关的功能useTool。在495号线找到：

self.useTool = function (tool) {
    ...
    var url = '/calc/analyzer/home/';
    var postData = self.analyzePostData();
    ...
        case 'analyze':
            url += 'analyze';
    ...
    $.ajax({
            url: url,
            data: JSON.stringify(postData),
            type: 'POST',
            dataType: 'json',
            contentType: 'application/json; charset=utf-8'

使用$.ajax(...重建requests请求 url = /calc/analyzer/home/analyze与您的不同。
这可能是您的请求未被识别的原因！
但是使用此url结果：找不到页面

默认情况下contentType: 'application/json不支持requests 无法回答这个问题。
相关SO答案：how to POST multipart

Python请求内容不包含响应

1 个答案: