Python请求 - 如何使用post方法？

时间：2018-03-11 05:09:55

标签： python python-requests

如何从学术文章列表中查询？使用Python和请求

搜索名称Maria Helena

网站：http://bibliotecadigital.fgv.br/ojs/index.php/abpt/search/search

2 个答案:

答案 0 :(得分：2)

您可以尝试以下步骤：

  1. 使用您的浏览器访问该网页，并使用Google开发者工具检查提出的请求。

    2. 请注意，响应是HTML，因此您可能需要使用像美丽汤这样的抓取库来处理数据。可能有一个json api（左侧作为练习）？

    Google Inspector

    1. 提出请求并不算太糟糕（完全实现留作练习）：

      requests.post（“http://bibliotecadigital.fgv.br/ojs/index.php/abpt/search/search”，...）

      2. Form Data

      请注意，我将有效负载作为练习留下，但如果需要，可以随意询问提示（尝试搜索如何使用Google中的请求输入表单数据）

答案 1 :(得分：1)

import json

import requests
from bs4 import BeautifulSoup

URL = 'http://bibliotecadigital.fgv.br/ojs/index.php/abpt/search/search'


def get_search_result(query):
    r = requests.post(URL, data={'query': query})
    r.raise_for_status()
    return r.text


def parse_search_result(html):
    soup = BeautifulSoup(html, 'lxml')
    div = soup.find('div', {'id': 'results'})

    result = []
    for row in div.find_all('tr', {'valign': 'top'}):
        edition, title, links = row.find_all('td')
        all_links = {}
        for link in links.find_all('a'):
            all_links[link.text.strip().lower()] = link['href']

        result.append({
            'edition': edition.a['href'],
            'title': title.text,
            **all_links
        })
    return result


if __name__ == '__main__':
    search_result = get_search_result('Maria Helena')
    parsed_result = parse_search_result(search_result)

    print(len(parsed_result))
    print(json.dumps(parsed_result, indent=4))

控制台：

25
[
    {
        "edition": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/issue/view/1055",
        "title": "Considera\u00e7\u00f5es s\u00f4bre o \"Sistema de reabilita\u00e7\u00e3o dos incapacitados\" da Gr\u00e3-Bretanha",
        "resumo": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/13779",
        "pdf": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/13779/12679"
    },
    {
        "edition": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/issue/view/1084",
        "title": "Como transmitir aos pais os resultados dos testes",
        "resumo": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/14685",
        "pdf": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/14685/13582"
    },
    {
        "edition": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/issue/view/1060",
        "title": "Resumo da participa\u00e7\u00e3o do Brasil ao V Congresso Interamericano de Psicologia Aplicada",
        "pdf": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/13915/12814"
    },
    {
        "edition": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/issue/view/1097",
        "title": "Estudo evolutivo dos n\u00edveis de segrega\u00e7\u00e3o perceptiva na crian\u00e7a",
        "pdf": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/14989/13886"
    },
    {
        "edition": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/issue/view/1058",
        "title": "An\u00e1lise das concord\u00e2ncias e discord\u00e2ncias entre a s\u00edntese da entrevista e a s\u00edntese final, no processo de orienta\u00e7\u00e3o",
        "resumo": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/13879",
        "pdf": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/13879/12779"
    },
    {
        "edition": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/issue/view/1102",
        "title": "The Psychological Development of the Child",
        "resumo": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/15168",
        "pdf": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/15168/14054"
    },
    {
        "edition": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/issue/view/1094",
        "title": "Impress\u00f5es do XVII Congresso Internacional de Psicologia e da 71\u00aa Conven\u00e7\u00e3o Anual da Associa\u00e7\u00e3o Americana de Psicologia",
        "pdf": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/14952/13850"
    },
    {
        "edition": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/issue/view/1111",
        "title": "Perspectivas atuais da Psicologia da Educa\u00e7\u00e3o",
        "pdf": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/15740/14599"
    },
    {
        "edition": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/issue/view/1101",
        "title": "Realidade do mundo atual e posi\u00e7\u00e3o do jovem",
        "pdf": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/15142/14027"
    },
    {
        "edition": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/issue/view/1058",
        "title": "Aspectos da readapta\u00e7\u00e3o profissional",
        "pdf": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/13877/12777"
    },
    {
        "edition": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/issue/view/1094",
        "title": "A dislexia e o problema da lateralidade",
        "pdf": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/14949/13847"
    },
    {
        "edition": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/issue/view/1110",
        "title": "Psicologia Educacional",
        "resumo": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/15704",
        "pdf": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/15704/14568"
    },
    {
        "edition": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/issue/view/1118",
        "title": "Psicological testing",
        "resumo": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/16094",
        "pdf": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/16094/14916"
    },
    {
        "edition": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/issue/view/1092",
        "title": "Estudo comparativo das aspira\u00e7\u00f5es profissionais dos jovens e das profiss\u00f5es aconselhadas pelo Instituto de Sele\u00e7\u00e3o e Orienta\u00e7\u00e3o Profissional",
        "pdf": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/14835/13735"
    },
    {
        "edition": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/issue/view/1110",
        "title": "Fontes de tendenciosidade cognitiva nas rela\u00e7\u00f5es interpessoais",
        "pdf": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/15693/14557"
    },
    {
        "edition": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/issue/view/1082",
        "title": "Problemas de linguagem e comunica\u00e7\u00e3o",
        "pdf": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/14659/13561"
    },
    {
        "edition": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/issue/view/1118",
        "title": "Lideran\u00e7a e din\u00e2mica de grupo",
        "pdf": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/16085/14907"
    },
    {
        "edition": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/issue/view/1107",
        "title": "An\u00e1lise e contr\u00f4le do teste de Brosson em grupos de adolescente",
        "pdf": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/15460/14344"
    },
    {
        "edition": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/issue/view/1100",
        "title": "Readings in Child Development and Personality",
        "resumo": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/15118",
        "pdf": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/15118/14004"
    },
    {
        "edition": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/issue/view/1099",
        "title": "Cognition: Theory, Research, Promise",
        "resumo": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/15074",
        "pdf": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/15074/13965"
    },
    {
        "edition": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/issue/view/1088",
        "title": "Princ\u00edpios psicol\u00f3gicos b\u00e1sicos da linguagem",
        "pdf": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/14758/13656"
    },
    {
        "edition": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/issue/view/1099",
        "title": "A car\u00eancia afetiva e sua repercuss\u00e3o na adapta\u00e7\u00e3o escolar",
        "pdf": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/15066/13958"
    },
    {
        "edition": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/issue/view/1098",
        "title": "A livre atividade art\u00edstica e o desenvolvimento do adolescente",
        "pdf": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/15057/13949"
    },
    {
        "edition": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/issue/view/1097",
        "title": "ADAPTATION - LEVEL THEORY",
        "resumo": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/15000",
        "pdf": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/15000/13897"
    },
    {
        "edition": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/issue/view/1115",
        "title": "Esteri\u00f3tipos em rela\u00e7\u00e3o a alunos de Psicologia num campus universit\u00e1rio",
        "pdf": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/16011/14836"
    }
]
相关问题
最新问题