在Python中发送多个POST数据

Question

我有一个Python代码，可以向网站发送POST请求，读取响应并对其进行过滤。对于我使用的POST数据（'number'，'11111'），它完美无缺。但是，我想创建一个包含100个不同数字的txt文件，如1111,2222,3333,4444 ...然后为每个数字发送POST请求。你能帮我解决一下如何在Python中做到这一点吗？

import urllib
from bs4 import BeautifulSoup

headers = {
    'Accept':'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
    'Origin': 'http://mahmutesat.com/python.aspx',
    'User-Agent': 'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.17 (KHTML, like Gecko)  Chrome/24.0.1312.57 Safari/537.17',
    'Content-Type': 'application/x-www-form-urlencoded',
    'Referer': 'http://mahmutesat.com/python.aspx',
    'Accept-Encoding': 'gzip,deflate,sdch',
    'Accept-Language': 'en-US,en;q=0.8',
    'Accept-Charset': 'ISO-8859-1,utf-8;q=0.7,*;q=0.3'
}

class MyOpener(urllib.FancyURLopener):
    version = 'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.57 Safari/537.17'

myopener = MyOpener()
url = 'http://mahmutesat.com/python.aspx'
# first HTTP request without form data
f = myopener.open(url)
soup = BeautifulSoup(f)
# parse and retrieve two vital form values
viewstate = soup.select("#__VIEWSTATE")[0]['value']
eventvalidation = soup.select("#__EVENTVALIDATION")[0]['value']
viewstategenerator = soup.select("#__VIEWSTATEGENERATOR")[0]['value']

formData = (
    ('__EVENTVALIDATION', eventvalidation),
    ('__VIEWSTATE', viewstate),
    ('__VIEWSTATEGENERATOR',viewstategenerator),
    ('number', '11111'),
    ('Button', 'Sorgula'),   
)

encodedFields = urllib.urlencode(formData)
# second HTTP request with form data
f = myopener.open(url, encodedFields)

soup = BeautifulSoup(f.read())

name=soup.findAll('input',{'id':'name_field'})

for eachname in name:
   print eachname['value']

Answer 1

1 - 以下是有关如何创建文件的示例：

f = open('test.txt','w')

这将打开test.txt文件进行写入（'w'）（如果已经有数据，则会被删除，但如果要添加它，请写入：f = open('test.txt','a')）或创建一个，如果它还不存在。请注意，这将发生在您当前的工作目录中，如果您希望它在特定目录中，请在文件名中包含完整目录路径，例如：

f = open('C:\\Python\\test.txt','w')

2 - 然后将所需数据写入/附加到此文件，例如：

for i in range(1,101):
     f.write(str(i*1111)+'\n')

这将从1111到111100

写出100个数字作为字符串

3 - 您应该始终关闭文件：

f.close()

4 - 现在，如果您想阅读此文件＆＃39; test.txt＆＃39;：

f = open('C:\\Python\\test.txt','r')
for i in f:
print i,
file.close()

这很简单，

您需要阅读python中的文件I / O：

https://docs.python.org/2.7/tutorial/inputoutput.html#reading-and-writing-files

请确保在此文档中为您选择正确的Python版本。

Answer 2

如果您的文件有数据：

＆＃34; sample.txt的＆＃34;

1111,2222,3333,4444,5555,6666,7777,8888,......(and so on)

要阅读文件内容，您可以使用文件open操作：

import itertools

#open the file for read
with open("sample.txt", "r") as fp:
     values = fp.readlines()

#Get the values split with ","
data = [map(int, line.split(",")) for line in values]
numbers = list(itertools.chain(*data)) #Ensuring if its having many lines then concatenate 

现在，请将其用作：

for number in numbers:
    formData = (
        ('__EVENTVALIDATION', eventvalidation),
        ('__VIEWSTATE', viewstate),
        ('__VIEWSTATEGENERATOR',viewstategenerator),
        ('number', str(number)), # Here you use the number obtained
        ('Button', 'Sorgula'),   
    )

    encodedFields = urllib.urlencode(formData)
    # second HTTP request with form data
    f = myopener.open(url, encodedFields)

    soup = BeautifulSoup(f.read())

    name=soup.findAll('input',{'id':'name_field'})

    for eachname in name:
       print eachname['value']

Answer 3

使用字典，您可以非常轻松地处理多个请求。

导入请求

values = {

    '__EVENTVALIDATION': event_validation, 
    '__LASTFOCUS': '', 
    '__VIEWSTATE': view_state, 
    '__VIEWSTATEGENERATOR': '6264FB8D', 
    'ctl00$ContentPlaceHolder1$ButGet': 'Get Report',
    'ctl00$ContentPlaceHolder1$Ddl_Circles': 'All Circles', 
    'ctl00$ContentPlaceHolder1$Ddl_Divisions': '-- Select --', 
    'ctl00$ContentPlaceHolder1$TxtTin': tin_num, 
    'ctl00$ContentPlaceHolder1$dropact': 'all' 

}



headers_1 = {

        'Origin': 'https://www.apct.gov.in', 
        'User-Agent': user_agent,
        'Cookie': cookie_1, 
        'Accept-Encoding': 'gzip, deflate, br', 
        'Referer': url_1,
        'Content-Type': 'application/x-www-form-urlencoded', 
        'Upgrade-Insecure-Requests': '1' 

}


try:
    req = requests.post(url_1, data=values, headers=headers_1)