Question

我正在尝试以编程方式从网络上下载“ .csv”文件。我收到带有以下代码的200响应。

var list = new List<KeyValuePair<string, string>>() {
new KeyValuePair<string, string>("60", "f1d34cc0-9b50-4a78-be78-d5facfcccfb7"), }; 

listItem["Title"] = "do something";
listItem["ParentID"] = list;
listItem.Update();
context.ExecuteQuery();

在浏览器中提交此URL时，将自动下载“ .csv”文件。我正在尝试让我的应用程序将文件存储在本地。我不知道文件的结束位置，因此我很难保存它。

我添加了HTTP Live Headers扩展，该扩展为我提供了

import requests
url = 'http://eoddata.com/data/filedownload.aspx?e=NYSE&sd=20180722&ed=20180722&d=5&k=xh2mnktdc4&o=d&ea=1&p=0'
user, password = '..', '..'
resp = requests.get(url, auth=(user, password))

对于保存目标文件的任何帮助，我们深表感谢。

编辑：我想在标题响应中指出这一行：GET /data/filedownload.aspx?e=AMEX&sd=20180719&ed=20180719&d=5&k=n4vic5se64&o=d&ea=1&p=0 HTTP/1.1 Host: eoddata.com Accept: text/css,*/*;q=0.1 Accept-Encoding: gzip, deflate Accept-Language: en-US,en;q=0.9 Cookie: ASP.NET_SessionId=drbcwljllaf3xdwdlvqzfvl1;EODDataAdmin=01438A3826CBEEC6085BF7B27AEB1BE6AF337FFBF3E3E33AA1D882D39FB92555E8370DD45631B3F8CAD5289C5C46DB28C99B44DC8253A78D659EB165ADC55A1D6E4C20FA81051908E6BE82143EA5398662779546BE4C31C6F32833C5558F23C9A30D0CA16A703CB939FC6D77A0FB0CB017DDEEA832A1A0CC228B61777295C58CA52AF78B472B9D2FC19E78770DEBD346E9B05F02C65F96EE22C3E55B083FC4A1311FA12B223CC2899D6166E1284AD95E DNT: 1 Referer: http://eoddata.com/download.aspx User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36 HTTP/1.1 200 OK Cache-Control: private content-disposition: attachment;filename="AMEX_20180719.csv" Content-Length: 93840 Content-Type: text/plain Date: Wed, 25 Jul 2018 13:54:31 GMT Server: Microsoft-IIS/7.5 X-AspNet-Version: 4.0.30319 X-Powered-By: ASP.NET是否有所作为，还是没有？

Answer 1

由于您的请求似乎正在处理中，因此您剩下要做的就是下载文件。这是我通常使用请求下载文件的方式：

import requests

def download_file(url, user, password, file_path):
    with requests.get(url, stream = True, auth = (user, password)) as response:
        with open(file_path, 'wb') as f:
            for chunk in response.iter_content(chunk_size = 1024):
                f.write(chunk)

在这种情况下，请不要忘记以.csv结尾的文件路径。

Answer 2

尝试一下：

import requests, urllib
from tqdm import tqdm

def download_file(url):

    filename = "FILENAME.csv"
    print("Downloading %s ..." % filename)

    with open(filename, "wb") as handle:
        for data in tqdm(response.iter_content()):
            handle.write(data)

    print("Download complete for %s!" % filename)

username = "USERNAME"
password = "PASSWORD"

data_credentials = {'user_id': username, 'password': password}
headers = {'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36'}

login_url = "https://mywebsite.com/login/"

requests_session = requests.Session() # create a requests Session

response = requests_session.post(login_url, headers=headers, data=data_credentials) # log in to the requests Session so that you can reuse it

download_url = "http://www.downloadmyfile.com/something.csv"

response = requests_session.get(download_url, timeout=(15,15))

download_file(response.url) # run function to download file

验证后使用Python请求下载CSV文件

2 个答案: