无法使用webclient下载文件,但我可以从网站手动下载

时间:2018-04-07 05:21:43

标签: c# asp.net

private void download_nse()
{
        string source_location = "https://www.nseindia.com/content/historical/DERIVATIVES/2015/DEC/fo23DEC2015bhav.csv.zip";
        Uri uu = new Uri(source_location);

        using (WebClient fileReader = new WebClient())
        {
            try
            {
                var ua = "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36";
                fileReader.Headers.Add(HttpRequestHeader.UserAgent, ua);
                fileReader.Headers["Accept"] = "/";
                fileReader.DownloadFile(uu, @"d:\nse\notworking.zip");
                fileReader.Dispose();
            }
            catch
            {

            }
        }
        source_location = "https://www.nseindia.com/content/historical/DERIVATIVES/2016/JAN/fo05JAN2016bhav.csv.zip";
        using (WebClient fileReader = new WebClient())
        {
            try
            {
                var ua = "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36";
                fileReader.Headers.Add(HttpRequestHeader.UserAgent, ua);
                fileReader.Headers["Accept"] = "/";
                fileReader.DownloadFile(source_location, @"d:\nse\working.zip");
            }
            catch
            {

            }
        }
    }

1 个答案:

答案 0 :(得分:0)

我检查了您提供的网址,看到当您选择“日期(DD-MM-YYYY)”和“选择报告”下拉选项并点击“获取数据”时,会有一个GET请求使用如下三个参数发送到https://www.nseindia.com/ArchieveSearch:“?h_filetype = fobhav& date = 02-04-2018& section = FO”。并且此GET请求返回:

<p class="archive_title">F&O - Bhavcopy for 02-04-2018 </p><br>
<br><table cellpadding=5>
<tr>
<td class=t0><a href=/content/historical/DERIVATIVES/2018/APR/fo02APR2018bhav.csv.zip target=new>fo02APR2018bhav.csv.zip</a></td></tr>
<tr>
</table>
</td></tr>
</table>

其中包含下载文件的链接。因此,当您在没有GET的情况下发送POST请求时,它会失败。 您可以先发送GET请求,然后使用返回的链接发送POST,如下所示:

        string source_location2 = "https://www.nseindia.com/ArchieveSearch";
        Uri uu2 = new Uri(source_location2);

        using (WebClient fileReader = new WebClient())
        {
            try
            {
                var ua = "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36";
                fileReader.Headers.Add(HttpRequestHeader.UserAgent, ua);
                fileReader.Headers["Accept"] = "/";
                fileReader.QueryString.Add("h_filetype", "fobhav");
                fileReader.QueryString.Add("date", "02-04-2018");
                fileReader.QueryString.Add("section", "FO");
                var response = fileReader.DownloadString(uu2);
                //using Html Agility Pack to parse the response and get the download link. you need to add this package through nuget package manager to project for this code to work.
                var htmlDoc = new HtmlDocument();
                htmlDoc.LoadHtml(response);
                var fileLink = htmlDoc.DocumentNode.SelectSingleNode("//table//tr//td//a").Attributes["href"].Value;
                //now sending the actual file request...
                var fileReader2 = new WebClient();
                fileReader2.Headers.Add(HttpRequestHeader.UserAgent, ua);
                fileReader2.Headers["Accept"] = "/";
                fileReader2.DownloadFile(new Uri("https://www.nseindia.com" + fileLink), @"d:\notworking.zip");
                fileReader2.Dispose();

            }
            catch(Exception e)
            {
                throw e;
            }
        }