Question

我正在尝试使用以下代码访问网站的内容：

HttpClient httpClient = new HttpClient();
string htmlresult = "";

var response = await httpClient.GetAsync(url);

if (response.IsSuccessStatusCode)
{
    htmlresult = await response.Content.ReadAsStringAsync();
}

return htmlresult;

它给了我正确的html，除了https://www.yahoo.com，它可能会给我一个加密的字符串而不是简单的html，如下所示。

   ‹       Ä½ç–ãF¶.øÿ<»Ž4Kj“ð¦ÔÒ½÷ž·îÊO0$ Úž~÷   4@D™U:ëNgK"bÛÄïÿõr¯4^ô

如何从此加密文本中获取简单的html？

Answer 1

Yahoo使用Accept-Encoding: gzip, deflate, br，因此您的案例中的内容是g-zipped。快速修复代码 - 启用自动解压缩：

private async Task<String> GetUrl(string url)
{
    HttpClientHandler handler = new HttpClientHandler()
    {
        AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate
    };

    HttpClient httpClient = new HttpClient(handler);

    string htmlresult = "";

    var response = await httpClient.GetAsync(url);

    if (response.IsSuccessStatusCode)
    {
        htmlresult = await response.Content.ReadAsStringAsync();
    }

    return htmlresult;
}

提取网站plain html

1 个答案: