Question

<noscript><strong>Javascript is enabled in your web browser on a specific site while implementing web crawling. If you disable JavaScript, this text will change.</strong></noscript>

我检查了是否添加了相同的标签。对于包含这个标签的站点，即使你用axios请求获取站点url，html值也不会完全进来。我该如何解决这个问题？（我使用了谷歌翻译。抱歉。）

https://vibe.naver.com/chart/total <= 本网站

Answer 1

不知道你有什么问题，但是这段代码应该可以工作

       const axios = require('axios');
       axios.get("https://vibe.naver.com/chart/total").then(html => {
        console.log(html.data)
})

使用node-fetch的方法

const fetch = require("node-fetch");
fetch("https://vibe.naver.com/chart/total").then(result => result.text()).then(result => {
console.log(result)
})

如果您没有获得整个网站，则可能是反机器人措施

Answer 2

我不太确定您使用的是哪种编程语言，或者您是否正在使用一种编程语言。

但是您将无法从 Naver 获得任何后端资源；您只会看到前端 html 代码，以及一堆指向后端源代码的链接。

这是我从检查 HTML 中得到的一小部分：（全部是一行）

<!DOCTYPE html><html lang="ko"><head><meta charset="utf-8"><meta http-equiv="X-UA-Compatible" content="IE=edge"><meta name="viewport" content="width=device-width,initial-scale=1,user-scalable=no"><meta name="format-detection" content="telephone=no"><meta name="description" content="좋아하는 음악은 물론, 좋아할 음악까지 들려주는 취향 저격 뮤직 서비스"><meta name="apple-itunes-app" content="app-id=1338631589"><meta property="og:type" content="website"><meta property="og:image" content="https://music-phinf.pstatic.net/20201019_118/16031004163365Jexk_JPEG/about.jpg"><meta property="og:title" content="NAVER VIBE(바이브)"><meta property="og:description" content="#나만을 위한 음악이 듣고 싶을 땐 #내돈내듣 VIBE."><meta property="og:url" content="https://vibe.naver.com"><meta name="twitter:card" content="summary_large_image"><meta name="twitter:image" content="https://music-phinf.pstatic.net/20201019_118/16031004163365Jexk_JPEG/about.jpg"><meta name="twitter:title" content="NAVER VIBE(바이브)"><meta name="twitter:site" content="NAVER VIBE(바이브)"><meta name="twitter:creator" content="VIBE"><meta name="twitter:description" content="#나만을 위한 음악이 듣고 싶을 땐 #내돈내듣 VIBE."><link rel="shortcut icon" href="/favicon.ico" type="image/x-icon"><link rel="icon" href="/favicon.ico" type="image/x-icon"><link rel="apple-touch-icon" href="/images/icon/icon_120x120.png"><link rel="apple-touch-icon" sizes="120x120" href="/images/icon/icon_120x120.png"><link rel="apple-touch-icon" sizes="152x152" href="/images/icon/icon_152x152.png"><link rel="apple-touch-icon" sizes="167x167" href="/images/icon/icon_167x167.png"><link rel="apple-touch-icon" sizes="180x180" href="/images/icon/icon_180x180.png"><title>VIBE (바이브)</title><link href="/20210611235311_resources/css/common.314db81d.css" rel="prefetch"><link href="/20210611235311_resources/js/common.435f4ddf.js" rel="prefetch"><link href="/20210611235311_resources/css/app.f381a573.css" rel="preload" as="style"><link href="/20210611235311_resources/css/chunk-vendors.598bb52f.css" rel="preload" as="style"><link href="/20210611235311_resources/js/app.54f33238.js" rel="preload" as="script"><link href="/20210611235311_resources/js/chunk-vendors.b833ef5d.js" rel="preload" as="script"><link href="/20210611235311_resources/css/chunk-vendors.598bb52f.css" rel="stylesheet"><link href="/20210611235311_resources/css/app.f381a573.css" rel="stylesheet"><script type="text/javascript" src="https://gc.kis.v2.scr.kaspersky-labs.com/FD126C42-EBFA-4E12-B309-BB3FDD723AC1/main.js?attr=6Lte6dwVvrAUmIXAmrO3pKxbbcRLcOyA1czuw93D8LByqZTbj1qR4t2QiZ9_JGMjj_RlN8QK-X9i0K0eVweFfA" charset="UTF-8"></script><link rel="stylesheet" crossorigin="anonymous" href="https://gc.kis.v2.scr.kaspersky-labs.com/E3E8934C-235A-4B0E-825A-35A08381A191/abn/main.css?attr=aHR0cHM6Ly92aWJlLm5hdmVyLmNvbS9jaGFydC90b3RhbA"/></head><body><noscript><strong>자바스크립트가 비활성화 되어있어 화면을 표시할 수 없습니다. 브라우저 설정에서 자바스크립트를 활성화 후 다시 시도해 주세요.</strong></noscript><div id="app" class="app"></div><script src="/20210611235311_resources/js/chunk-vendors.b833ef5d.js"></script><script src="/20210611235311_resources/js/app.54f33238.js"></script></body></html>

如何从站点获取html数据

2 个答案: