Question

我想画一个条形图，说明一位Google学术学者的h指数每年如何变化。为了计算这一点，我需要每年每篇论文的被引用次数和每年的h指数。

我设法在作者个人资料页面上获得了图形。以艾伯特·爱因斯坦（Albert Einstein）的Google学者为例https://scholar.google.com/citations?user=qc6CJjYAAAAJ&hl=en，我获得了右侧每年的引文计数图，但这是不正确的。我真正想要的是，当您单击一篇论文时，有一个“年度总引用量”图表。我在Python中使用BeautifulSoup和Selenium包。我目前最大的困难是：如果您检查作者的html代码，则每篇论文的内容都是隐藏的，如何单击每篇论文并访问每篇论文每年的总引用量图表？

这是我为右边的图形所做的

def get_citation_by_year(url):
    s = soup(str(urllib.request.urlopen(url).read()), 'lxml')
    print(s)
    #print(s.title.text) #whose google scholar is this?
    years = list(map(int, [i.text for i in s.find_all('span', {'class':'gsc_g_t'})]))
    citation_number = list(map(int, [i.text for i in s.find_all('span', {'class':'gsc_g_al'})]))
    final_chart_data = dict(zip(years, citation_number))
    df = pd.DataFrame({'Year': years, 'Cited_By': citation_number})
    return(df)

点击showmore botton以显示最多文章：

def get_citation_byarticle_byyear(url):
    #quote_page is an URL of google scholar page of a specific author
    quote_page = url
    page = urlopen(quote_page)
    # Click Show more 
    chrome_options = Options()  
    chrome_options.add_argument("--headless")

    driver = webdriver.Chrome(chrome_options=chrome_options, executable_path=r"/Users/upcrown/Desktop/chromedriver") #need to download ChromeDriver, http://chromedriver.chromium.org/downloads

    driver.implicitly_wait(30)
    driver.get(url)

    python_button = driver.find_element_by_xpath('//*[@id="gsc_bpf_more"]')
    python_button.click() #click fhsu link

    time.sleep(5)
    # Selenium hands the page source to Beautiful Soup
    s = BeautifulSoup(driver.page_source, "html.parser")

    year = list(map(str, [i.text for i in s.find_all('span', {'class': 'gsc_a_h gsc_a_hc gs_ibl'})])) ##string not int because some are ''

    #find the paper
    #paper = soup.find_all("a", attrs={"class": "gsc_a_at"})
    paper = list(map(str, [i.text for i in s.find_all('a', {'class': 'gsc_a_at'})]))
    #find the citations 
    #citations = soup.find_all("a", attrs={"class":"gsc_a_ac gs_ibl"})
    citations = list(map(str, [i.text for i in s.find_all('a', {'class': 'gsc_a_ac gs_ibl'})]))

尝试过的其他工具：R“学者”软件包，没有每篇论文每年的引文计数，只有每年的引文计数。 Windows应用程序：发布或灭亡（相同的问题）。 Scopus API（没有Google作者的所有文章的完整列表）

Answer 1

当您点击其中一篇文章时，您可以使用第三方解决方案（如 SerpApi）访问“弹出窗口”或引用。这是一个免费试用的付费 API。

示例 Python 代码（也可在其他库中使用）：

from serpapi import GoogleSearch

params = {
  "api_key": "SECRET_API_KEY",
  "engine": "google_scholar_author",
  "hl": "en",
  "author_id": "qc6CJjYAAAAJ",
  "citation_id": "qc6CJjYAAAAJ:qyhmnyLat1gC",
  "view_op": "view_citation"
}

search = GoogleSearch(params)
results = search.get_dict()

示例 JSON 输出：

"citation": {
  "title": "Rosen (1935)",
  "authors": "A Einstein, B Podolsky",
  "publication_date": "1964",
  "journal": "Physical Review",
  "volume": "47",
  "pages": "777",
  "total_citations": {
    "cited_by": {
      "total": 20216,
      "link": "https://scholar.google.com/scholar?oi=bibs&hl=en&cites=8174092782678430881,4810886029029668500,2204829022686080230&as_sdt=5",
      "serpapi_link": "https://serpapi.com/search.json?cites=8174092782678430881%2C4810886029029668500%2C2204829022686080230&engine=google_scholar&hl=en",
      "cites_id": "8174092782678430881,4810886029029668500,2204829022686080230"
    },
    "table": [
      {
        "year": 1983,
        "citations": 68
      },
      {
        "year": 1984,
        "citations": 62
      },
      ...
    ]
  },
  "scholar_articles": [
    {
      "title": "Can quantum-mechanical description of physical reality be considered complete?",
      "link": "https://scholar.google.com/scholar?oi=bibs&cluster=8174092782678430881&btnI=1&hl=en",
      "authors": "A Einstein, B Podolsky, N Rosen - Physical review, 1935",
      "cited_by": {
        "total": 20195,
        "link": "https://scholar.google.com/scholar?oi=bibs&hl=en&cites=8174092782678430881&as_sdt=5",
        "serpapi_link": "https://serpapi.com/search.json?cites=8174092782678430881&engine=google_scholar&hl=en",
        "cites_id": "8174092782678430881"
      },
      "related_pages_link": {
        "link": "https://scholar.google.com/scholar?oi=bibs&hl=en&q=related:odSh4BM2cHEJ:scholar.google.com/"
      },
      "versions": {
        "total": 96,
        "link": "https://scholar.google.com/scholar?oi=bibs&hl=en&cluster=8174092782678430881",
        "serpapi_link": "https://serpapi.com/search.json?cluster=8174092782678430881&engine=google_scholar&hl=en",
        "cluster_id": "8174092782678430881"
      }
    },
    {
      "title": "Podolsky B Rosen N 1935",
      "link": "https://scholar.google.com/scholar?oi=bibs&cluster=4810886029029668500&btnI=1&hl=en",
      "authors": "A Einstein - Phys. Rev",
      "cited_by": {
        "total": 48,
        "link": "https://scholar.google.com/scholar?oi=bibs&hl=en&cites=4810886029029668500&as_sdt=5",
        "serpapi_link": "https://serpapi.com/search.json?cites=4810886029029668500&engine=google_scholar&hl=en",
        "cites_id": "4810886029029668500"
      },
      "related_pages_link": {
        "link": "https://scholar.google.com/scholar?oi=bibs&hl=en&q=related:lPYzr1qzw0IJ:scholar.google.com/"
      }
    },
    {
      "title": "RosenN",
      "link": "https://scholar.google.com/scholar?oi=bibs&cluster=2204829022686080230&btnI=1&hl=en",
      "authors": "PB EinsteinA - Canquantum mechanical …",
      "cited_by": {
        "total": 16,
        "link": "https://scholar.google.com/scholar?oi=bibs&hl=en&cites=2204829022686080230&as_sdt=5",
        "serpapi_link": "https://serpapi.com/search.json?cites=2204829022686080230&engine=google_scholar&hl=en",
        "cites_id": "2204829022686080230"
      },
      "related_pages_link": {
        "link": "https://scholar.google.com/scholar?oi=bibs&hl=en&q=related:5uysf1AgmR4J:scholar.google.com/"
      }
    }
  ]
}

页面截图：

您可以查看documentation了解更多详情。

免责声明：我在 SerpApi 工作。

如何在Google学术搜索中通过网络抓取每年每篇论文的引文数量？

1 个答案: