使用R从Google学者那里获取论文

时间:2015-03-13 10:03:14

标签: r web-scraping google-scholar

使用google-scholar和R,我想知道是谁引用了某篇论文。

现有的包(如scholar)面向H指数分析:研究人员的统计数据。

我想给目标纸作为输入。一个示例网址是:

https://scholar.google.co.uk/scholar?oi=bibs&hl=en&cites=12939847369066114508

然后R应该抓住这些引文页面(谷歌学者分页这些),然后返回一系列引用目标的论文(最多500个或更多引用)。然后我们在标题中搜索关键词,列表期刊和引用作者等。

有关如何做到这一点的任何线索?或者是从字面上刮下每一页? (我可以使用复制和粘贴进行一次性操作)。

似乎这应该是一个通常有用的功能,如播种系统评论,所以有人将它添加到一个包可能会增加他们的H: - )

2 个答案:

答案 0 :(得分:0)

尽管有很多可用的Google's API,但基于Google Scholar的API却不可用。因此,尽管在Google Scholar页面上创建网络爬虫可能并不困难,但我不知道它在多大程度上是非法的。选中this

答案 1 :(得分:0)

或者,您可以使用第三方解决方案,例如 SerpApi。这是一个免费试用的付费 API。我们为您处理代理、解析验证码并解析所有丰富的结构化数据。

示例 Python 代码(也可在其他库中使用):

from serpapi import GoogleSearch

params = {
  "api_key": "secret_api_key",
  "engine": "google_scholar",
  "hl": "en",
  "cites": "12939847369066114508"
}

search = GoogleSearch(params)
results = search.get_dict()

示例 JSON 输出:

{
  "position": 1,
  "title": "Lavaan: An R package for structural equation modeling and more. Version 0.5–12 (BETA)",
  "result_id": "HYlMgouq9VcJ",
  "type": "Pdf",
  "link": "https://users.ugent.be/~yrosseel/lavaan/lavaanIntroduction.pdf",
  "snippet": "Abstract In this document, we illustrate the use of lavaan by providing several examples. If you are new to lavaan, this is the first document to read … 3.1 Entering the model syntax as a string literal … 3.2 Reading the model syntax from an external file …",
  "publication_info": {
    "summary": "Y Rosseel - Journal of statistical software, 2012 - users.ugent.be",
    "authors": [
      {
        "name": "Y Rosseel",
        "link": "https://scholar.google.com/citations?user=0R_YqcMAAAAJ&hl=en&oi=sra",
        "serpapi_scholar_link": "https://serpapi.com/search.json?author_id=0R_YqcMAAAAJ&engine=google_scholar_author&hl=en",
        "author_id": "0R_YqcMAAAAJ"
      }
    ]
  },
  "resources": [
    {
      "title": "ugent.be",
      "file_format": "PDF",
      "link": "https://users.ugent.be/~yrosseel/lavaan/lavaanIntroduction.pdf"
    }
  ],
  "inline_links": {
    "serpapi_cite_link": "https://serpapi.com/search.json?engine=google_scholar_cite&q=HYlMgouq9VcJ",
    "cited_by": {
      "total": 10913,
      "link": "https://scholar.google.com/scholar?cites=6338159566757071133&as_sdt=2005&sciodt=0,5&hl=en",
      "cites_id": "6338159566757071133",
      "serpapi_scholar_link": "https://serpapi.com/search.json?as_sdt=2005&cites=6338159566757071133&engine=google_scholar&hl=en"
    },
    "related_pages_link": "https://scholar.google.com/scholar?q=related:HYlMgouq9VcJ:scholar.google.com/&scioq=&hl=en&as_sdt=2005&sciodt=0,5",
    "versions": {
      "total": 27,
      "link": "https://scholar.google.com/scholar?cluster=6338159566757071133&hl=en&as_sdt=2005&sciodt=0,5",
      "cluster_id": "6338159566757071133",
      "serpapi_scholar_link": "https://serpapi.com/search.json?as_sdt=2005&cluster=6338159566757071133&engine=google_scholar&hl=en"
    },
    "cached_page_link": "https://scholar.googleusercontent.com/scholar?q=cache:HYlMgouq9VcJ:scholar.google.com/&hl=en&as_sdt=2005&sciodt=0,5"
  }
},
...

查看documentation了解更多详情。

免责声明:我在 SerpApi 工作。