木偶:从网站获取localStorage

时间:2019-04-17 10:18:27

标签: javascript node.js web-scraping local-storage puppeteer

我需要使用Puppeteer来保存网站保存的所有数据:cookie和localStorage(例如,登录后)。我已经阅读了所有Puppeteer文档,但是找不到关于localStorage的任何信息。

enter image description here

我可以获取cookie,但是我不知道可以获取localStorage。例如:

const puppeteer = require('puppeteer');

(async () => {
  const browser = await puppeteer.launch()
  const page = await browser.newPage()
  await page.goto('https://www.twitter.com/')

  //
  // code for login
  //

  const returnedCookie = await page.cookies();  
  console.log(returnedCookie)
  // const localStorage = ??
  // console.log(localStorage)

  await browser.close()
})()

3 个答案:

答案 0 :(得分:1)

我找到了方法:

const puppeteer = require('puppeteer');

(async () => {
  const browser = await puppeteer.launch()
  const page = await browser.newPage()
  await page.goto('https://www.twitter.com/')

  //
  // code for login
  //

  const returnedCookie = await page.cookies();  
  console.log(returnedCookie)

  await page.waitFor( 10000 );
  const localStorageData = await page.evaluate(() => {
    let json = {};
    for (let i = 0; i < localStorage.length; i++) {
      const key = localStorage.key(i);
      json[key] = localStorage.getItem(key);
    }
    return json;
  });

  console.log(localStorageData)

  await browser.close()
})()

答案 1 :(得分:1)

更适合我的方式

const localStorage = await page.evaluate(() => Object.assign({}, window.localStorage));

答案 2 :(得分:0)

Guido的答案有效,但是如果您只想从本地/会话存储中返回一个值(而不是整个本地/会话存储对象),则可以使用:

int contain(int big[], int size_b, int small[], int size_s) { int contains, k; for(int i = 0; i < size_b; i++){ contains = 1; if(big[i] == small[0]){ k = 1; for(int j = i + 1; j < size_b; j++){ if(k >= size_s - 1) { break; } if(big[j] != small[k]){ contains = 0; break; } k++; } if(contains) { return i; } } } return -1; }

注意:如果为空,它将返回const localStorage = await page.evaluate(() => localStorage.getItem("myKey"));

木偶5.3.1。