首页 文章

节点js Puppeteer转到页面数组

提问于
浏览
0

我尝试从我的数组中逐页进行,但得到这个:

(node:4196)MaxListenersExceededWarning:检测到可能的EventEmitter内存泄漏 . 添加了11个请求监听器 . 使用emitter.setMaxListeners()增加限制(节点:4196)MaxListenersExceededWarning:检测到可能的EventEmitter内存泄漏 . 11个帧分离的听众adde d . 使用emitter.setMaxListeners()增加限制(节点:4196)MaxListenersExceededWarning:检测到可能的EventEmitter内存泄漏 . 11个生命周期事件监听器添加了ed . 使用emitter.setMaxListeners()增加限制(节点:4196)UnhandledPromiseRejectionWarning:错误:协议错误(Page.navigate):目标已关闭 . 在Promise(D:\ Kutz \ irrParse \ node_modules \ puppeteer \ lib \ Connection.js:198:56)在CDPSession.send的新Promise()处(D:\ Kutz \ irrParse \ node_modules \ puppeteer \ lib \ Connection.js) :197:12)在Page.goto(D:\ Kutz \ irrParse \ node_modules \ puppeteer \ lib \ Page.js)导航(D:\ Kutz \ irrParse \ node_modules \ puppeteer \ lib \ Page.js:520:39) :500:7)在Array.forEach()的uniqueLinks.forEach(D:\ Kutz \ irrParse \ scrape.js:26:16)D:\ Kutz \ irrParse \ scrape.js:25:15 at at process . _tickCallback(internal / process / next_tick.js:118:7)(node:4196)UnhandledPromiseRejectionWarning:未处理的承诺拒绝 . 这个错误源于在没有catch块的情况下抛出异步函数,或者拒绝未使用.catch()处理的promise . (r弹出ID:1)(节点:4196)[DEP0018]弃用警告:不推荐使用未处理的拒绝承诺 . 将来,promise拒绝未处理的离子将使用非零退出代码终止Node.js进程 . (node:4196)UnhandledPromiseRejectionWarning:错误:超出导航超时:在Promise.then超过30000ms(D:\ Kutz \ irrParse \ node_modules \ puppeteer \ lib \ NavigatorWatcher.js:71:21)at

const puppeteer = require("puppeteer");
var forEach = require('async-foreach').forEach;


const url = "https://reddit.com/r/programming";
const linkSelector = ".content a.title";

(async () => {
  // Launch chrome process
  const browser = await puppeteer.launch({headless: true});
  const page = await browser.newPage();

  await page.goto(url, { waitUntil: "load" });

  // This runs the `document.querySelectorAll` within the page and passes
  // the result to function
  const links = await page.$$eval(linkSelector, links => {
    return links.map((link) => link.href);
  });

  // Make sure we get the unique set of links only
  const uniqueLinks = [...links];
  //console.log(uniqueLinks[0]);

  uniqueLinks.forEach(async (link) => {
    await page.goto(link, { waitUntil: "load" });
  });

  // Kill the browser process
  await browser.close();
})();

错误抛出forEach()

1 回答

  • 2

    不幸的是, Array.prototype.forEach 's iterator function is not executed in an async manner as you would expect when defining it as async. Using a for loop should work for what you'正在努力做到 .

    for (let i = 0; i < uniqueLinks.length; i ++) {
      await page.goto(uniqueLinks[i], { waitUntil: "load" });
    }
    

相关问题