我尝试从我的数组中逐页进行,但得到这个:
(node:4196)MaxListenersExceededWarning:检测到可能的EventEmitter内存泄漏 . 添加了11个请求监听器 . 使用emitter.setMaxListeners()增加限制(节点:4196)MaxListenersExceededWarning:检测到可能的EventEmitter内存泄漏 . 11个帧分离的听众adde d . 使用emitter.setMaxListeners()增加限制(节点:4196)MaxListenersExceededWarning:检测到可能的EventEmitter内存泄漏 . 11个生命周期事件监听器添加了ed . 使用emitter.setMaxListeners()增加限制(节点:4196)UnhandledPromiseRejectionWarning:错误:协议错误(Page.navigate):目标已关闭 . 在Promise(D:\ Kutz \ irrParse \ node_modules \ puppeteer \ lib \ Connection.js:198:56)在CDPSession.send的新Promise()处(D:\ Kutz \ irrParse \ node_modules \ puppeteer \ lib \ Connection.js) :197:12)在Page.goto(D:\ Kutz \ irrParse \ node_modules \ puppeteer \ lib \ Page.js)导航(D:\ Kutz \ irrParse \ node_modules \ puppeteer \ lib \ Page.js:520:39) :500:7)在Array.forEach()的uniqueLinks.forEach(D:\ Kutz \ irrParse \ scrape.js:26:16)D:\ Kutz \ irrParse \ scrape.js:25:15 at at process . _tickCallback(internal / process / next_tick.js:118:7)(node:4196)UnhandledPromiseRejectionWarning:未处理的承诺拒绝 . 这个错误源于在没有catch块的情况下抛出异步函数,或者拒绝未使用.catch()处理的promise . (r弹出ID:1)(节点:4196)[DEP0018]弃用警告:不推荐使用未处理的拒绝承诺 . 将来,promise拒绝未处理的离子将使用非零退出代码终止Node.js进程 . (node:4196)UnhandledPromiseRejectionWarning:错误:超出导航超时:在Promise.then超过30000ms(D:\ Kutz \ irrParse \ node_modules \ puppeteer \ lib \ NavigatorWatcher.js:71:21)at
const puppeteer = require("puppeteer");
var forEach = require('async-foreach').forEach;
const url = "https://reddit.com/r/programming";
const linkSelector = ".content a.title";
(async () => {
// Launch chrome process
const browser = await puppeteer.launch({headless: true});
const page = await browser.newPage();
await page.goto(url, { waitUntil: "load" });
// This runs the `document.querySelectorAll` within the page and passes
// the result to function
const links = await page.$$eval(linkSelector, links => {
return links.map((link) => link.href);
});
// Make sure we get the unique set of links only
const uniqueLinks = [...links];
//console.log(uniqueLinks[0]);
uniqueLinks.forEach(async (link) => {
await page.goto(link, { waitUntil: "load" });
});
// Kill the browser process
await browser.close();
})();
错误抛出forEach()
1 回答
不幸的是,
Array.prototype.forEach
's iterator function is not executed in an async manner as you would expect when defining it as async. Using a for loop should work for what you'正在努力做到 .