首页 文章

在Selenium Python中使用Keys.PAGE_DOWN滚动

提问于
浏览
1

你好每个人都可以帮助我滚动https://www.grainger.com/category/black-pipe-fittings/pipe-fittings/pipe-tubing-and-fittings/plumbing/ecatalog/N-qu1?searchRedirect=products

我想用这个滚动它

actions = ActionChains(browser)
actions.send_keys(Keys.PAGE_DOWN)
actions.perform()

直到它到达卷轴的底部,它会找到一个元素“加载更多”

loadMoreButton = browser.find_element_by_css_selector(
                        ".btn.list-view__load-more.list-view__load-more--js")
loadMoreButton.click()

然后ponce点击它所具有的加载更多按钮再次执行滚动操作,然后再次执行loadmore操作,直到加载更多按钮不可用 .

我必须使用此页面向下操作,因为元素不会加载,直到页面滚动到元素,如果有人可以建议一些解决方案将有很大的帮助

2 回答

  • 0

    @PedroLobito我试图退出产品链接,你可以帮我这个


    在这种情况下不需要 selenium ,只需 sniff xhr 通过 developer tools 请求并直接进入黄金( json ) .


    url 产品结构如下:

    https://www.x.com/product/anything-Item#
    

    只需在网址末尾的json object中添加 Item # 值,例如:

    • https://www.x.com/product/anything-5P540

    • https://www.x.com/product/anything-5P541

    ...


    py3 示例(对于 py2 ,只需更改 format 语法):

    import json
    import requests
    
    main_cat = "WP7115916"
    sub_cat = "4836"
    
    x = requests.get(f"https://www.x.com/product/tableview/GRAINGER-APPROVED-Square-Head-Plugs-{main_cat}/_/N-qu1?searchRedirect=products&breadcrumbCatId={sub_cat}&s_pp=false").json()
    
    for p in x['records']:
        for childs in p['children']:
            for item in json.loads(childs['collapseValues']):
                url = f"https://www.x.com/product/lol-{item['sku']}"
                print(url)
    

    https://www.x.com/product/lol-5P540
    https://www.x.com/product/lol-5P541
    https://www.x.com/product/lol-5P542
    https://www.x.com/product/lol-5P543
    https://www.x.com/product/lol-5P544
    https://www.x.com/product/lol-5P545
    https://www.x.com/product/lol-5P546
    https://www.x.com/product/lol-5P547
    https://www.x.com/product/lol-5P548
    ...
    
  • 0

    要滚动页面 https://www.grainger.com/category/black-pipe-fittings/pipe-fittings/pipe-tubing-and-fittings/plumbing/ecatalog/N-qu1?searchRedirect=products ,直到它到达页面底部,它将找到一个文本为 View More 的元素,然后单击元素直到该元素不可用,您可以使用以下解决方案:

    • 代码块:
    from selenium import webdriver
    from selenium.webdriver.common.by import By
    from selenium.webdriver.support.ui import WebDriverWait
    from selenium.webdriver.support import expected_conditions as EC
    from selenium.common.exceptions import StaleElementReferenceException
    from selenium.common.exceptions import TimeoutException
    
    options = webdriver.ChromeOptions() 
    options.add_argument("start-maximized")
    options.add_argument('disable-infobars')
    browser=webdriver.Chrome(chrome_options=options, executable_path=r'C:\Utility\BrowserDrivers\chromedriver.exe')
    browser.get("https://www.grainger.com/category/black-pipe-fittings/pipe-fittings/pipe-tubing-and-fittings/plumbing/ecatalog/N-qu1?searchRedirect=products")
    while True:
        try:
            browser.execute_script("return arguments[0].scrollIntoView(true);", WebDriverWait(browser,10).until(EC.visibility_of_element_located((By.XPATH, "//a[@class='btn list-view__load-more list-view__load-more--js' and normalize-space()='View More']"))))
            browser.execute_script("arguments[0].click();", WebDriverWait(browser,10).until(EC.element_to_be_clickable((By.XPATH, "//a[@class='btn list-view__load-more list-view__load-more--js' and normalize-space()='View More']"))))
            print("View More button clicked")
        except (TimeoutException, StaleElementReferenceException) as e:
            print("No more View More buttons")
            break
    browser.quit()
    
    • 控制台输出:
    View More button clicked
    View More button clicked
    No more View More buttons
    

相关问题