Python 3 Web Scrape和Beautiful Soup-Java 学习之路

我开始使用Python和美味的汤 .

我正在练习使用以下代码： -

import requests  
r = requests.get('https://www.autobarn.com.au/car-care-touring-accessories/car-care/washes?dir=asc&limit=48&order=name')

from bs4 import BeautifulSoup  
soup = BeautifulSoup(r.text, 'lxml')  
results = soup.find_all('div', class_='product-details')

records = []  
for result in results:  
    SKU = result.find('small',class_='text-muted').text.strip()
    DESC = result.find('strong').text.strip().upper()
    PRICE = result.find ('span',class_='price')
    URL = result.find('a')['href']
    records.append((SKU, DESC, PRICE, URL))

import pandas as pd  
df = pd.DataFrame(records, columns=['SKU','DESCRIPTION', 'RRP', 'URL'])  
df.to_csv('d:\\WEB SCRAPE TEST 4.csv', index=False, encoding='utf-8')

这可以很好地获取我所追求的信息 .

但是，对于价格而言，它会拖累所有周围的HTML信息 .

例如span class =“price”id =“product-price-1242”span class =“price”> $ 6.99

这似乎是由两个相同的标签引起的： - span class ='price'span class ='price'

虽然我可以在csv文件中清理价格数据，但有没有办法优化代码以获取价格？

提前致谢

2 回答

你可以这样做：

PRICE = result.find('span',class_='price').find('span',class_='price').text

您还必须决定如何处理没有可用价格的情况 . 也许是这样的：

if result.find('span',class_='price') is None:
    PRICE = "N/A"
else:
    PRICE = result.find('span',class_='price').find('span',class_='price').text

回复于 2024-05-17T17:28:34+08:00

试试这个 . 它应该解决这个问题：

import requests
from bs4 import BeautifulSoup  

url = 'https://www.autobarn.com.au/car-care-touring-accessories/car-care/washes?dir=asc&limit=48&order=name'

r = requests.get(url)
soup = BeautifulSoup(r.text, 'lxml')  
for result in soup.find_all('div', class_='product-details'): 
    SKU = result.find('small',class_='text-muted').text.strip()
    DESC = result.find('strong').text.strip().upper() 

    try:
        PRICE = result.select_one("[id^='product-price-'] span").text
    except AttributeError: PRICE = ""

    URL = result.find('a')['href']
    print(SKU, DESC, PRICE, URL)

回复于 2024-05-17T17:28:34+08:00

Python 3 Web Scrape和Beautiful Soup

2 回答

相关问题