如何正确地使用 beautifulsoup 来抓取网页元素？

正如 Manos Kounelakis 所建议的，您可能正在寻找的是BeautifulSoup HTML 元素的属性。此外，根据类而不是text元素来拆分 html 更为自然，因为元素对应于屏幕上的每个可视卡片单元。以下是一些可以相当漂亮地打印信息的代码：cardrowcard

import requests
from bs4 import BeautifulSoup as bs

url = "https://ihgfdelhifair.in/mis/Exhibitors"

page = requests.get(url)
soup = bs(page.text, features="html5lib")

cards = soup.find_all("div", class_="card")

for element in cards:
    title = element.find("h4").text
    other_info = [" ".join(elem.text.split()) for elem in element.find_all("p")]
    print("Title:", title)
    for info in other_info:
        print(info)
    print("-" * 80)

您正在抓取的页面使用 JavaScript 呈现。这意味着当您通过 HTTP(S) GET 访问 HTML 时，它可能尚未完全准备好进行解析。

如果您使用硒，您可能会获得更好的结果，您可以这样做：

from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver import ChromeOptions

options = ChromeOptions()
options.add_argument("--headless=true")
url = "https://ihgfdelhifair.in/mis/Exhibitors"

labels = {
    "email",
    "contact",
    "state",
    "city"
}

def text(e):
    if t := e.text.strip():
        return t
    if (t := e.get_attribute("textContent")) is not None:
        return t.strip()
    return ""

with webdriver.Chrome(options) as driver:
    driver.get(url)
    for card in WebDriverWait(driver, 10).until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, "div.card"))):
        wait = WebDriverWait(card, 10)
        if title := wait.until(EC.presence_of_element_located((By.CSS_SELECTOR, "h4"))):
            print("Title :", text(title))
        for p in wait.until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, "p"))):
            t = text(p).split()
            if t[0].lower() in labels:
                print(*t)
        print()

示例输出（为简洁起见，不完整）：

Title : 1 ARTIFACT DECOR (INDIA)
Email : [email protected]
Contact Person : SHEENU
State : UTTAR PRADESH
City : AGRA

Title : 10G HOUSE OF CRAFT
Email : [email protected]
Contact Person : MR. SUSHIL KUMAR
State : UTTAR PRADESH
City : MORADABAD

Title : 2 S COLLECTION
Email : [email protected],[email protected]
Contact Person : MR. SHARIQ AHMAD SIDDIQUI
State : UTTAR PRADESH
City : MORADABAD