使用Python Selenium进行AmCharts抓取

该图形不在Selenium view端口中,因此首先我们必须处理垂直滚动到所需图形的问题,然后我看到了 //*[name()='tspan'] xpath包含水平和垂直值,这些值也存在于UI中。

代码:

driver_path = r'D:\\chromedriver.exe'

driver = webdriver.Chrome(driver_path)
driver.maximize_window()
wait = WebDriverWait(driver, 30)

driver.get("https://eg.pricena.com/en/product/oppo-reno-5g-price-in-egypt")

driver.execute_script("window.scrollTo(0, 1000)")

ele = driver.find_element_by_xpath("//*[name()='svg']")
driver.execute_script("arguments[0].scrollIntoView(true);", ele)

char_val = []
for elem in driver.find_elements(By.XPATH, "//*[name()='tspan']"):
    print(elem.text)

进口:

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

输出:

Mar
May
Jul
Sep
Nov
Mar
May
Jul
Sep
Nov
5,000
5,500
6,000
6,500
7,000
7,500
8,000

Process finished with exit code 0

你需要补充 return 给你的 execute_script 调用以访问脚本中的值,例如。 driver.execute_script("return AmCharts.charts[0].dataProvider")

完整代码如下:

from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait

driver_path = r"path/to/chromedriver"

driver = webdriver.Chrome(driver_path)
driver.maximize_window()
wait = WebDriverWait(driver, 30)

driver.get("https://eg.pricena.com/en/product/oppo-reno-5g-price-in-egypt")

# scroll into the div so that the chart will render
driver.execute_script("document.getElementById('product_pricechart').scrollIntoView()")

# wait until the chart div has been rendered before accessing the data provider
wait.until(lambda x: x.find_element_by_class_name("amcharts-chart-div").is_displayed())

# display chart data
print(driver.execute_script("return AmCharts.charts[0].dataProvider"))

driver.close()