Request fully javascript rendered html source from a website and find all iframe tags

前端未结

关注

 1  738

I am currently trying to use selenium and BeautifulSoup to retrieve all iframe tags from a website. The problem is I am not getting all the iframes because there are inner h

相关标签:

1条回答

借酒劲吻你

2020-12-12 00:23

You can get all the <iframe> tags exclusively through Selenium with the following code block :

from selenium import webdriver

browser = webdriver.Firefox(executable_path=r'C:\Utility\BrowserDrivers\geckodriver.exe')
browser.get("https://reddit.com")
frames_tag = browser.find_elements_by_tag_name("iframe")
frames_xpath = browser.find_elements_by_xpath("//iframe")
frames_css = browser.find_elements_by_css_selector("iframe")
print("Frames detected through iframe tag are %s" %frames_tag)
print("Frames detected through xpath are %s" %frames_xpath)
print("Frames detected through css are %s" %frames_css)
browser.quit()

The output on my console is :

Frames detected through iframe tag are [<selenium.webdriver.firefox.webelement.FirefoxWebElement (session="98594106-54a6-4941-a6ab-cd9d92e9afa2", element="ead39d06-0e39-4b40-9425-a86a1fe88d4f")>, <selenium.webdriver.firefox.webelement.FirefoxWebElement (session="98594106-54a6-4941-a6ab-cd9d92e9afa2", element="1ce10f29-a620-4ce6-90e1-9da563046c70")>, <selenium.webdriver.firefox.webelement.FirefoxWebElement (session="98594106-54a6-4941-a6ab-cd9d92e9afa2", element="ba8493e4-8246-47a0-9ed4-3f51b8c0f133")>, <selenium.webdriver.firefox.webelement.FirefoxWebElement (session="98594106-54a6-4941-a6ab-cd9d92e9afa2", element="19c0f134-c243-47bd-96d1-6b06ff66a011")>, <selenium.webdriver.firefox.webelement.FirefoxWebElement (session="98594106-54a6-4941-a6ab-cd9d92e9afa2", element="95d78fa6-fb4f-4b7c-89c5-9b85965f0e4c")>, <selenium.webdriver.firefox.webelement.FirefoxWebElement (session="98594106-54a6-4941-a6ab-cd9d92e9afa2", element="e6d2d931-1f35-432f-8825-052e244fe798")>]
Frames detected through xpath are [<selenium.webdriver.firefox.webelement.FirefoxWebElement (session="98594106-54a6-4941-a6ab-cd9d92e9afa2", element="ead39d06-0e39-4b40-9425-a86a1fe88d4f")>, <selenium.webdriver.firefox.webelement.FirefoxWebElement (session="98594106-54a6-4941-a6ab-cd9d92e9afa2", element="1ce10f29-a620-4ce6-90e1-9da563046c70")>, <selenium.webdriver.firefox.webelement.FirefoxWebElement (session="98594106-54a6-4941-a6ab-cd9d92e9afa2", element="ba8493e4-8246-47a0-9ed4-3f51b8c0f133")>, <selenium.webdriver.firefox.webelement.FirefoxWebElement (session="98594106-54a6-4941-a6ab-cd9d92e9afa2", element="19c0f134-c243-47bd-96d1-6b06ff66a011")>, <selenium.webdriver.firefox.webelement.FirefoxWebElement (session="98594106-54a6-4941-a6ab-cd9d92e9afa2", element="95d78fa6-fb4f-4b7c-89c5-9b85965f0e4c")>, <selenium.webdriver.firefox.webelement.FirefoxWebElement (session="98594106-54a6-4941-a6ab-cd9d92e9afa2", element="e6d2d931-1f35-432f-8825-052e244fe798")>]
Frames detected through css are [<selenium.webdriver.firefox.webelement.FirefoxWebElement (session="98594106-54a6-4941-a6ab-cd9d92e9afa2", element="ead39d06-0e39-4b40-9425-a86a1fe88d4f")>, <selenium.webdriver.firefox.webelement.FirefoxWebElement (session="98594106-54a6-4941-a6ab-cd9d92e9afa2", element="1ce10f29-a620-4ce6-90e1-9da563046c70")>, <selenium.webdriver.firefox.webelement.FirefoxWebElement (session="98594106-54a6-4941-a6ab-cd9d92e9afa2", element="ba8493e4-8246-47a0-9ed4-3f51b8c0f133")>, <selenium.webdriver.firefox.webelement.FirefoxWebElement (session="98594106-54a6-4941-a6ab-cd9d92e9afa2", element="19c0f134-c243-47bd-96d1-6b06ff66a011")>, <selenium.webdriver.firefox.webelement.FirefoxWebElement (session="98594106-54a6-4941-a6ab-cd9d92e9afa2", element="95d78fa6-fb4f-4b7c-89c5-9b85965f0e4c")>, <selenium.webdriver.firefox.webelement.FirefoxWebElement (session="98594106-54a6-4941-a6ab-cd9d92e9afa2", element="e6d2d931-1f35-432f-8825-052e244fe798")>]

0 讨论(0)