Selenium takes lots of time to get dynamic page of given URL

前端 未结 2 1979
傲寒
傲寒 2021-01-27 06:51

I am doing a Project in Java. In this project I have to work with DOM. For that I first load a dynamic page of any given URL, by using Selenium. Then I parse them using

2条回答
  •  逝去的感伤
    2021-01-27 07:41

    Try this:

    public static void main(String[] args) throws IOException {
    
        // Selenium
        WebDriver driver = new FirefoxDriver(createFirefoxProfile());
        driver.get("ANY URL HERE");
        String html_content = driver.getPageSource();
        driver.close();
    
        // Jsoup makes DOM here by parsing HTML content
        // OPERATIONS USING DOM TREE
    }
    
    private static FirefoxProfile createFirefoxProfile() {
        File profileDir = new File("/tmp/firefox-profile-dir");
        if (profileDir.exists())
            return new FirefoxProfile(profileDir);
        FirefoxProfile firefoxProfile = new FirefoxProfile();
        File dir = firefoxProfile.layoutOnDisk();
        try {
            profileDir.mkdirs();
            FileUtils.copyDirectory(dir, profileDir);
        } catch (IOException e) {
            e.printStackTrace();
        }
        return firefoxProfile;
    }
    

    The createFireFoxProfile() method creates a profile if one doesn't exist. It uses if a profile already exists. So selenium doesn't need to create the profile-dir structure each and every time.

提交回复
热议问题