Python get all the contents from a website to html file

Question

someone please help, i want to transfer all to contents from url to a html file can someone help me please? I have to use user-agent too!

Answer 1

because I don't know what site you need scrape so I say a few wasy

if site contains JS frontend and for laoding needed waiting then I recommend you use requests_html module which has method for rendering content

from requests_html import HTMLSession

url = "https://some-url.org"

with HTMLSession() as session:
    response = session.get(url)
    response.html.render() #  rendering JS code
    content = response.html.html #  full content

if site doesn't use JS for frontent then requests module is really good choice for you

import requests

url = "https://some-url.org"

response = requests.get(url)
content = response.content #  html content in bytes()

else you can use selenium webdriver but it works few slowly for python