Using Python and BeautifulSoup (saved webpage source codes into a local file)

Question

I am using Python 2.7 + BeautifulSoup 4.3.2.

I am trying to use Python and BeautifulSoup to pick up information on a webpage. Because the webpage is in the company w

Answer 1

The best way to open a local file with BeautifulSoup is to pass it an open file handler directly. http://www.crummy.com/software/BeautifulSoup/bs4/doc/#making-the-soup

from bs4 import BeautifulSoup

soup = BeautifulSoup(open("C:\\example.html"), "html.parser")

for city in soup.find_all('span', {'class' : 'city-sh'}):
    print(city)

Answer 2

With Chandan's help, the problem has been solved. All the credits shall go to him. :)

the "urllib2.url" is useless here.

from bs4 import BeautifulSoup
import re
# import urllib2

url = "C:\example.html"
page = open(url)
soup = BeautifulSoup(page.read())

cities = soup.find_all('span', {'class' : 'city-sh'})

for city in cities:
    print city

Answer 3

You can try using lxml parser also. Here is an example for your html data.

from lxml.html import fromstring
import lxml.html as PARSER

data = open('example.html').read()
root = PARSER.fromstring(data)

for ele in root.getiterator():
    if ele.tag == "td":
        print ele.text_content()

o/p: port_new_cape 452 South May 09, 1997 Jan 23, 2009 12:05 pm