How do I extract table data in pairs using BeautifulSoup?

后端 未结 2 1834
無奈伤痛
無奈伤痛 2021-01-04 22:47

My data sample :

Google07/11/2001
相关标签:
2条回答
  • 2021-01-04 23:15

    Here is small variation of Sean answer if you need exactly what you wrote in question,

    table = soup.find("table", id = "history")
    
    rows = table.findAll('tr')
    
    data = ['|'.join([td.findNext(text=True) for td in tr.findAll("td")]) for tr in rows]
    print data
    
    0 讨论(0)
  • 2021-01-04 23:30

    List comprehension will make it easier:

    table = soup.find("table", id = "history")
    rows = table.findAll('tr')
    data = [[td.findChildren(text=True) for td in tr.findAll("td")] for tr in rows]
    # data now contains:
    [[u'Google', u'07/11/2001'],
     [u'Apple', u'27/08/2001'],
     [u'Microsoft', u'01/11/1991']]
    
    # If the data may contain extraneous whitespace you can clean it up
    # Additional processing could also be done - but once you hit much more
    # complex than this later maintainers, yourself included, will thank you
    # for using a series of for loops that call clearly named functions to perform
    # the work.
    data = [[u"".join(d).strip() for d in l] for l in data]
    
    # If you want to store it joined as name | company
    # then simply follow that up with:
    data = [u"|".join(d) for d in data]
    

    The list comprehension is basically a reverse for loop with aggregation:

    [[td.findNext(text=True) for td in tr.findAll("td")] for tr in rows]
    

    translates into*:

    final_list = []
    intermediate_list = []
    
    for tr in rows:
        for td in tr.findAll("td")
            intermediate_list.append(td.findNext(text=True))
    
        final_list.append(intermediate_list)
        intermediate_list = []
    
    data = final_list
    

    * Roughly - we are leaving out the awesomeness involving generators not building intermediate lists, since I can't add generators right now without cluttering the example.

    0 讨论(0)
提交回复
热议问题