Beautiful Soup returns empty list

Question

I am new to webscrapping. So I have been given a task to extract data from : Here

I am choosing dataset of \"comments\". Below is my code for scrapping.

Answer 1

Even though you were able to see the 'tbody', class_ = 'TableBody-kSbjpE jGqIxa' in the element inspector, the request that you make does not contain this class. See for yourself print(soup.prettify()). This is most likely because you're not requesting the correct url.

This may be not something you're aware of, but as a fyi: You don't actually need to scrape using BeautifulSoup, you can get a list of all the available datasets from the API. Once you have it installed and configured, you can get the dataset: kaggle datasets download -d . Here's more info if you wish to proceed with the API instead: https://github.com/Kaggle/kaggle-api