Keep getting attribute error using Praw to scrape specific search term in subreddit

試著忘記壹切 提交于 2021-02-11 12:28:01

问题


I'm an extreme novice. My goal is to scrape reddit posts and comments from the subreddit r/Coronavirus by the searchterm "smokers". I keep getting "AttributeError: 'MoreComments' object has no attribute 'body'" referring to the "commentsDict["Body"].append(topLevelComments.body)" line. There are two other lines using the (topLevelComments.author, .score, and .body) that keep causing it to crash. When I comment out all of the lines with ".append(topLevelComments. )it returns: ValueError("arrays must all be same length") I'm losing my mind as this code worked fine 2 days ago. Please help, code below. I commented out the lines causing trouble, but not sure what to do about error 2 either. One step at a time though:

commentsDict = {"Post" : [], "Post Score" : [], "No of Comments":[], "Post Date":[], \
                "Body":[],"Score":[],"Date":[],"Author":[], "id":[], "p_auth":[], "Post body":[]}

for submission in reddit.subreddit("Coronavirus").search("smoker"):
    submission.comment_sort = 'new'
    topLevelComments = list(submission.comments)
    for topLevelComments in submission.comments:
        commentsDict["Post"].append(submission.title)#title of post with comment
        commentsDict["Post Score"].append(submission.score)
        commentsDict["Post body"].append(submission.selftext)
        commentsDict["id"].append(submission.id)
        commentsDict["p_auth"].append(submission.author)
        commentsDict["No of Comments"].append(submission.num_comments)
        date = submission.created_utc
        timestamp = datetime.datetime.fromtimestamp(date)
        commentsDict["Post Date"].append(timestamp.strftime('%Y-%m-%D %H:%M:%S'))
        #commentsDict["Body"].append(topLevelComments.body)
        #commentsDict["Score"].append(topLevelComments.score)
        #date = topLevelComments.created
        timestamp = datetime.datetime.fromtimestamp(date)
        commentsDict["Date"].append(timestamp.strftime('%Y-%m-%D %H:%M:%S'))
        #commentsDict["Author"].append(topLevelComments.author)

commentsDF = pd.DataFrame(commentsDict)

commentsDF.to_csv('smoker_covid.csv', index=True) 

来源:https://stackoverflow.com/questions/61415483/keep-getting-attribute-error-using-praw-to-scrape-specific-search-term-in-subred

标签
易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!