问题
I am scraping a web site using selenium "https://www.medline.com/catalog/category-products.jsp?itemId=Z05-CA02_03&N=111079+4294770643&iclp=Z05-CA02_03"
For single page and single product i am able to scrape by passing the product url but i am trying to do so by selenium i.e auto selection of product an page after select all the product one by one and it should move to next page and after opening product details page it should scrape which is done by beautiful soup here is product url from the base url "https://www.medline.com/product/SensiCare-Powder-Free-Nitrile-Exam-Gloves/SensiCare/Z05-PF00342?question=&index=P1&indexCount=1"
Here is my code:
chromeOptions = webdriver.ChromeOptions()
chromeOptions.add_experimental_option('useAutomationExtension', False)
driver = webdriver.Chrome(executable_path='C:/Users/ptiwar34/Documents/chromedriver.exe', chrome_options=chromeOptions, desired_capabilities=chromeOptions.to_capabilities())
driver.get("https://www.medline.com/catalog/category-products.jsp?itemId=Z05-CA02_03&N=111079+4294770643&iclp=Z05-CA02_03")
while True:
try:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//div[contains(@class, 'resultGalleryViewRow')]//div[@class='medGridProdTitle']//a[contains(@href]"))).click()
print("Clicked for next page")
except TimeoutException:
print("No more pages")
break
driver.quit()
Here it does not throws error
It does not open page for each product , I want to open each product in new tab and after scrapping it delete and open the new tab for a new product
回答1:
From the webpage https://www.medline.com/catalog/category-products.jsp?itemId=Z05-CA02_03&N=111079+4294770643&iclp=Z05-CA02_03 to open each product in new tab and scrap it you have to induce WebDriverWait for the number_of_windows_to_be(2) and you can use the following Locator Strategies:
Code Block:
from selenium import webdriver from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.common.by import By from selenium.webdriver.support import expected_conditions as EC import time chrome_options = webdriver.ChromeOptions() chrome_options.add_argument("start-maximized") driver = webdriver.Chrome(options=chrome_options, executable_path=r'C:\WebDrivers\chromedriver.exe') driver.get("https://www.medline.com/catalog/category-products.jsp?itemId=Z05-CA02_03&N=111079+4294770643&iclp=Z05-CA02_03") my_hrefs = [my_elem.get_attribute("href") for my_elem in WebDriverWait(driver, 5).until(EC.visibility_of_all_elements_located((By.XPATH, "//div[contains(@class, 'resultGalleryViewRow')]//div[@class='medGridProdTitle']//a")))] windows_before = driver.current_window_handle # Store the parent_window_handle for future use for my_href in my_hrefs: driver.execute_script("window.open('" + my_href +"');") WebDriverWait(driver, 10).until(EC.number_of_windows_to_be(2)) # Induce WebDriverWait for the number_of_windows_to_be 2 windows_after = driver.window_handles new_window = [x for x in windows_after if x != windows_before][0] # Identify the newly opened window driver.switch_to.window(new_window) # switch_to the new window time.sleep(3) # perform your webscrapping here print(driver.title) # print the page title or your perform your webscrapping driver.close() # close the window driver.switch_to.window(windows_before) # switch_to the parent_window_handle driver.quit() #quit your programConsole Output:
SensiCare Powder-Free Nitrile Exam Gloves | Medline Industries, Inc. MediGuard Vinyl Synthetic Exam Gloves | Medline Industries, Inc. CURAD Stretch Vinyl Exam Gloves | Medline Industries, Inc. CURAD Nitrile Exam Gloves | Medline Industries, Inc. SensiCare Ice Blue Powder-Free Nitrile Exam Gloves | Medline Industries, Inc. MediGuard Synthetic Exam Gloves | Medline Industries, Inc. Accutouch Synthetic Exam Gloves | Medline Industries, Inc. Aloetouch Ice Powder-Free Nitrile Exam Gloves | Medline Industries, Inc. Aloetouch 3G Powder-Free Synthetic Exam Gloves | Medline Industries, Inc. SensiCare Powder-Free Stretch Vinyl Sterile Exam Gloves | Medline Industries, Inc. CURAD Powder-Free Textured Latex Exam Gloves | Medline Industries, Inc. Accutouch Chemo Nitrile Exam Gloves | Medline Industries, Inc. Aloetouch 12" Powder-Free Nitrile Exam Gloves | Medline Industries, Inc. Ultra Stretch Synthetic Exam Gloves | Medline Industries, Inc. Generation Pink 3G Synthetic Exam Gloves | Medline Industries, Inc. SensiCare Extended Cuff Powder-Free Nitrile Exam Gloves | Medline Industries, Inc. Eudermic MP High-Risk Powder-Free Latex Exam Gloves | Medline Industries, Inc. Aloetouch Powder-Free Latex Exam Gloves | Medline Industries, Inc. CURAD Powder-Free Nitrile Exam Gloves | Medline Industries, Inc. Medline Sterile Powder-Free Latex Exam Gloves | Medline Industries, Inc. SensiCare Silk Powder-Free Nitrile Exam Gloves | Medline Industries, Inc. Medline Sterile Powder-Free Latex Exam Glove Pairs | Medline Industries, Inc. MediGuard 2.0 Nitrile Exam Gloves | Medline Industries, Inc. Designer Boxed Vinyl Exam Gloves | Medline Industries, Inc.
You can find a relevant discussion in How to open multiple hrefs within a webtable to scrape through selenium
回答2:
You do not need selenium to scrape the products, instead, simple requests will suffice. This solution can simply iterate over the pagination and return all product listings for each page:
from bs4 import BeautifulSoup as soup
d = soup(requests.get('https://www.medline.com/catalog/category-products.jsp?itemId=Z05-CA02_03&N=111079+4294770643&iclp=Z05-CA02_03').text, 'html.parser')
def get_p(_d):
return [{'img':i.img['src'], 'link':i.a['href'], 'title':i.find('div', {'class':'medGridProdTitle'}).text, } for i in _d.find_all('div', {'class':'product'})]
r = [get_p(d)]+[get_p(soup(requests.get(f'https://www.medline.com{i["href"]}').text, 'html.parser')) for i in d.find('div', {'class':'pagination'}).find_all('a')]
Output:
[[{'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_08/PF00342/PF00342_PRI05.JPG?quality=high&width=180&height=180', 'link': '/product/SensiCare-Powder-Free-Nitrile-Exam-Gloves/SensiCare/Z05-PF00342;ecomsessionid=8YLYmH_8oOnAQOELjXzF4KY84kOVBXYqYWmHXXtR.OC2-b2b?question=&index=P1&indexCount=1', 'title': '\nSensiCare Powder-Free Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_02/CA02_03_02_05/PF00350/PF00350_PRI04.JPG?quality=high&width=180&height=180', 'link': '/product/MediGuard-Vinyl-Synthetic-Exam-Gloves/MediGuard/Z05-PF00350;ecomsessionid=8YLYmH_8oOnAQOELjXzF4KY84kOVBXYqYWmHXXtR.OC2-b2b?question=&index=P2&indexCount=2', 'title': '\nMediGuard Vinyl Synthetic Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_02/CA02_03_02_03/PF00348/PF00348_PRI05.JPG?quality=high&width=180&height=180', 'link': '/product/CURAD-Stretch-Vinyl-Exam-Gloves/Curad/Z05-PF00348;ecomsessionid=8YLYmH_8oOnAQOELjXzF4KY84kOVBXYqYWmHXXtR.OC2-b2b?question=&index=P3&indexCount=3', 'title': '\nCURAD Stretch Vinyl Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_03/PF00331/PF00331_PRI03.JPG?quality=high&width=180&height=180', 'link': '/product/CURAD-Nitrile-Exam-Gloves/Non-Navigable-For-Boost/Z05-PF00331;ecomsessionid=8YLYmH_8oOnAQOELjXzF4KY84kOVBXYqYWmHXXtR.OC2-b2b?question=&index=P4&indexCount=4', 'title': '\nCURAD Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_08/PF56141/PF56141_PRI10.JPG?quality=high&width=180&height=180', 'link': '/product/SensiCare-Ice-Blue-Powder-Free-Nitrile-Exam-Gloves/SensiCare/Z05-PF56141;ecomsessionid=8YLYmH_8oOnAQOELjXzF4KY84kOVBXYqYWmHXXtR.OC2-b2b?question=&index=P5&indexCount=5', 'title': '\nSensiCare Ice Blue Powder-Free Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_02/CA02_03_02_05/PF19768/PF19768_PRI02.JPG?quality=high&width=180&height=180', 'link': '/product/MediGuard-Synthetic-Exam-Gloves/MediGuard/Z05-PF19768;ecomsessionid=8YLYmH_8oOnAQOELjXzF4KY84kOVBXYqYWmHXXtR.OC2-b2b?question=&index=P6&indexCount=6', 'title': '\nMediGuard Synthetic Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_02/CA02_03_02_01/PF00344/PF00344_PRI04.JPG?quality=high&width=180&height=180', 'link': '/product/Accutouch-Synthetic-Exam-Gloves/Accutouch/Z05-PF00344;ecomsessionid=8YLYmH_8oOnAQOELjXzF4KY84kOVBXYqYWmHXXtR.OC2-b2b?question=&index=P7&indexCount=7', 'title': '\nAccutouch Synthetic Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_02/PF00327/PF00327_PRI03.JPG?quality=high&width=180&height=180', 'link': '/product/Aloetouch-Ice-Powder-Free-Nitrile-Exam-Gloves/Aloetouch/Z05-PF00327;ecomsessionid=8YLYmH_8oOnAQOELjXzF4KY84kOVBXYqYWmHXXtR.OC2-b2b?question=&index=P8&indexCount=8', 'title': '\nAloetouch Ice Powder-Free Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_02/CA02_03_02_02/PF00345/PF00345_PRI05.JPG?quality=high&width=180&height=180', 'link': '/product/Aloetouch-3G-Powder-Free-Synthetic-Exam-Gloves/Aloetouch/Z05-PF00345;ecomsessionid=8YLYmH_8oOnAQOELjXzF4KY84kOVBXYqYWmHXXtR.OC2-b2b?question=&index=P9&indexCount=9', 'title': '\nAloetouch 3G Powder-Free Synthetic Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_07/CA02_03_07_01/PF00370/PF00370_PRI02.JPG?quality=high&width=180&height=180', 'link': '/product/SensiCare-Powder-Free-Stretch-Vinyl-Sterile-Exam-Gloves/Sensicare/Z05-PF00370;ecomsessionid=8YLYmH_8oOnAQOELjXzF4KY84kOVBXYqYWmHXXtR.OC2-b2b?question=&index=P10&indexCount=10', 'title': '\nSensiCare Powder-Free Stretch Vinyl Sterile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_03/CA02_03_03_02/PF00356/PF00356_PRI05.JPG?quality=high&width=180&height=180', 'link': '/product/CURAD-Powder-Free-Textured-Latex-Exam-Gloves/Non-Navigable-For-Boost/Z05-PF00356;ecomsessionid=8YLYmH_8oOnAQOELjXzF4KY84kOVBXYqYWmHXXtR.OC2-b2b?question=&index=P11&indexCount=11', 'title': '\nCURAD Powder-Free Textured Latex Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_01/PF00326/PF00326_PRI01.JPG?quality=high&width=180&height=180', 'link': '/product/Accutouch-Chemo-Nitrile-Exam-Gloves/Accutouch/Z05-PF00326;ecomsessionid=8YLYmH_8oOnAQOELjXzF4KY84kOVBXYqYWmHXXtR.OC2-b2b?question=&index=P12&indexCount=12', 'title': '\nAccutouch Chemo Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_06/CA02_03_06_01/PF00367/PF00367_PRI01.JPG?quality=high&width=180&height=180', 'link': '/product/Aloetouch-12-Powder-Free-Nitrile-Exam-Gloves/Aloetouch/Z05-PF00367;ecomsessionid=8YLYmH_8oOnAQOELjXzF4KY84kOVBXYqYWmHXXtR.OC2-b2b?question=&index=P13&indexCount=13', 'title': '\nAloetouch 12" Powder-Free Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_02/CA02_03_02_07/PF00353/PF00353_PRI03.JPG?quality=high&width=180&height=180', 'link': '/product/Ultra-Stretch-Synthetic-Exam-Gloves/Ultra/Z05-PF00353;ecomsessionid=8YLYmH_8oOnAQOELjXzF4KY84kOVBXYqYWmHXXtR.OC2-b2b?question=&index=P14&indexCount=14', 'title': '\nUltra Stretch Synthetic Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_02/CA02_03_02_04/PF00349/PF00349_PRI07.JPG?quality=high&width=180&height=180', 'link': '/product/Generation-Pink-3G-Synthetic-Exam-Gloves/Generation-Pink/Z05-PF00349;ecomsessionid=8YLYmH_8oOnAQOELjXzF4KY84kOVBXYqYWmHXXtR.OC2-b2b?question=&index=P15&indexCount=15', 'title': '\nGeneration Pink 3G Synthetic Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_08/PF00339/PF00339_PRI05.JPG?quality=high&width=180&height=180', 'link': '/product/SensiCare-Extended-Cuff-Powder-Free-Nitrile-Exam-Gloves/SensiCare/Z05-PF00339;ecomsessionid=8YLYmH_8oOnAQOELjXzF4KY84kOVBXYqYWmHXXtR.OC2-b2b?question=&index=P16&indexCount=16', 'title': '\nSensiCare Extended Cuff Powder-Free Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_03/CA02_03_03_04/PF00358/PF00358_PRI04.JPG?quality=high&width=180&height=180', 'link': '/product/Eudermic-MP-High-Risk-Powder-Free-Latex-Exam-Gloves/Eudermic-MP/Z05-PF00358;ecomsessionid=8YLYmH_8oOnAQOELjXzF4KY84kOVBXYqYWmHXXtR.OC2-b2b?question=&index=P17&indexCount=17', 'title': '\nEudermic MP High-Risk Powder-Free Latex Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_03/CA02_03_03_01/PF00354/PF00354_PRI03.JPG?quality=high&width=180&height=180', 'link': '/product/Aloetouch-Powder-Free-Latex-Exam-Gloves/Aloetouch/Z05-PF00354;ecomsessionid=8YLYmH_8oOnAQOELjXzF4KY84kOVBXYqYWmHXXtR.OC2-b2b?question=&index=P18&indexCount=18', 'title': '\nAloetouch Powder-Free Latex Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_03/PF61953/PF61953_PRI01.JPG?quality=high&width=180&height=180', 'link': '/product/CURAD-Powder-Free-Nitrile-Exam-Gloves/Curad/Z05-PF61953;ecomsessionid=8YLYmH_8oOnAQOELjXzF4KY84kOVBXYqYWmHXXtR.OC2-b2b?question=&index=P19&indexCount=19', 'title': '\nCURAD Powder-Free Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_08/CA02_03_08_02/PF00373/PF00373_PRI01.JPG?quality=high&width=180&height=180', 'link': '/product/Medline-Sterile-Powder-Free-Latex-Exam-Gloves/Medline/Z05-PF00373;ecomsessionid=8YLYmH_8oOnAQOELjXzF4KY84kOVBXYqYWmHXXtR.OC2-b2b?question=&index=P20&indexCount=20', 'title': '\nMedline Sterile Powder-Free Latex Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_08/PF00338/PF00338_PRI04.JPG?quality=high&width=180&height=180', 'link': '/product/SensiCare-Silk-Powder-Free-Nitrile-Exam-Gloves/SensiCare/Z05-PF00338;ecomsessionid=8YLYmH_8oOnAQOELjXzF4KY84kOVBXYqYWmHXXtR.OC2-b2b?question=&index=P21&indexCount=21', 'title': '\nSensiCare Silk Powder-Free Nitrile Exam Gloves'}, {'img': '/media/catalog/sku/MDS/MDS194034_PRI03.JPG?quality=high&width=180&height=180', 'link': '/product/Medline-Sterile-Powder-Free-Latex-Exam-Glove-Pairs/Medline/Z05-PF00372;ecomsessionid=8YLYmH_8oOnAQOELjXzF4KY84kOVBXYqYWmHXXtR.OC2-b2b?question=&index=P22&indexCount=22', 'title': '\nMedline Sterile Powder-Free Latex Exam Glove Pairs'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_06/PF21246/PF21246_PRI03.JPG?quality=high&width=180&height=180', 'link': '/product/MediGuard-20-Nitrile-Exam-Gloves/MediGuard/Z05-PF21246;ecomsessionid=8YLYmH_8oOnAQOELjXzF4KY84kOVBXYqYWmHXXtR.OC2-b2b?question=&index=P23&indexCount=23', 'title': '\nMediGuard 2.0 Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_02/CA02_03_02_10/PF13890/PF13890_PRI03.JPG?quality=high&width=180&height=180', 'link': '/product/Designer-Boxed-Vinyl-Exam-Gloves/Medline/Z05-PF13890;ecomsessionid=8YLYmH_8oOnAQOELjXzF4KY84kOVBXYqYWmHXXtR.OC2-b2b?question=&index=P24&indexCount=24', 'title': '\nDesigner Boxed Vinyl Exam Gloves'}], [{'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_09/PF00343/PF00343_PRI03.JPG?quality=high&width=180&height=180', 'link': '/product/Venom-Nonsterile-Powder-Free-Nitrile-Exam-Gloves/Venom/Z05-PF00343;ecomsessionid=j9Vn9v-IKFBcIb1m4Mvtv0eGE-m7t9iR0CPpgwsx.OC1-b2b?question=&index=P25&indexCount=25', 'title': '\nVenom Nonsterile Powder-Free Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_03/CA02_03_03_07/PF00361/PF00361_PRI02.JPG?quality=high&width=180&height=180', 'link': '/product/Spruce-Nonsterile-Powder-Free-Latex-Exam-Gloves/Spruce/Z05-PF00361;ecomsessionid=j9Vn9v-IKFBcIb1m4Mvtv0eGE-m7t9iR0CPpgwsx.OC1-b2b?question=&index=P26&indexCount=26', 'title': '\nSpruce Nonsterile Powder-Free Latex Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_10/PF21875/PF21875_PRI04.JPG?quality=high&width=180&height=180', 'link': '/product/Solstice-Powder-Free-Nitrile-Exam-Gloves/Medline/Z05-PF21875;ecomsessionid=j9Vn9v-IKFBcIb1m4Mvtv0eGE-m7t9iR0CPpgwsx.OC1-b2b?question=&index=P27&indexCount=27', 'title': '\nSolstice Powder-Free Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_05/PF00333/PF00333_PRI05.JPG?quality=high&width=180&height=180', 'link': '/product/Generation-Pink-Pearl-Nitrile-Exam-Gloves/Generation-Pink/Z05-PF00333;ecomsessionid=j9Vn9v-IKFBcIb1m4Mvtv0eGE-m7t9iR0CPpgwsx.OC1-b2b?question=&index=P28&indexCount=28', 'title': '\nGeneration Pink Pearl Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_10/PF149180/PF149180_PRI02.JPG?quality=high&width=180&height=180', 'link': '/product/FitGuard-Prime-Nitrile-Exam-Gloves/Medline/Z05-PF149180;ecomsessionid=j9Vn9v-IKFBcIb1m4Mvtv0eGE-m7t9iR0CPpgwsx.OC1-b2b?question=&index=P29&indexCount=29', 'title': '\nFitGuard Prime Nitrile Exam Gloves'}, {'img': '/media/catalog/sku/MDS/MDS198314_PRI03.JPG?quality=high&width=180&height=180', 'link': '/product/Medline-Sterile-Powder-Free-Nitrile-Exam-Glove-Pairs/Medline/Z05-PF00369;ecomsessionid=j9Vn9v-IKFBcIb1m4Mvtv0eGE-m7t9iR0CPpgwsx.OC1-b2b?question=&index=P30&indexCount=30', 'title': '\nMedline Sterile Powder-Free Nitrile Exam Glove Pairs'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_06/CA02_03_06_01/PF00368/PF00368_PRI01.JPG?quality=high&width=180&height=180', 'link': '/product/Aloetouch-9-Powder-Free-Nitrile-Single-Exam-Gloves/Aloetouch/Z05-PF00368;ecomsessionid=j9Vn9v-IKFBcIb1m4Mvtv0eGE-m7t9iR0CPpgwsx.OC1-b2b?question=&index=P31&indexCount=31', 'title': '\nAloetouch 9" Powder-Free Nitrile Single Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_10/PF64087/PF64087_PRI04.JPG?quality=high&width=180&height=180', 'link': '/product/SmartGuard-Powder-Free-Nitrile-Exam-Gloves/Non-Navigable-For-Boost/Z05-PF64087;ecomsessionid=j9Vn9v-IKFBcIb1m4Mvtv0eGE-m7t9iR0CPpgwsx.OC1-b2b?question=&index=P32&indexCount=32', 'title': '\nSmartGuard Powder-Free Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_02/CA02_03_02_10/PF112126/PF112126_PRI07.JPG?quality=high&width=180&height=180', 'link': '/product/Glide-On-Powder-Free-Vinyl-Exam-Gloves/Medline/Z05-PF112126;ecomsessionid=j9Vn9v-IKFBcIb1m4Mvtv0eGE-m7t9iR0CPpgwsx.OC1-b2b?question=&index=P33&indexCount=33', 'title': '\nGlide-On Powder-Free Vinyl Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_02/CA02_03_02_10/PF84819/PF84819_PRI05.JPG?quality=high&width=180&height=180', 'link': '/product/FitGuard-Stretch-Vinyl-Exam-Gloves/Medline/Z05-PF84819;ecomsessionid=j9Vn9v-IKFBcIb1m4Mvtv0eGE-m7t9iR0CPpgwsx.OC1-b2b?question=&index=P34&indexCount=34', 'title': '\nFitGuard Stretch Vinyl Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_13/PF129312/PF129312_PRI04.JPG?quality=high&width=180&height=180', 'link': '/product/CS-Pro-16-Cuff-Nitrile-Exam-Gloves/Branded-Nonsterile-Nitrile-Exam-Gloves/Z05-PF129312;ecomsessionid=j9Vn9v-IKFBcIb1m4Mvtv0eGE-m7t9iR0CPpgwsx.OC1-b2b?question=&index=P35&indexCount=35', 'title': '\nCS Pro 16" Cuff Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_10/PF84816/PF84816_PRI03.JPG?quality=high&width=180&height=180', 'link': '/product/FitGuard-Nitrile-Exam-Gloves/Medline/Z05-PF84816;ecomsessionid=j9Vn9v-IKFBcIb1m4Mvtv0eGE-m7t9iR0CPpgwsx.OC1-b2b?question=&index=P36&indexCount=36', 'title': '\nFitGuard Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_02/PF81028/PF81028_PRI06.JPG?quality=high&width=180&height=180', 'link': '/product/Medline-Professional-Nitrile-Exam-Gloves-with-Aloe/Aloetouch/Z05-PF81028;ecomsessionid=j9Vn9v-IKFBcIb1m4Mvtv0eGE-m7t9iR0CPpgwsx.OC1-b2b?question=&index=P37&indexCount=37', 'title': '\nMedline Professional Nitrile Exam Gloves with Aloe'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_06/CA02_03_06_02/PF149184/PF149184_PRI01.JPG?quality=high&width=180&height=180', 'link': '/product/Medline-Extended-Cuff-Nitrile-Exam-Glove-Pairs/Medline/Z05-PF149184;ecomsessionid=j9Vn9v-IKFBcIb1m4Mvtv0eGE-m7t9iR0CPpgwsx.OC1-b2b?question=&index=P38&indexCount=38', 'title': '\nMedline Extended Cuff Nitrile Exam Glove Pairs'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_06/CA02_03_06_02/PF149183/PF149183_PRI01.JPG?quality=high&width=180&height=180', 'link': '/product/Medline-Extended-Cuff-Nitrile-Exam-Glove-Singles/Medline/Z05-PF149183;ecomsessionid=j9Vn9v-IKFBcIb1m4Mvtv0eGE-m7t9iR0CPpgwsx.OC1-b2b?question=&index=P39&indexCount=39', 'title': '\nMedline Extended Cuff Nitrile Exam Glove Singles'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_10/PF84818/PF84818_PRI04.JPG?quality=high&width=180&height=180', 'link': '/product/FitGuard-Ultra-Powder-Free-Nitrile-Exam-Gloves/Medline/Z05-PF84818;ecomsessionid=j9Vn9v-IKFBcIb1m4Mvtv0eGE-m7t9iR0CPpgwsx.OC1-b2b?question=&index=P40&indexCount=40', 'title': '\nFitGuard Ultra Powder-Free Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_05/PF00334/PF00334_PRI03.JPG?quality=high&width=180&height=180', 'link': '/product/Generation-Pink-Powder-Free-Nitrile-Exam-Gloves/Generation-Pink/Z05-PF00334;ecomsessionid=j9Vn9v-IKFBcIb1m4Mvtv0eGE-m7t9iR0CPpgwsx.OC1-b2b?question=&index=P41&indexCount=41', 'title': '\nGeneration Pink Powder-Free Nitrile Exam Gloves'}, {'img': '/media/catalog/sku/MDS/MDS198214_PRI03.JPG?quality=high&width=180&height=180', 'link': '/product/Medline-Sterile-Powder-Free-Non-Latex-Nitrile-Exam-Gloves/Medline/Z05-PF65609;ecomsessionid=j9Vn9v-IKFBcIb1m4Mvtv0eGE-m7t9iR0CPpgwsx.OC1-b2b?question=&index=P42&indexCount=42', 'title': '\nMedline Sterile Powder-Free Non-Latex Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_13/PF152511/PF152511_PRI03.JPG?quality=high&width=180&height=180', 'link': '/product/FitGuard-Touch-Nitrile-Exam-Gloves/Branded-Nonsterile-Nitrile-Exam-Gloves/Z05-PF152511;ecomsessionid=j9Vn9v-IKFBcIb1m4Mvtv0eGE-m7t9iR0CPpgwsx.OC1-b2b?question=&index=P43&indexCount=43', 'title': '\nFitGuard Touch Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_03/PF42081/PF42081_PRI04.JPG?quality=high&width=180&height=180', 'link': '/product/CURAD-White-Nitrile-Exam-Gloves/Non-Navigable-For-Boost/Z05-PF42081;ecomsessionid=j9Vn9v-IKFBcIb1m4Mvtv0eGE-m7t9iR0CPpgwsx.OC1-b2b?question=&index=P44&indexCount=44', 'title': '\nCURAD White Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_02/CA02_03_02_02/PF101925/PF101925_PRI01.JPG?quality=high&width=180&height=180', 'link': '/product/Aloetouch-3G-Synthetic-Exam-Gloves-CA-Only/Aloetouch/Z05-PF101925;ecomsessionid=j9Vn9v-IKFBcIb1m4Mvtv0eGE-m7t9iR0CPpgwsx.OC1-b2b?question=&index=P45&indexCount=45', 'title': '\nAloetouch 3G Synthetic Exam Gloves - CA Only'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_02/CA02_03_02_05/PF101932/PF101932_PRI01.JPG?quality=high&width=180&height=180', 'link': '/product/MediGuard-Vinyl-Synthetic-Exam-Gloves-CA-Only/MediGuard/Z05-PF101932;ecomsessionid=j9Vn9v-IKFBcIb1m4Mvtv0eGE-m7t9iR0CPpgwsx.OC1-b2b?question=&index=P46&indexCount=46', 'title': '\nMediGuard Vinyl Synthetic Exam Gloves - CA Only'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_02/CA02_03_02_03/PF101930/PF101930_PRI01.JPG?quality=high&width=180&height=180', 'link': '/product/CURAD-Stretch-Vinyl-Exam-Gloves-CA-Only/Curad/Z05-PF101930;ecomsessionid=j9Vn9v-IKFBcIb1m4Mvtv0eGE-m7t9iR0CPpgwsx.OC1-b2b?question=&index=P47&indexCount=47', 'title': '\nCURAD Stretch Vinyl Exam Gloves - CA Only'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_03/PF164126/PF164126_HRE01.JPG?quality=high&width=180&height=180', 'link': '/product/CURAD-Germshield-Nitrile-Exam-Gloves/Curad/Z05-PF164126;ecomsessionid=j9Vn9v-IKFBcIb1m4Mvtv0eGE-m7t9iR0CPpgwsx.OC1-b2b?question=&index=P48&indexCount=48', 'title': '\nCURAD Germshield Nitrile Exam Gloves'}], [{'img': '/media/catalog/CA02/CA02_03/CA02_03_02/CA02_03_02_01/PF101924/PF101924_PRI01.JPG?quality=high&width=180&height=180', 'link': '/product/Accutouch-Synthetic-Exam-Gloves-CA-Only/Accutouch/Z05-PF101924;ecomsessionid=sX4N8IskisxTruFKifHnaQM2wDnvaTBQFFnD7YP3.OC8-b2b?question=&index=P49&indexCount=49', 'title': '\nAccutouch Synthetic Exam Gloves - CA Only'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_03/CA02_03_03_01/PF84222/PF84222_PRI04.JPG?quality=high&width=180&height=180', 'link': '/product/Medline-Professional-Latex-Exam-Gloves/Aloetouch/Z05-PF84222;ecomsessionid=sX4N8IskisxTruFKifHnaQM2wDnvaTBQFFnD7YP3.OC8-b2b?question=&index=P50&indexCount=50', 'title': '\nMedline Professional Latex Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_02/CA02_03_02_10/PF101935/PF101935_PRI02.JPG?quality=high&width=180&height=180', 'link': '/product/Vinyl-Exam-Gloves-CA-Only/Medline/Z05-PF101935;ecomsessionid=sX4N8IskisxTruFKifHnaQM2wDnvaTBQFFnD7YP3.OC8-b2b?question=&index=P51&indexCount=51', 'title': '\nVinyl Exam Gloves - CA Only'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_02/CA02_03_02_05/PF101931/PF101931_PRI01.JPG?quality=high&width=180&height=180', 'link': '/product/MediGuard-Synthetic-Exam-Gloves-CA-Only/MediGuard/Z05-PF101931;ecomsessionid=sX4N8IskisxTruFKifHnaQM2wDnvaTBQFFnD7YP3.OC8-b2b?question=&index=P52&indexCount=52', 'title': '\nMediGuard Synthetic Exam Gloves - CA Only'}, {'img': '/media/catalog/sku/EQP/EQPT4121_PRI01.JPG?quality=high&width=180&height=180', 'link': '/product/EQPT-Powder-Free-Disposable-Latex-Gloves/Branded-Nonsterile-PF-Latex-Exam-Gloves/Z05-PF146919;ecomsessionid=sX4N8IskisxTruFKifHnaQM2wDnvaTBQFFnD7YP3.OC8-b2b?question=&index=P53&indexCount=53', 'title': '\nEQPT Powder-Free Disposable Latex Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_02/CA02_03_02_10/PF101934/PF101934_PRI01.JPG?quality=high&width=180&height=180', 'link': '/product/Designer-Boxed-Vinyl-Exam-Gloves-CA-Only/Medline/Z05-PF101934;ecomsessionid=sX4N8IskisxTruFKifHnaQM2wDnvaTBQFFnD7YP3.OC8-b2b?question=&index=P54&indexCount=54', 'title': '\nDesigner Boxed Vinyl Exam Gloves - CA Only'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_02/PF84142/PF84142_PRI04.JPG?quality=high&width=180&height=180', 'link': '/product/Medline-Professional-Lightly-Textured-Aloe-Exam-Gloves/Aloetouch/Z05-PF84142;ecomsessionid=sX4N8IskisxTruFKifHnaQM2wDnvaTBQFFnD7YP3.OC8-b2b?question=&index=P55&indexCount=55', 'title': '\nMedline Professional Lightly Textured Aloe Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_02/CA02_03_02_04/PF111036/PF111036_PRI02.JPG?quality=high&width=180&height=180', 'link': '/product/CA-Generation-Pink-3G-Synthetic-Exam-Gloves/Generation-Pink/Z05-PF111036;ecomsessionid=sX4N8IskisxTruFKifHnaQM2wDnvaTBQFFnD7YP3.OC8-b2b?question=&index=P56&indexCount=56', 'title': '\nCA - Generation Pink 3G Synthetic Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_02/CA02_03_02_10/PF113869/PF113869_PRI02.JPG?quality=high&width=180&height=180', 'link': '/product/Glide-On-Powder-Free-Vinyl-Exam-Gloves-CA-Only/Medline/Z05-PF113869;ecomsessionid=sX4N8IskisxTruFKifHnaQM2wDnvaTBQFFnD7YP3.OC8-b2b?question=&index=P57&indexCount=57', 'title': '\nGlide-On Powder-Free Vinyl Exam Gloves - CA Only'}, {'img': '/media/catalog/sku/546/546846_PRI01.JPG?quality=high&width=180&height=180', 'link': '/product/ProVision-Select-Powder-Free-Exam-Gloves/Medline/Z05-PF106917;ecomsessionid=sX4N8IskisxTruFKifHnaQM2wDnvaTBQFFnD7YP3.OC8-b2b?question=&index=P58&indexCount=58', 'title': '\nProVision Select Powder-Free Exam Gloves'}, {'img': '/media/catalog/sku/OAT/OAT4583_PRI02.JPG?quality=high&width=180&height=180', 'link': '/product/Restore-Sense-Powder-Free-Nitrile-Exam-Gloves-with-maxOat/SensiCare/Z05-PF188723;ecomsessionid=sX4N8IskisxTruFKifHnaQM2wDnvaTBQFFnD7YP3.OC8-b2b?question=&index=P59&indexCount=59', 'title': '\nRestore Sense Powder-Free Nitrile Exam Gloves with maxOat+'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_08/PF166969/PF166969_PRI02.JPG?quality=high&width=180&height=180', 'link': '/product/Restore-Touch-Nitrile-Exam-Gloves-with-Oatmeal/SensiCare/Z05-PF166969;ecomsessionid=sX4N8IskisxTruFKifHnaQM2wDnvaTBQFFnD7YP3.OC8-b2b?question=&index=P60&indexCount=60', 'title': '\nRestore Touch Nitrile Exam Gloves with Oatmeal'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_10/PF189775/PF189775_PRI01.JPG?quality=high&width=180&height=180', 'link': '/product/Woodland-Friends-Nitrile-Exam-Gloves/Medline/Z05-PF189775;ecomsessionid=sX4N8IskisxTruFKifHnaQM2wDnvaTBQFFnD7YP3.OC8-b2b?question=&index=P61&indexCount=61', 'title': '\nWoodland Friends Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_08/PF149181/PF149181_PRI04.JPG?quality=high&width=180&height=180', 'link': '/product/VersaShield-Extended-Cuff-Powder-Free-Nitrile-Exam-Gloves/SensiCare/Z05-PF149181;ecomsessionid=sX4N8IskisxTruFKifHnaQM2wDnvaTBQFFnD7YP3.OC8-b2b?question=&index=P62&indexCount=62', 'title': '\nVersaShield Extended Cuff Powder-Free Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_08/PF178153/PF178153_HRE01.JPG?quality=high&width=180&height=180', 'link': '/product/Generation-Pink-Sense-Powder-Free-Nitrile-Exam-Gloves/SensiCare/Z05-PF178153;ecomsessionid=sX4N8IskisxTruFKifHnaQM2wDnvaTBQFFnD7YP3.OC8-b2b?question=&index=P63&indexCount=63', 'title': '\nGeneration Pink Sense Powder-Free Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_10/PF146918/PF146918_PRI01.JPG?quality=high&width=180&height=180', 'link': '/product/EQPT-Nitrile-Powder-Free-Disposable-Gloves/Medline/Z05-PF146918;ecomsessionid=sX4N8IskisxTruFKifHnaQM2wDnvaTBQFFnD7YP3.OC8-b2b?question=&index=P64&indexCount=64', 'title': '\nEQPT Nitrile Powder-Free Disposable Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_10/PF105686/PF105686_PRI09.JPG?quality=high&width=180&height=180', 'link': '/product/Critical-Response-Powder-Free-Nitrile-Exam-Gloves/Medline/Z05-PF105686;ecomsessionid=sX4N8IskisxTruFKifHnaQM2wDnvaTBQFFnD7YP3.OC8-b2b?question=&index=P65&indexCount=65', 'title': '\nCritical Response Powder-Free Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_03/PF149182/PF149182_PRI04.JPG?quality=high&width=180&height=180', 'link': '/product/VersaShield-Powder-Free-Nitrile-Exam-Gloves/Curad/Z05-PF149182;ecomsessionid=sX4N8IskisxTruFKifHnaQM2wDnvaTBQFFnD7YP3.OC8-b2b?question=&index=P66&indexCount=66', 'title': '\nVersaShield Powder-Free Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_10/PF84817/PF84817_PRI04.JPG?quality=high&width=180&height=180', 'link': '/product/FitGuard-Select-Powder-Free-Nitrile-Exam-Gloves/Medline/Z05-PF84817;ecomsessionid=sX4N8IskisxTruFKifHnaQM2wDnvaTBQFFnD7YP3.OC8-b2b?question=&index=P67&indexCount=67', 'title': '\nFitGuard Select Powder-Free Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_08/PF88156/PF88156_PRI12.JPG?quality=high&width=180&height=180', 'link': '/product/Restore-Powder-Free-Nitrile-Exam-Gloves-with-Oatmeal/SensiCare/Z05-PF88156;ecomsessionid=sX4N8IskisxTruFKifHnaQM2wDnvaTBQFFnD7YP3.OC8-b2b?question=&index=P68&indexCount=68', 'title': '\nRestore Powder-Free Nitrile Exam Gloves with Oatmeal'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_06/PF159466/PF159466_PRI04.JPG?quality=high&width=180&height=180', 'link': '/product/MediGuard-ES-Powder-Free-Nitrile-Exam-Gloves/MediGuard/Z05-PF159466;ecomsessionid=sX4N8IskisxTruFKifHnaQM2wDnvaTBQFFnD7YP3.OC8-b2b?question=&index=P69&indexCount=69', 'title': '\nMediGuard ES Powder-Free Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_10/PF114581/PF114581_PRI06.JPG?quality=high&width=180&height=180', 'link': '/product/FitGuard-Touch-Powder-Free-Nitrile-Exam-Gloves/Medline/Z05-PF114581;ecomsessionid=sX4N8IskisxTruFKifHnaQM2wDnvaTBQFFnD7YP3.OC8-b2b?question=&index=P70&indexCount=70', 'title': '\nFitGuard Touch Powder-Free Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_08/PF145694/PF145694_PRI09.JPG?quality=high&width=180&height=180', 'link': '/product/Restore-Sense-Powder-Free-Nitrile-Exam-Gloves-with-Oatmeal/SensiCare/Z05-PF145694;ecomsessionid=sX4N8IskisxTruFKifHnaQM2wDnvaTBQFFnD7YP3.OC8-b2b?question=&index=P71&indexCount=71', 'title': '\nRestore Sense Powder-Free Nitrile Exam Gloves with Oatmeal'}, {'img': '/media/catalog/sku/EQP/EQPT4131_PRI01.JPG?quality=high&width=180&height=180', 'link': '/product/EQPT-Powder-Free-Disposable-Vinyl-Gloves/Medline/Z05-PF146920;ecomsessionid=sX4N8IskisxTruFKifHnaQM2wDnvaTBQFFnD7YP3.OC8-b2b?question=&index=P72&indexCount=72', 'title': '\nEQPT Powder-Free Disposable Vinyl Gloves'}]]
回答3:
Documentation of BeautifulSoup says that (https://www.crummy.com/software/BeautifulSoup/bs4/doc/#find)
If find() can’t find anything, it returns None
That means that finding div with class medSkuProductListExpanded productFamilyOrderInfo wasn't successfull, you should probably print things out around this and double check criteria by which you're searching.
回答4:
The error occurs because the following selector div = soup.find('div', {'class' matched nothing and thus returned NoneType. When searching for multiple classes the following syntax can be used:
div = soup.find('div', {'class': 'medSkuProductListExpanded productFamilyOrderInfo'})
right_table = div.find('table', {'class': ['actualDataTable', 'table-striped']})
div1 = soup.find('div', {'class': ['medTabAreaContent', 'Basic']})
right_table1 = div1.find('table', {'class': 'table-striped'})
if the element should match all classes css selectors can be used: https://www.crummy.com/software/BeautifulSoup/bs4/doc/#css-selectors
来源：https://stackoverflow.com/questions/57640584/how-to-open-each-product-within-a-website-in-a-new-tab-for-scrapping-using-selen