The Dormouse's story
""" soup = BeautifulSoup(html_doc, 'html.parser') # Using a web page content fetched with requests import requests response = requests.get('http://example.com') soup = BeautifulSoup(response.content, 'html.parser') ``` ## Navigating the Parse Tree ### Accessing Tags ```python # Access the title element title_tag = soup.title # Access the name of the tag print(title_tag.name) # Access the text within the title tag print(title_tag.string) ``` ### Finding All Instances of a Tag ```python # Find all 'a' tags a_tags = soup.find_all('a') ``` ### Accessing Attributes ```python # Access the first 'p' tag p_tag = soup.find('p') # Access its 'class' attribute p_class = p_tag['class'] ``` ## Searching the Tree ### find and find_all ```python # Find the first 'a' tag first_a_tag = soup.find('a') # Find all 'a' tags with a specific class specific_a_tags = soup.find_all('a', class_='sister') # Find using a CSS selector css_select_tags = soup.select('p.myclass') ``` ### Getting Data from Tags ```python # Get text content from a tag for tag in soup.find_all('a'): print(tag.get_text()) # Get a specific attribute value for tag in soup.find_all('a'): print(tag.get('href')) ``` ## Practical Example: Extracting Data from a Page ```python # Assuming you have fetched a page with product listings for product in soup.find_all('div', class_='product'): name = product.h2.text price = product.find('span', class_='price').text print(f'Product: {name}, Price: {price}') ``` `Beautiful Soup` is designed to make your web scraping code more human-readable and concise. It's incredibly effective at parsing messy web HTML content and offers both simplicity and flexibility, making it accessible for beginners yet powerful enough for advanced users. This guide covers basic functionalities, but `Beautiful Soup` supports a wide array of parsing and navigating techniques, making it a versatile tool in your web scraping toolkit. Beautiful Soup's flexibility and ease of use make it an excellent choice for both novice programmers and professional developers needing to extract information from the web, parse documents, or scrape data from websites efficiently.