Simple Web Scraping Using requests, Beautiful Soup, and lxml in Python

Get started with web scraping in Python

When it comes to web scraping in Python, the first package we think about is Scrapy. However, Scrapy is more suitable for larger scraping projects. Besides, there is a learning curve for Scrapy, which takes time. For simple scraping issues where you only need to get data from a single webpage directly, it can be rather overkilled to use Scrapy. In this case, we can use the requests plus Beautiful Soup/lxml packages to scrape the content you need…