Learn to use word boundaries and lookahead assumptions to solve practical problems — In this post, we will showcase a selection of special applications for regular expressions in Python, aimed at resolving real-world challenges. As we will see in our examples, we can use word boundaries and lookahead assumptions to achieve more accurate matching. …

Learn a handy tool to check package dependencies in Python — Writing a requirements.txt file from scratch for your application can be cumbersome because you need to sort out what packages need to be installed and what are their dependencies. Worse still, it’s tedious to resolve conflicting dependencies, especially on legacy systems when an older version of pip is used. Luckily…

Learn to build and publish both Python packages and Docker images with Cloud Build — In previous posts, we introduced how to store Python packages and Docker images in Google Artifact Registry, respectively. In this post, we will try to connect these two techniques and introduce how to install private Python packages in Artifact Registry when building Docker images with Cloud Build. …

Learn a handy tip to test your web application on the phone — When developing a web application with Angular, or any other framework, we need to test it on our phones frequently. Even though we can use the “device toolbar” simulator in Chrome for testing, sometimes we still need to use a real phone for testing. …

Learn how to scrape JavaScript webpages with a smart proxy — In this post, we will introduce how to use the Zyte Smart proxy with Splash which is integrated with the Scrapy web scraping framework. We will learn how to set up the Zyte smart proxy, how to use it with Splash, and how to set up a headless proxy tool…

Learn a simple way to perform smooth transitions — Container Registry has been deprecated since May 15, 2023. In order to get the latest updates, features, and support for the services, we should transition from Container Registry to Artifact Registry. In this post, we will first introduce the differences between Container Registry and Artifact Registry and then the procedures…

Learn a simple way to build a production-ready Splash server in the Cloud — In previous posts, we introduced how to start Splash locally using Docker. In production, we would want to put the Splash server somewhere all our scraping projects can access. Since Splash is Dockerized, a simple solution is to start a Splash microservice in Cloud Run. It may seem simple to…

Learn a new way to store Docker images on the Google Cloud Platform — We used to store Docker images in the Container Registry on the Google Cloud Platform (GCP). …

Learn different ways to use proxies for web scraping with Splash — In web scraping, we often need to use proxies because of IP blocking or geolocation-sensitive data which means the scraped data can change depending on the geolocations of the IPs making the requests. It is straightforward to specify proxies for regular web scraping. …