Overview Description
Instruct ChatGPT to write a robust, production-ready python web scraper utilizing requests, BeautifulSoup, and rate-limiting rules.
Prompt Command
74 wordsAct as an expert Python developer. Write a complete, production-ready Python script that scrapes product information (title, price, availability, rating) from a paginated e-commerce website. The script should use requests and BeautifulSoup. It must include robust error handling, retry logic for failed requests using a backoff strategy, custom headers (User-Agent rotation), and strict rate-limiting (e.g., delay between requests) to avoid getting blocked. Please document the code thoroughly and explain how to run it.
Tip: Click the copy button to copy the prompt text.
Execution Sample Output
A representation of what the AI outputs when this prompt is executed:
# Example Scrape Success Output Log
import requests
from bs4 import BeautifulSoup
# Script running results:
# [SUCCESS] Fetching page 1...
# [SUCCESS] Parsing product listing data.
# [INFO] Scraped items count: 25. Delaying 2.0s...
# Output structure:
# {"title": "AI Coding Book", "price": "$29.99", "availability": "In Stock"}
Associated tags
Similar Prompts
Related Recommendations
Explore additional top-rated prompt parameters for similar platform criteria.