In this guide, you will learn how to write a Python script that visits a website, grabs product names and prices, and saves everything into a spreadsheet file. No experience needed — we explain every step.

You can find the complete source code for this project in this GitHub repository.

🧰 What You Need Before Starting

• Python 3 installed on your computer (download here)
• A code editor — VS Code works great (download here)
• Your terminal (the black window where you type commands)
• A basic understanding of Python (variables and for loops)

Step 1 — Install the Tools

We need to install 3 extra tools for Python. Open your terminal and copy-paste this line:

pip install requests beautifulsoup4 pandas

Then press Enter and wait for it to finish.

📦

What are these tools?

• requests — Goes to a website and downloads its content (like a browser, but in code)
• beautifulsoup4 — Reads the downloaded content and helps you find specific information in it
• pandas — Puts your data into a table and saves it as a spreadsheet (.csv file)

Step 2 — Download the Website Content

First, we tell Python which website to visit and download its content.

import requests
from bs4 import BeautifulSoup
import pandas as pd

# The website you want to scrape
URL = "https://toscrape.com"

# This tells the website our script is a normal browser (not a robot)
headers = {
    "User-Agent": "Mozilla/5.0"
}

# Visit the website and download the page
response = requests.get(URL, headers=headers)

# If something goes wrong, Python will tell us
response.raise_for_status()

print("Page downloaded successfully!")

💡

The User-Agent line is important. Without it, some websites will refuse our script because they think it's a robot.

Step 3 — Find the Products on the Page

Every website is built with HTML — a language that describes what goes where on a page. We use BeautifulSoup to read that HTML and find the products.

# Read the HTML of the page
soup = BeautifulSoup(response.text, "html.parser")

# Find all product cards on the page
# On toscrape.com, each product is inside an <article> tag with class "product_pod"
products = soup.find_all("article", class_="product_pod")

print(f"We found {len(products)} products!")

🔍

How do you know which class name to use?

1. Open the website in Chrome or Firefox
2. Right-click on a product → click Inspect
3. Look for the HTML tag and the class="..." name around that product
4. Use that class name in your code

Step 4 — Get the Name and Price of Each Product

Now we go through each product one by one and grab its name and price.

# This list will store all our products
data = []

for product in products:
    # Find the name of the product
    name = product.h3.a if product.h3 else None

    # Find the price of the product
    price = product.find("p", class_="price_color")

    # Save the name and price (or "N/A" if not found)
    # We use name["title"] because the full product name is stored in the title attribute
    data.append({
        "name": name["title"] if name and name.has_attr("title") else "N/A",
        "price": price.get_text(strip=True) if price else "N/A"
    })

# Show the first 3 results to check everything looks good
print(data[:3])

Step 5 — Save Everything to a Spreadsheet

Finally, we take all the data we collected and save it as a .csv file (which you can open in Excel or Google Sheets).

# Turn the list into a table
df = pd.DataFrame(data)

# Save the table as a CSV file
df.to_csv("products.csv", index=False, encoding="utf-8")

print(f"✅ Done! {len(df)} products saved to products.csv")

⚠️ Common Problems & How to Fix Them