LegosTracker

1. Overview

brickeconomy.py reads set numbers from set_list.txt and drives a visible Chromium window with Playwright. For each set it searches BrickEconomy, opens the matching page and extracts the chart rows embedded in the page script: the value history goes to <set>_history.csv and, when present, the four-band price series to <set>_new.csv. It then reads the details, pricing, quick buy, predictions, facts and subtheme panels into Pydantic models and appends them to lego_sets.csv. brickeconomy.ipynb holds the same code with the install commands.

general.ipynb loads the Rebrickable tables in datasets/ (colors, inventories, parts, sets, themes and their links) and plots grouped counts. analyse.ipynb cleans the scraped data/lego_sets.csv and the data/001-1_*.csv files for the 1965 set 001-1, fits linear and gradient boosting regressions on the price history, projects them to 2030, runs a Monte Carlo simulation on log returns and charts yearly gains and the four price bands. price.ipynb takes a personal mycollection.csv, looks up retail and current values on BrickEconomy with requests and BeautifulSoup, and writes updated_mycollection.csv.

3. Getting started

brickeconomy.ipynb installs the scraper dependencies:

pip install playwright asyncio pydantic aiohttp
playwright install
python brickeconomy.py

The script writes its CSV files in the current directory; analyse.ipynb reads them from data/. The notebooks need pandas, NumPy, Matplotlib, seaborn and scikit-learn (installed by the first cell of analyse.ipynb), plus requests and BeautifulSoup for price.ipynb:

pip install pandas matplotlib seaborn scikit-learn numpy
jupyter notebook

4. Repository guide

• brickeconomy.py is the Playwright scraper for BrickEconomy set pages.
• set_list.txt lists the set numbers to scrape (001-1 to 0016-1 and 002-1).
• data/ holds scraped output: lego_sets.csv and one _history.csv per set, plus 001-1_new.csv.
• datasets/ holds the Rebrickable exports as CSV and .csv.gz; the README documents their columns, and images/downloads_schema_v3.webp shows the schema.
• general.ipynb explores the Rebrickable catalog.
• analyse.ipynb cleans scraped prices and models the history of set 001-1.
• price.ipynb updates the values of the sets in mycollection.csv.

5. Features

• scrape BrickEconomy value history, price bands and set panels for a list of sets;
• explore Rebrickable colors, inventories, parts, sets and themes;
• compare scraped prices across sets (lowest, highest, average);
• fit linear and gradient boosting regressions on a set’s price history and project them;
• simulate future prices with a Monte Carlo model on log returns;
• update the current value of a personal collection.

6. Limits

• There is no requirements file; dependencies come from !pip install cells in the notebooks.
• The scraper depends on BrickEconomy’s page structure and CSS selectors, runs a visible browser and stops with an error when a search returns no link.
• Output files are written in the working directory, not in data/, so they must be moved before running analyse.ipynb.
• The scraped data covers nine sets; the price models work on a single set (001-1) and are illustrations, not forecasts.
• price.ipynb reads only the first table of the BrickEconomy search page and defines a TAX_RATE constant that it never uses.
• The Rebrickable exports are a dated snapshot; the repository has no script to refresh them.

7. Sources

• GitHub repository
• README
• brickeconomy.py
• analyse.ipynb
• general.ipynb
• price.ipynb
• Exploring the data of LEGO history, linked from the README

flowchart LR
  repo[("GitHub repository")]
  local["Local machine"]
  repo -->|"clone and run"| local