# BlueLaminate

> A CS2 skin-market intelligence platform — ingests live marketplace listings across **CSFloat**, **cs.money**, and **skin.land**, tracks prices and float values in Postgres, and mines the catalogue for profitable **trade-up contracts**.

![.NET](https://img.shields.io/badge/.NET-10.0-512BD4?logo=dotnet&logoColor=white)
![Python](https://img.shields.io/badge/Python-3.x-3776AB?logo=python&logoColor=white)
![Postgres](https://img.shields.io/badge/PostgreSQL-skintracker-4169E1?logo=postgresql&logoColor=white)
![Docker](https://img.shields.io/badge/Docker-compose-2496ED?logo=docker&logoColor=white)
![Grafana](https://img.shields.io/badge/Observability-Grafana%20LGTM-F46800?logo=grafana&logoColor=white)

---

## ✨ What it does

| Capability | Where | Notes |
|---|---|---|
| **CSFloat ingestion** | `BlueLaminate.Scraper` + CLI | Official API. Catalogue-driven and global incremental sweeps, paced off rate-limit headers. |
| **cs.money scraping** | `worker/csmoney_worker.py` | Cloudflare-walled. A Python `nodriver` worker fleet driven by the C2 (pull model). |
| **skin.land scraping** | `worker/skinland_worker.py` | Nuxt SSR behind Cloudflare. Same worker plumbing, market-specific scrape. |
| **Cross-market presence** | `MarketPresenceService` | "Where is this skin/instance listed, and at what price?" across all sources. |
| **Trade-up engine** | `BlueLaminate.Core/Tradeups` | Finds profitable 10-input contracts from live listings — exact float math, guaranteed vs. expected-value ranking, StatTrak universes, interactive TUI. |
| **Price history & dedup** | `BlueLaminate.EFCore` | Postgres data model with per-source checkpoints, `SkinInstance` fingerprinting, and `price_history`. |
| **Observability** | `monitoring/` | Standalone Grafana LGTM stack (Loki / Grafana / Tempo / Mimir) wired via OTLP. |

---

## 🏗️ Architecture

```
                       ┌──────────────────────────────────────────┐
                       │              PostgreSQL                    │
                       │   (skintracker schema, least-priv role)    │
                       └───────────────▲────────────────▲──────────┘
                                       │                │
           ┌───────────────────────────┘                └────────────────┐
           │                                                              │
  ┌────────┴─────────┐                                          ┌─────────┴──────────┐
  │  BlueLaminate.Cli │  CSFloat (official API)                 │  BlueLaminate.C2    │
  │  sync / sweep /   │─────────────────────────────────────►  │  (ASP.NET control   │
  │  find-tradeups    │                                          │   plane)            │
  └────────┬──────────┘                                          └─────────▲──────────┘
           │                                                               │ jobs / results
           │ shares                                                        │ (X-Worker-Token)
           ▼                                                     ┌─────────┴──────────┐
  ┌────────────────────┐                                         │  Python workers     │
  │  BlueLaminate.Core  │  ◄── composition root, all shared       │  nodriver + IPRoyal │
  │  (ingest, tradeups, │      logic (AddBlueLaminateCore)         │  cs.money / skin.land│
  │   market presence)  │                                         └─────────────────────┘
  └────────────────────┘
```

- **`BlueLaminate.Core`** — the single composition root. Every frontend wires itself with `AddBlueLaminateCore(configuration)` and reuses the same ingest/tradeup/presence services. No duplicated logic.
- **`BlueLaminate.Scraper`** — typed CSFloat API client + the CSGO-API skin catalogue client.
- **`BlueLaminate.EFCore`** — Postgres data layer and EF Core migrations (auto-applied on C2 boot).
- **`BlueLaminate.Cli`** — `System.CommandLine` operator tools.
- **`BlueLaminate.C2`** — hands cs.money/skin.land scrape jobs to the Python worker fleet and ingests their results.
- **`worker/`** — Python `nodriver` workers. .NET Selenium gets insta-challenged by Cloudflare; `nodriver` drives Chromium directly over CDP and passes. Each replica mints its own IPRoyal sticky residential exit IP.

---

## 🚀 Quick start

### Prerequisites

- [.NET 10 SDK](https://dotnet.microsoft.com/download)
- **PostgreSQL** with a `skintracker` database
- Python 3.x + Chrome/Edge (only if you run the cs.money / skin.land workers)
- Docker (optional — for the one-command C2 + worker stack)
- A **CSFloat API key** for the CSFloat paths (`CSFLOAT_API_KEY`)

### 1. Database

Run the schema/role hardening **once** as a superuser (edit the password placeholders first):

```powershell
psql -U postgres -d skintracker -f db/01_schema_and_roles.sql
```

The C2 applies EF migrations automatically on boot (`AutoMigrate=true`); for the CLI alone, run `dotnet ef database update --project BlueLaminate\BlueLaminate.EFCore`.

### 2. Build

```powershell
dotnet build BlueLaminate/BlueLaminate.slnx
```

### 3. Run the CLI

```powershell
# One-time: pull the CS2 skin catalogue (throttled to once a month)
dotnet run --project BlueLaminate\BlueLaminate.Cli -- sync-skins

# Ingest live CSFloat listings (reads CSFLOAT_API_KEY)
$env:CSFLOAT_API_KEY="your-key"
dotnet run --project BlueLaminate\BlueLaminate.Cli -- sweep-catalog

# Mine profitable trade-up contracts (interactive TUI)
dotnet run --project BlueLaminate\BlueLaminate.Cli -- find-tradeups --min-profit 2.00
```

#### CLI commands

| Command | Purpose |
|---|---|
| `sync-skins` | Load the CS2 skin catalogue from the CSGO-API dataset and upsert it. |
| `fetch-listings` | Fetch active CSFloat listings for one skin and print them (no DB write). |
| `sweep-listings` | Global incremental sweep of active CSFloat listings into the DB. |
| `sweep-catalog` | Catalogue-driven sweep: query each skin by `def_index`+`paint_index`, split by wear band. |
| `find-tradeups` | Find profitable 10-input trade-up contracts from live listings, ranked best-first. |

Add `--help` to any command for its full option set.

### 4. Run the marketplace workers (cs.money / skin.land)

The browser/Cloudflare scrapers run as Python workers that pull jobs from the C2.

**Containerized (one command):**

```powershell
# Defaults: 1 cs.money worker + 1 skin.land worker. Postgres is external (host).
docker compose up --build
```

- C2 → http://localhost:5080 (`/health`, `/jobs/*`, `/market/*`)
- Worker noVNC → http://localhost:6080/vnc.html (watch the browser; solve a Cloudflare challenge by hand if one appears)

Set the worker mix and proxy via env / a `.env` file next to `docker-compose.yml`:

```powershell
$env:CSMONEY_WORKERS=0; $env:SKINLAND_WORKERS=3
$env:IPROYAL_USERNAME="..."; $env:IPROYAL_PASSWORD="..."
docker compose up --build
```

**Local (without Docker):**

```powershell
# terminal 1 — the C2 (from repo root)
dotnet run --project BlueLaminate\BlueLaminate.C2     # http://localhost:5080

# terminal 2 — a worker
cd worker
py -m venv .venv; .venv\Scripts\Activate.ps1
pip install -r requirements.txt
$env:WORKER_TOKEN="dev-worker-token"                  # must match the C2's WorkerToken
python csmoney_worker.py                               # or skinland_worker.py
```

See [`DOCKER.md`](DOCKER.md) and [`worker/README.md`](worker/README.md) for the full configuration matrix, scaling notes, and proxy details.

---

## ⚙️ Configuration

Key settings (env vars override `appsettings.json`):

| Var | Purpose |
|---|---|
| `ConnectionStrings__SkinTracker` / `SKINTRACKER_CONN` | Postgres connection string. |
| `CSFLOAT_API_KEY` | CSFloat official API key (CLI sweep/fetch). |
| `WorkerToken` / `WORKER_TOKEN` | Shared secret between C2 and workers. |
| `MaxPagesPerJob`, `MinResweepHours` | C2 sweep budget / re-sweep floor (metered-proxy cost control). |
| `IPROYAL_USERNAME` / `IPROYAL_PASSWORD` / `IPROYAL_COUNTRY` | Residential proxy for workers. |

> 🔒 Secrets (API keys, proxy credentials, DB passwords) belong in environment variables, a gitignored `.env`, or .NET user secrets — never committed.

---

## 📊 Observability

A standalone Grafana LGTM stack lives under [`monitoring/`](monitoring/README.md) (deployed to a Proxmox LXC). Apps emit OpenTelemetry; the CLI ships with a compact console exporter and is wired to swap in OTLP.

---

## 📁 Repository layout

```
BlueLaminate/            # .NET solution
  BlueLaminate.Core/     #   shared logic + composition root
  BlueLaminate.Scraper/  #   CSFloat API + skin catalogue clients
  BlueLaminate.EFCore/   #   Postgres data layer + migrations
  BlueLaminate.Cli/      #   operator CLI (System.CommandLine)
  BlueLaminate.C2/       #   ASP.NET control plane for the worker fleet
  BlueLaminate.Tests/    #   xUnit tests
worker/                  # Python nodriver workers (cs.money, skin.land)
db/                      # SQL schema + role hardening + audit queries
monitoring/              # Grafana LGTM observability stack
docker-compose.yml       # one-command C2 + worker bring-up
```