bob
dc7c3f99ae
Brings up the pull-model scraper: the .NET C2 hands skin+wear jobs to Python nodriver workers that scrape cs.money and post results back, plus the supporting Core/EFCore data model, migrations, and docker-compose orchestration. IPRoyal proxying lets workers scale horizontally with a distinct residential exit IP each: every worker process mints its own sticky session at startup, and an in-process forwarding proxy injects the gateway auth so Chromium talks only to an auth-free localhost endpoint (zero CDP). On a Cloudflare challenge a worker rotates to a fresh session/IP and re-warms. Verified end-to-end against live IPRoyal: distinct US residential exits per worker and IP rotation on demand. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
36 lines
1.1 KiB
Docker
36 lines
1.1 KiB
Docker
# cs.money worker: headful Chromium (nodriver) under a virtual display, with noVNC
|
|
# so you can open a browser into the container and solve a Cloudflare challenge by hand
|
|
# if one ever appears. Build context is the repo root (see docker-compose.yml).
|
|
FROM python:3.13-slim
|
|
|
|
# chromium + a virtual X display + VNC bridge + the fonts/libs Chromium needs.
|
|
RUN apt-get update && apt-get install -y --no-install-recommends \
|
|
chromium \
|
|
xvfb \
|
|
x11vnc \
|
|
novnc \
|
|
websockify \
|
|
ca-certificates \
|
|
fonts-liberation \
|
|
dumb-init \
|
|
&& rm -rf /var/lib/apt/lists/*
|
|
|
|
WORKDIR /app
|
|
COPY worker/requirements.txt ./
|
|
RUN pip install --no-cache-dir -r requirements.txt
|
|
COPY worker/worker.py worker/entrypoint.sh ./
|
|
RUN chmod +x entrypoint.sh
|
|
|
|
ENV BROWSER_PATH=/usr/bin/chromium \
|
|
CHROME_NO_SANDBOX=1 \
|
|
DISPLAY=:99 \
|
|
SOLVE_SECONDS=45 \
|
|
PYTHONUNBUFFERED=1
|
|
|
|
|
|
# noVNC web UI (browse http://localhost:6080/vnc.html to watch / solve a challenge).
|
|
EXPOSE 6080
|
|
|
|
# dumb-init reaps the Xvfb/x11vnc/websockify children cleanly.
|
|
ENTRYPOINT ["dumb-init", "--", "./entrypoint.sh"]
|