Add cs.money worker stack with per-worker IPRoyal residential proxy

Brings up the pull-model scraper: the .NET C2 hands skin+wear jobs to Python nodriver workers that scrape cs.money and post results back, plus the supporting Core/EFCore data model, migrations, and docker-compose orchestration.

IPRoyal proxying lets workers scale horizontally with a distinct residential exit IP each: every worker process mints its own sticky session at startup, and an in-process forwarding proxy injects the gateway auth so Chromium talks only to an auth-free localhost endpoint (zero CDP). On a Cloudflare challenge a worker rotates to a fresh session/IP and re-warms. Verified end-to-end against live IPRoyal: distinct US residential exits per worker and IP rotation on demand.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
bob
2026-05-31 15:03:31 -05:00
parent eb5fb0dac7
commit dc7c3f99ae
82 changed files with 8354 additions and 571 deletions

View File

@@ -8,23 +8,29 @@
</PropertyGroup>
<ItemGroup>
<None Update="appsettings.json" CopyToOutputDirectory="PreserveNewest" />
<!-- Design-time only (the IDesignTimeDbContextFactory reads it for `dotnet ef`).
Kept in the build output but NOT published: otherwise it flows transitively
into a consumer's publish (e.g. BlueLaminate.C2) and collides with that
project's own appsettings.json (NETSDK1152). -->
<None Update="appsettings.json"
CopyToOutputDirectory="PreserveNewest"
CopyToPublishDirectory="Never" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="EFCore.NamingConventions" Version="10.0.1" />
<PackageReference Include="EFCore.NamingConventions" />
<!-- Pin the runtime EF Core version so it flows transitively to consumers
(the Design package is PrivateAssets=all and won't). Keeps the version
the library compiles against in sync with what the CLI links. -->
<PackageReference Include="Microsoft.EntityFrameworkCore.Relational" Version="10.0.8" />
<PackageReference Include="Microsoft.EntityFrameworkCore.Design" Version="10.0.8">
<PackageReference Include="Microsoft.EntityFrameworkCore.Relational" />
<PackageReference Include="Microsoft.EntityFrameworkCore.Design">
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
<PrivateAssets>all</PrivateAssets>
</PackageReference>
<PackageReference Include="Microsoft.Extensions.Configuration.EnvironmentVariables" Version="10.0.8" />
<PackageReference Include="Microsoft.Extensions.Configuration.Json" Version="10.0.8" />
<PackageReference Include="Microsoft.Extensions.Configuration.UserSecrets" Version="10.0.8" />
<PackageReference Include="Npgsql.EntityFrameworkCore.PostgreSQL" Version="10.0.2" />
<PackageReference Include="Microsoft.Extensions.Configuration.EnvironmentVariables" />
<PackageReference Include="Microsoft.Extensions.Configuration.Json" />
<PackageReference Include="Microsoft.Extensions.Configuration.UserSecrets" />
<PackageReference Include="Npgsql.EntityFrameworkCore.PostgreSQL" />
</ItemGroup>
</Project>