Skip to content

Scaling scenarios: 70, 500, 2 000 and 10 000 guests

The owner deciding whether to accept an event, and the engineer who has to prepare for it.

  • 70 guests, one station: works as deployed. Top up the provider balance and follow the event playbook.
  • 500 guests: no code changes, but four preparations are mandatory — a higher provider quota, a dedicated server, a second station and a real burst load test.
  • 2 000 guests: one large server still works, with small code changes: more worker slots, a longer delivery deadline, flood handling for pack creation, a live fallback provider.
  • 10 000 guests or several events at once: a re-architecture — PostgreSQL, workers on several machines, object storage with a lifecycle rule, per-event tenancy.
  • Everything below the measured line is an estimate or a proposal; the formulas are shown.

All inputs come from Capacity and Cost.

Quantity Formula Inputs today
Packs guests × participation rate participation is unknown — measure it; plan with 100 %
Image requests packs × 4.3 4.29 measured per pack
Provider quota needed per day image requests ÷ 0.7 keep the warning threshold (70 %) as head-room
AI cost packs × $0.42, plus 20 % reserve $0.417 measured per pack
Stations peak guests per hour ÷ station throughput 40–60 guests per hour per station (estimate)
Parallel packs needed peak packs per hour ÷ 60 about 60 s per pack
Memory for the bot process parallel packs × 1.4 GB + 0.5 GB 1.42 GB peak measured for one pack

Worked numbers used below assume every guest takes part and a six-hour event whose peak hour is twice the average (peak packs per hour = packs ÷ 6 × 2).

Guests Packs Image requests Quota needed AI cost + 20 % Peak packs / hour Stations Parallel packs
70 70 about 300 1 000 (entry tier) about $35 about 25 1 1
500 500 about 2 150 about 3 100 about $250 about 170 3–4 3
800 800 about 3 440 about 4 900 about $400 about 270 5–7 5
2 000 2 000 about 8 600 about 12 300 about $1 000 about 670 12–16 11–12
10 000 10 000 about 43 000 about 61 400 about $5 000 about 3 300 55–80 about 55

For any other size, for example 800 guests: 800 packs × 4.3 = about 3 440 image requests (well above the entry quota of 1 000 per day); 800 × $0.42 = $336, about $400 with reserve; a worst-case ceiling of 800 × $1.50 = $1 200 if every job hit its cost cap. Infrastructure for 800, row by row:

Item 800 guests
Provider daily image quota at least about 4 900 (3 440 ÷ 0.7), so the entry tier of 1 000 is not enough; set IMAGE_RPD_LIMIT to the real figure
Prepaid balance about $400, more if the flat-cut detector raises the average to $0.50 per pack (then about $480)
Server dedicated, about 8 vCPU / 16 GB: five parallel packs × 1.4 GB ≈ 7 GB plus head-room for the API and the system
WORKER_SLOTS 5, with the bot unit’s MemoryHigh about 7.5G and MemoryMax about 8G
Stations 5–7 tablets at the peak (about 270 packs per hour ÷ 40–60 per station). Time per guest at the tablet was never measured: at 120 s per guest a station serves 30 per hour and the peak needs 9. Measure at the rehearsal and recount
Delivery deadline at 5 slots the 15-minute deadline holds about 70 queued jobs — enough if the stations above are not exceeded
Before the event fallback provider key, burst load test on this server, Telegram pack creation measured at 4–5 per minute

All of it is an estimate from the formulas above; nothing above one pack at a time has been measured.

Nothing to change. This is the pilot configuration.

  • Server: the existing 2 vCPU / 4 GB droplet, WORKER_SLOTS=2.
  • Quota: about 300 image requests, 30 % of the entry tier.
  • Money: about $30 of generation; the load-test report recommends a prepaid balance of at least $50 to cover test packs and retries.
  • Follow the Event playbook.

Risk: low. Open items are the missing fallback provider and the shared droplet.

No code changes are required. Four preparations are.

# Change How Effort Risk if skipped
1 Raise the provider’s daily image quota to at least about 3 100 requests Move the provider account to a higher tier (or split across projects, see the 2 000 scenario); set IMAGE_RPD_LIMIT to the real figure Hours to days — depends on the provider Generation stops for the day at about 230 packs
2 Dedicated server, not shared New droplet of 4 vCPU / 8 GB or more; deploy from scratch with the existing scripts (Deploy & rollback); point the API domain at it Half a day Neighbour’s load or web-server mistake hits the event
3 Three parallel packs WORKER_SLOTS=3; raise the unit’s MemoryHigh/MemoryMax to fit 3 × 1.4 GB (about 5 GB); CUT_MAX_PARALLEL stays at the vCPU count One hour Queue grows past the 15-minute deadline in the peak hour
4 Burst load test on that server tools/load_test.py with a stream and a burst, on a separate test key, with a stated budget Two hours and a few dollars Memory and provider limits under parallel load are unknown today
5 Three or four stations Open the same kiosk address on each tablet Logistics only Lines at the stand
6 Fallback provider key Configure the second provider and run one live pack through it A day, including account verification A provider outage is a full outage
7 Prepaid balance Top up to the event estimate plus reserve; enable auto top-up if the provider offers it Minutes Auto-pause in the middle of the event
8 Measure Telegram pack creation at three packs per minute Part of step 4 if run against a real test bot One hour Unknown flood limits

SQLite, the single bot process and long polling stay as they are.

Risk: medium. The largest unknowns are the provider’s limits at this request rate and memory with three packs in flight.

A single large server is still the simplest answer — memory per parallel pack is the sizing rule — but the code needs a few changes.

Infrastructure

Change Detail
One dedicated server of about 16 vCPU / 32 GB 12 parallel packs × 1.4 GB ≈ 17 GB plus head-room; CPU for cutting and export
Provider quota of about 12 000 image requests per day and a matching per-minute limit Negotiate the tier in advance; consider several provider projects with separate keys
A second, working AI provider Mandatory at this size
Offsite database backups Today backups stay on the same server
External uptime check on /health None exists in the repository

Code changes (small to medium)

Change Why Effort
Make the delivery deadline and delay notice configurable A peak of 670 packs per hour will queue; 15 minutes is hard-coded Small
Show guests an honest waiting estimate when the queue is long The promise “a couple of minutes” breaks under backlog Small
Handle Telegram flood limits in pack creation with backoff Not handled specially today Small
Rotate between several provider keys One key’s quota and per-minute limit Medium
Split the worker into its own service on the same host Restarting the bot should not interrupt 12 running packs; lets the worker scale separately Medium
Optional: move to PostgreSQL One writer is still enough here, but this is the last size where it is Medium
Optional: a second bot for pack creation Only if step “flood limits” shows one bot cannot create packs fast enough; pack names end in the creating bot’s username Medium

Operations: a rehearsal with a real burst, two people on call, and a staffed “help desk” at the stand for failed packs.

Risk: high until the load test has been run at the target rate. Nothing in the repository proves throughput above one pack per minute.

10 000 guests, or several events at the same time

Section titled “10 000 guests, or several events at the same time”

This is a different system. The current design assumes one machine, one file database, one bot and one event configuration.

flowchart LR
  subgraph Edge
    CDN["CDN for kiosk static files"]
    LB["Load balancer and TLS"]
  end
  subgraph App["Stateless services"]
    API1["API instances"]
    BOTW["Bot with webhooks"]
    W1["Worker pool, many hosts"]
  end
  PG[("PostgreSQL")]
  Q["Queue: Postgres SKIP LOCKED or a broker"]
  OBJ[("Object storage with 72 h lifecycle rule")]
  AI1["AI provider A, several keys"]
  AI2["AI provider B"]
  TG["Telegram, several bots"]
  CDN --> LB --> API1
  API1 --> PG
  API1 --> OBJ
  BOTW --> PG
  PG --- Q
  Q --> W1
  W1 --> OBJ
  W1 --> AI1
  W1 --> AI2
  W1 --> TG
Area Today Needed Effort
Database SQLite file PostgreSQL; every BEGIN IMMEDIATE section becomes a row lock or a transaction Large
Queue jobs table polled by one process Queue with competing consumers (SELECT … FOR UPDATE SKIP LOCKED or a broker) Medium
Workers Tasks inside the bot process Separate worker service, about 55 parallel packs across hosts Large
Files Local folder with an hourly cleanup loop Object storage with a lifecycle rule; the 72-hour limit must still be enforced and provable Medium
Telegram updates Long polling, one instance Webhooks behind the load balancer Medium
Pack creation One bot Several bots or a measured single-bot limit Unknown — measure first
AI One provider, one key Several keys and two live providers, with per-key quota tracking Medium
Tenancy One event per installation event_id on sessions, jobs and usage; per-event branding, budgets, limits and statistics Large — see Multi-tenant and white label
Rate limiters and lockouts Process memory Shared store Small
Kiosk static files Served by the API CDN Small
Observability Log files, Telegram alerts, admin page Metrics, dashboards, alerting on queue age and failure rate Medium
Neighbouring project Same droplet Fully separate infrastructure Small

Risk: very high as a single step. Take it in the order database → worker service → storage → tenancy, with a load test after each.

Several small events at once, without the rebuild

Section titled “Several small events at once, without the rebuild”

Two or three events of pilot size on the same day do not need the rebuild. Run one installation per event: its own server (or directory and port), its own bot, its own .env, its own kiosk branding. The deploy scripts already support a fresh server. Costs: one bot and one provider project per event, and no combined statistics.

  • Consent before the photo, the 72-hour limit, one pack per account and “the station never waits” are product rules, not scaling parameters.
  • The cost cap per job and the auto-pause on an empty balance keep working and matter more as volume grows.
  • Test with mocks; pay for a live load test once, with a budget, on a separate key.
  • Capacity — the measurements behind these numbers.
  • Cost — the money side.
  • Roadmap — which of these changes are planned.

Last verified: 2026-11-13 against ai-sticker-pack@b715418 and ai-stickers-landing@e350ff5.