Secrets

Who is this for

The owner or the engineer who creates credentials for a new installation, rotates one after a leak or a staff change, or needs to know who must hold which secret.

TL;DR

• Server-side secrets live in one file: /opt/ai-sticker-pack/bot/.env (owner sticker, mode 600). Landing secrets live in Vercel environment variables. Nothing secret is in git.
• This page never shows a value. Neither should a commit, a log line, a chat message or a screenshot.
• Rotating the bot token or an AI key means: create the new value, edit bot/.env, restart, test, revoke the old value.
• LEADS_SECRET exists on two sides. During rotation the landing falls back to direct send, so leads are not lost, but they miss the database. Keep the window to seconds.
• With APP_ENV=prod the services refuse to start with a missing token, AI key, alert chat or admin password, or with a weak LEADS_SECRET.
• Edit secrets with an editor (nano), not with a command that puts the value into shell history.

Inventory

Server: /opt/ai-sticker-pack/bot/.env

Name	What it protects	Created in	Who needs it
TELEGRAM_BOT_TOKEN	Full control of the Telegram bot: reading guest messages, sending messages, creating sticker packs	BotFather	Owner, Engineer
GEMINI_API_KEY	Paid image and vision generation on the primary provider; spends the prepaid balance and the daily quota	AI provider console (Google AI Studio)	Owner, Engineer
OPENAI_API_KEY	Paid generation on the fallback provider	AI provider console (OpenAI)	Owner, Engineer
ADMIN_PANEL_USER, ADMIN_PANEL_PASSWORD	The web admin panel at /admin: guest names, pack links, retry, reset, balance override	Generated by the operator	Owner, Operator on site, Engineer
LEADS_SECRET	POST /leads: who may write leads into the database and the leads group	Generated by the operator	Engineer (also set in Vercel)
LEADS_IP_SALT	Makes the stored hash of a visitor address irreversible	Generated by the operator	Engineer

Not secrets, but private identifiers that must not be published: ADMIN_CHAT_ID, ADMIN_USER_IDS, LEADS_CHAT_ID, LEADS_ADMIN_IDS. They decide who receives alerts and who may run admin commands.

Landing: Vercel environment variables

Name	What it protects	Notes
LEADS_SECRET	Same value as on the server	Server only, never NEXT_PUBLIC_
TELEGRAM_BOT_TOKEN	The bot that posts the request straight into the group on the fallback route	Server only. May be the same bot as the product bot or a separate one; check before rotating.
TELEGRAM_CHAT_ID	Private identifier of the leads group	Vercel, server-side variable (never NEXT_PUBLIC_); the same group as the product’s LEADS_CHAT_ID
LEADS_ENDPOINT, NEXT_PUBLIC_SITE_URL	Not secret	NEXT_PUBLIC_SITE_URL is public by design

Workstation and accounts

Secret	What it protects
SSH private key for root@<server>	The whole server, including bot/.env, the database and guest photos
Local bot/.env of a developer	Whatever keys it holds; often the production AI key
TEST_GEMINI_API_KEY (local bot/.env)	A separate provider project for test scripts, with its own budget
Telegram account that owns the bot in BotFather	Can revoke and re-issue the bot token
AI provider console account	Billing, keys, quota tier
DigitalOcean, Vercel, DNS provider and GitHub accounts	Server, landing, domains, source code

Use two-factor authentication on every account in this table. Hand over accounts, not just values, when the owner changes.

How each secret is created

Secret	How
TELEGRAM_BOT_TOKEN	In Telegram, open BotFather → /newbot → choose a name and a username. BotFather answers with the token. The username without @ goes into TELEGRAM_BOT_USERNAME.
GEMINI_API_KEY	Google AI Studio → create an API key in the project that has billing and the paid tier. The prepaid balance and the daily image quota belong to the project, not to the key.
TEST_GEMINI_API_KEY	Same console, but in a different project, so tests cannot spend the guests’ balance or quota.
OPENAI_API_KEY	OpenAI console → API keys.
ADMIN_PANEL_PASSWORD	Any password manager, 12+ random characters. Example generator: python3 -c "import secrets; print(secrets.token_urlsafe(18))".
LEADS_SECRET	python3 -c "import secrets; print(secrets.token_urlsafe(32))" — at least 24 characters are required in prod.
LEADS_IP_SALT	python3 -c "import secrets; print(secrets.token_urlsafe(24))".
SSH key	ssh-keygen -t ed25519 on the workstation; add the public key to the server through the hosting panel or ~/.ssh/authorized_keys.

Generate values on your own machine and paste them directly into nano on the server or into the Vercel form. Do not send them through a chat.

Rotation

General pattern on the server:

nano /opt/ai-sticker-pack/bot/.env       # replace the value, save
bash /opt/ai-sticker-pack/deploy/update.sh --yes

update.sh restarts both services and checks /health. See Deploy and rollback for what a restart does to jobs in flight. Rotate outside event hours whenever you can.

Bot token

Revoking the token stops the bot at once. Everything else (database, packs, pack links) is unaffected: pack names depend on the bot username, not on the token.

1. Wait for an empty queue: curl -s localhost:8000/health shows "active_jobs":0.
2. In BotFather: /mybots → the bot → API Token → Revoke current token. Copy the new token.
3. On the server: nano /opt/ai-sticker-pack/bot/.env, replace TELEGRAM_BOT_TOKEN, save.
4. bash /opt/ai-sticker-pack/deploy/update.sh --yes. Both services need the token: the API sends the “generating” message to the guest.
5. If the landing’s fallback route uses the same bot: Vercel → Settings → Environment Variables → TELEGRAM_BOT_TOKEN → new value → Redeploy.
6. Update every local bot/.env that held the old token.

Verify:

journalctl -u sticker-bot -n 30 --no-pager -o cat | grep -E 'STARTUP|Unauthorized|Conflict'

Expected: STARTUP lines, no Unauthorized, no Conflict. Send /stats to the bot from an admin account; it answers. Between steps 2 and 4 the bot is silent and running jobs cannot create packs.

AI keys (GEMINI_API_KEY, OPENAI_API_KEY)

1. Create the new key in the same project, so the balance and the quota tier stay the same.
2. nano /opt/ai-sticker-pack/bot/.env, replace the value.
3. systemctl restart sticker-bot (only the bot process calls the AI provider), or update.sh --yes.
4. Generate one test pack from the tablet. Grant yourself an extra pack first: admin panel → your row → Разрешить ещё раз (“allow one more”).
5. Delete the old key in the provider console.
6. Update local .env files. Make sure TEST_GEMINI_API_KEY is still a different key: if it equals GEMINI_API_KEY, production refuses to start.

Verify: the pack link arrives; journalctl -u sticker-bot --since -10min | grep -E 'FAILED|BILLING_EXHAUSTED|RATE_LIMIT' is empty.

LEADS_SECRET (two sides)

The server accepts exactly one secret, so there is always a short window in which the two sides disagree. What happens in that window (deliverRequest.ts, leads_api.py):

• The server answers 401. The landing then sends the request straight to the Telegram group, with the first line ⚠️ Запасной канал: сервер заявок не ответил — заявки нет в базе (request_id …) (“fallback channel: the leads server did not answer — the request is not in the database”). The request is not lost, but it has no status buttons and is not in the database.
• That fallback works only if TELEGRAM_BOT_TOKEN and TELEGRAM_CHAT_ID are set in Vercel. Without them the request is lost and the visitor sees an error.
• After 5 wrong-secret requests from one calling address within 10 minutes the server answers 429 to that address for 10 minutes. The landing does not fall back on 429; the visitor sees “try later”. So the window must stay well under five requests.

Order that keeps the window to seconds:

1. In Vercel, confirm that TELEGRAM_BOT_TOKEN and TELEGRAM_CHAT_ID exist for Production.
2. Generate the new secret.
3. Vercel → Settings → Environment Variables → LEADS_SECRET → paste the new value → Save. A saved variable is not active until the next deployment.
4. Open an SSH session and open the file: nano /opt/ai-sticker-pack/bot/.env. Replace LEADS_SECRET but do not save yet.
5. Vercel → Deployments → … → Redeploy. Wait until the new deployment is live.
6. The moment it is live: save in nano, then systemctl restart sticker-api. Only the API reads this secret.
7. Send a test request from the form on pack.eventtalks.uz.

Verify:

journalctl -u sticker-api --since -5min --no-pager -o cat | grep -E 'LEAD_NEW|LEAD_REJECTED'

Expected: LEAD_NEW #<N> … and a card with status buttons in the group. LEAD_REJECTED неверный секрет (“wrong secret”) after step 6 means the two values differ; compare them. Process any ⚠️ Запасной канал messages that arrived during the window by hand: see Leads missing or fallback.

Doing it the other way round (server first, then Vercel) also loses nothing, but the window lasts as long as the Vercel build.

LEADS_IP_SALT

1. nano /opt/ai-sticker-pack/bot/.env, replace the value.
2. systemctl restart sticker-api.

Side effect: visitor address hashes stored with existing leads no longer match hashes computed after the change, so the same visitor looks like a new one from that point. Per-visitor rate-limit counters live in memory and reset on every API restart anyway. Repeat-request detection is unaffected: it matches the Telegram username or the phone number, not the hash. The salt must not be empty while LEADS_SECRET is set; in prod that stops the API at start.

Admin panel password

1. nano /opt/ai-sticker-pack/bot/.env, replace ADMIN_PANEL_PASSWORD (and ADMIN_PANEL_USER if needed).
2. systemctl restart sticker-api.
3. Tell the people who use the panel through a private channel.

Verify:

curl -s -o /dev/null -w '%{http_code}\n' https://stickers.eventtalks.uz/admin   # 401 without credentials

Then log in with the new password in a browser. Five failed logins from one address within 10 minutes block that address for 10 minutes (ADMIN_LOGIN_BLOCKED in the API log); restarting the API clears the block.

SSH key

Add the new public key to /root/.ssh/authorized_keys, confirm that login works in a second session, then remove the old key line. Never close the working session before the new key is proven.

After a suspected leak

1. Rotate the leaked secret now, using the procedure above. Rotation is the fix; deleting the message that leaked it is not.
2. If bot/.env as a whole leaked, rotate everything in it: bot token, both AI keys, admin password, LEADS_SECRET, LEADS_IP_SALT.
3. Check spend in the AI provider console and the admin panel audit log (/admin, section Журнал действий (audit) — “action log”) for actions you did not make.
4. If a secret reached git history, rotating it is still required; rewriting history does not un-leak it.

What must never happen

• A secret committed to git, in any repository, including the documentation repository.
• A secret printed to a log, a terminal recording, a chat, a ticket or a screenshot. Do not cat the .env file during a screen share.
• A secret in a Vercel variable whose name starts with NEXT_PUBLIC_. Those values are shipped to every browser.
• A secret on a command line (setenv LEADS_SECRET …, curl -H "…: <value>"). It lands in shell history and in the process list. Read it from the file into a variable instead, as the test command in Deploy and rollback does.
• The production AI key used for experiments. Use TEST_GEMINI_API_KEY and EVENT_MODE=1.
• bot/.env readable by other users. It must stay sticker:sticker, mode 600.
• A copy of the production .env left in a downloads folder, a cloud drive or a mail attachment.

Safeguards in the code

Safeguard	Where
bot/.env, .env, bot/data/, *.db, *.bak-*, logs/ and out/ are ignored by git	.gitignore
The upload takes only committed files and then refuses the archive if it contains .env, a venv, data/, logs/, sessions/, debug_sheets/, .bak files or .git/	deploy/push.ps1, tests/test_deploy.py
setup.sh never touches bot/.env; push.ps1 and update.sh only re-apply mode 600 and the owner	deploy/setup.sh, deploy/update.sh
prod refuses to start without the bot token, bot username, an AI key, ADMIN_CHAT_ID and ADMIN_PANEL_PASSWORD; with LEADS_SECRET shorter than 24 characters; with an empty LEADS_IP_SALT	bot/config.py
prod refuses to start when GEMINI_API_KEY equals TEST_GEMINI_API_KEY; the bot and the API never read the test key	bot/config.py, bot/services/event_guard.py
Secrets are compared in constant time; wrong admin logins and wrong leads secrets are rate limited (5 per 10 minutes per address)	bot/admin_panel.py, bot/leads_api.py
Admin POST actions need the X-Admin-Action header (CSRF protection) and are written to the audit log	bot/admin_panel.py
/health returns version, mode and counters, never a secret	bot/api.py
Lead logs contain only the lead number, request_id, locale and UTM source, never name, username or phone	bot/leads_api.py
The landing reads LEADS_SECRET and the bot token only on the server side	src/lib/deliverRequest.ts
Values are stripped of stray whitespace and carriage returns when read	bot/config.py

Where to go next

• Configuration — the non-secret settings in the same file.
• Deploy and rollback — creating bot/.env on a new server.
• Security — the threat model behind these safeguards.
• Leads missing or fallback — what a secret mismatch looks like in production.

---

Last verified: 2026-11-13 against ai-sticker-pack@b715418 and ai-stickers-landing@e350ff5.