Deployment

Target

The pilot runs under the dedicated OVH OS user heyaira with rootless Docker:

• application checkout: /srv/heyaira/app;
• protected token files: /srv/heyaira/app/secrets;
• Compose project: /srv/heyaira/app/compose.yaml;
• PostgreSQL data: /srv/heyaira/app/postgres-data;
• application loopback port: 127.0.0.1:18080;
• public MCP endpoint: one address per project, https://mcp.heyaira.eu/p/<project-slug>/mcp (the shared https://mcp.heyaira.eu/mcp is deprecated but still serves and still issues new grants).

The application container and PostgreSQL container must not publish PostgreSQL to a public host interface. The production .env file and project token file stay outside Git and are readable only by heyaira.

Bridge enrollment needs HEYAIRA_BRIDGE_ENROLLMENT_SECRET and HEYAIRA_BRIDGE_TOKEN_SECRET (see the enrollment contract). Server startup does not check them. With either one empty the server still starts, the /bridge/v1/enrollment/* and /bridge/v1/tokens/* routes answer 503 bridge_enrollment_unavailable, and heyaira-admin recover-bridge-token refuses to run. Nothing checks their length or that they differ, so set two independent random values. Production startup does enforce the OpenBao settings for the phone push-token vault (HEYAIRA_OPENBAO_ADDR and HEYAIRA_OPENBAO_TOKEN_FILE; see BRIDGE_OPENBAO).

DNS and HTTPS

The heyaira.eu zone is now managed by Cloudflare. The active MCP record is:

mcp.heyaira.eu.  A  51.83.159.109  (Proxied)

Cloudflare's edge certificate is active and the zone is currently using Full encryption. The host administrator can run the repository's ops/caddy-heyaira.sh helper as root. It creates a recoverable backup, adds only the HeyAira site when absent, validates the full configuration, and reloads Caddy. Its resulting site block is:

mcp.heyaira.eu {
    reverse_proxy 127.0.0.1:18080
}

For the current Dockerized Caddy deployment, the active site block uses the private relay 172.21.0.1:18081 instead. Keep the bootstrap script's direct loopback example only for hosts where Caddy runs directly on the host; do not replace an existing relay route without validating the complete Caddy setup.

The script must be run on the OVH host from a checkout of this repository, for example:

cd /srv/heyaira/app
sudo ./ops/caddy-heyaira.sh

The heyaira runtime user intentionally cannot run this command. It must be executed through a separate root-admin channel; do not use Voyana credentials. After Caddy obtains its origin certificate, change Cloudflare SSL/TLS encryption from Full to Full (strict).

Caddy provides the public TLS certificate. Do not expose port 18080 directly; it is the private edge-to-container hop.

Client attribution

/register and /authorize admit at most a small share of their global caps per client network (src/heyaira/client_source.py). The app works out the client from the headers. It believes CF-Connecting-IP only when the rightmost public X-Forwarded-For entry is a Cloudflare address. Caddy writes that entry by default, as the address that connected to it. A request that reached the origin without Cloudflare is keyed on its own address. Any Caddy or relay in front of the app must keep forwarding X-Forwarded-For. Without it, every request falls into one shared bucket, and the app logs client_source_unattributed once.

uvicorn runs with proxy_headers=False, so the app sees the real TCP peer. The public scheme comes from X-Forwarded-Proto instead. The app accepts it only from a loopback or private peer, and only as http or https. Caddy sets it to https, and passes Host through unchanged. Every URL the app builds from the request, such as Starlette's trailing-slash redirect, then stays on https. The OAuth issuer and every endpoint and consent URL come from HEYAIRA_PUBLIC_URL, never from the request.

Refreshing the Cloudflare ranges:

1. Compare CLOUDFLARE_NETWORKS in src/heyaira/client_source.py with https://www.cloudflare.com/ips-v4 and https://www.cloudflare.com/ips-v6.
2. Update the list if it differs, and set CLOUDFLARE_NETWORKS_CHECKED to today. tests/test_client_source.py::test_built_in_cloudflare_ranges_are_not_stale fails 180 days after that date.
3. To change the ranges in production without a release, set HEYAIRA_CLOUDFLARE_NETWORKS in .env to the full list of CIDRs, comma or space separated. It replaces the built-in list, it is passed to the server container by compose.yaml, and recreating server applies it. A malformed, private or implausibly wide network stops the server at startup.

Application deployment

Immutable production release guard

Production releases must use the repository's tracked ops/deploy-heyaira.sh wrapper. Do not copy an ad-hoc deployment command into the server or run docker compose up directly for a release. The wrapper requires the exact production overlay ops/openbao-production-compose.yaml; it rejects a missing or different override before creating a release directory or changing running services.

The same override is passed to the initial compose config, build, start, health check and rollback commands. A rollback source that does not contain the overlay is rejected before the new release is started. This is required because the base compose.yaml intentionally leaves the OpenBao settings empty; starting it without the production overlay fails closed and can leave both the new release and an attempted rollback unhealthy.

From a standalone checkout, as the dedicated heyaira user:

./ops/deploy-heyaira.sh \
  --source /secure/heyaira-source \
  --sha <full-40-character-commit-sha> \
  --app-root /srv/heyaira \
  --config-dir /srv/heyaira/config \
  --compose-project app \
  --compose-override ops/openbao-production-compose.yaml \
  --compose-extra /srv/heyaira/config/compose.restart.yaml \
  --services server

A release must say what it may recreate. --services NAME... builds and starts only those services (docker compose up -d --no-deps); every other container stays running untouched. OpenBao and PostgreSQL cannot be named: recreating OpenBao seals the vault, which only the owner may undo, and the database is never recreated by a release. --all-services recreates every other active service; it is refused while an OpenBao container is running, and it always names those services explicitly with --no-deps, never PostgreSQL or OpenBao. Only a true first bootstrap, where neither a PostgreSQL nor an OpenBao container exists in any state, uses a bare up -d (bootstrap=first-start). --compose-extra FILE (repeatable) keeps a host-local overlay such as the restart-policy file in every compose call, including rollback.

A release that starts server-beat or server-beat-runner is refused unless ops/server-beat-repair-compose.yaml is one of the --compose-extra files (docs/SERVER_BEAT_RUNBOOK.md, Activation). Pass it relative (--compose-extra ops/server-beat-repair-compose.yaml, the release's own copy); any file with that name must have exactly the release's content, so an empty or stale same-named file is refused before anything changes. The overlay must also win: before anything changes, the script renders the effective configuration (docker compose config --format json with exactly the release's files) and refuses when any environment value or volume mount the overlay contributes to the Beat services (pulse directory, pulse token, SES configuration, repair broker socket, restart switch, and their mounts) is overridden by another Compose file. This check needs python3 on the host. After the start the script checks that each Beat container's com.docker.compose.project.config_files label lists one of those exact overlay paths and waits up to 120 seconds for one pulse-out work item that succeeded after the new containers started and was completed by the deployed runner itself (its worker ID server-beat-runner-<container hostname> on the succeeded work event; a success by another runner sharing the database does not count); otherwise it rolls back. The pulse-out goes to the external Watchdog on Cloudflare, which this host does not control: when every attempt of the deployed runner in that window failed only with WatchdogUnreachable (a network failure or a Cloudflare 5xx, after the runner claimed the job, found the beat fresh and read its token), the release is accepted with pulse_out=watchdog-unreachable and a warning, instead of being rolled back (or, after a migration, stranded at rollback=refused) by a Cloudflare outage. Confirm a successful pulse-out once Cloudflare answers. A 4xx answer, a stale beat, a missing token or no attempt at all still fails the release. The 120 seconds are a real deadline: every query gets at most the remaining time, both as PostgreSQL statement_timeout/lock_timeout and as a process timeout, so a lock on control_work_items cannot hold the deploy. Every other database query of the script is bounded to 30 seconds the same way. PostgreSQL and OpenBao are never revision-checked: neither carries the release revision, so a release does not recreate them.

Release images, rollback and redeploy

Every built service runs an immutable per-commit image, heyaira-<service>:<sha> (compose.yaml), labelled org.opencontainers.image.revision=<sha>. The script pins those tags with a generated Compose overlay in every call it makes, so a release, its rollback and a redeploy each start exactly their own image. After the start it checks, for every service, both the HEYAIRA_SERVICE_REVISION the container reports and that the container's image ID is the ID behind the release's tag. The variable alone proves nothing: before per-commit tags a rollback restarted the newly built image with the old variable (audit b6260428, P1-3).

Before it changes anything, the script snapshots the image each service is running under a tag of this deployment, heyaira-<service>:rollback-<new sha> (rollback_snapshot[...]). A snapshot claims nothing about which release built the image: services of one project can run different releases. The canonical tag heyaira-<service>:<previous sha> is created only when the running image's revision label names that release. The rollback handler exists before the release starts, so a start that fails half-way (some containers already replaced) rolls back too. A failed release rolls back to exactly the snapshots (up -d --no-deps --no-build, never a rebuild) and stops services that were not running before. It prints rollback=verified only when every rolled-back container runs its snapshot image ID and every stopped service has no running container; a failed start or stop, or any mismatch, prints rollback=failed.

--redeploy --sha <sha> (no --source) starts an existing release under releases/<sha> again, for example a deliberate rollback to an earlier commit; a normal deploy of an existing release is refused and points to it. It reuses that release's images when their revision label names the release, and otherwise (removed, unlabelled or re-pointed tag) rebuilds them from the immutable release directory. It verifies, rolls back and switches current like a normal release.

Old code never runs on a newer schema. Every service refuses to start when the database has an applied migration the code does not know: not one of its migrations/*.sql files and not a renamed historical name listed in migrations/retired-names.txt (heyaira.db.refuse_newer_schema). Comparing the complete set, not the greatest name, also catches a later release's new migration that sorts between older files. New migrations take the next free number (tests/test_schema_guard.py enforces unique, gapless numbers; the existing duplicates are grandfathered), and a rename adds the old name to retired-names.txt. The script applies the same rule before it rolls back (the failed release may already have migrated: rollback=refused, see "A failed release that already migrated" below) and before a --redeploy.

Migrations

heyaira.db.apply_migrations runs at the start of every database service, in one transaction under one advisory lock, so concurrent starts apply each file once.

• Every lock a migration waits for (an ALTER TABLE behind a long transaction, for example) is bounded: lock_timeout 5 s, after which the whole transaction rolls back and is retried, six attempts with a growing pause. Then the service stops with MigrationLockTimeout and nothing applied; find the blocking session in pg_locks/pg_stat_activity. Only the wait for another process's migration lock is unbounded, and that process's own lock waits are bounded the same way. Statements themselves have no time limit: a migration may legitimately run long.
• schema_migrations.checksum records the SHA-256 of each file when it is applied. A service refuses to start (MigrationChecksumError, naming the files) when an applied file's content changed since. Never edit a merged migration: restore its content and put the change in a new file with the next number. A disposable development database that applied a draft can simply be recreated. The column is the runner's own bookkeeping, added in place like the table itself, not a numbered migration, so older releases still run on the schema and ignore it. Rows without a checksum (applied before this, or by an older release) are filled in from the current file on the next start; retired names have no file and are never compared.

Restore point before a migration

A release that carries a migration the database has not applied first writes a dump, because a migration is the one change a rollback to the previous images cannot undo. The script compares the release's migrations/*.sql with schema_migrations after the database-client check and before anything is built, tagged or started, and when a file is new it:

1. writes pg_dump -Fc to APP_ROOT/predeploy/heyaira-<sha>-<UTC time>.dump (mode 0600, through a .part file that is renamed only once verified);
2. checks that the file is at least 10 KiB, starts with the PGDMP header, that pg_restore --list reads it and lists schema_migrations, and that pg_restore -f /dev/null reads every block of it (a cut-off dump still lists fine);
3. logs restore_point_path, restore_point_bytes, restore_point_sha256, restore_point_entries and restore_point_verified=pg_restore-list-ok.

Any failure (database unreadable, directory not writable, pg_dump error or timeout, a dump that is too small, not a dump, or unreadable) stops the deployment with refusing release ...; nothing was built or started, and no migration was applied. The new release directory is removed, so the same command can be run again after the cause is fixed. A release with no new migration, a database without schema_migrations, and a release that starts no database service need no dump and behave as before. A bootstrap (no PostgreSQL container exists) needs none only when no <compose project>_postgres-data volume exists either; with that volume present, nothing can be dumped before the release migrates it, so the release is refused until PostgreSQL is running or --skip-restore-point is given. Whatever else fails inside this step, the script removes the new release directory on exit so the same command can be run again. The dump covers PostgreSQL only; backups to R2 and their schedule are unchanged.

--skip-restore-point "REASON" deploys without the dump. It needs a non-empty one-line reason (at most 300 characters, no control characters), prints it, verbatim, in the preflight and as restore_point=skipped reason=..., and is meant only for a deployment the owner approved without one; copy the reason into the RESULT. --dry-run shows the directory and the reason but, like the rest of the dry run, does not touch the database.

A failed release that already migrated

A release applies its migrations when its first database service starts. If it then fails (health, revision, image or pulse check), the previous release usually cannot run on the migrated schema, so the script does not start it: it prints rollback=refused and stops. Nothing is rolled back: the failed release's containers keep running, and current still names the previous release. Recover forward, in this order:

1. Failure outside the code (configuration, .env, a host overlay, the health port, Cloudflare): fix it and start the same release again with --redeploy --sha <failed sha> and the same --services. Its release directory exists, and the schema guard accepts it because it knows every applied migration.
2. Failure in the code: commit the fix on top of the failed commit, keeping its migrations byte-identical (a changed applied file is refused, see Migrations), with any schema correction in a new migration, and deploy that commit normally. Its rollback source is still the previous release, so if it fails too it again stops at rollback=refused instead of starting old code on the new schema.
3. Only when neither is possible (a migration destroyed data): the owner decides to restore the newest verified backup taken before the release (backup runbook, "Restore procedure" and "Credentials after a production restore") and then --redeploy --sha <previous sha>. Everything written after that backup is lost.

Never delete rows from schema_migrations or revert a migration by hand to get past the guard: the code that would then start does not match the schema it runs on.

A release also may not leave a running database client behind the migrations it applies. Before anything is built or started (release and --redeploy alike), the script takes the release's migrations/*.sql that are not yet in schema_migrations. When there are any, every database client container the release does not recreate must already know them. The script finds the project's containers in every state by the Compose project label (so inactive profiles and services no longer in compose.yaml count too) and judges each container, every replica, on its own: - it counts if it runs, or is stopped with a restart policy other than no (it comes back by itself); - it is a database client if it belongs to server, backup, server-beat or server-beat-runner, to a service of the active profiles that depends on postgres or sets HEYAIRA_DB_HOST, or if its own configuration sets HEYAIRA_DB_HOST or carries a postgres dependency label (only variable names are read, never values); - what it knows is its own migrations/*.sql and retired-names.txt, the set its startup check uses: read inside the running container, or from its image in a throwaway docker run --rm --network none when it is stopped or cannot be executed. Unreadable counts as not knowing.

Otherwise the release is refused (SchemaAheadError on the client's next restart) with the --services list to use instead, and the release directory this run created is removed, so the corrected command can be run at once. A release that applies no new migration is never refused for this, so a server-only deploy stays possible. retention is not a database client: it never connects to PostgreSQL. This check needs python3 on the host.

First deploy with this script, from the current production state (releases started by the operator script, containers running app-<service>:latest, rollback copies tagged app-<service>:rollback-<id>): no manual step. The script snapshots each running container's image as heyaira-<service>:rollback-<new sha> before it builds, and a rollback restarts that snapshot. The unlabelled app-<service>:latest images get no canonical release tag (their provenance is unknown); a later --redeploy of such an old release rebuilds it from its release directory. The old app-<service>:* tags are no longer used by the script; remove them once the new release is accepted.

The transition deploy must be scoped: run it with --services naming the application services, never --all-services and never a manual docker compose up. This change and #107 change PostgreSQL's Compose configuration (its environment no longer carries the release revision, and it gains a restart policy), so any unscoped Compose start would recreate the database container. The script never names PostgreSQL in a release, but a scoped transition keeps that guarantee independent of the script version on the host. Keep at least the current and previous heyaira-<service>:<sha> images when pruning; an earlier release whose image was pruned can still be redeployed (it is rebuilt from its release directory).

The base compose.yaml now carries restart: unless-stopped for every long-running service, PostgreSQL and the server included, so a host reboot brings the database back without the host-local /srv/heyaira/config/compose.restart.yaml. That overlay still wins where it sets a policy (later Compose files override earlier ones), so keep passing it with --compose-extra until it is retired; it then changes nothing.

Every service in compose.yaml logs through Docker's json-file driver with rotation (10 MB per file, five files), so a crash loop cannot fill the disk. Application services carry a memory limit (server 1 GB, backup 768 MB, retention 384 MB, server-beat-runner 512 MB, server-beat 256 MB): a leaking process is killed and restarted by its restart policy instead of starving the host. PostgreSQL has no memory limit on purpose (an OOM kill of the database is worse than memory pressure). A release never recreates PostgreSQL, so its new logging setting applies only when the owner recreates it; the application services pick theirs up with their next release.

Run the deterministic preflight first when preparing a release:

./ops/deploy-heyaira.sh \
  --source /secure/heyaira-source \
  --sha <full-40-character-commit-sha> \
  --app-root /srv/heyaira \
  --config-dir /srv/heyaira/config \
  --compose-project app \
  --compose-override ops/openbao-production-compose.yaml \
  --compose-extra /srv/heyaira/config/compose.restart.yaml \
  --services server \
  --dry-run

The external /srv/heyaira/bin/deploy-heyaira.sh installation must be kept identical to this tracked script. Updating the script on the host is a separate reviewed operational step; never weaken the mandatory overlay check to accommodate an older release. The guard tests cover missing, incorrect and valid overlays and assert that rejected preflights do not create release directories.

For the initial host bootstrap only, before the first immutable release is installed, the operator may prepare the protected paths and validate the Compose project as heyaira:

cd /srv/heyaira/app
chmod 600 .env
mkdir -p secrets && chmod 700 secrets
docker compose up -d --build
docker compose ps
curl --fail http://127.0.0.1:18080/healthz

These direct Compose commands are not a production release procedure. Once the immutable wrapper is installed, all production starts, updates and rollbacks must go through ops/deploy-heyaira.sh with the required override.

Before treating a rebuilt server as ready, run the unauthenticated Bridge route canary. It performs no write and must be run against the exact container that has just started:

python scripts/verify_bridge_routes.py --base-url http://127.0.0.1:18080

The canary expects healthz=200, a method rejection (405) for a heartbeat GET, and authentication rejection (401) for protected Bridge routes. A 404 is always a failed deployment, even when /healthz is healthy. Stop the rollout and restore the last verified application checkout/image before reloading or changing the public edge. Do not enable APNs or copy Apple credentials until this check passes and an authenticated Bridge E2E check has also passed.

Create a project token once, writing it to a protected path. The command never prints the token:

docker compose exec server heyaira-admin create-project \
  --name "HeyAira" \
  --repository-url "https://github.com/h8v6/heyaira" \
  --slug "heyaira" \
  --description "What the project is about" \
  --token-label "primary" \
  --token-file /run/heyaira-secrets/heyaira.token

--slug gives the project its own MCP address, printed as mcp_url=https://mcp.heyaira.eu/p/<slug>/mcp. Set or change the slug, name or description of an existing project with:

docker compose exec server heyaira-admin update-project \
  --project-id "<project-id>" --slug "voyana" --description "What the project is about"

A slug is lowercase letters, digits and hyphens, unique on the instance. It is part of every grant's resource binding, so changing an existing slug requires --replace-slug and every client connected to the old address must reconnect.

The token is then entered into the lab's protected MCP client configuration. It must not be put into GitHub, /docs, shell history, or a chat message.

Project access is administered separately from the MCP protocol. Grant a stable identity a project role with:

docker compose exec server heyaira-admin add-membership \
  --project-id "<project-id>" \
  --subject "github:user:<numeric-id>" \
  --role contributor

Bind another GitHub repository to the project without copying its contents:

docker compose exec server heyaira-admin link-repository \
  --project-id "<project-id>" \
  --github-owner "<owner>" \
  --github-repository "<repository>" \
  --repository-url "https://github.com/<owner>/<repository>" \
  --default-branch main

GitHub App installation and repository API operations remain a separate integration; these commands only maintain HeyAira's project bindings. After installing the read-only GitHub App on the repository, bind its installation ID to the project:

docker compose exec server heyaira-admin link-repository \
  --project-id "<project-id>" \
  --github-owner "<owner>" \
  --github-repository "<repository>" \
  --repository-url "https://github.com/<owner>/<repository>" \
  --github-installation-id "<installation-id>" \
  --default-branch main

Store the App private key outside Git, for example at /srv/heyaira/app/secrets/github-app.pem with mode 0600, and set these production environment variables:

HEYAIRA_GITHUB_APP_ID=<numeric-app-id>
HEYAIRA_GITHUB_APP_PRIVATE_KEY_FILE=/run/heyaira-secrets/github-app.pem
HEYAIRA_GITHUB_WEBHOOK_SECRET_FILE=/run/heyaira-secrets/github-webhook.secret

For a repeatable setup, use the single connection command below after the one-time GitHub App installation. It validates the App ID, protected key and webhook secret, mints an installation token, checks repository access through GitHub, and then writes the project binding. It never prints a token or key.

docker compose exec server heyaira-admin github-connect \
  --project-id "<project-id>" \
  --github-owner "<owner>" \
  --github-repository "<repository>" \
  --github-installation-id "<installation-id>"

Use --check-only --json to verify an existing installation without changing the database. A successful setup prints github_connection=connected; any failed check exits with status 2 and identifies only the failing check. The Bridge can invoke this local command under its host policy; the server Bridge API does not expose arbitrary remote shell execution.

The server mints short-lived installation tokens on demand. Never put the private key, an installation token, or a personal access token in the repository, a task, or a chat message. Configure the GitHub App webhook URL as https://mcp.heyaira.eu/github/webhooks; HeyAira validates X-Hub-Signature-256, deduplicates X-GitHub-Delivery, and stores bounded event and workflow-run metadata only. Subscribe the GitHub App to Pull request events as well as Workflow run, with Pull requests: Read permission. HeyAira acts on pull_request closed only; merges and closes are learned from that signed delivery, with no new GitHub API request. Rebuild the server and beat images together; no migration or schema-driven DB-client redeploy is required. After deployment, merge one test PR and verify publication_checked history with source: github_webhook.

Revoking access

Two operator commands end access at once. Each runs in one transaction and prints what it revoked.

A leaked or retired project token, together with everything granted through it in any project: OAuth grants issued on its consent, unused authorization codes, its memberships, catalog editor grants, registered phones, the Mac bindings it created, and the work it dispatched or that runs on those bindings (those tasks become blocked, never accepted). A consent, code exchange, refresh or dispatch in progress at that moment either finishes first and is revoked with the rest, or finds the token gone:

docker compose exec server heyaira-admin revoke-token \
  --project-id "<project-id>" --label "<token-label>"

Use --token-id when several tokens share a label. The command refuses to revoke the last owner or admin token of any project the token administers; create another first, or pass --allow-last to leave that project without one on purpose. A contributor token is never held back by this rule.

A lost, sold or compromised Mac. This stops its offered and running executions, including a verifier offered or a dispatch committed while the command runs (their tasks become blocked, never accepted), revokes its transport tokens and pending rotations, every project binding, the identity itself and the paired node with its credentials:

docker compose exec server heyaira-admin revoke-bridge-identity \
  --project-id "<enrollment-project-id>" --bridge-identity-id "<identity-id>"

Repeating either command is harmless. A revoked Mac enrolls again as a new identity; nothing is restored.

Bridge expired-token recovery

Routine Bridge operation rotates the transport credential before expiry. Use operator recovery only when no valid active token remains. On the server, write the replacement to a new protected path; the command refuses to replace a healthy active token and never prints the secret:

docker compose exec server heyaira-admin recover-bridge-token \
  --project-id "<project-id>" \
  --bridge-id "<bridge-id>" \
  --token-file /run/heyaira-secrets/bridge-recovery.json

Transfer that file through the authorized administrative channel, then stop the running Bridge process and pipe the file to the matching reviewed client:

heyaira-bridge token recover --server-url https://mcp.heyaira.eu \
  < /secure/local/path/bridge-recovery.json

The client validates the project and Bridge binding through the authenticated status route before changing Keychain state. It refuses recovery if a local rotation journal contains a pending successor. Remove the transferred file after successful validation, restart the Bridge, and verify both transport status and the legacy heartbeat separately. Never paste the file into logs, HeyAira memory, task descriptions, or work receipts.

Notification project copies

Migration 036 lets a task event be stored once for its project (node_id null) as well as once per Mac, so a project with no Mac still reaches its phones and inbox. The same change lets the Mac feed hand out a cursor that also carries the event id, so more than 500 events in one instant are paged without loss (see Who gets a task event). A server from before 036 can read neither format, so the change is rolled out in two deploys:

1. Deploy the code with HEYAIRA_NOTIFICATIONS_PROJECT_COPIES unset (off). Migration 036 runs. The server writes exactly what the previous release wrote, and it can already read project copies and id-carrying cursors. Rolling back to the previous release is safe, because nothing new has been written.
2. Set HEYAIRA_NOTIFICATIONS_PROJECT_COPIES=true on the server service and redeploy it. To roll back, unset the variable and redeploy the code from step 1, not an older release. That code delivers the pushes already queued on project copies, and it reads the cursors the Macs have stored. The Bridge needs no change in either step.

If a release from before 036 ran with project copies in the database (a rollback past step 1), its push worker cannot read them. It fails their pushes with delivery_configuration_error, and failed is final. Once step 1's code or later is running again, queue those pushes again:

docker compose exec server heyaira-admin requeue-project-pushes

The command prints project_pushes_requeued=<count>. It only touches failed pushes of project copies with that error code, and only for phones that are still active, so running it twice does nothing the second time. The statement it runs is REQUEUE_PROJECT_PUSHES_SQL in src/heyaira/admin.py.

A Mac that was handed an id-carrying cursor and then talks to a release from before 036 gets 400 invalid_cursor until the server is back on step 1's code or later. That is why the rollback target after step 2 is step 1.

Verification

Run the synthetic smoke test from a machine outside the OVH host after DNS and HTTPS are active:

.venv/bin/python scripts/smoke_mcp.py \
  --url https://mcp.heyaira.eu/p/heyaira/mcp \
  --token-file /secure/local/path/heyaira-token \
  --profile /secure/local/path/heyaira-public-profile.json

The acceptance check includes unauthenticated rejection, MCP discovery, memory write/read, task state, work receipt persistence, and stale-version rejection.

For the local independent-client continuity check, run:

.venv/bin/python scripts/multi_client_smoke.py \
  --url http://127.0.0.1:18080/mcp \
  --token-file secrets/heyaira.token \
  --profile /secure/local/path/heyaira-local-test-profile.json

The test uses two separate MCP client sessions and synthetic data only. It does not share a transcript or a scratch file between the simulated sessions.

Binding activation and rollback

The compatible new pair is the server at the BIND-FIX result commit and the client scripts from that same commit, using an operator-selected profile whose resource, assigned instance_id, and project UUID are pinned. Activation is ordered: apply migrations 006 and 007, set explicit HEYAIRA_INSTANCE_ID, HEYAIRA_RESOURCE (or leave it empty so it derives from the explicit public URL), and HEYAIRA_ENVIRONMENT, deploy the server, refresh client tool schemas, verify server_identity against the profile for programmatic clients, run the synthetic smoke suite, reconnect the web clients through OAuth so their credentials receive the instance binding, and only then enable writes.

The compatible rollback pair is the pre-binding server/client revision 45dd81e6f7effeb28c019cee833d90346ca6cbf7 with its legacy static-token profile. Rollback requires redeploying that server revision together with the matching pre-binding clients; selecting an old client profile while leaving the new server live only produces the documented refusal. Migration 006 is additive: do not drop its tables, delete audit history, or rewrite existing domain rows during rollback. Returning to a server without the binding guard removes the protection, so it is an operational compatibility rollback rather than a security-equivalent state.