Workflow stall worker evidence¶

Task: 84c5a67b-f760-4069-a452-d8bef64d65c8, frozen version 2. Execution: ab3b58d4-571a-49b6-bd79-82db09616c24. Repository: h8v6/heyaira; branch: heyaira/84c5a67b. Source and starting base: f79688a3c170fd06e6754122193566a7606f5492. No carried candidate or unrelated changes existed. No subagents were used.

Commands ran in /Users/jablonskib/HeyAira-agents/.heyaira-worktrees/heyaira/84c5a67b-worker, unless the base archive directory is specified. Python is the prepared Python 3.12.7; PYTHONPATH=src selects this checkout, and TMPDIR=$PWD/.pytest_cache/stall-tmp stays inside the writable worktree. Git commands use the existing Xcode Git binary through PATH; no installation. Initial /usr/bin/git emitted denied xcrun cache-write warnings, so subsequent Git reads used /Applications/Xcode.app/Contents/Developer/usr/bin/git directly.

Frozen checklist¶

K1: python -m pytest -q <S1 test file> on HEAD and on a base checkout with only the new test file added.
K2: python -m pytest -q <new and changed test files>.
K3: python -m compileall -q src/heyaira; git diff --check f79688a3c170fd06e6754122193566a7606f5492..HEAD.
K4: git diff --name-only f79688a3c170fd06e6754122193566a7606f5492..HEAD.

Implementation¶

The missing shared judgement allowed queued/running stages to look ordinary after their limits. stalls.py now judges the five rules from loaded inputs, with UTC anchors, environment-configurable limits and scheduled-sleep suppression. workflow.py loads executions, authorized node health and execution-specific events in two queries per batch; recovery scans 64-workflow keyset pages. Existing alert-copy/delivery uniqueness deduplicates the integer-epoch key. The same evaluator serves workflow reads, task summaries, continuity's oldest 20 stalls and Server Beat resolution. Push and e-mail use fixed text. No migrations, phase/lease/reaper changes, dependency installation, publication, production mutation or task-status changes were made.

K1: base negative control and S1 names¶

An archive checkout was created with:

git archive f79688a3c170fd06e6754122193566a7606f5492 -o .pytest_cache/stall-base.tar
mkdir -p .pytest_cache/stall-base
tar -xf .pytest_cache/stall-base.tar -C .pytest_cache/stall-base
cp tests/test_stalls.py .pytest_cache/stall-base/tests/test_stalls.py
cd .pytest_cache/stall-base
PYTHONPATH=src TMPDIR="$PWD/../stall-tmp" python -m pytest -q tests/test_stalls.py

Git archive, extraction and test copy succeeded. Only the new S1 file was added to the source archive. Pytest exited 2, with:

ERROR collecting tests/test_stalls.py
from heyaira import stalls
ImportError: cannot import name 'stalls' from 'heyaira'
(.pytest_cache/stall-base/src/heyaira/__init__.py)
1 error in 0.22s

An earlier attempted shell heredoc could not create its temporary file; that empty-test run exited 5 (no tests ran). It was not counted as the negative control. The corrected run above loaded the real 36-case test file.

The implementation working tree passed PYTHONPATH=src TMPDIR="$PWD/.pytest_cache/stall-tmp" python -m pytest -q tests/test_stalls.py: exit 0, 36 passed in 0.27s. HEAD verification is recorded below after the implementation commit.

python -m pytest -q --collect-only tests/test_stalls.py exited 0, 36 tests collected in 0.17s. Each S1 case is named here:

tests/test_stalls.py::test_rule_threshold_and_triggering_anchor[offer_unclaimed]
tests/test_stalls.py::test_rule_threshold_and_triggering_anchor[start_stuck]
tests/test_stalls.py::test_rule_threshold_and_triggering_anchor[lease_lost]
tests/test_stalls.py::test_rule_threshold_and_triggering_anchor[over_deadline]
tests/test_stalls.py::test_rule_threshold_and_triggering_anchor[bridge_away]
tests/test_stalls.py::test_bridge_away_precedes_other_rules[queued]
tests/test_stalls.py::test_bridge_away_precedes_other_rules[claimed]
tests/test_stalls.py::test_bridge_away_precedes_other_rules[preparing]
tests/test_stalls.py::test_bridge_away_precedes_other_rules[running]
tests/test_stalls.py::test_bridge_away_precedes_other_rules[verifying]
tests/test_stalls.py::test_non_stalling_phases[ready]
tests/test_stalls.py::test_non_stalling_phases[waiting_capacity]
tests/test_stalls.py::test_non_stalling_phases[awaiting_verification]
tests/test_stalls.py::test_non_stalling_phases[changes_requested]
tests/test_stalls.py::test_non_stalling_phases[blocked]
tests/test_stalls.py::test_non_stalling_phases[done]
tests/test_stalls.py::test_non_stalling_phases[cancelled]
tests/test_stalls.py::test_non_stalling_phases[failed]
tests/test_stalls.py::test_non_stalling_phases[interrupted]
tests/test_stalls.py::test_non_stalling_phases[retry_wait]
tests/test_stalls.py::test_asleep_never_stalls_even_with_other_overdue_anchors[queued]
tests/test_stalls.py::test_asleep_never_stalls_even_with_other_overdue_anchors[claimed]
tests/test_stalls.py::test_asleep_never_stalls_even_with_other_overdue_anchors[preparing]
tests/test_stalls.py::test_asleep_never_stalls_even_with_other_overdue_anchors[running]
tests/test_stalls.py::test_asleep_never_stalls_even_with_other_overdue_anchors[verifying]
tests/test_stalls.py::test_sleep_presence_boundary_is_separate_from_heartbeat_age_limit
tests/test_stalls.py::test_claim_and_session_timestamps_belong_to_active_execution[claimed]
tests/test_stalls.py::test_claim_and_session_timestamps_belong_to_active_execution[preparing]
tests/test_stalls.py::test_environment_overrides[offer_unclaimed]
tests/test_stalls.py::test_environment_overrides[start_stuck]
tests/test_stalls.py::test_environment_overrides[lease_lost]
tests/test_stalls.py::test_environment_overrides[over_deadline]
tests/test_stalls.py::test_environment_overrides[bridge_away]
tests/test_stalls.py::test_utc_serialization_and_epoch_normalization
tests/test_stalls.py::test_missing_or_different_execution_is_not_used
tests/test_stalls.py::test_skip_hint_is_bounded_and_does_not_expose_secrets

K2: changed-file suite and frozen-scope conflict¶

PYTHONPATH=src TMPDIR="$PWD/.pytest_cache/stall-tmp" python -m pytest -q \
  tests/test_stalls.py tests/test_stall_batching.py tests/test_stall_resolution.py \
  tests/test_stall_pipeline.py tests/test_notification_docs_contract.py

Working-tree result: exit 1, 1 failed, 60 passed, 6 skipped in 3.20s. The failure is:

tests/test_notification_docs_contract.py::test_docs_check_cli_runs_without_application_package_on_pythonpath
AssertionError: contracts/mcp-tools.json is stale; run `python scripts/export_mcp_contract.py`

The task mandates adding one sentence to each of the three MCP tool descriptions. The existing checker compares descriptions in the generated contract. A read-only build_contract() comparison exited 0 and found exactly these differences:

[('continuity_context', ['description']), ('task_list', ['description']), ('workflow_get', ['description'])]

K4 excludes contracts/mcp-tools.json. The worker preserved K4's closed scope and did not regenerate that file or weaken the documentation checker. This is a frozen-requirement conflict, not a database exception. K2 fails with failure kind criterion; it is not claimed as passed.

All six real-PostgreSQL regression cases were collected and skipped because HEYAIRA_EXECUTION_E2E_DATABASE_URL was not supplied:

tests/test_stall_pipeline.py::test_s2_recovery_enqueues_one_stall_and_one_phone_delivery_after_two_runs
tests/test_stall_pipeline.py::test_s3_claim_and_session_start_resolve_the_stall_alert
tests/test_stall_pipeline.py::test_s4_mcp_reads_show_stalled_and_healthy_workflows
tests/test_stall_pipeline.py::test_stall_escalation_is_queued_once_and_resolves_on_rule_or_anchor_change
tests/test_stall_pipeline.py::test_loader_filters_old_execution_events_and_skip_reasons
tests/test_stall_pipeline.py::test_loader_does_not_expose_unbound_foreign_node_health

These cases use the existing real PostgreSQL/HTTP fixture, with in-process ASGI transport; no listening socket or simulated database is substituted. S2–S4, the database escalation path and the full suite remain unverified locally.

Supplemental command:

PYTHONPATH=src TMPDIR="$PWD/.pytest_cache/stall-tmp" python -m pytest -q \
  tests/test_stall_resolution.py tests/test_bridge_presence.py tests/test_workflow_view.py \
  tests/test_unclaimed_offers.py tests/test_task_alert_escalation.py

Exit 0, 62 passed, 40 skipped in 5.26s; all skips reported missing isolated/disposable PostgreSQL. This does not substitute for the K2 suite.

K3 and K4 preparation¶

python -m compileall -q src/heyaira exited 0, no output. git diff --check exited 0, no output. The frozen range checks are recorded below after the implementation commit, rather than treating a precommit HEAD-at-base range as meaningful verification.

Blockers and next actor¶

what: K2's contract test fails because required tool-description changes need a generated file that K4 forbids changing. who_moves: coordinator. needed: a revised frozen task that authorizes contracts/mcp-tools.json and adjusts K4's permitted paths, or otherwise resolves this explicit conflict.

what: six required database regression cases are skipped; no isolated PostgreSQL URL is configured in the worker sandbox. who_moves: coordinator, then Bridge environment operator/CI. needed: a prepared isolated PostgreSQL environment for the named tests; no worker install.

After the scope is resolved, the first executable action is python scripts/export_mcp_contract.py, then rerun the changed-file suite. Independent verification, full CI, acceptance and publication remain with Bridge/the server. Confidence is high for the tested pure judgement, batching and epoch comparison; database behavior is prepared but not locally verified.

Committed implementation verification¶

Tested HEAD: 17fa1a1c5fcaf5aeaa97a5c0d37ab6518b6dd902, with parent f79688a3c170fd06e6754122193566a7606f5492. The implementation commit succeeded; Git's automatic pack-refs/commit-graph maintenance was denied by the sandbox (packed-refs.lock and commit-graph temporary directory). HEAD and its parent were read back, and git status --short was empty. These maintenance warnings did not prevent the local commit.

K1 HEAD¶

PYTHONPATH=src TMPDIR="$PWD/.pytest_cache/stall-tmp" python -m pytest -q tests/test_stalls.py

Exit 0. Actual output:

....................................                                     [100%]
36 passed in 0.33s

K2 HEAD¶

PYTHONPATH=src TMPDIR="$PWD/.pytest_cache/stall-tmp" python -m pytest -q tests/test_stalls.py tests/test_stall_batching.py tests/test_stall_resolution.py tests/test_stall_pipeline.py tests/test_notification_docs_contract.py

Exit 1. Actual output:

...................................................ssssss.F........      [100%]
=================================== FAILURES ===================================
______ test_docs_check_cli_runs_without_application_package_on_pythonpath ______

    def test_docs_check_cli_runs_without_application_package_on_pythonpath():
        environment = os.environ.copy()
        environment.pop("PYTHONPATH", None)
        result = run(
            [sys.executable, "scripts/check_docs_contract.py"],
            cwd=ROOT,
            env=environment,
            capture_output=True,
            text=True,
            check=False,
        )
>       assert result.returncode == 0, result.stderr
E       AssertionError: contracts/mcp-tools.json is stale; run `python scripts/export_mcp_contract.py`
E
E       assert 1 == 0
E        +  where 1 = CompletedProcess(args=['/Users/jablonskib/.local/share/mise/installs/python/3.12.7/bin/python', 'scripts/check_docs_co...'], returncode=1, stdout='', stderr='contracts/mcp-tools.json is stale; run `python scripts/export_mcp_contract.py`\n').returncode

tests/test_notification_docs_contract.py:63: AssertionError
=========================== short test summary info ============================
SKIPPED [1] tests/test_stall_pipeline.py:38: set HEYAIRA_EXECUTION_E2E_DATABASE_URL to an isolated PostgreSQL database
SKIPPED [1] tests/test_stall_pipeline.py:59: set HEYAIRA_EXECUTION_E2E_DATABASE_URL to an isolated PostgreSQL database
SKIPPED [1] tests/test_stall_pipeline.py:72: set HEYAIRA_EXECUTION_E2E_DATABASE_URL to an isolated PostgreSQL database
SKIPPED [1] tests/test_stall_pipeline.py:94: set HEYAIRA_EXECUTION_E2E_DATABASE_URL to an isolated PostgreSQL database
SKIPPED [1] tests/test_stall_pipeline.py:119: set HEYAIRA_EXECUTION_E2E_DATABASE_URL to an isolated PostgreSQL database
SKIPPED [1] tests/test_stall_pipeline.py:135: set HEYAIRA_EXECUTION_E2E_DATABASE_URL to an isolated PostgreSQL database
FAILED tests/test_notification_docs_contract.py::test_docs_check_cli_runs_without_application_package_on_pythonpath
1 failed, 60 passed, 6 skipped in 4.06s

K3 compile¶

python -m compileall -q src/heyaira

Exit 0. Actual output:

(no output)

K3 range¶

/Applications/Xcode.app/Contents/Developer/usr/bin/git diff --check f79688a3c170fd06e6754122193566a7606f5492..HEAD

Exit 0. Actual output:

(no output)

K4 range¶

/Applications/Xcode.app/Contents/Developer/usr/bin/git diff --name-only f79688a3c170fd06e6754122193566a7606f5492..HEAD

Exit 0. Actual output:

docs/reference/mcp-tools.md
docs/reference/notifications.md
docs/reports/84c5a67b-workflow-stalls-evidence.md
scripts/check_docs_contract.py
src/heyaira/bridge.py
src/heyaira/server.py
src/heyaira/server_beat.py
src/heyaira/stalls.py
src/heyaira/workflow.py
tests/test_notification_docs_contract.py
tests/test_stall_batching.py
tests/test_stall_pipeline.py
tests/test_stall_resolution.py
tests/test_stalls.py

Per-criterion worker record¶

criterion_id: K1
outcome: pass
evidence_path: docs/reports/84c5a67b-workflow-stalls-evidence.md
feedback: python -m pytest -q tests/test_stalls.py: HEAD exit 0, 36 passed; base archive exit 2, missing stalls module.

criterion_id: K2
outcome: fail
evidence_path: docs/reports/84c5a67b-workflow-stalls-evidence.md
feedback: Changed-file pytest: exit 1, 60 passed, 6 database skips, 1 stale MCP-contract test failure; K4 forbids refreshing contracts/mcp-tools.json.

criterion_id: K3
outcome: pass
evidence_path: docs/reports/84c5a67b-workflow-stalls-evidence.md
feedback: compileall exit 0; git diff --check source..HEAD exit 0, no output.

criterion_id: K4
outcome: pass
evidence_path: docs/reports/84c5a67b-workflow-stalls-evidence.md
feedback: git diff --name-only source..HEAD exit 0; all 14 paths allowed and no migrations.

This addendum changes evidence only; no implementation or test file changes followed the tested snapshot. The final result SHA and repeated final HEAD checks are recorded through task_result. This worker report is not independent acceptance. No subagents were used.