Skip to content

test: stop integration suites failing on container teardown timeout - #466

Merged
tianzhou merged 2 commits into
mainfrom
claude/sharp-cray-ihgohl
Oct 9, 2026
Merged

tianzhou merged 2 commits into
mainfrom
claude/sharp-cray-ihgohl

Conversation

@tianzhou

@tianzhou tianzhou commented Oct 9, 2026

Copy link
Copy Markdown
Member

Problem

Integration CI fails intermittently on green code. Pulling the failed-job logs for every red Run Tests run over the last two months, the flaky failures are all the same line:

Error: Hook timed out in 10000ms.
 ❯ src/connectors/__tests__/mariadb.integration.test.ts:171:3   (afterAll)

Seen on MariaDB twice (2026-10-09, runs 37904209709 and 37896731506) and on postgres-ssh once (2026-09-28, run 36420437936). In each case every test passed and only the suite's afterAll timed out. The other red runs in that window were genuine failures on WIP branches, not flakes.

Root cause

Teardown is Testcontainers' stop(), which runs docker stop -t 0 followed by docker rm -v (removing the container's anonymous data volume). All integration suites run in parallel against one Docker daemon. While Oracle (1.2GB image) or SQL Server is still pulling/extracting, removing a sibling container plus its volume can exceed Vitest's default 10s hookTimeout.

Every beforeAll in these suites already carries an explicit 60–360s timeout; every afterAll relied on the default. The postgres-ssh case confirms it is the container stop and not the connector: that suite's afterAll only calls container.stop().

Changes

  • vitest.config.ts: hookTimeout: 60_000 for the integration project, with a comment explaining why. Setup hooks keep their explicit per-suite timeouts.
  • IntegrationTestBase.cleanup(): stop the container in a finally so a failed disconnect() can't leak a running container into the next suite.
  • .claude/skills/testing/SKILL.md: troubleshooting entry for the symptom and cause.

Verification

  • tsc --noEmit passes.
  • pnpm test:unit: 1201 tests pass.
  • Docker-free integration suites (multi-sqlite-sources, tool-metadata, api/sources) pass under the new config.
  • Container suites could not be run in the authoring sandbox (no Docker); CI on this PR is the confirmation.

Not done on purpose: limiting integration file concurrency would also reduce the contention but slows CI noticeably; the timeout fix addresses the failure directly.

🤖 Generated with Claude Code

https://claude.ai/code/session_01TZPfFnRTrHzHhsCzjrZjgM


Generated by Claude Code

Recent CI failures on green code (MariaDB twice, postgres-ssh once) all
read "Hook timed out in 10000ms" at the suite's afterAll. Teardown is
Testcontainers' stop(), i.e. `docker stop -t 0` + `docker rm -v`. The
suites share one Docker daemon, and while Oracle (1.2GB) or SQL Server is
still pulling/extracting its image, removing a sibling container with its
data volume can exceed Vitest's default 10s hookTimeout. Every beforeAll
already carries an explicit 60-360s timeout; afterAll relied on the
default.

- vitest.config.ts: hookTimeout 60s for the integration project.
- IntegrationTestBase.cleanup(): stop the container in a finally so a
  disconnect failure can't leak a running container.
- testing skill: document the symptom and cause.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TZPfFnRTrHzHhsCzjrZjgM
Copilot AI balanced review requested due to automatic review settings October 9, 2026 08:56

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The functional changes are focused and correct; only a minor overstatement in a code comment remains.

1 open finding
What changed in this PR

Improves integration-test teardown reliability under Docker contention.

Changes:

  • Raises integration hook timeout to 60 seconds.
  • Ensures containers stop after disconnect failures.
  • Documents teardown timeout troubleshooting.
File Description
vitest.config.ts Configures integration hook timeout.
src/​connectors/​__tests__/​shared/​integration-test-base.ts Makes container cleanup resilient to disconnect failures.
.claude/​skills/​testing/​SKILL.md Documents the CI failure mode and convention.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/connectors/__tests__/shared/integration-test-base.ts Outdated
A pending disconnect() never reaches the finally, so the container-stop
guarantee covers thrown errors, not hangs; say so.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TZPfFnRTrHzHhsCzjrZjgM
@tianzhou
tianzhou merged commit 8b5d45a into main Oct 9, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants