Skip to content

Cancelling connect() during the SSLRequest transport setup leaves an unretrieved ConnectionError #1397

Open

Description

@alex-feel
  • asyncpg version: 0.32.0 (also reproduced on 0.31.0)
  • PostgreSQL version: none needed for the reproduction below
  • Do you use a PostgreSQL SaaS? If so, which? Can you reproduce
    the issue with a local PostgreSQL install?
    : no SaaS
  • Python version: 3.12.8 and 3.13.1
  • Platform: Linux and Windows (selector and proactor loops)
  • Do you use pgbouncer?: no
  • Did you install asyncpg with pip?: yes, the PyPI wheels
  • If you built asyncpg locally, which version of Cython did you use?: n/a
  • Can the issue be reproduced under both asyncio and
    uvloop?
    : asyncio; not tried under uvloop

When asyncpg.connect() with the default sslmode=prefer is cancelled while loop.create_connection() is still setting up the transport for the SSLRequest exchange, asyncio later logs:

Future exception was never retrieved
future: <Future finished exception=ConnectionError('unexpected connection_lost() call')>

_create_ssl_connection() creates the TLSUpgradeProto inside loop.create_connection(). When that call is cancelled after the protocol exists, the transport is closed and TLSUpgradeProto.connection_lost(None) sets the error on on_data, which nothing awaits any more. 0.32.0 handles the same situation for the connected future in __connect_addr() ("The protocol can exist before create_connection() returns"), but not for on_data.

Since 0.32.0 the pool runs into it by itself: Pool._maintain_min_size() opens a replacement connection after a dropped one, and Pool.close() cancels that task. With an unresponsive server the cancellation can land in this window, so closing a pool during a database outage can log the error above.

Reproduction with asyncpg alone, against a TCP server that never answers; the cancellation is tried after 1 to 5 loop iterations. The script pins the selector loop on Windows so the iteration numbers below are stable; the proactor loop shows the same at iterations 3 and 4:

import asyncio
import gc
import sys

import asyncpg


async def main():
    loop = asyncio.get_running_loop()
    reports = []
    loop.set_exception_handler(lambda _loop, context: reports.append(context))

    async def never_answer(reader, writer):
        try:
            while await reader.read(65536):
                pass
        finally:
            writer.close()

    server = await asyncio.start_server(never_answer, '127.0.0.1', 0)
    port = server.sockets[0].getsockname()[1]
    dsn = f'postgresql://user:secret@127.0.0.1:{port}/db'
    for steps in range(1, 6):
        task = asyncio.create_task(asyncpg.connect(dsn, timeout=5))
        for _ in range(steps):
            await asyncio.sleep(0)
        task.cancel()
        try:
            await task
        except asyncio.CancelledError:
            pass
        await asyncio.sleep(0.05)
        gc.collect()
        found = [f"{c['message']}: {c.get('exception')!r}" for c in reports]
        print(f'cancel after {steps} loop iteration(s): {found or "clean"}')
        reports.clear()
    server.close()
    await server.wait_closed()


if sys.platform == 'win32':
    asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())
asyncio.run(main())

Output with asyncpg 0.32.0 on Python 3.13.1 (0.31.0 prints the same):

cancel after 1 loop iteration(s): clean
cancel after 2 loop iteration(s): clean
cancel after 3 loop iteration(s): clean
cancel after 4 loop iteration(s): ["Future exception was never retrieved: ConnectionError('unexpected connection_lost() call')"]
cancel after 5 loop iteration(s): ["Future exception was never retrieved: ConnectionError('unexpected connection_lost() call')"]

Fix in #1398: cancel on_data, or retrieve its exception, when create_connection() fails or is cancelled, the way __connect_addr() does for connected.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions