Skip to content

Closes #1472: Implement in-memory config backend for config overrides in context manager - #1503

Open
beamerblvd wants to merge 1 commit into
libgit2:masterfrom
beamerblvd:add-in-memory-config-overrides
Open

beamerblvd wants to merge 1 commit into
libgit2:masterfrom
beamerblvd:add-in-memory-config-overrides

Conversation

@beamerblvd

Copy link
Copy Markdown
Contributor

This implements #1472. When DefaultConfig and RepositoryConfig are constructed, they now construct an in-memory config backend with level ConfigLevel.APP that, initially, is unused. When instances of these classes are entered as a context manager, that in-memory backend is given sole write priority. When instances of these classes are exited as a context manager, that in-memory backend in cleared, and the default write priority is restored.

This pull request is the final subset of the changes in #1473 and a replacement for that pull request, which I will now close.

…errides in context manager

This implements libgit2#1472. When `DefaultConfig` and `RepositoryConfig` are constructed, they now construct an in-memory config backend with level `ConfigLevel.APP` that, initially, is unused. When instances of these classes are entered as a context manager, that in-memory backend is given sole write priority. When instances of these classes are exited as a context manager, that in-memory backend in cleared, and the default write priority is restored.
@jdavid

jdavid commented Oct 21, 2026

Copy link
Copy Markdown
Member

Kimi's review...

Blocker: PyPy test-suite hang (4 CI jobs × 6h timeouts)

gh pr checks 1503 shows exactly four failed jobs — linux-amd-glibc, linux-arm-glibc, macos-intel, windows-x64 — each canceled after ~6 hours. Those four are precisely the platforms that run PyPy tests (pp311); every platform without PyPy tests passes, and master's CI is green.

Digging into the job logs, all four die in the same place: the pp311 session completes test_filter.py::test_filter_buffered and then produces no output for 6 hours (an orphaned python process is killed at cancellation). All 7 CPython sessions and all config tests, including the new ones, pass on PyPy before the hang.

I reproduced it locally with the exact CI build (portable pypy3.11-v8.0.0-linux64, libgit2 1.9.7):

  • pypy3 -m pytest test/test_config.py test/test_filter.py → hangs; faulthandler dump shows the main thread stuck inside gc.collect() in the filter fixture teardown (test/test_filter.py:60), with the stack running through weakref-callback machinery (_weakrefset._remove)
  • test/test_filter.py alone on the PR → passes; test/test_filter.py test/test_index.py → passes
  • the same subset on master under the same interpreter → 56 passed in ~7s

So: the config tests create garbage that, once PyPy's collector gets to it during the fixture's explicit gc.collect(), deadlocks — plausibly where PyPy runs weakref callbacks/finalizers against the PR's new CFFI objects (ffi.new_handled backends kept alive from C, 16 new extern "Python" callbacks). pygit2's own weakref callback at pygit2/filter.py:139 is in the immediate neighborhood. The root cause needs the author's investigation; the recipe above reproduces it in ~60s. Whatever the fix, the PR cannot merge with PyPy hanging — and I'd suggest adding -o faulthandler_timeout=120 (or pytest-timeout) to the cibuildwheel test command so future hangs dump a stack instead of burning 6 hours.

Bug: get returns the first value of a multivar

pygit2/config.py:1577 returns self._read_data[key][0]. libgit2's file backend, libgit2's own in-memory backend (config_mem.c), and git CLI semantics all return the last value (git_config_list_get keeps the last appended entry; verified empirically: file backend config['x.y'] → 'last', the PR's APP backend → 'first'). Inside an override context, plain reads and get_bool/get_int silently disagree with what the same values would yield from a file. One-line fix: index [-1].

Bug: nested with config: silently writes through to disk

enter/exit (pygit2/config.py:828-867) have no re-entrancy tracking. Verified behavior:

  with cfg:                    # write order → (APP,)
      cfg['a.b'] = 'outer1'
      with cfg:                # no-op
          cfg['a.c'] = 'inner'
      # inner __exit__ already restored write order to (LOCAL,) and cleared the backend
      cfg['a.d'] = 'outer2'    # ← persisted to .git/config

a.d landed in .git/config while the user still believes they're in an override context; the outer context's earlier overrides were also erased by the inner exit. A depth counter (only the outermost exit clears/restores), or explicitly rejecting nested entry, would fix it. At minimum this needs documenting since the current docstrings advertise re-entry.

Minor issues

  • pygit2/config.py:1597 — _config_memory_backend_set mutates _write_data without the write lock its own docstring promises (unlike set_multivar/del/del_multivar). Same for _config_memory_backend_get (no read lock). Low practical risk while cffi holds the GIL, but the inconsistency is misleading.
  • pygit2/config.py:1410 — clear() drops _c_entries/_iterators while libgit2 may still hold raw pointers into that cdata (e.g. a ConfigEntry from get_entry alive across context exit, freed later → use-after-free window). Same lifetime-bug class as Refactor ConfigEntry and related iterators for safer memory management #1499, which this project fixed recently.
  • pygit2/_libgit2/ffi.pyi — GitConfigIteratorC.next's first parameter should be Pointer[GitConfigBackendEntryC], not the struct itself; del doesn't match the real field name (del, reachable only via setattr/getattr).
  • pygit2/config.py _set_write_order docstring has a mismatched-backtick typo ( { GIT_CONFIG_LEVEL_APP }` ``).
  • Entries stored by the APP backend keep lowercased names, so iteration shows core.override1 even if set as core.oVeRrIdE1; file backends preserve written case. Cosmetic.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants