Skip to content

[BUG] reindex --full overwrites entity.created_at with the reindex timestamp for notes without a frontmatter created:` — irreversible #1772

Closed

Description

@rirkeyou-maker

For notes that don't declare created: in frontmatter, bm reindex --full (and the full project-index path generally) sets entity.created_at to the moment the reindex runs, destroying the original creation timestamp. The chain:

  1. indexing/input_file_adaptation.py — build_index_input_files() hardcodes created_at=None for every file, even though file creation time (st_ctime) is available and is in fact used elsewhere (services/file_service.py builds created_at from stat_result.st_ctime).
  2. indexing/batch_indexer.py — passes ctime=file.created_at to the markdown parser, i.e. always None on this path.
  3. markdown/entity_parser.py — with ctime=None and no frontmatter created:, the fallback is datetime.now().
  4. services/entity_service.py — _apply_markdown_entity_fields() unconditionally assigns entity.created_at = markdown.created.

updated_at keeps the file mtime, so afterwards created_at > updated_at (inverted). In one run, every note in a project without frontmatter created: received created_at equal to the reindex moment to the millisecond, while the underlying files' creation times were hours older and untouched. Across the whole installation, hundreds of entity rows are inverted this way. The original value cannot be recovered from the database.

Side effect: find_related-style time-window filtering (created_at >= since) silently returns wrong results. Main search-result ordering is unaffected (it uses search_index.updated_at).

Aggravating factor: the recommended remedy for index inconsistency is reindex --full, so users hit this repeatedly while trying to fix other index problems.

Impact

Any installation with notes lacking frontmatter created: loses real creation timestamps on every full reindex. Data loss is silent and irreversible.

Steps to reproduce

  1. Create a note without created: in frontmatter; note its file creation time.
  2. Wait (or edit it so updated_at < reindex time), then run bm reindex --full.
  3. SELECT title, created_at, updated_at FROM entity WHERE created_at > updated_at;

Actual

created_at equals the reindex timestamp (later than updated_at), regardless of the file's real creation time.

Expected

created_at is preserved (from the existing entity row, or from the file's creation time, which is available via st_ctime) when frontmatter does not declare created:.

Versions

basic-memory 0.23.2, Windows 11.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions