Skip to content

fix: support source re-ingestion and removal - #30

Open
Aaqibhafeezkhan wants to merge 1 commit into
openapi:mainfrom
Aaqibhafeezkhan:fix/18-source-lifecycle
Open

Aaqibhafeezkhan wants to merge 1 commit into
openapi:mainfrom
Aaqibhafeezkhan:fix/18-source-lifecycle

Conversation

@Aaqibhafeezkhan

Copy link
Copy Markdown

Closes #18

@francescobianco francescobianco left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @Aaqibhafeezkhan, this is a substantial contribution and it closes one of the most important gaps in gknowledge.

Until now, ingestion was a one-way street: if a document was updated, ingested with the wrong model, or simply did not belong in the project, the only fix was to start a new knowledge base. With this PR the knowledge base becomes something you can actually maintain over time, which is the whole point of the "evolving knowledge base" idea.

What makes it solid:

  • Provenance-aware cleanup. _detach_source deletes only the pages that the source alone supported, while pages shared with other sources are kept and just lose the [[source]] link. A shared concept page never disappears because one of its sources was removed.
  • No stale state left behind. Jobs, review items, mentions.json entries, the index, the overview and the log are all updated, so the wiki, the database and the cache stay consistent.
  • Safe by design. _managed_path refuses paths outside sources/ and knowledge/, both actions are blocked while an ingestion is running, and Remove asks for confirmation with a destructive-styled dialog.
  • Tests for both flows, including the tricky case of a page shared between two sources.

Great work, and thank you again!

@francescobianco

Copy link
Copy Markdown
Member

@Aaqibhafeezkhan since you now know the source鈫抪age lifecycle better than anyone, here are the open issues that build directly on this PR, in case you want to keep going:

None of this is required for merging #30. They're just the places where your work pays off most. And of course, the invitation to join https://github.com/openapi/contributors stands 馃檪

@Aaqibhafeezkhan

Copy link
Copy Markdown
Author

@Aaqibhafeezkhan since you now know the source鈫抪age lifecycle better than anyone, here are the open issues that build directly on this PR, in case you want to keep going:

None of this is required for merging #30. They're just the places where your work pays off most. And of course, the invitation to join https://github.com/openapi/contributors stands 馃檪

Thanks @francescobianco Will check them out :)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Re-ingest and remove a source

2 participants