RagKnowledge is cleaned up on runtime initialization

[JustinFeng]

Is your feature request related to a problem? Please describe.

Our team developed a service to manage agent RagKnowledge, it interacts with the RagKnowledge instance under agent runtime directly

we support loading a txt/pdf file from a remote URL, and creating knowledge via ragKnowledge#processFile, at the moment we set the content.metadata.source to be the URL string, here is how the content file in db looks like in our case

{
   "text":"I like txt file, because it's plain\n",
   "metadata":{
      "type":"txt",
      "source":"https://........./knowledge/test.txt",
      "isShared":false
   }
}

However we found in the recent change, that there is a cleanup operation to remove knowledge that's missing local source file
https://github.com/elizaOS/eliza/blob/main/packages/core/src/runtime.ts#L560

and this will clean up all the rag-knowledge that created from a remote file

Describe the solution you'd like

Can we either set some flag in the knowledge metadata or detect the source string format to prevent cleaning up such remote sourced knowledge, so that there is more flexibility in knowledge management

Describe alternatives you've considered

• we have considered touching a fake file or removing the source string from the metadata to bypass the cleanup, but none of those is an elegant or robust way

I would like to hear what's your thoughts on this, or if there is some way to avoid such knowledge from being cleaned up without changing the current logic, cheers

[github-actions]

Hello @JustinFeng! Welcome to the elizaOS community. Thank you for opening your first issue; we appreciate your contribution. You are now an elizaOS contributor!