Support for multiple knowledges per session

[raychz]

Support for Multiple Knowledge Bases per Session

Current Situation

Currently, Hollama allows users to select a single knowledge entry per session. This limits the flexibility and depth of information that can be incorporated into a single conversation.

Feature Request

We propose adding support for selecting and using multiple knowledges within a single session. This would allow users to combine different sources of information, creating more comprehensive and versatile conversational contexts.

Proposed Functionality

1. User Interface Updates:

   • Modify the knowledge selection dropdown to allow multiple selections.

2. Logic Changes:

   • Modify the request payload creation to incorporate information from all selected knowledges.

Benefits

1. More comprehensive responses by combining multiple sources of information.
2. Greater flexibility in tailoring sessions to specific needs.
3. Improved ability to handle complex, multi-domain queries.

Implementation Considerations

• Should there be a limit on the number of knowledge bases that can be selected? I don't think there should be.
• How might this affect the model's context window and response generation time? This is where allowing the user to edit the Ollama num_ctx param would come in handy. See Ollama docs.
• Does Ollama support multiple CONTEXT tags? Here's a relevant snippet from ollama.ts.

	const firstMessage = session.messages[0]
	if (firstMessage.knowledge) {
		payload.prompt = `
			<CONTEXT
				name="${firstMessage.knowledge.name}"
				id="${firstMessage.knowledge.id}"
				updatedAt="${firstMessage.knowledge.updatedAt}"
			>
				${firstMessage.knowledge.content}
			</CONTEXT>

			${payload.prompt}
		`;
	}

• The Message interface would need to be updated to support a list of Knowledges. Maybe these two should be decoupled? Or better yet, maybe only the Session interface should contain the list of Knowledge's. Here's the relevant snippet from sessions.ts:

export interface Message {
	role: 'user' | 'ai' | 'system';
	content: string;
	knowledge?: Knowledge;  // Remove this one?
}

export interface Session {
	id: string;
	model: string;
	messages: Message[];
	context: number[];
	knowledge?: Knowledge;  // Make this an array of Knowledges?
}

Questions for Discussion

1. Are there any concerns about the complexity this might add for new users?
2. Should this be an "advanced" feature, or available to all users by default?

Thanks for your consideration @fmaclen! I might have time to take this one on soon if you'd like, but I know you're working on a major UI overhaul, so maybe I'll wait until that one is merged in.

[fmaclen]

@raychz thanks for the detailed suggestion!

The way I was thinking about this feature is as follows:

• A user can add "context" to any session at any time, meaning it can be added at the beginning in a "New session" or in the middle of an existing one.
• Context could be one or more sources such as:
  • Copy/paste a block of text
  • Paste a link to a website (which Hollama will attempt to extract the text from) [*]
  • Choose/drag-and-drop a text file
• All of this can be added "on the fly" in a session, a user shouldn't need to create Knowledge first.
• Then, in the Knowledge section a user can find the same UI component that's present in Session, except that it can be saved as Knowledge to be re-used later.

Should there be a limit on the number of knowledge bases that can be selected? I don't think there should be.

Yeah, agreed. Wouldn't set a limit.

How might this affect the model's context window

This is an important detail, adding a bunch of context at once could easily exceed the context window of the model.

This is where allowing the user to edit the Ollama num_ctx param would come in handy.

In general, I'd like to avoid expecting the user to know which value to set num_ctx.

It'd be great if we can count the tokens of the current context + prompt (#7) before submitting the form. Looks like there is an ongoing discussion about it: ollama/ollama#1716

If Ollama doesn't have a tokenizer we can use, can we use a 3rd party one?

The other caveat is that each model has a different context window, does Ollama report this value via the API?

---

Here's where I'm at:

1. The first issue should only implement "multiple text blocks" in the Knowledge view.
2. We should add 2 new issues for supporting link parsing and file upload also in the Knowledge view.
3. And then a 3rd issue which implements a way to add all of these data sources in the Session window without having to go through the Knowledge view.
4. The token count/context window feature should probably be done separately in Show session token count #7.

I might have time to take this one on soon if you'd like, but I know you're working on a major UI overhaul, so maybe I'll wait until that one is merged in.

The UI overhaul should have an impact on some of these features but should be minimal for tasks 1 and 4.

---

[*] I actually have a prototype Chrome extension that extracts the text from the current tab, should make that repo public here's the repo:

[fmaclen]

Here's a rough mockup for adding multiple text boxes in a particular "Knowledge":

• By default "New knowledge" has an empty text box
• Clicking the [T] adds a new box

[fmaclen]

This feature will likely be covered by #116