Try embedding a knowledge base on a preexisting model.
$ python wikibot.py
Hello I am wikibot! Ask me anything.
>
Write your question. For example:
> What does the fox say?
- homemade zig matrices
- cold start with supervised learning
- no self attention
- no tokenizer
- 128 character context window
- get through entire dataset once
- Take less space than the dataset (14 gb)
- Correct responses
- reinforcement learning to behave like a "chatbot", or should I say... wikibot.
- the wikibot doesn't have to be smart, maybe I can train it locally?
- how to teach the wikibot about itself
- how to handle the large dataset
- getting correct responses. maybe many slightly different wording of wikipedia would work?
- learning while you are prompting it
- wikipedia dataset
- llm from scratch
- tokenization
- gpt 2 from scratch
- let's reproduce gpt-2
- find tuning llama
- llama2.c
- memory
- Reinforcement learning walkkthrough (the best I've found yet)
- Voice output
- Voice input