Notebook finetuning Orpheus-TTS

[Etherll]

No description provided.

[Erland366]

Looks good, I tested them in colab and it runs fine.

I'll edit them later on so for the installation part, it'll stay updated with how Unsloth install libraries (it keeps changing :( )

Thank you so much for the PR

[kadirnar]

@Etherll Is this just lora training? What should we do for finetuning or pretraining?

[kadirnar]

@Etherll @Erland366 Is the inference code faulty? Because the emotion feature isn't working.

Output: https://voca.ro/1D3xYHviDI6N

I tried this demo and the emotion feature works. However, the unsloth code doesn't produce the output I want.

Demo: https://huggingface.co/spaces/MohamedRashad/Orpheus-TTS

[Etherll]

@Etherll Is this just lora training? What should we do for finetuning or pretraining?

For full finetuning just add full_finetuning = True as in
model, tokenizer = FastLanguageModel.from_pretrained(
model_name = "canopylabs/orpheus-3b-0.1-pretrained",
max_seq_length= 2048, # Choose any for long context!
dtype = dtype,
full_finetuning = True,
token = "hf_...", # use one if using gated models like meta-llama/Llama-2-7b-hf
)
for continued pretraining follow the docs

[Etherll]

@Etherll @Erland366 Is the inference code faulty? Because the emotion feature isn't working.

Output: https://voca.ro/1D3xYHviDI6N

I tried this demo and the emotion feature works. However, the unsloth code doesn't produce the output I want.

Demo: https://huggingface.co/spaces/MohamedRashad/Orpheus-TTS

inference code works fine but pretrained model does not have emotive tags maybe one sample in dataset is not enough