Default to f16 model memory k/v in llm CLI and InferenceSessionConfig

[KerfuffleV2]

Using 32bit values doesn't increase quality in a measurable way, see discussion here: ggerganov/llama.cpp#1593

The current default doubles memory consumption/size of prompt caches without a measurable upside. This pull changes the default to 16bit.

Allows --float16 for backward compatibility.

Adds --no-float16 flag to allow using 32bit memory.

---

Don't know if it's just me but I can't even run llm llama infer --help from the main branch:

The application panicked (crashed).
Message:  Command infer: Argument names must be unique, but 'vocabulary_path' is in use by more than one argument or group

[LLukas22]

LGTM!

But shouldn't we also change the defaults for the InferenceSessionConfig to make this the default for all libraries using llm?

---

Regarding the --help parameter the application only panics in debug mode, if you run it in release mode via cargo run --release -- llama infer -help it works as expected. I will create an issue for this.

[KerfuffleV2]

@LLukas22 Thanks! I missed that one.

Regarding the --help parameter the application only panics in debug mode

It's really weird that compiling in release affects argument parsing. I'd never have guessed to try a different build type.

[LLukas22]

Yeah clap seams to validate duplicated parameters only in debug mode. 🤷

Thanks for changing the defaults, everything looks good now. 👍