Add QK=128 q4_1 for GPTQ

[qwopqwop200]

Currently, GPTQ is converted to q4_1(QK=32) format. However, this is very inefficient considering that GPTQ generally recommends a QK value of 128.
And to solve this, I created a new q4_1 (qk=128) called q4_2.
Applying this gives an approximate 20% speed improvement and a 17% reduction in memory usage on GPTQ.
I'm sure this implementation will make llama run faster and more robustly.

Currently, I don't have an ARM cpu, so ARM may not work because I haven't tested the operation.

[qwopqwop200]

make gptq model (not support act-order)
https://github.com/qwopqwop200/GPTQ-for-LLaMa
python llama.py ./llama-hf/llama-7b c4 --wbits 4 --true-sequential --groupsize 128 --save llama7b-4bit-128g.pt

use python 3.10

cd llama.cpp
make

# obtain the original LLaMA model weights and place them in ./models
ls ./models
tokenizer_checklist.chk tokenizer.model  llama7b-4bit-128g.pt

# install Python dependencies
python3 -m pip install -r requirements.txt

# convert the 7B model to ggml int4 format
python3 ./convert.py ./models/llama7b-4bit-128g.pt --outtype q4_2 --outfile ./models/llama7b-4bit-128g-ggjt.bin

# run the inference
./main -m ./models/llama7b-4bit-128g-ggjt.bin -n 128

[ggerganov]

Can you provide perplexity comparison between original Q4_1 quantization and the proposed one?

[qwopqwop200]

Currently, some parts of my experiment seem to be wrong (set ctx=1024). In particular, the result of the current experiment is that the result is worse than that of Q4_1. Because of this, I will close the current issue.