Code

SOTA 2-bit quants#4773

Code

Merged

ikawrakow merged 17 commits intomasterggml-org/llama.cpp:masterfrom ik/iq2_2.06bpwggml-org/llama.cpp:ik/iq2_2.06bpwCopy head branch name to clipboardJan 8, 2024

+902-1

Commits

Commits on Jan 8, 2024

• iq2_xxs: basics

  Kawrakow

  committed
  4af2488

  Copy full SHA for 4af2488

• iq2_xxs: scalar and AVX2 dot products

  Show description for 7ef6389
  Kawrakow

  committed
  7ef6389

  Copy full SHA for 7ef6389

• iq2_xxs: ARM_NEON dot product

  Show description for 7b72318
  Kawrakow

  committed
  7b72318

  Copy full SHA for 7b72318

• iq2_xxs: WIP Metal

  Show description for d383f00
  Kawrakow

  committed
  d383f00

  Copy full SHA for d383f00

• iq2_xxs: Metal dot product now works

  Show description for dd29610
  Kawrakow

  committed
  dd29610

  Copy full SHA for dd29610

• iq2_xxs: slighty faster dot product

  Show description for 1c96aa0
  Kawrakow

  committed
  1c96aa0

  Copy full SHA for 1c96aa0

• iq2_xxs: slighty faster dot product

  Show description for e211fad
  Kawrakow

  committed
  e211fad

  Copy full SHA for e211fad

• iq2_xxs: even faster Metal dot product

  Show description for 065cc8c
  Kawrakow

  committed
  065cc8c

  Copy full SHA for 065cc8c

• iq2_xxs: dequantize CUDA kernel - fix conflict with master

  Kawrakow

  committed
  06e6908

  Copy full SHA for 06e6908

• iq2_xxs: quantized CUDA dot product (MMVQ)

  Show description for 8240521
  Kawrakow

  committed
  8240521

  Copy full SHA for 8240521

• iq2_xxs: slightly faster CUDA dot product

  Show description for c19d0d0
  Kawrakow

  committed
  c19d0d0

  Copy full SHA for c19d0d0

• iq2_xxs: add to llama ftype enum

  Kawrakow

  committed
  fd42737

  Copy full SHA for fd42737

• iq2_xxs: fix MoE on Metal

  Kawrakow

  committed
  47ae9b8

  Copy full SHA for 47ae9b8

• Fix missing MMQ ops when on hipBLAS

  Show description for 61c0405
  Kawrakow

  committed
  61c0405

  Copy full SHA for 61c0405

• Fix bug in qequantize_row_iq2_xxs

  Show description for 7db967e
  Kawrakow

  committed
  7db967e

  Copy full SHA for 7db967e

• Fixing tests

  Kawrakow

  committed
  5684d79

  Copy full SHA for 5684d79

• PR suggestion

  Kawrakow

  committed
  bad5f7f

  Copy full SHA for bad5f7f