Why is the CPU usage so high when searching in Faiss::gpu??

[YoungjaeDev]

Summary

Platform

OS: Ubuntu 20

Faiss version: c00fe25

Installed from: compiled

Faiss compilation options:

cmake -DFAISS_ENABLE_GPU=ON -DCUDAToolkit_ROOT=/usr/local/cuda -DCMAKE_CUDA_ARCHITECTURES="86" -DFAISS_ENABLE_PYTHON=ON -DBUILD_SHARED_LIBS=ON -DPython_EXECUTABLE=/usr/bin/python3 -DCMAKE_BUILD_TYPE=Release -B build .

Running on:

• CPU
• GPU

Interface:

• C++
• Python

Reproduction instructions

After creating indexflatIP (GPU) in python, I read it in C++ at 33ms intervals (to match real time), but it takes a lot of CPU load (1500%). Is that what it is?????

def write_index(self, index_path):
        """
        인덱스를 파일로 저장한다
        index_path: 인덱스 파일 경로
        """
        # 인덱스를 파일로 저장한다
        if self.gpu:
            index_cpu = faiss.index_gpu_to_cpu(self.index)
            faiss.write_index(index_cpu, index_path)
        else:
            faiss.write_index(self.index, index_path)

int main() {
    // Read index
    faiss::Index* index = faiss::read_index("index.faiss");
    assert(index != nullptr);

    // Convert CPU index to GPU index
    faiss::gpu::StandardGpuResources res;
    faiss::gpu::GpuIndexFlatIP* gpu_index = reinterpret_cast<faiss::gpu::GpuIndexFlatIP*>(faiss::gpu::index_cpu_to_gpu(&res, 0, index));
    assert(gpu_index != nullptr);
    
    // print cpu, gpu_index info
    printf("index->ntotal: %ld\n", index->ntotal);
    printf("gpu_index->ntotal: %ld\n", gpu_index->ntotal);

    std::mt19937 rng;
    std::uniform_real_distribution<> distrib;
    
    int count = 0;
    // Load test
    while(true) 
    {
        // dim=512, nq=1 random vector
        int d = 512;
        int nq = 1;
        float* xq = new float[d * nq];
        for (int i = 0; i < nq; i++) {
            for (int j = 0; j < d; j++)
                xq[d * i + j] = distrib(rng);
            xq[d * i] += i / 1000.;
        }

        // topk = 4
        int k = 4;

        // search xq
        long* I = new long[k * nq];
        float* D = new float[k * nq];

        gpu_index->search(nq, xq, k, D, I);

        // print results
        if (count % 10 == 0) {  // 예: 10번에 한 번만 결과 출력
            printf("I results=\n");
            for (int i = 0; i < 4; i++) {
                printf("I[%d] = %ld\n", i, I[i]);
            }
            printf("D results=\n");
            for (int i = 0; i < 4; i++) {
                printf("D[%d] = %f\n", i, D[i]);
            }
        }
        count++;
        
        // delete
        delete[] xq;
        delete[] I;
        delete[] D;

        // sleep 33ms
        std::this_thread::sleep_for(std::chrono::milliseconds(33));
    }

    return 0;
}

Should I use a different vector method? I would be grateful if you could introduce me to other methods and advice.

Thank yoU

[mdouze]

It is unlikely that this comes from the search function because a single query should be fast + normally the GPU search uses only 1 CPU core. Please try on a larger batch of vectors.

[YoungjaeDev]

@mdouze
You're welcome. When you mention a single query, are you saying that it's executed on the CPU? In that case, what would be the typical data size for GPU execution? Currently, I have nb=37, nq=1, dim=512.
thank you

[YoungjaeDev]

@mdouze

The range you're talking about is about 10,000 to 100,000. Is it right?

[mdouze]

With 37 database vectors and 1 query, the problem size is so small that it does not make sense to use a GPU. GPU brute force search is faster.

[YoungjaeDev]

you mean. CPU brute force search is faster. not GPU brute force search is faster. Right?

[mdouze]

yes sorry

[YoungjaeDev]

Sorry, but is there a way to intentionally limit omp when building Cmake????

[mdouze]

I don't understand the question.