feat: Add nitro vulkan to support AMD GPU/ APU and Intel Arc GPU

[hiro-v]

Describe Your Changes

• Add vulkan nitro to support AMD GPU and Intel Arc version
• Verify Jan with nitro vulkan can run on AMD GPU
• Verify Jan with nitro vulkan can run on Intel Arc GPU
• Add setting toggle boolean to enable Vulkan GPU
• Add logic to use nitro binary in linux-vulkan and windows-vulkan
• Add settings for a number as INT () which we will use as environment variables VULKAN_DEVICE_INDEX=<index> when executing nitro vulkan.
• Update settings frontend with jan/settings/settings.json
  
  

Fixes Issues

• Feat: Support OpenCL and ADM Radeon #1608
• Intel Arc Support #1926
• epic: Jan supports multiple CPU/GPU Hardware optimizations #905
• feat: Inference engine for Intel iGPU and intel GPU #915
• epic: support AMD using Vulkan #913

Self Checklist

• Added relevant comments, esp in complex areas
• Updated docs (for bug fixes / features)
• Created issues for follow-up changes or refactoring needed

[hiro-v]

Report on real machine with AMD Ryzen 5 and AMD Radeon GPU (6600)

Machine spec

Performance
==> Summary: AMD GPU consistently yield 300% better performance compared to AMD Ryzen CPU (which is really good already)

• Tiny Llama 1.1B

  • AMD GPU
    
  • AMD Ryzen CPU
    

• Mistral 7B Q4

  • AMD GPU
    
  • AMD Ryzen CPU
    

[louis-menlo]

Updated:

• vulkan enabled: should be a boolean value located in the root of settings.json.
• gpus_in_use: reuse, and it should be shared between Cuda & Vulkan settings while resetting when switching between them.
• Do not use the VULKAN_DEVICE_INDEX variable; instead, use GGML_VULKAN_DEVICE. Only support using one selected GPU at a time, and both Nvidia and AMD GPUs should work.
• Handle crash on app launch due to undefined value (settings.json > vulkan)
• nvidia-smi yield error when vulkan enabled (wrong device index)
• Wrong Linux vulkan executable binary download url

In progress:

• GPU devices list

[louis-menlo]

Update:

• GPU devices list

NOTES: There is a limitation with the current approach; GPU VRAM information is not retrieved. Will investigate further.

The screenshots are for demo purposes; please see the AMD performance above.

[louis-menlo]

Known issue:
Enable Vulkan support on unsupported GPU device would cause this error. The GPU list would be empty in App > Settings.

[louis-menlo]

Experimental feature: 0.4.7

[louis-menlo]

LGTM