Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: improvementUI GPU acceleration #1990

Merged
merged 1 commit into from
Feb 13, 2024
Merged

feat: improvementUI GPU acceleration #1990

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion web/containers/GPUDriverPromptModal/index.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ const GPUDriverPrompt: React.FC = () => {
<Modal open={showNotification} onOpenChange={openChanged}>
<ModalContent>
<ModalHeader>
<ModalTitle>
<ModalTitle className="pr-4 leading-relaxed">
Checking for machine that does not meet the requirements.
</ModalTitle>
</ModalHeader>
Expand Down
53 changes: 44 additions & 9 deletions web/containers/Layout/BottomBar/index.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,13 +56,24 @@
const setShowSelectModelModal = useSetAtom(showSelectModelModalAtom)
const [serverEnabled] = useAtom(serverEnabledAtom)

const calculateGpuMemoryUsage = (gpu: Record<string, never>) => {

Check warning on line 59 in web/containers/Layout/BottomBar/index.tsx

View workflow job for this annotation

GitHub Actions / test-on-macos 

'calculateGpuMemoryUsage' is assigned a value but never used

Check warning on line 59 in web/containers/Layout/BottomBar/index.tsx

View workflow job for this annotation

GitHub Actions / test-on-ubuntu 

'calculateGpuMemoryUsage' is assigned a value but never used

Check warning on line 59 in web/containers/Layout/BottomBar/index.tsx

View workflow job for this annotation

GitHub Actions / test-on-windows (mcafee) 

'calculateGpuMemoryUsage' is assigned a value but never used

Check warning on line 59 in web/containers/Layout/BottomBar/index.tsx

View workflow job for this annotation

GitHub Actions / test-on-windows (default-windows-security) 

'calculateGpuMemoryUsage' is assigned a value but never used

Check warning on line 59 in web/containers/Layout/BottomBar/index.tsx

View workflow job for this annotation

GitHub Actions / test-on-windows (bit-defender) 

'calculateGpuMemoryUsage' is assigned a value but never used
const total = parseInt(gpu.memoryTotal)
const free = parseInt(gpu.memoryFree)
if (!total || !free) return 0
return Math.round(((total - free) / total) * 100)
}

const calculateUtilization = () => {
let sum = 0
const util = gpus.map((x) => {
return Number(x['utilization'])
})
util.forEach((num) => {
sum += num
})
return sum
}

return (
<div className="fixed bottom-0 left-16 z-20 flex h-12 w-[calc(100%-64px)] items-center justify-between border-t border-border bg-background/80 px-3">
<div className="flex flex-shrink-0 items-center gap-x-2">
Expand Down Expand Up @@ -126,15 +137,39 @@
<SystemItem name="Mem:" value={`${ram}%`} />
</div>
{gpus.length > 0 && (
<div className="flex items-center gap-x-2">
{gpus.map((gpu, index) => (
<SystemItem
key={index}
name={`GPU ${gpu.id}:`}
value={`${gpu.utilization}% Util, ${calculateGpuMemoryUsage(gpu)}% Mem`}
/>
))}
</div>
<Tooltip>
<TooltipTrigger>
<div className="flex cursor-pointer items-center">
<SystemItem
name={`${gpus.length} GPU `}
value={`${calculateUtilization()}% `}
/>
</div>
</TooltipTrigger>
{gpus.length > 1 && (
<TooltipContent
side="top"
sideOffset={10}
className="min-w-[240px]"
>
<span>
{gpus.map((gpu, index) => (
<div
key={index}
className="flex items-center justify-between"
>
<div>
<span>{gpu.name}</span>
<span>{gpu.vram}MB VRAM</span>
</div>
<span>{gpu.utilization}%</span>
</div>
))}
</span>
<TooltipArrow />
</TooltipContent>
)}
</Tooltip>
)}
{/* VERSION is defined by webpack, please see next.config.js */}
<span className="text-xs text-muted-foreground">
Expand Down
2 changes: 1 addition & 1 deletion web/hooks/useSettings.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ export const useSettings = () => {
((settings.nvidia_driver?.exist && !settings.cuda?.exist) ||
!settings.nvidia_driver?.exist)
) {
setShowNotification(true)
setShowNotification(false)
}

// Check if run_mode is 'gpu' or 'cpu' and update state accordingly
Expand Down
4 changes: 2 additions & 2 deletions web/screens/LocalServer/index.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -383,8 +383,8 @@ const LocalServerScreen = () => {
xmlns="http://www.w3.org/2000/svg"
>
<path
fill-rule="evenodd"
clip-rule="evenodd"
fillRule="evenodd"
clipRule="evenodd"
d="M9.00033 17.3337C13.6027 17.3337 17.3337 13.6027 17.3337 9.00033C17.3337 4.39795 13.6027 0.666992 9.00033 0.666992C4.39795 0.666992 0.666992 4.39795 0.666992 9.00033C0.666992 10.9978 1.36978 12.8311 2.54157 14.2666L0.910703 15.9111C0.390085 16.436 0.758808 17.3337 1.49507 17.3337H9.00033ZM5.25033 7.33366C5.25033 6.87342 5.62342 6.50033 6.08366 6.50033H11.917C12.3772 6.50033 12.7503 6.87342 12.7503 7.33366C12.7503 7.7939 12.3772 8.16699 11.917 8.16699H6.08366C5.62342 8.16699 5.25033 7.7939 5.25033 7.33366ZM6.08366 9.83366C5.62342 9.83366 5.25033 10.2068 5.25033 10.667C5.25033 11.1272 5.62342 11.5003 6.08366 11.5003H8.58366C9.0439 11.5003 9.41699 11.1272 9.41699 10.667C9.41699 10.2068 9.0439 9.83366 8.58366 9.83366H6.08366Z"
fill="#2563EB"
/>
Expand Down
246 changes: 189 additions & 57 deletions web/screens/Settings/Advanced/index.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,32 @@ import {
ChangeEvent,
} from 'react'

import { openExternalUrl } from '@janhq/core'

import { fs } from '@janhq/core'
import { Switch, Button, Input } from '@janhq/uikit'
import {
Switch,
Button,
Input,
Select,
Checkbox,
SelectContent,
SelectGroup,
SelectPortal,
SelectLabel,
SelectTrigger,
SelectValue,
Tooltip,
TooltipArrow,
TooltipContent,
TooltipTrigger,
} from '@janhq/uikit'

import { AlertTriangleIcon, AlertCircleIcon } from 'lucide-react'

import ShortcutModal from '@/containers/ShortcutModal'

import { toaster } from '@/containers/Toast'
import { snackbar, toaster } from '@/containers/Toast'

import { FeatureToggleContext } from '@/context/FeatureToggle'

Expand All @@ -40,6 +60,12 @@ const Advanced = () => {
const { readSettings, saveSettings, validateSettings, setShowNotification } =
useSettings()

const selectedGpu = gpuList
.filter((x) => gpusInUse.includes(x.id))
.map((y) => {
return y['name']
})

const onProxyChange = useCallback(
(event: ChangeEvent<HTMLInputElement>) => {
const value = event.target.value || ''
Expand Down Expand Up @@ -77,6 +103,7 @@ const Advanced = () => {
}

const handleGPUChange = (gpuId: string) => {
// TODO detect current use GPU nvidia or AMD
let updatedGpusInUse = [...gpusInUse]
if (updatedGpusInUse.includes(gpuId)) {
updatedGpusInUse = updatedGpusInUse.filter((id) => id !== gpuId)
Expand Down Expand Up @@ -127,66 +154,171 @@ const Advanced = () => {

{/* CPU / GPU switching */}
{!isMac && (
<div className="flex w-full items-start justify-between border-b border-border py-4 first:pt-0 last:border-none">
<div className="flex-shrink-0 space-y-1.5">
<div className="flex gap-x-2">
<h6 className="text-sm font-semibold capitalize">Nvidia GPU</h6>
<div className="flex w-full flex-col items-start justify-between border-b border-border py-4 first:pt-0 last:border-none">
<div className="flex items-start justify-between">
<div className="space-y-1.5">
<div className="flex gap-x-2">
<h6 className="text-sm font-semibold capitalize">
GPU Acceleration
</h6>
</div>
<p className="pr-8 leading-relaxed">
Enable to enhance model performance by utilizing your devices
GPU for acceleration. Read{' '}
<span>
{' '}
<span
className="cursor-pointer text-blue-600"
onClick={() =>
openExternalUrl(
'https://jan.ai/guides/troubleshooting/gpu-not-used/'
)
}
>
troubleshooting guide
</span>{' '}
</span>{' '}
for further assistance.
</p>
</div>
<p className="leading-relaxed">
Enable GPU acceleration for Nvidia GPUs.
</p>
</div>
<Switch
checked={gpuEnabled}
onCheckedChange={(e) => {
if (e === true) {
saveSettings({ runMode: 'gpu' })
setGpuEnabled(true)
setShowNotification(false)
setTimeout(() => {
validateSettings()
}, 300)
} else {
saveSettings({ runMode: 'cpu' })
setGpuEnabled(false)
}
}}
/>
</div>
)}
{/* Directory */}
{gpuEnabled && (
<div className="mt-4">
<label className="block text-sm font-medium text-gray-700">
Select GPU(s)
</label>
<div className="mt-2 space-y-2">
{gpuList.map((gpu) => (
<div key={gpu.id}>
<input
type="checkbox"
id={`gpu-${gpu.id}`}
name="gpu"
value={gpu.id}
checked={gpusInUse.includes(gpu.id)}
onChange={() => handleGPUChange(gpu.id)}
{gpuList.length > 0 && !gpuEnabled && (
<Tooltip>
<TooltipTrigger>
<AlertCircleIcon size={20} className="mr-2 text-yellow-600" />
</TooltipTrigger>
<TooltipContent
side="right"
sideOffset={10}
className="max-w-[240px]"
>
<span>
Disabling GPU Acceleration may result in reduced
performance. It is recommended to keep this enabled for
optimal user experience.
</span>
<TooltipArrow />
</TooltipContent>
</Tooltip>
)}

<Tooltip>
<TooltipTrigger>
<Switch
checked={gpuEnabled}
onCheckedChange={(e) => {
if (e === true) {
saveSettings({ runMode: 'gpu' })
setGpuEnabled(true)
setShowNotification(false)
snackbar({
description: 'Successfully turned on GPU Accelertion',
type: 'success',
})
setTimeout(() => {
validateSettings()
}, 300)
} else {
saveSettings({ runMode: 'cpu' })
setGpuEnabled(false)
snackbar({
description: 'Successfully turned off GPU Accelertion',
type: 'success',
})
}
}}
/>
<label htmlFor={`gpu-${gpu.id}`}>
{' '}
{gpu.name} (VRAM: {gpu.vram} MB)
</label>
</div>
))}
</TooltipTrigger>
{gpuList.length === 0 && (
<TooltipContent
side="right"
sideOffset={10}
className="max-w-[240px]"
>
<span>
Your current device does not have a compatible GPU for
monitoring. To enable GPU monitoring, please ensure your
device has a supported Nvidia or AMD GPU with updated
drivers.
</span>
<TooltipArrow />
</TooltipContent>
)}
</Tooltip>
</div>

{gpuEnabled && (
<div className="mt-2 w-full rounded-lg bg-secondary p-4">
<label className="mb-1 inline-block font-medium">
Choose GPU
</label>
<Select value={selectedGpu.join()}>
<SelectTrigger className="w-[340px] bg-white">
<SelectValue placeholder="Select GPU">
<span className="line-clamp-1 w-full pr-8">
{selectedGpu.join()}
</span>
</SelectValue>
</SelectTrigger>
<SelectPortal>
<SelectContent className="w-[400px] px-1 pb-2">
<SelectGroup>
<SelectLabel>Nvidia</SelectLabel>
<div className="px-4 pb-2">
<div className="rounded-lg bg-secondary p-3">
{gpuList
.filter((gpu) =>
gpu.name.toLowerCase().includes('nvidia')
)
.map((gpu) => (
<div
key={gpu.id}
className="my-1 flex items-center space-x-2"
>
<Checkbox
id={`gpu-${gpu.id}`}
name="gpu-nvidia"
className="bg-white"
value={gpu.id}
checked={gpusInUse.includes(gpu.id)}
onCheckedChange={() =>
handleGPUChange(gpu.id)
}
/>
<label
className="flex w-full items-center justify-between"
htmlFor={`gpu-${gpu.id}`}
>
<span>{gpu.name}</span>
<span>{gpu.vram}MB VRAM</span>
</label>
</div>
))}
</div>
{/* Warning message */}
{gpuEnabled && gpusInUse.length > 1 && (
<div className="mt-2 flex items-start space-x-2 text-yellow-500">
<AlertTriangleIcon
size={16}
className="flex-shrink-0"
/>
<p className="text-xs leading-relaxed">
If multi-GPU is enabled with different GPU models
or without NVLink, it could impact token speed.
</p>
</div>
)}
</div>
</SelectGroup>

{/* TODO enable this when we support AMD */}
</SelectContent>
</SelectPortal>
</Select>
</div>
)}
</div>
)}
{/* Warning message */}
{gpuEnabled && gpusInUse.length > 1 && (
<p className="mt-2 italic text-red-500">
If enabling multi-GPU without the same GPU model or without NVLink, it
may affect token speed.
</p>
)}

<DataFolder />
{/* Proxy */}
<div className="flex w-full items-start justify-between border-b border-border py-4 first:pt-0 last:border-none">
Expand Down
Loading