feat: Jan Hub Revamp

[louis-menlo]

Describe Your Changes

This PR introduces support for /model/sources in Jan Model Hub, implemented using cortex.cpp.

With this endpoint support, the Jan frontend can now remove pre-populated models and legacy HuggingFace operations. All requests are now routed to the cortex.cpp server. So that:

As a user, I can now access model variants from the Cortexso Hub, along with improved model descriptions.

Demo

New Model Hub Revamp

This is a full flow of this new model hub revamp

New model search outlook

This new model search outlook functions more effectively with fuzzy searching. It supports either recent searched models or Hugging Face repository ID or URL.

Download HuggingFace models

Users can now see the search status and model founds instead of just getting an error message as before.

Download Cortexso hub models

Users can input the repository ID of the cortexso repository to work with multiple branches (model versions).

Cloud models

Instead of listing all remote models as before, which was confusing, we've created a long list that needs to be grouped. We're introducing a new remote model detail page to help organize the information.

New sorting options

Now users can sort models by Most Downloaded or Newest

Quick download action

Users can conveniently select model versions right on the download button, bypassing the need to go to the model detail page.

New Deeplink update

Now, instead of displaying the model dialog, it directs users to the model detail page.

Hub Banner settings

Users can now customize the Model Hub with banner settings

Fixes Issues

• Closes feat: New Jan Hub UI #4170
• Closes roadmap: Jan's Hub is revamped and uses Cortex's /model/sources #3910

Self Checklist

• Revamped Model Hub UI
• Added model source details page
• Download various model quantizations
• Remote provider listing
• Initialize model sources

Changes

This pull request includes significant changes to the model management system, the integration of a new dropdown menu component, and some utility function updates. The most important changes are detailed below:

Model Management Enhancements:

• Added new methods getSources, addSource, and deleteSource to the ModelExtension class for managing model sources (core/src/browser/extensions/model.ts).
• Introduced ModelSource interface and related types to define the structure of model sources (core/src/types/model/modelSource.ts).
• Updated ICortexAPI and CortexAPI to include methods for handling model sources (extensions/model-extension/src/cortex.ts). [1] [2]

Dropdown Menu Component:

• Added a new Dropdown component using @radix-ui/react-dropdown-menu, including styles and functionality (joi/src/core/Dropdown/index.tsx, joi/src/core/Dropdown/styles.scss). [1] [2]
• Updated package.json to include the new dependency for the dropdown menu (joi/package.json).

Utility Function Updates:

• Changed utility function toGibibytes to toGigabytes in several files for consistency in memory size conversion (web/containers/Layout/BottomPanel/SystemMonitor/TableActiveModel/index.tsx, web/containers/Layout/BottomPanel/SystemMonitor/index.tsx). [1] [2] [3] [4]

These changes improve the flexibility and functionality of the model management system and enhance the user interface with a new dropdown component.

[github-actions]

Barecheck - Code coverage report

Total: 67.62%

Your code coverage diff: -0.21% ▾

Uncovered files and lines

File	Lines
core/src/browser/extensions/model.ts	15
joi/src/core/Dropdown/index.tsx	13, 24, 30
web/containers/Layout/BottomPanel/SystemMonitor/index.tsx	55, 94, 102, 108, 115-116, 153, 157
web/containers/Layout/BottomPanel/SystemMonitor/TableActiveModel/index.tsx	51-53
web/containers/ModelDropdown/index.tsx	126-127, 137, 146, 150, 152, 154-155, 157-158, 160-163, 165, 187-188, 198, 202, 217-218, 225-228, 230, 232, 251, 256, 266, 273, 276-277, 298, 322, 345, 366, 375, 381, 389, 392, 394, 398, 400-405, 408, 411, 468-469, 471, 497, 505, 507, 532, 537-538, 540, 544-546, 548-549, 551, 561-564, 595, 603, 605
web/hooks/useEngineManagement.ts	40-41, 43, 64, 77-78, 80, 90, 92, 99, 106-107, 109, 119, 121, 128, 137, 142, 144, 150, 152, 154-157, 161-162, 165-166, 169, 180, 193, 197, 199, 205, 207-209, 211, 215-216, 219-220, 223, 232, 243-244, 246, 254, 257-258, 266, 270, 282, 284-285, 288, 290-291, 293-294, 307, 309-310, 313, 315-317, 319-320, 332, 334-335, 338, 340-342, 344-345, 354, 356-357, 360, 362-364, 366-367, 379, 381-382, 385, 387-389, 391-392, 403, 405-406, 409, 411, 416-417, 419-420, 429-430, 432, 434, 437, 440, 444
web/hooks/useModels.ts	48, 62, 64, 68, 89-90
web/screens/Thread/ThreadCenterPanel/ChatBody/OnDeviceStarterScreen/index.tsx	66, 99, 105, 132, 153-154, 156, 179, 188, 190, 218, 241, 243, 274, 300, 308, 312, 316, 321-322, 351
web/screens/Thread/ThreadCenterPanel/FileUploadPreview/index.tsx	14-15, 17-19
web/screens/Thread/ThreadCenterPanel/TextMessage/DocMessage.tsx	16, 22
web/screens/Thread/ThreadCenterPanel/TextMessage/MarkdownTextMessage.tsx	32, 36-37, 41, 49-52, 54, 58, 65-67, 73, 76-78, 80, 85-86, 89, 139, 141-142, 147-148, 207, 222
web/screens/Thread/ThreadCenterPanel/TextMessage/index.tsx	31-36, 38, 40, 44, 46, 50
web/utils/converter.ts	13, 21-22
web/utils/modelEngine.ts	68, 70, 72, 74, 76, 78, 80, 82, 84, 86