Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v2.17.7 #650

Merged
merged 32 commits into from
Mar 18, 2025
Merged
Changes from 1 commit
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
6588549
Limit queue size, add logs
dbobrenko Feb 27, 2025
61b2676
Fix timers logging; code clean up
dbobrenko Feb 27, 2025
762bc99
Add scoring queue rate-limit delay by default
dbobrenko Feb 27, 2025
5a45f56
Add min wait time to scoring queue
dbobrenko Feb 27, 2025
307a1ca
Format code
dbobrenko Feb 28, 2025
7aa9052
Add gemma model and change llm_model to list
bkb2135 Mar 14, 2025
b992a27
Linting
bkb2135 Mar 14, 2025
44001c2
Remove class method
bkb2135 Mar 14, 2025
071e725
Adjust for LLM_MODEL being a list
bkb2135 Mar 14, 2025
4fa07ce
Add logic for deleting hf models
bkb2135 Mar 14, 2025
1904ff7
Move to cpu then delete
bkb2135 Mar 14, 2025
64adfd8
Merge branch 'staging' into fix/SN1-411-api-throttle
dbobrenko Mar 14, 2025
2438ad2
Make embedding class var instead of globalvar
dbobrenko Mar 14, 2025
f3ea57e
Merge branch 'main' of https://github.com/macrocosm-os/prompting into…
bkb2135 Mar 17, 2025
4625e04
Merge pull request #631 from macrocosm-os/fix/SN1-411-api-throttle
dbobrenko Mar 17, 2025
6142366
Remove unused pipeline
bkb2135 Mar 17, 2025
2280b6f
Load model based on settings config
bkb2135 Mar 17, 2025
7eacb91
Update pyproject.toml
bkb2135 Mar 17, 2025
e8b18aa
Rename llama key
bkb2135 Mar 17, 2025
4be892e
Update Lock File
bkb2135 Mar 17, 2025
b3151bd
Format gemma messages before inferencing
bkb2135 Mar 17, 2025
4610cd4
Create 3 classes
bkb2135 Mar 17, 2025
4ca49f7
Alternate Assistant and User
bkb2135 Mar 17, 2025
fc04646
Merge pull request #649 from macrocosm-os/main
bkb2135 Mar 18, 2025
03d1633
Precommit Fix
richwardle Mar 18, 2025
f7b8b45
Update Gemma Syntax
bkb2135 Mar 18, 2025
fbf3611
Replace Gemma with Mistral Model
bkb2135 Mar 18, 2025
25b9586
Simplify Unload Model
richwardle Mar 18, 2025
b86049a
Merge pull request #644 from macrocosm-os/SN1-437-implement-gemma-3-2…
bkb2135 Mar 18, 2025
a1099b2
Use Async Cacheing
bkb2135 Mar 18, 2025
0a0c675
Bump Version Number
bkb2135 Mar 18, 2025
6aec8b0
Merge pull request #651 from macrocosm-os/fix/staging
bkb2135 Mar 18, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Rename llama key
  • Loading branch information
bkb2135 committed Mar 17, 2025
commit e8b18aaed6385ddb22df9640eacfff14aa6c78c1
2 changes: 1 addition & 1 deletion prompting/llms/hf_llm.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ def __init__(
"""Deterministic HuggingFace model."""
self._device = device
self.sampling_params = {} if sampling_params is None else sampling_params
if settings.shared_settings.LLM_TYPE.get(model_id) == "llama":
if settings.shared_settings.LLM_TYPE.get(model_id) == "text-generation":
self.model: PreTrainedModel = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.float16,
Expand Down
Loading