Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Report filename and file size in modelLoader #122

Closed
kerthcet opened this issue Sep 3, 2024 · 3 comments · Fixed by #124
Closed

Report filename and file size in modelLoader #122

kerthcet opened this issue Sep 3, 2024 · 3 comments · Fixed by #124
Assignees
@kerthcet
Labels
cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. needs-kind Indicates a PR lacks a label and requires one. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
Milestone
v0.1.0

Comments

@kerthcet
Copy link
Member

kerthcet commented Sep 3, 2024

What would you like to be cleaned:

Here's the current output of model loader:

Skipping virtualenv creation, as specified in config file.
2024-09-03 14:49:03,464 - __main__ - INFO - loading models from modelhub takes 0:03:15.848986s
Start to download model TheBloke/Llama-2-7B-GGUF
Skipping virtualenv creation, as specified in config file.
2024-09-03 14:45:45,397 - __main__ - INFO - loading models from modelhub takes 0:11:10.482161s
Start to download model TheBloke/Llama-2-7B-GGUF
Error from server (BadRequest): container "model-runner" in pod "llamacpp-speculator-0" is waiting to start: PodInitializing

If we're downloading GGUF files, there's no file name and the file size, we should report that as well.

Why is this needed:

Better observation.
@kerthcet kerthcet added the cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. label Sep 3, 2024
@InftyAI-Agent InftyAI-Agent added needs-triage Indicates an issue or PR lacks a label and requires one. needs-kind Indicates a PR lacks a label and requires one. needs-priority Indicates a PR lacks a label and requires one. labels Sep 3, 2024
@kerthcet
Copy link
Member Author

kerthcet commented Sep 3, 2024

/milestone v0.1.0

@InftyAI-Agent InftyAI-Agent added this to the v0.1.0 milestone Sep 3, 2024
@kerthcet
Copy link
Member Author

kerthcet commented Sep 4, 2024

/assign

@kerthcet
Copy link
Member Author

kerthcet commented Sep 4, 2024

I think file size is not necessary because usually we know the size very well. And calculating the size takes times, if oneday it costs no time like supported by the hub library, we can support this.

@kerthcet kerthcet mentioned this issue Sep 4, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@kerthcet kerthcet

Labels
cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. needs-kind Indicates a PR lacks a label and requires one. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
Projects
None yet
Milestone
v0.1.0
Development

Successfully merging a pull request may close this issue.

Add verbose log to modelLoader kerthcet/llmaz
2 participants
@kerthcet @InftyAI-Agent