Fault-tolerant, highly scalable GPU orchestration, and a machine learning framework designed for training models with billions to trillions of parameters

🤖 🔥 Language-to-actions engine

Numbers every LLM developer should know

OmniXAI: A Library for eXplainable AI

Burn is a new comprehensive dynamic Deep Learning Framework built using Rust with extreme flexibility, compute efficiency and portability as its primary goals.

Distribute and run LLMs with a single file.

Postgres with GPUs for ML/AI apps.

A language for constraint-guided and efficient LLM programming.

AirLLM 70B inference with single 4GB GPU

Finetune Llama 3.3, DeepSeek-R1, Reasoning, Phi-4 & Gemma 2 LLMs 2x faster with 70% less memory

A fast inference library for running LLMs locally on modern consumer-class GPUs

Argilla is a collaboration tool for AI engineers and domain experts to build high-quality datasets

Code and documents of LongLoRA and LongAlpaca (ICLR 2024 Oral)

Build, customize and control you own LLMs. From data pre-processing to fine-tuning, xTuring provides an easy way to personalize open-source LLMs. Join our discord community: https://discord.gg/TgHX…

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

🪢 Open source LLM engineering platform: LLM Observability, metrics, evals, prompt management, playground, datasets. Integrates with LlamaIndex, Langchain, OpenAI SDK, LiteLLM, and more. 🍊YC W23

Self-hosted AI coding assistant

An Extensible Toolkit for Finetuning and Inference of Large Foundation Models. Large Models for All.

4 bits quantization of LLaMA using GPTQ

Code for the ICML 2023 paper "SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot".

An open-source visual programming environment for battle-testing prompts to LLMs.

LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.

The LLM Evaluation Framework

A library for advanced large language model reasoning

Serving multiple LoRA finetuned LLM as one

🔍 LangKit: An open-source toolkit for monitoring Large Language Models (LLMs). 📚 Extracts signals from prompts & responses, ensuring safety & security. 🛡️ Features include text quality, relevance m…

ChatdollKit enables you to make your 3D model into a chatbot

the LLM vulnerability scanner