vllm-project / llm-compressor Public

Notifications You must be signed in to change notification settings
Fork 35
Star 431

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

431 stars 35 forks Branches Tags Activity

Notifications

Name		Name	Last commit message	Last commit date
Latest commit History 1,833 Commits
.github		.github
docker		docker
docs		docs
examples		examples
src/llmcompressor		src/llmcompressor
tests		tests
utils		utils
.MAINTAINERS		.MAINTAINERS
.gitignore		.gitignore
CONTRIBUTING.md		CONTRIBUTING.md
DEVELOPING.md		DEVELOPING.md
LICENSE		LICENSE
MANIFEST.in		MANIFEST.in
Makefile		Makefile
README.md		README.md
pyproject.toml		pyproject.toml
setup.py		setup.py

Repository files navigation

LLM Compressor

A library for compressing large language models utilizing the latest techniques and research in the field for both training aware and post training techniques. The library is designed to be flexible and easy to use on top of PyTorch and HuggingFace Transformers, allowing for quick experimentation.