A high-throughput and memory-efficient inference and serving engine for LLMs

Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

An open-source NLP research library, built on PyTorch.

The open-source tool for building high-quality datasets and computer vision models

Learn how to design large-scale systems. Prep for the system design interview. Includes Anki flashcards.

Top-level directory for documentation and general content

Sparsity-aware deep learning inference runtime for CPUs

ML model optimization product to accelerate inference.

Neural network model repository for highly sparse and sparse-quantized models with matching sparsification recipes

Libraries for applying sparsification recipes to neural networks with a few lines of code, enabling faster and smaller models