Kickstart your MLOps initiative with a flexible, robust, and productive Python package.

The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery 🧑‍🔬

Social Distancing Detector using deep learning and capable to run on edge AI devices such as NVIDIA Jetson, Google Coral, and more.

An Optimized Speech-to-Text Pipeline for the Whisper Model Supporting Multiple Inference Engine

Guide for fine-tuning Llama/Mistral/CodeLlama models and more

A collection of postmortems. Sorry for the delay in merging PRs!

Inspect: A framework for large language model evaluations

A lightweight, low-dependency, unified API to use all common reranking and cross-encoder models.

Calculate token/s & GPU memory requirement for any LLM. Supports llama.cpp/ggml/bnb/QLoRA quantization

The Art of Debugging

Official Repository for "Eureka: Human-Level Reward Design via Coding Large Language Models" (ICLR 2024)

Reference implementation for DPO (Direct Preference Optimization)

YaRN: Efficient Context Window Extension of Large Language Models

Mamba SSM architecture

S-LoRA: Serving Thousands of Concurrent LoRA Adapters

The goal of this project is to enable users to create cool web demos using the newly released OpenAI GPT-3 API with just a few lines of Python.

LLM Frontend for Power Users.

📋 A list of open LLMs available for commercial use.

Machine Learning Engineering Open Book

🤖 The free, Open Source alternative to OpenAI, Claude and others. Self-hosted and local-first. Drop-in replacement for OpenAI, running on consumer-grade hardware. No GPU required. Runs gguf, transf…

Get up and running with Llama 3.2, Mistral, Gemma 2, and other large language models.

Making large AI models cheaper, faster and more accessible

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficie…

Running large language models on a single GPU for throughput-oriented scenarios.

The standard data-centric AI package for data quality and machine learning with messy, real-world data and labels.

Semantic cache for LLMs. Fully integrated with LangChain and llama_index.

Medusa: Simple Framework for Accelerating LLM Generation with Multiple Decoding Heads

A CLI that writes your git commit messages for you with AI