The n-gram Language Model

The official evaluation suite and dynamic data release for MixEval.

RewardBench: the first evaluation tool for reward models.

What would you do with 1000 H100s...

Puzzles for learning Triton

18.06 course at MIT

Robust recipes to align language models with human and AI preferences

Scalable training for dense retrieval models.

Python pdb for multiple processes

Machine Learning Engineering Open Book

Scripts for fine-tuning Meta Llama with composable FSDP & PEFT methods to cover single/multi-node GPUs. Supports default & custom datasets for applications such as summarization and Q&A. Supporting…

Train transformer language models with reinforcement learning.

The simplest, fastest repository for training/finetuning medium-sized GPTs.

The TinyLlama project is an open endeavor to pretrain a 1.1B Llama model on 3 trillion tokens.

Expanding natural instructions

Ongoing research training transformer models at scale

Provides end-to-end model development pipelines for LLMs and Multimodal models that can be launched on-prem or cloud-native.

Reference implementation for DPO (Direct Preference Optimization)

Evaluation framework for your Retrieval Augmented Generation (RAG) pipelines

Inference code for Llama models

[NeurIPS 2023] MeZO: Fine-Tuning Language Models with Just Forward Passes. https://arxiv.org/abs/2305.17333

🇫🇷 Oh my tmux! My self-contained, pretty & versatile tmux configuration made with ❤️

This repository is to prepare for Machine Learning interviews.

Instruct-tune LLaMA on consumer hardware

An interactive exploration of Transformer programming.

AI tools for research

Language Modeling with the H3 State Space Model