Efficient Triton Kernels for LLM Training

Ring attention implementation with flash attention

A repository for research on medium sized language models.

[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.

Scalable Diffusion Models with State Space Backbone

Mamba SSM architecture

A PyTorch implementation of the paper "All are Worth Words: A ViT Backbone for Diffusion Models".

Audiocraft is a library for audio processing and generation with deep learning. It features the state-of-the-art EnCodec audio compressor / tokenizer, along with MusicGen, a simple and controllable…

[NeurIPS 2023] Riemannian Residual Neural Networks (https://arxiv.org/abs/2006.10254)

🧬 Generative modeling of regulatory DNA sequences with diffusion probabilistic models 💨

Implementation of MagViT2 Tokenizer in Pytorch

Consistency Distilled Diff VAE

Official implementation of VQ-Diffusion

RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference,…

Dataset of GPT-2 outputs for research in detection, biases, and more

Package to compute Mauve, a similarity score between neural text and human text. Install with `pip install mauve-text`.

Cramming the training of a (BERT-type) language model into limited compute.

Open clone of OpenAI's unreleased WebText dataset scraper. This version uses pushshift.io files instead of the API for speed.

Fast and memory-efficient exact attention

Simple Text-Generator with OpenAI gpt-2 Pytorch Implementation

CUDA Library Samples

Transformers with Arbitrarily Large Context

Official PyTorch Implementation of "Scalable Diffusion Models with Transformers"

Reparameterized Discrete Diffusion Models for Text Generation

PyTorch implementation for Score-Based Generative Modeling through Stochastic Differential Equations (ICLR 2021, Oral)

Application of the L2HMC algorithm to simulations in lattice QCD.