Fast and memory-efficient exact attention

An Easy-to-use, Scalable and High-performance RLHF Framework (70B+ PPO Full Tuning & Iterative DPO & LoRA & Mixtral)

A high-throughput and memory-efficient inference and serving engine for LLMs

Visual self-questioning for large vision-language assistant.

Implementation for the paper "Reliable Visual Question Answering Abstain Rather Than Answer Incorrectly" (ECCV 2022: https//arxiv.org/abs/2204.13631).

A data generation pipeline for creating semi-realistic synthetic multi-object videos with rich annotations such as instance segmentation masks, depth maps, and optical flow.

VMamba: Visual State Space Models，code is based on mamba

EfficientSAM: Leveraged Masked Image Pretraining for Efficient Segment Anything

A PyTorch-based Speech Toolkit

A Repository for Single- and Multi-modal Speaker Verification, Speaker Recognition and Speaker Diarization

Example models using DeepSpeed

[EMNLP'2024 Findings] Explore generated documents for enhanced IR with LLMs. We enhance BM25 to surpass strong dense retriever on many datasets.

[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.

Mamba SSM architecture

Train transformer language models with reinforcement learning.

PyTorch code for Learning Cooperative Visual Dialog Agents using Deep Reinforcement Learning

{KFAC,EKFAC,Diagonal,Implicit} Fisher Matrices and finite width NTKs in PyTorch

ICCV 2023 Paper Global Features are All You Need for Image Retrieval and Reranking Official Repository

Easily turn large sets of image urls to an image dataset. Can download, resize and package 100M urls in 20h on one machine.

US VISA (ais.usvisa-info.com) appointment re-scheduler - Colombian adaptation

A curated list of AWESOME papers, datasets and tutorials within Multimodal Knowledge Graph.

Several data modalities for KBs (visual, numerical, temporal, etc.)