MiniCPM-V 2.6: A GPT-4V Level MLLM for Single Image, Multi Image and Video on Your Phone

RLAIF-V: Aligning MLLMs through Open-Source AI Feedback for Super GPT-4V Trustworthiness

Train transformer language models with reinforcement learning.

Get up and running with Llama 3.1, Mistral, Gemma 2, and other large language models.

Web2Code: A Large-scale Webpage-to-Code Dataset and Evaluation Framework for Multimodal LLMs

Efficient Multi-modal Models via Stage-wise Visual Context Compression

The official Github page for "Evaluating the Quality of Hallucination Benchmarks for Large Vision-Language Models"

Xwin-LM: Powerful, Stable, and Reproducible LLM Alignment

[EMNLP 2024] Multi-modal code generation problems.

The official Meta Llama 3 GitHub site

💫 [CVPR 2024] LiDAR4D: Dynamic Neural Fields for Novel Space-time View LiDAR Synthesis

The official repo of our work "Pensieve: Retrospect-then-Compare mitigates Visual Hallucination"

Official implementation for the paper "DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models"

PyTorch code and models for the DINOv2 self-supervised learning method.

[CVPR 2024 Highlight🔥] Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understanding

[CVPR 2024 Highlight] OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation

✨✨Latest Advances on Multimodal Large Language Models

[CVPR 2024] Retrieval-Augmented Image Captioning with External Visual-Name Memory for Open-World Comprehension

LAVIS - A One-stop Library for Language-Vision Intelligence

huggingface mirror download

[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.

[CVPR 2024 Highlight] Visual Point Cloud Forecasting

[CVPR 2024 Highlight] Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding

Open-sourced codes for MiniGPT-4 and MiniGPT-v2 (https://minigpt-4.github.io, https://minigpt-v2.github.io/)

Official repository for "IntentQA: Context-aware Video Intent Reasoning" from ICCV 2023.