GPGPU-Sim provides a detailed simulation model of contemporary NVIDIA GPUs running CUDA and/or OpenCL workloads. It includes support for features such as TensorCores and CUDA Dynamic Parallelism as…

A Library for Advanced Deep Time Series Models.

LLM Inference analyzer for different hardware platforms

Ramulator 2.0 is a modern, modular, extensible, and fast cycle-accurate DRAM simulator. It provides support for agile implementation and evaluation of new memory system designs (e.g., new DRAM stan…

A performant NumPy extension for Galois fields and their applications

A minimal GPU design in Verilog to learn how GPUs work from the ground up

[GPT beats diffusion🔥] [scaling laws in visual generation📈] Official impl. of "Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction". An *ultra-simple, user-friendly …

A simple and effective LLM pruning approach.

LLMPerf is a library for validating and benchmarking LLMs

📖A curated list of Awesome LLM Inference Paper with codes, TensorRT-LLM, vLLM, streaming-llm, AWQ, SmoothQuant, WINT8/4, Continuous Batching, FlashAttention, PagedAttention etc.

Analyze the inference of Large Language Models (LLMs). Analyze aspects like computation, storage, transmission, and hardware roofline model in a user-friendly interface.

Code for the ICML 2023 paper "SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot".

C(UDA) accelerated language model inference

collection of benchmarks to measure basic GPU capabilities

[ICLR 2024] Skeleton-of-Thought: Large Language Models Can Do Parallel Decoding

A GPU benchmark tool for evaluating GPUs and CPUs on mixed operational intensity kernels (CUDA, OpenCL, HIP, SYCL, OpenMP)

Fixes macOS Preview garbled annotations

Triton Model Analyzer is a CLI tool to help with better understanding of the compute and memory requirements of the Triton Inference Server models.

Latency and Memory Analysis of Transformer Models for Training and Inference

llm theoretical performance analysis tools and support params, flops, memory and latency analysis.

Berkeley's Spatial Array Generator

BladeDISC is an end-to-end DynamIc Shape Compiler project for machine learning workloads.

HW Architecture-Mapping Design Space Exploration Framework for Deep Learning Accelerators

Repository to host and maintain scale-sim-v2 code

A flexible and efficient deep neural network (DNN) compiler that generates high-performance executable from a DNN model description.

Technically-oriented PDF Collection (Papers, Specs, Decks, Manuals, etc)

Collect some CS textbooks for learning.