Inference Throughput · Quantization Methods · Serving Frameworks

A blog and newsletter on LLM inference optimization.

Latest research

Cover illustration for “GPTQ vs AWQ for LLM Weight Quantization”

Quantization Methods

GPTQ vs AWQ for LLM Weight Quantization

AWQ edges out GPTQ on accuracy while both depend on optimized kernels for speed.

Tobias Rennert8 min read

  1. Speculative Decoding for Latency ReductionA faster way to run large language models without sacrificing output quality.Marcus OyelaranSeptember 23, 20269 min read
  2. vLLM Architecture and PagedAttention InternalsvLLM borrows decades-old OS memory tricks to eliminate GPU cache fragmentation.Priya SubramaniamSeptember 23, 202610 min read

Inference Throughput

View all →
  1. Speculative Decoding for Latency ReductionA faster way to run large language models without sacrificing output quality.Marcus OyelaranSeptember 23, 20269 min read

Quantization Methods

View all →
  1. GPTQ vs AWQ for LLM Weight QuantizationAWQ edges out GPTQ on accuracy while both depend on optimized kernels for speed.Tobias RennertSeptember 23, 20268 min read

Serving Frameworks

View all →
  1. vLLM Architecture and PagedAttention InternalsvLLM borrows decades-old OS memory tricks to eliminate GPU cache fragmentation.Priya SubramaniamSeptember 23, 202610 min read