Open Inference Review

Serving Frameworks

Latest research

Cover illustration for “vLLM Architecture and PagedAttention Internals”

Serving Frameworks

vLLM Architecture and PagedAttention Internals

vLLM borrows decades-old OS memory tricks to eliminate GPU cache fragmentation.

Priya Subramaniam10 min read

    Serving Frameworks

    View all →
    1. vLLM Architecture and PagedAttention InternalsvLLM borrows decades-old OS memory tricks to eliminate GPU cache fragmentation.Priya SubramaniamSeptember 23, 202610 min read