Open Inference Review

Inference Throughput

Latest research

Cover illustration for “Speculative Decoding for Latency Reduction”

Inference Throughput

Speculative Decoding for Latency Reduction

A faster way to run large language models without sacrificing output quality.

Marcus Oyelaran9 min read

    Inference Throughput

    View all →
    1. Speculative Decoding for Latency ReductionA faster way to run large language models without sacrificing output quality.Marcus OyelaranSeptember 23, 20269 min read