GPTQ vs AWQ for LLM Weight Quantization

A 70B parameter model in FP16 needs 140GB of VRAM just to sit in memory, which puts it out of reach for anyone without a rack of A100s. Quantize it to 4-bit and that number drops to roughly 35-40GB, small enough to load on a single RTX 4090 or one A100. That drop, on the order of 75% off the original size, comes at a cost of maybe 1-5% in quality depending on the method and bit depth chosen. Two algorithms dominate this space, GPTQ and AWQ, and they get there through entirely different mechanics. GGUF is a file format built for CPU and Apple Silicon inference, while GPTQ and AWQ are quantization algorithms, and comparing a format to an algorithm is a category error that shows up constantly in casual discussion of this topic.

What GPTQ does: second-order optimization applied layer by layer

GPTQ came out of work by Frantar, Ashkboos, Hoefler, and Alistarh, accepted at ICLR 2023 (arXiv:2210.17323). The method builds on an earlier framework called Optimal Brain Quantization, and the core move is to quantize weights one at a time, not all at once, and after each one gets rounded down to 4-bit, the remaining unquantized weights get adjusted to make up for the error just introduced.

That adjustment step relies on second-order information, specifically the Hessian matrix, which tells the algorithm how sensitive the model's output is to changes in each weight. A weight sitting in a high-curvature region of the loss landscape gets treated carefully, because a small error there produces a large downstream effect. A weight in a flatter region can absorb more rounding error without much consequence. The process is applied selectively rather than as a blanket rule across a layer. It's a running correction, weight by weight, guided by gradient sensitivity, which is a meaningfully more expensive computation than plain rounding but buys back a lot of the accuracy that naive quantization would throw away.

Protecting the 1% of weights that drive most of the output

AWQ came out of Song Han's group at MIT, first posted to arXiv in June 2023, and it went on to win the MLSys 2024 Best Paper Award. The starting observation is blunt: most weights in a transformer layer barely matter, and a small slice, something like 1%, accounts for most of the damage when quantization goes wrong.

Finding that 1% is harder. AWQ doesn't look at the weights themselves to decide what's important, it looks at activations. During a calibration pass, the algorithm tracks which input channels consistently produce large activation values, and treats those channels as salient regardless of how large or small the corresponding weights happen to be. It would be easy to assume the biggest weights matter most, but AWQ's premise is that a channel driving big activations determines salience more than whatever weight magnitude sits on it.

Keeping those salient weights at higher precision while dropping everything else to 4-bit would work, but it breaks the clean, uniform memory layout that makes 4-bit inference fast on GPUs. So AWQ does something more clever: before quantization even happens, it rescales the salient channels with a per-channel scaling factor, math that's exactly reversible, and folds that scaling back into the stored weights at inference time. Every weight ends up at 4-bit, but the salient ones have been repositioned in the numerical range where quantization error hurts them least.

The mechanical difference that determines everything downstream: Hessian correction vs. activation-aware scaling

Both algorithms land in the same broad category, W4A16: weights stored in INT4, activations left in FP16 during inference. That shared design is what makes a head-to-head comparison fair in the first place, and it's why both methods beat naive rounding by a wide margin on speed and memory alike.

Past that similarity, the two diverge sharply in when they intervene. GPTQ works after the fact. It quantizes a weight, measures the error that move just introduced, and pushes that error into the weights that haven't been touched yet, a correction applied directly to weight values. AWQ works before the fact. It never corrects weight values after quantizing, instead it changes how the weights are expressed in the first place, through the per-channel scaling described above, so that the rounding step which follows has less room to do damage.

That difference carries a practical consequence around calibration data. GPTQ's correction step is built to match the specific outputs of the calibration set. A narrow or unrepresentative calibration set can leave the model overfit to that particular domain, accurate on the calibration data's style of prompt and shakier outside it. AWQ, because it's measuring activation statistics rather than fitting to output values, doesn't carry that same overfitting risk in the same way, since it's describing a general property of the channels rather than matching a specific target.

Accuracy benchmarks at 70B scale, and where AWQ leads and GPTQ holds

On the Open LLM Leaderboard V2 with Llama-3.1, a paper from Red Hat AI and IST Austria found AWQ scoring an average of 27.40 on Llama-3.1-8B-Instruct against GPTQ's 26.53, with the unquantized base model at 27.62. AWQ scored clearly higher on IFEval (78.25 vs. 76.30) and GPQA (5.21 vs. 4.04), while GPTQ actually came out slightly ahead on BBH, 28.91 against 27.20. At 70B scale, the same comparison held roughly steady: AWQ averaged 41.09 against GPTQ's 40.58, with the base model at 41.66, a modest gap but one that appeared consistently across most sub-benchmarks.

That same paper's third finding cuts against the tidy version of this story: a well-tuned GPTQ variant beat AWQ outright on certain real-world tasks. So the advantage isn't universal, and it depends heavily on calibration quality and how close the task domain sits to what the calibration set covered.

The benchmark pattern above is consistent with broader findings in the quantization literature: AWQ tends to hold closer to baseline accuracy at 4-bit, while GPTQ can show more variation across scales and tasks. 4-bit quantization introduces a real accuracy cost, and the gap between the two methods varies depending on model scale and task domain.

Throughput benchmarks: why the inference kernel matters more than the format label

A JarvisLabs benchmark from January 2026, run on Qwen2.5-32B-Instruct on an H200 GPU, makes a point that's easy to miss if you only look at the quantization method's name. The FP16 baseline set a throughput reference point, with higher Pass@1 on code generation. AWQ without the Marlin kernel managed only 67 tokens per second, slower than FP16, despite using a quarter of the memory. Add the Marlin kernel to that same AWQ model and throughput jumped to 741 tokens per second, a substantial multiple of improvement from the kernel change alone, nothing else about the quantized weights changed. GPTQ followed the same pattern: 276 tokens per second without Marlin, 712 tokens per second with it. BitsandBytes, tested alongside, held onto the best perplexity of the group at 6.67, though it lagged both AWQ and GPTQ on raw speed once Marlin entered the picture.

The reason a naive quantized model runs slower than FP16 comes down to where time actually gets spent. Standard GPTQ or AWQ inference without an optimized kernel loads INT4 weights from HBM, dequantizes them back to FP16, and only then runs the matrix multiplication. That dequantize step turns into a memory bandwidth bottleneck, with compute units sitting idle waiting on data to arrive rather than doing math. Marlin fixes that by fusing dequantization directly into the matrix multiplication step, so INT4 weights feed straight into the compute unit without a separate conversion pass in between.

A 2026 SGLang paper (arXiv:2509.22944) backs this up with numbers across several model sizes: AWQ with an appropriate kernel delivered a 2.4× speedup over FP16 on Llama2-7B and a 2.9× speedup on Qwen3-32B, with the gains varying by model size and architecture. The pattern across all these benchmarks points to one conclusion: the quantization algorithm sets the accuracy ceiling, but the kernel decides whether that accuracy ever turns into usable speed.

Memory savings in practice: what 4-bit does to VRAM budgets

AWQ's INT4 output cuts GPU memory by roughly half compared to FP16, with a quality hit small enough to disappear into benchmark noise on most tasks. GPTQ shows similar gains in absolute terms. A study on an e-commerce small language model (arXiv:2510.21970) found GPTQ 4-bit quantization brought total VRAM down from 3.27GB to 1.93GB, a 41% reduction, with the parameter-only footprint dropping from 2.30GB to 0.96GB.

At 70B scale, both algorithms are roughly the same, somewhere around 35-40GB, close enough that memory budget alone rarely settles the choice between them. The quality gap in that range tends to run about 1-2% in benchmark scores, which matters for some workloads and is background noise for others.

What that memory drop actually buys, in concrete terms, is needing one consumer card instead of multiple A100s. A 70B model that once required a multi-GPU server to load can run on a single RTX 4090 once quantized to 4-bit, which is the detail that matters most to anyone actually budgeting hardware rather than reading benchmark tables.

Tooling and ecosystem: active and deprecated libraries for projects starting today

AutoGPTQ, for years the default library for running GPTQ, was archived in April 2025. AutoAWQ has been deprecated too, with the vLLM team's llm-compressor taking over as the recommended path forward. Anyone starting a new project on either of these older libraries is building on ground that's no longer being maintained for security or compatibility, even though both still function for basic inference today.

The current landscape looks different. GPTQModel, forked originally from AutoGPTQ, has diverged enough to stand as its own project: faster quantization, lower memory use during the quantization process itself, more accurate defaults, and more accurate defaults. It also covers a wider hardware surface than its predecessor, running on Linux, macOS, and Windows 11, with support for AMD ROCm, Apple Silicon, Intel and AMD CPUs, and Intel's Datacenter Max and Arc GPU lines, plus multimodal models. For AWQ, llm-compressor from the vLLM team is the current recommendation, with Hugging Face Transformers and TensorRT-LLM both offering supported paths as well.

On the serving side, vLLM supports both formats and offers the Marlin kernel for each, which, given the throughput numbers above, is close to a requirement rather than a nice-to-have. Hugging Face Transformers offers a supported path for both formats. GPTQ still holds a larger library of pre-quantized models on Hugging Face, a legacy of its earlier arrival, but AWQ has caught up fast and now ships alongside GGUF as a default option for many new model releases. For a project starting today, the honest read is that both algorithms are live and well supported, and the choice comes down less to which one is "better" and more to which serving stack and hardware target a given deployment is already built around.