AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration

Discovered that protecting the top 1% of salient weights based on activation magnitude preserves model perplexity, enabling hardware-friendly uniform 4-bit weight-only quantization without backpropagation.

Directly guides architectural decisions, alignment strategy, and serving infrastructure for task-text-generation.

Assumptions

• Empirical distribution regularity holds and target domain adheres to pretraining linguistic/visual support

Limitations

• Resource scaling, inference memory requirements, and alignment robustness vary with model size and hardware topology