GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers

Devised an exact second-order error compensation algorithm based on inverse Hessian matrix updates, quantizing 175B parameter models to 3-bit or 4-bit weights in roughly 4 GPU hours.

Directly guides architectural decisions, alignment strategy, and serving infrastructure for task-text-generation.

Assumptions

• Empirical distribution regularity holds and target domain adheres to pretraining linguistic/visual support

Limitations

• Resource scaling, inference memory requirements, and alignment robustness vary with model size and hardware topology