SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models

Designed a mathematically equivalent per-channel scaling transformation smoothing activation outliers onto weights, enabling true W8A8 INT8 matrix multiplication on Tensor Cores.

Directly guides architectural decisions, alignment strategy, and serving infrastructure for task-text-generation.

Assumptions

• Empirical distribution regularity holds and target domain adheres to pretraining linguistic/visual support

Limitations

• Resource scaling, inference memory requirements, and alignment robustness vary with model size and hardware topology