> ML_LITERATURE // ETHAYARAJH-2024-KTO-MODEL-ALIGNMENT-PROSPECT-THEORY_v1.0
KTO: Model Alignment as Prospect Theoretic Optimization
Kawin Ethayarajh, Winnie Xu, Niklas Muennighoff, Dan Jurafsky, Christopher D. Manning · International Conference on Machine Learning (ICML) (2024)
algorithm2024industry-standardthirdPartyReproduced
Principal Contribution
Formulated Kahneman-Tversky Optimization (KTO), directly aligning LLMs using unpaired binary positive/negative signals rather than paired preferences.
Operational Relevance
Directly guides architectural decisions, alignment strategy, and serving infrastructure for task-text-generation.
Assumptions
- Empirical distribution regularity holds and target domain adheres to pretraining linguistic/visual support
Limitations
- Resource scaling, inference memory requirements, and alignment robustness vary with model size and hardware topology
Connected Algorithms, Architectures & Tools
Related Algorithms:
Related Architectures:
Implementing Libraries: