Skip to main content

> ML_LITERATURE // LIN-2022-TRUTHFULQA-MEASURING-HOW-MODELS-MIMIC-HUMAN-FALSEHOODS_v1.0

TruthfulQA: Measuring How Models Mimic Human Falsehoods

Stephanie Lin, Jacob Hilton, Owain Evans · Annual Meeting of the Association for Computational Linguistics (ACL) (2022)

benchmark2022foundationalartifactsAvailable
arXiv:2109.07958Back to Literature

Principal Contribution

Constructed TruthfulQA: 817 questions across 38 categories designed to elicit common human superstitions, misconceptions, and conspiracy theories from LLMs.

Operational Relevance

Serves as qualified theoretical and empirical reference for task-question-answering.

Assumptions

  • Mathematical convexity, regularity, and empirical consistency hold across problem configurations

Limitations

  • Theoretical bounds and benchmark saturation characteristics vary across model architectures and training scales

Connected Algorithms, Architectures & Tools

Related Algorithms:
Related Architectures:
Implementing Libraries: