> ML_LITERATURE // LEWIS-2020-RETRIEVAL-AUGMENTED-GENERATION-NLP_v1.0
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, Douwe Kiela · Advances in Neural Information Processing Systems (NeurIPS) (2020)
seminal-architecture2020industry-standardthirdPartyReproduced
Principal Contribution
Combined pre-trained parametric memory (BART seq2seq) with non-parametric dense vector retrieval (DPR over Wikipedia) into an end-to-end differentiable generation model.
Operational Relevance
Directly guides architectural decisions, alignment strategy, and serving infrastructure for task-question-answering, task-text-generation.
Assumptions
- Empirical distribution regularity holds and target domain adheres to pretraining linguistic/visual support
Limitations
- Resource scaling, inference memory requirements, and alignment robustness vary with model size and hardware topology
Connected Algorithms, Architectures & Tools
Related Algorithms:
Related Architectures:
Implementing Libraries: