> ML_LITERATURE // LIU-2023-VISUAL-INSTRUCTION-TUNING-LLAVA_v1.0
Visual Instruction Tuning (LLaVA)
Haotian Liu, Chunyuan Li, Qingyang Wu, Yong Jae Lee · Advances in Neural Information Processing Systems (NeurIPS) (2023)
seminal-architecture2023industry-standardthirdPartyReproduced
Principal Contribution
Connected a CLIP vision encoder to Vicuna LLM via a simple linear projection matrix trained on GPT-4 generated multimodal instruction-following conversations.
Operational Relevance
Serves as qualified reference for implementing task-multimodal, task-question-answering in production systems.
Assumptions
- Underlying spatio-temporal continuity and domain distributional stability hold
Limitations
- Performance scaling and computational footprint depend on receptive field depth, sequence length, and resolution
Connected Algorithms, Architectures & Tools
Related Algorithms:
Related Architectures:
Implementing Libraries: