인문학
사회과학
자연과학
공학
의약학
농수해양학
예술체육학
복합학
DBpia AI
AI 에이전트
AI 아이디어
AI 리더
AI 뷰어
Citeasy
크롬 서지관리 다운로드
지원사업
학술연구/단체지원/교육 등 연구자 활동을 지속하도록 DBpia가 지원하고 있어요.
커뮤니티
연구자들이 자신의 연구와 전문성을 널리 알리고, 새로운 협력의 기회를 만들 수 있는 네트워킹 공간이에요.
논문 기본 정보
- 자료유형
- 학술대회자료
- 저자정보
- 발행연도
- 2024.11
- 수록면
- 870 - 874 (5page)
이용수
초록· 키워드
In Knowledge-Based Visual Question Answering (KB-VQA), which requires external knowledge for accurate question answering, significant advancements have been made through the use of Large Language Models (LLMs). BLIP-2, a popular multimodal LLM, employs a single-layer Q-Former for visual feature extraction and cross-modal interactions but faces challenges with complex reasoning tasks.
To overcome these limitations, we propose integrating the Multimodal Co-Attention Network (MCAN), which uses a multi-layered approach to enhance the interaction between visual and textual inputs. Additionally, we introduce Question-Aware Prompts during fine-tuning, combining Answer Candidates with confidence scores and Answer-Aware Examples from past cases. This improves the model's ability to interpret questions accurately and generate more contextually appropriate answers.
Experimental results on KB-VQA datasets show a 6.9% improvement in accuracy compared to baseline models, demonstrating the effectiveness of our approach in handling complex multimodal reasoning tasks.
상세정보 수정요청해당 페이지 내 제목·저자·목차·페이지To overcome these limitations, we propose integrating the Multimodal Co-Attention Network (MCAN), which uses a multi-layered approach to enhance the interaction between visual and textual inputs. Additionally, we introduce Question-Aware Prompts during fine-tuning, combining Answer Candidates with confidence scores and Answer-Aware Examples from past cases. This improves the model's ability to interpret questions accurately and generate more contextually appropriate answers.
Experimental results on KB-VQA datasets show a 6.9% improvement in accuracy compared to baseline models, demonstrating the effectiveness of our approach in handling complex multimodal reasoning tasks.
정보가 잘못된 경우 알려주세요!
목차
- Abstract
- I. 서론
- II. 본론
- Ⅲ. 실험
- Ⅳ. 결론 및 향후 연구 방향
- 참고문헌
참고문헌
참고문헌 신청최근 본 자료
