The Metadata is LLaVA: Evaluating Generative AI for Digital Records
Start Date
3-6-2025 10:15 AM
Presentation Summary
This presentation shares a pilot study to assess the multimodal AI tool LLaVA (Large Language and Vision Assistant) in generating descriptive metadata for digitized photographs. The study focuses on evaluation methods for AI-generated descriptions utilizing structured rubrics, which will assess the accuracy, bias, usability, and overall effectiveness of the AI tool. The presentation will also cover the ethics and best practices of using AI-generated metadata for sensitive content. Attendees will gain insights into practical methods for critically evaluating AI-generated outputs, equipping them with adaptable, relevant assessment techniques that may be useful in various library workflow settings.
Presenters Bio
Rebecca Bakker is the Digital Collections Librarian at the Digital Collections Center at FIU Libraries. She manages various digital repositories, supports grant-funded initiatives, and collaborates closely with community partners to preserve and provide access to unique cultural and historical collections. Her current research focuses on integrating AI technologies into digital archiving workflows and enhancing discoverability through metadata innovation. She has presented on topics ranging from oral history transcription to repository migration and digital scholarship.
Adaleen Ferreiro is a third-year undergraduate student at FIU, studying History and Communication Arts, with a particular interest in material culture. She is currently a Humanities Edge intern at the Digital Collections Center at FIU Libraries, where she works on metadata and digital archive projects and is learning about best practices for ensuring long-term accessibility of digital materials. After graduation, Adaleen hopes to pursue a graduate degree in history, with potential specializations in digital humanities and historical preservation.
The Metadata is LLaVA: Evaluating Generative AI for Digital Records
This presentation shares a pilot study to assess the multimodal AI tool LLaVA (Large Language and Vision Assistant) in generating descriptive metadata for digitized photographs. The study focuses on evaluation methods for AI-generated descriptions utilizing structured rubrics, which will assess the accuracy, bias, usability, and overall effectiveness of the AI tool. The presentation will also cover the ethics and best practices of using AI-generated metadata for sensitive content. Attendees will gain insights into practical methods for critically evaluating AI-generated outputs, equipping them with adaptable, relevant assessment techniques that may be useful in various library workflow settings.
