Barry Menglong Yao is a PhD student in Computer Science at Virginia Tech, advised by Dr. Lifu Huang. Before that, he was a master’s student of Computer Science at the University at Buffalo, advised by Dr. Changyou Chen. He was also a graduate research assistant in Dr. David Doermann’s Federated Learning project. He received the Best Paper Award Honorable Mention from SIGIR 2023 and the Best MS Research Award from the CSE department at the University at Buffalo.
Check my CV .
PhD in Computer Science, Present
M.S. in Computer Science and Engineering, 2022
University at Buffalo, The State University of New York
B.S. in Computer Science and Technology, 2016
We propose the end-to-end multimodal fact-checking and explanation generation, where the input is a claim and a large collection of web sources, including articles, images, videos, and tweets, and the goal is to assess the truthfulness of the claim by retrieving relevant evidences and predicting a truthfulness label (i.e., support, refute and not enough information), and to generate a rationalization statement to explain the reasoning and ruling process. To support this research, we construct MOCHEG, a large-scale dataset consisting of 21,184 claims where each claim is annotated with a truthfulness label and ruling statement, with 43,148 text evidences and 15,375 image evidences. To establish baseline performances on MOCHEG, we experiment with several state-of-the-art neural architectures on the three pipelined subtasks: multimodal evidence retrieval, claim verification and explanation generation, and demonstrate that the performance of the state-of-the-art end-to-end multimodal fact-checking does not provide satisfactory outcomes. To the best of our knowledge, we are the first to build the benchmark dataset and solutions for end-to-end multimodal fact-checking and explanation generation.