Multilingual Multimodal Knowledge Graph (M3KG) and Its Application in Smart Nation
Project Name: Multilingual Multimodal Knowledge Graph (M3KG) and Its Application in Smart Nation
Project Sponsor: National Research Foundations
Grant Amount: S$500,000
Duration: 3 Years
Description: Knowledge Graph (KG) encodes human knowledge in the form of (Subject, Predicate, Object), such as (LeeHsienLoong, PrimeMinisterOf, Singapore). With these clauses, we would be able to infer that Lee Hsien Long is the Prime Minister of Singapore, and his father is Mr. Lee Kuan Yew, and his wife is Ho Ching. KG has become popular after Google announced to support KG in late 2000, and the incorporation of KG in their search engine in 2012 to support semantic search. As a result, it has promoted more in-depth research in academia and industry. While text-based KG is very useful, it is also important to extract knowledge from non-English text, and the increasing amount of images and videos. This project aims to build multilingual multimodal KG from large-scale multilingual multimodal resources available on the Web and social media. To accomplish this, we develop techniques to extract entities and their relations from such resources and construct multilingual multimodal QA. The task is challenging but is useful to support a wide range of inferencing tasks, such as multimedia question-answering, multimodal product analytics, and event detection.
