EY has developed a multimodal retrieval-augmented generation framework that retrieves charts, tables, engineering diagrams, equations and images alongside text, connecting them through a knowledge graph to produce more complete and contextualized answers. Conventional RAG systems are built mainly to retrieve text, but EY’s research says most implementations overlook useful data because enterprise documents often place critical facts in illustrative content. The approach does not affect the underlying LLM but changes how enterprise content is prepared, indexed, related and supplied to the model at inference time. The work grew out of limitations EY encountered in client projects, said Dipanjan Sengupta, EY Global Delivery Services Consulting distinguished technologist and AI engineering leader. The framework separates textual and illustrative content into different ingestion pipelines, stores them in separate vector indexes, and makes each content element a node in a knowledge graph with weighted relationships. A white paper describes three methods for building those relationships: deterministic keyword matching, semantic similarity based on embeddings, and machine learning-based inference for implicit associations. A “gleaning” process looks for missing links, resolves ambiguous entities and identifies related information across documents, resulting in a heterogeneous graph that can support multi-hop reasoning.