Artificial intelligence has created a governance problem for banks, as models offering the greatest gains in fraud detection, credit assessment, customer service, compliance and operations are often the hardest to reduce to a human-readable explanation. While explainability matters, an explanation can be plausible without being complete, stable without being correct, or easy to read without faithfully describing the mechanism that produced the decision. The emerging supervisory view treats explainability as one control among many, with frameworks from the Bank for International Settlements, the Financial Stability Board and NIST emphasising governance, validation, resilience, data quality and accountability. Traditional model governance assumes a model can usually be bounded, but generative and agentic AI weaken those assumptions by responding differently to small changes in prompts, incorporating external tools, generating unstructured outputs and operating across tasks for which they were not explicitly trained. In 2026, the OCC’s revised model-risk guidance explicitly excluded generative AI and agentic AI from scope because of their novelty and rapid evolution, while indicating that the banking agencies intended further work on banks’ use of AI. That exclusion is not a regulatory exemption but evidence that older model-risk frameworks do not map neatly onto newer systems.