A tech head at a mid-sized NBFC in Mumbai was blocked for three months by his compliance team from using AI due to RBI rules and customer data concerns, seeking a way to use AI tools without data leaving India. The article, set in Mumbai last month, explains that self-hosting AI has shifted from a niche topic to a common discussion among CTOs and CIOs, enabled by two tools: Ollama and Docker. The author notes that while he was sceptical about self-hosting two years ago due to painful hardware and unimpressive models, today’s open models like Llama 3.1, Mistral, Phi-3, Gemma, Qwen, and DeepSeek are "really good," with some running on a workstation costing less than three months of API bills. CTOs cite reasons for self-hosting: data cannot leave India or a specific data centre, they want response times independent of AWS outages, they want to test prompts without metered costs, and they are tired of pricing hostage situations. Ollama is described as the "Docker of language models," hiding model file formats, driver headaches, and Python environment chaos; users install it, run a model like Llama 3.1, and get a working API on port 11434. A Modelfile, similar to a Dockerfile, allows users to pick a base model, add a system prompt, and set parameters like temperature and context length, making prompts config rather