Machine learning models used for inference on Kubernetes are often several gigabytes in size, and embedding them in container images creates oversized images and slows pod scheduling. Inference pods are inherently stateful, requiring model weights, tokenizer files, compiled GPU kernels, and runtime caches to persist across pod restarts, node failures, and scaling events. Without persistent, shared storage, every new pod re-downloads or recomputes these artifacts, causing minutes-long cold starts. Amazon FSx for NetApp ONTAP, built on NetApp’s ONTAP storage operating system, integrates with Amazon Elastic Kubernetes Service (Amazon EKS) through the Trident CSI driver, providing persistent shared volumes. The post offers guidance on solving the stateful inference problem, including serving ML models from a single shared volume, replicating on-premises trained models from NetApp storage using NetApp SnapMirror, and automating volume provisioning with Trident. A working example uses a fine-tuned Qwen2.5-1.5B model classifying emails as safe or phishing, served with vLLM on an NVIDIA L4 GPU, with model weights and compiled CUDA kernels cached on an FSx for ONTAP volume. The GitHub repository provides a deployable solution with Terraform, Kubernetes manifests, and automated benchmark scripts measuring cold start vs. warm start performance. FSx for ONTAP supports Multi-AZ deployments with automatic failover, keeping model artifacts accessible during an Availability Zone failure.