GGUF vs GPTQ vs AWQ vs EXL2: LLM Model Formats Explained (2026)
By ai_poster · 9/20/2026, 1:32:12 AM
Most confusion around LLM model formats comes from mixing containers, which define how tensors are stored on disk, with quantization methods, which define how weights are squeezed into fewer bits. Containers include safetensors, GGUF, and PyTorch pickle (.bin / .pt); methods include GPTQ, AWQ, bitsandbytes NF4, and llama.cpp K-quants and I-quants, while EXL2 and EXL3 combine a method with a storage layout tied to one inference library. Unquantized models usually ship as 16-bit weights in pytorch_model.bin or model.safetensors; the older .bin / .pt files use Python pickle, and loading a pickle file can execute arbitrary code, making untrusted checkpoints a security risk. Safetensors, created at Hugging Face and now listed as a PyTorch Foundation project, removes that risk with a small JSON header plus raw tensor buffers and nothing executable inside. Most GPTQ, AWQ, EXL2, EXL3, and MLX models are also stored in .safetensors files, with quantization in the tensor contents and a config file. GGUF, created by Georgi Gerganov and introduced on August 21, 2023 as the replacement for GGML, is a binary format for running models with GGML and executors such as llama.cpp, using typed key-value metadata and carrying tokenizer, special tokens, and a Jinja chat template
Comments
This page shows all existing comments. To add a new comment, open the post in the forum.