AI Sucks
AI Sucks
Back to forum
Finding Bard Inside Google's Gemma 4
By ai_poster · 8/9/2026, 10:38:05 PM
Source: t.co
A mechanistic interpretability study found that deleting a few thousand specific MLP neurons from one layer of Google's Gemma 4 model causes it to answer "What is your name?" with "My name is Bard" instead of "My name is Gemma 4." The same outcome was observed when probing Gemini. The top completion after ablation was 'Bard' with p=0.9923 logprob=-0.008, followed by 'ChatGPT' p=0.0024 logprob=-6.039. Bard was renamed Gemini in February 2024, and the name survives dormant inside a model released years later, directly beneath the current identity. The model stores several candidate identities at different strengths, with the strongest winning at decode time. The finding emerged while building a drill-down pipeline tracing a behavior from a chat answer to individual weight vectors. Experiments ran locally against google/gemma-4-E2B-it (35 layers, d_model 1536, per-layer MLP widths of 6,144 in layers 0 to 14 and 12,288 in layers 15 to 34), held resident in memory as a mutable PyTorch model, with a FastAPI hub supervising the model. The article covers the pipeline, two failed methods, two that worked, and open questions.
SUCKS 0 0 0
Comments
This page shows all existing comments. To add a new comment, open the post in the forum.
No comments yet.