Hugging Face's logo
Jim Lai's picture
59

Jim Lai

grimjim
xzuyn's profile picture rityak's profile picture wintervim's profile picture
·

AI & ML interests

Experimenting primarily with 7B-12B parameter text completion models. Not all models are intended for direct end use, but aim for research and/or educational purposes. Recent Contributions: stabilized refusal direction ablation via Gram-Schmidt orthonormalization and norm-preserving interventions; confirmed reasoning transfer via model merger.

Recent Activity

updated a model 1 day ago
grimjim/Equatorium-v1-12B
published a model 1 day ago
grimjim/Equatorium-v1-12B
posted an update 2 days ago
After tinkering with Gemma Scope 2, I now have an mechanistic explanation of why Winsorization was as effective as it was in my ablation experiments on Gemma 3 12B Instruct. In short, the activation for the BOS token overwhelms everything else. Gemma Scope 2 deliberately did not train on the BOS token. Winsorization capped the magnitude of the BOS token, allowing the activations of other tokens to be compared. https://huggingface.co/google/gemma-scope-2-12b-it
View all activity

Organizations

Social Post Explorers's profile picture SillyTilly's profile picture Hugging Face Discord Community's profile picture Debased AI's profile picture Anthracite's profile picture Anthracite Core's profile picture smE's profile picture Hugging Face MCP Course's profile picture GrimJim's Merge Workshop's profile picture Tokenizers and Chat Templates Only's profile picture MCP-1st-Birthday's profile picture GrimJim's Experimental Workshop's profile picture