DeepSeek recently published https://arxiv.org/abs/2512.24880, a modification to the Transformer architecture that replaces the single residual stream with several residual connections that layers can read to/from, plus a computationally efficient correction to avoid exploding gradients.

They claim performance gains up to 27B parameters, with only minor reductions in training speed.

Will another frontier lab (Anthropic, OpenAI, DeepMind, Meta, Alibaba, etc.) use publically use mHC in a production model (whether weights open or not) during 2026?