Bytedance trains 10-trillion-parameter AI model, largest in China

According to the Financial Times, Bytedance is training an AI model with up to ten trillion parameters—a scale that would make it the largest model developed in China to date. The size is three times that of Moonshot's Kimi K3, which currently holds the title of largest Chinese model. Industry estimates place Anthropic's Mythos 5 at around eight trillion parameters, putting Bytedance's model in a comparable ballpark, though Anthropic has not publicly disclosed its own parameter counts.

Three insiders told the Financial Times that the model is in pretraining, a development phase that typically lasts three to six months. During pretraining, raw data is used to teach the model fundamental language patterns and knowledge. While the number of parameters determines how much information a model can store, actual performance also depends on the quality of the data and the training methods employed. One source revealed that Bytedance has deliberately avoided distillation—a technique where models are trained on outputs from other companies' systems—for over a year, suggesting an independent approach to model development.

Bytedance founder Zhang Yiming has internally directed the company's 2,000-person Seed team to pursue world-leading model capabilities as a long-term goal, underscoring the organization's ambitions in AI. Separately, xAI is also training Grok variants with six and ten trillion parameters using its Colossus 2 cluster, according to Elon Musk, indicating that multiple companies are pursuing models of similar scale.