FeiYuan

FeYuan

8 14 4

AI & ML interests

Multilingual & Code Intelligence

Recent Activity

posted an update about 14 hours ago

Drift-Constrained Optimization: Only Direction Matters in Fine-Tuning Instruct Models We usually ask: How can we SFT an instruct model without behavioral drift? We reverse the question: Given a fixed drift budget, how should we SFT? This predicts that changing *where* the model can move—not the supervision—can reverse fine-tuning failure. On Qwen3-8B and Qwen3-14B, it does. code: https://github.com/CONE-MT/DCO/tree/main/dco models: https://huggingface.co/collections/LLaMAX/dco

submitted a paper about 14 hours ago

Drift-Constrained Optimization: Only Direction Matters in Fine-Tuning Instruct Models

submitted a paper about 14 hours ago

Training Specialist Models without Reasoning Trajectories for Domain Expert Distillation

View all activity

Organizations

Posts 2

Post

Drift-Constrained Optimization: Only Direction Matters in Fine-Tuning Instruct Models

We usually ask:

How can we SFT an instruct model without behavioral drift?

We reverse the question:

Given a fixed drift budget, how should we SFT?

This predicts that changing *where* the model can move—not the supervision—can reverse fine-tuning failure.

On Qwen3-8B and Qwen3-14B, it does.

code: https://github.com/CONE-MT/DCO/tree/main/dco
models: https://huggingface.co/collections/LLaMAX/dco

Post

5059

Hi everyone, I'm excited to introduce our latest work, LLaMAX. 😁😁😁
LLaMAX is a powerful language model created specifically for multilingual scenarios. Built upon Meta's LLaMA series models, LLaMAX undergoes extensive training across more than 100 languages.

Remarkably, it enhances its multilingual capabilities without compromising its generalization ability, surpassing existing LLMs.

✨Highlights:

🎈 LLaMAX supports the 102 languages covered by Flores-101, and its performance in translating between low-resource languages far surpasses other decoder-only LLMs.

🎈 Even for languages not covered in Flores-200, LLaMAX still shows significant improvements in translation performance.

🎈 By performing simple SFT on English task data, LLaMAX demonstrates impressive multilingual transfer abilities in downstream tasks.

🎈 In our paper, we discuss effective methods for enhancing the multilingual capabilities of LLMs during the continued training phase.

We welcome you to use our model and provide feedback.

More Details:

🎉 Code: https://github.com/CONE-MT/LLaMAX/

🎉 Model: https://huggingface.co/LLaMAX/

View all Posts

Papers 13

models 0

None public yet

datasets 0

None public yet