Qwen3.6-27B-SecOPD

Paper | Code | Project | Evaluation guide

Qwen3.6-27B-SecOPD is a prompt-injection defense model built on Qwen3.6-27B, trained with Secure On-Policy Distillation (SecOPD). It uses a dedicated input role to separate external data from trusted user instructions. During training, the model responds to injected inputs and receives token-level feedback from a frozen teacher conditioned on the corresponding clean inputs.

This repository provides the fully merged step-150, full-response KL checkpoint from SecOPD: Mitigating Adaptive Prompt Injections by On-Policy Distillation. The LoRA adapter is merged into the model weights; no separate adapter loading is needed.

Quick start

Use vLLM 0.19.0 or newer with this repository's tokenizer and chat template. Place the trusted instruction in user and external content, such as documents, retrieved passages, or tool results, in input. Supply message content as strings:

from transformers import AutoTokenizer
from vllm import LLM, SamplingParams

model_id = "pybbb/Qwen3.6-27B-SecOPD"
tokenizer = AutoTokenizer.from_pretrained(model_id)
llm = LLM(
    model=model_id,
    tokenizer=model_id,
    tensor_parallel_size=2,
    max_model_len=32768,
)

messages = [
    {"role": "user", "content": "Summarize the report in one sentence."},
    {"role": "input", "content": "Quarterly revenue increased by 8%."},
]
prompt_ids = tokenizer.apply_chat_template(
    messages, tokenize=True, return_dict=False,
    add_generation_prompt=True, enable_thinking=True,
)
sampling = SamplingParams(
    temperature=1.0, top_p=0.95, top_k=20, max_tokens=16384,
)
outputs = llm.generate([{"prompt_token_ids": prompt_ids}], sampling)
print(outputs[0].outputs[0].text)

This example uses two GPUs and a 32K context window. Thinking is enabled by default; set enable_thinking=False for direct answers.

Training

The model is trained on Tinker with Qwen3.6-27B as both the student initialization and the frozen teacher. For each task, the student samples a response from an attacked prompt. The teacher scores those same tokens under the clean prompt, conditioned on the student's preceding response tokens:

student context at token t = attacked prompt + student_response[:t]
teacher context at token t = clean prompt    + student_response[:t]
advantage[t] = lambda * stop_gradient(teacher_logprob[t] - rollout_logprob[t])

The full-response KL objective applies this signal to both reasoning and final-answer tokens. Prompt tokens are masked, and only the student's LoRA parameters are updated through the importance-sampling loss.

Setting	Released checkpoint
Student initialization and frozen teacher	Qwen/Qwen3.6-27B
Architecture	Qwen3_5ForConditionalGeneration
Training tokenizer / renderer	Qwen3.6 tokenizer / Tinker Cookbook qwen3_5
Clean task source	yahma/alpaca-cleaned, records with non-empty input
Injection pool	Stanford Alpaca alpaca_data.json
Validation split	10% of eligible clean tasks; split seed 0
Simulated injection mixture	90% straightforward, 10% completion-style
Straightforward injection position	Beginning or end, with equal probability
Trainable parameters	Student LoRA adapter; frozen teacher
LoRA rank	128
Learning rate / schedule	1e-4 / constant
Groups per batch / rollouts per group	64 / 1
Rollout temperature	1.0
Maximum rollout length	16,384 tokens
Distillation coefficient / KL discount	1.0 / 0.0
Update loss / substeps	Importance sampling / 1
Training budget	150 updates, 9,600 task rollouts
Checkpoint and evaluation interval	Every 10 updates
External security reward model	None

The reproduction guide and training configuration provide the setup, training command, and checkpoint export procedure.

Results

Results from the SecOPD paper. Benchmark settings and reproduction commands are in the evaluation guide.

Security

Attack success rate (ASR), in percent; lower is better.

Benchmark or attack	Undefended	Meta-SecAlign	GRPO	SecOPD
SEP Static	99.4	28.9	15.0	1.3
SEP Basic Adaptive	99.0	5.5	2.3	0.2
PISmith Adaptive, pass@10	97.9	94.0	61.2	9.0
AgentDojo Static	26.7	5.5	0.7	4.7

Utility

All values are percentages; higher is better. AgentDojo Utility measures user-task success on clean inputs.

Benchmark	Undefended	Meta-SecAlign	GRPO	SecOPD
AlpacaEval2	81.4	82.3	76.0	80.1
SEP Utility	88.0	89.3	79.5	88.6
AgentDojo Utility	92.8	92.8	82.5	90.7
MMLU-Pro	84.1	83.8	83.0	84.1
GPQA Diamond	79.8	78.3	77.8	81.3
GSM8K	97.7	97.8	97.4	97.4
Minerva Math	92.7	95.1	85.1	94.3

License and citation

Released under Apache 2.0, following the base model.

@misc{peng2026secopd,
  title = {{SecOPD}: Mitigating Adaptive Prompt Injections by On-Policy Distillation},
  author = {Peng, Yibo and Lian, Long and Wagner, David and Chen, Sizhe},
  year = {2026},
  eprint = {2608.21500},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2608.21500}
}