RDR2: Training the Router

① a tree, no traversal→② score-based, both ways→③ LLM-guided instead→④ traversal learned→⑤ multimodal
the obstacle · standard RAG data gives a question and an answer, but no routing trajectory to imitate action curation · manufacture the trajectories from the question alone ① a question "who played wonder woman?" the answer is never used so any question set works ② build the state off-the-shelf retriever → top-k their originating documents → retrieval subtree s the same state the router sees at inference ③ a teacher writes the actions DeepSeek-v3 · sees (q, s) [EXP] 5: ===Television=== [ANS] 6,7: Cathy Lee Crosby… [REF] −1 one step's worth of actions, not a full trajectory ④ the training set ⟨ q , s , A ⟩ ASQA training questions 23,827 samples no human annotation anywhere supervised fine-tuning · plain next-token prediction ℒ = − log P(A | q, s) cross-entropy on the output tokens only Llama-3.1-8B-Instruct + LoRA · 3.5 epochs trained on single steps, never on whole trajectories the model learns "what to do in this state"; the iteration is left to inference the output format carries the node id "[ACTION] id: text_prefix" — every action is bound to a real node
17 / 32