AI - Tag - lilfry's library

High frequency interview: Bradley-Terry vs Plackett-Luce, what is the difference between reward modeling?

fry — Wed, 08 Apr 2026 14:20:00 +0800

In a word: BT is a special case of PL when the number of candidates is N=2. BT is lighter, more stable, and more suitable for existing RLHF/DPO; PL has more complete information, but the labeling and training costs are significantly higher.

Read YaRN: Long context expansion, the core is not to pull the window hard, but not to break the RoPE

fry — Tue, 07 Apr 2026 21:12:00 +0800

It would be a pity if we only regard YaRN as “another trick to pull the context from 4k to 64k/128k”.

What it really encounters is a more essential geometric problem than “window expansion”:

**The model is already accustomed to a position and phase system. Now that you forcefully increase the length, how can you make this system continue to work at a longer range while trying not to lose the positional resolution at close range? **

Once this question is clarified, YaRN’s design will appear very natural.

Read "Reasoning with Sampling": RL does not make the model smarter, it just redistributes reasoning capabilities

fry — Tue, 07 Apr 2026 20:55:00 +0800

The really dangerous thing about this paper is not that it proposes a new sampler, but that it asks a deeper interpretive question:

**The progress of reasoning model, is it that the model has learned new capabilities, or have we finally learned how to sample from old capabilities? **

If this question were asked correctly, much of the narrative about RL for reasoning would have to be rewritten.

Read OLMo 3: What really matters is not the new architecture, but the training signal

fry — Mon, 06 Apr 2026 19:00:00 +0800

When I read OLMo 3 recently, my biggest feeling was not that “the open source community has made a stronger model”, but that it made something clear that is often said very vaguely:

**What a large model will eventually look like is determined not by a few architectural fine-tunings, but by what training signals it repeatedly receives at each stage. **

This statement may sound naive, but if you really think about it, many questions about LLM will become clearer. Why do some models have good bases but cannot be pulled up after training? Why do some model context windows become longer, but they still don’t really take advantage of the long context? Why do some RL recipes seem advanced but have no obvious benefits in the end? The answer is often not “what has changed in the structure”, but “where does the gradient come from?”