It would be a pity if we only regard YaRN as “another trick to pull the context from 4k to 64k/128k”.

What it really encounters is a more essential geometric problem than “window expansion”:

**The model is already accustomed to a position and phase system. Now that you forcefully increase the length, how can you make this system continue to work at a longer range while trying not to lose the positional resolution at close range? **

Once this question is clarified, YaRN’s design will appear very natural.

The really dangerous thing about this paper is not that it proposes a new sampler, but that it asks a deeper interpretive question:

**The progress of reasoning model, is it that the model has learned new capabilities, or have we finally learned how to sample from old capabilities? **

If this question were asked correctly, much of the narrative about RL for reasoning would have to be rewritten.

When I read OLMo 3 recently, my biggest feeling was not that “the open source community has made a stronger model”, but that it made something clear that is often said very vaguely:

**What a large model will eventually look like is determined not by a few architectural fine-tunings, but by what training signals it repeatedly receives at each stage. **

This statement may sound naive, but if you really think about it, many questions about LLM will become clearer. Why do some models have good bases but cannot be pulled up after training? Why do some model context windows become longer, but they still don’t really take advantage of the long context? Why do some RL recipes seem advanced but have no obvious benefits in the end? The answer is often not “what has changed in the structure”, but “where does the gradient come from?”

Deep learning optimization algorithms: SGD, RMSProp, AdaGrad, Adam detailed explanation

In the training process of deep learning, optimization algorithms play a crucial role. They determine how the model parameters are updated according to the gradient of the loss function, thus affecting the model’s convergence speed and final performance. This article will introduce in depth several of the most commonly used and basic optimization algorithms: SGD, RMSProp, AdaGrad and Adam, and analyze their formulas, advantages and disadvantages.

Question description

You are given an array of strings, and you are asked to combine the anagrams together. The list of results can be returned in any order.

Example 1:

Input: strs = [“eat”, “tea”, “tan”, “ate”, “nat”, “bat”]

Output: [[“bat”],[“nat”,“tan”],[“ate”,“eat”,“tea”]]

explain:

There are no strings in strs that can be rearranged to form “bat”. The strings “nat” and “tan” are anagrams because they can be rearranged to form each other. The strings “ate” , “eat” and “tea” are anagrams because they can be rearranged to form each other. Example 2:

Question description

Given the root node root of a binary tree, return all duplicate subtrees.

For duplicate subtrees of the same type, you only need to return the root node of any one of them.

If two trees have the same structure and the same node values, they are considered duplicates.

https://assets.leetcode.com/uploads/2020/08/16/e1.jpg

Input: root = [1,2,3,4,null,2,4,null,null,4] Output: [[2,4],[4]]

Problem-solving ideas

Use postorder traversal + hashing!

Idea:

1. Give each subtree an “ID card” (a string traversed in post-order)
2. Record the number of occurrences of each subtree (hash, counter)

**Why does this method work? **

1. String description uniquely determines the subtree structure For example, "2,4,#,#,#" can only correspond to one binary tree (root->left->right)