Despite the fact that experimental neural scaling laws have substantially guided empirical progress in large-scale machine learning, no existing theory can quantitatively predict the exponents of these important laws for any modern LLM trained on natural languages. It is also unclear whether they can be beaten by training with other objectives. Matthieu Wyart will review the efforts of the collaboration in studying this phenomenon, including the first such theory in the case of data-limited scaling laws. Wyart will then introduce synthetic languages displaying a hidden hierarchy (a central notion in both linguistics and computer vision), and discuss how these languages are learnt by deep, but not shallow, nets. Scaling laws in these synthetic languages follow from the same principle as in natural languages when trained at the token level, but these languages can be learnt much more efficiently by machines predicting in an abstract space — justifying the success of joint embedding predictive architectures (JEPA) and other machines learning from their own latents. For more information please visit: https://www.simonsfoundation.org/events/simons-collaboration-on-the-physics-of-learning-and-neural-computation-annual-meeting-2026/