{
  "author": {
    "name": "Michael Frank Martin",
    "url": "https://www.symmetrybroken.com/about/"
  },
  "blyg": "0.4",
  "changelog": [
    {
      "at": "2026-01-03T16:05:25Z",
      "note": null,
      "version": 1
    },
    {
      "at": "2026-11-19T00:17:44Z",
      "note": "Corrected the positional-encoding labels, the attention formula and the credit for block-spin renormalization; removed the note on Bianconi and Connes",
      "version": 2
    }
  ],
  "content_hash": "sha256:02fd2a6105877321d080eb854810418096482d63f1be77db90bc6c3e48606bb0",
  "content_html": "\u003ch1 id=\"the-transformer-as-renormalization-group-flow\"\u003eThe Transformer as Renormalization Group Flow\u003c/h1\u003e\n\u003cp\u003eA recent essay and linked papers by Vishal Misra and colleagues, \u003ca href=\"https://medium.com/@vishalmisra/attention-is-bayesian-inference-578c25db4501\"\u003e\u0026ldquo;Attention is Bayesian Inference,\u0026rdquo;\u003c/a\u003e argues that the transformer attention mechanism implements a form of probabilistic inference — each layer asks questions and eliminates hypotheses until the posterior collapses onto the correct answer. The framework connects naturally to ideas I explored in \u003ca href=\"https://www.symmetrybroken.com/a-stationary-action-is-stable-information/\"\u003eA Stationary Action is Stable Information\u003c/a\u003e: systems find stable configurations where neighboring paths agree, where informational consensus forms.\u003csup id=\"fnref:1\"\u003e\u003ca href=\"https://www.symmetrybroken.com/transformer-as-renormalization-group-flow/#fn:1\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e1\u003c/a\u003e\u003c/sup\u003e\u003c/p\u003e\n\u003cp\u003eThe connection runs deeper than analogy. The mathematics of attention — softmax over dot products, weighted sums of values — maps precisely onto the partition functions and thermal expectations of statistical mechanics. When we stack these operations into deep networks, something remarkable emerges: the forward pass through a transformer implements \u003ca href=\"https://faculty.kfupm.edu.sa/phys/imnasser/Phase_transition/Maris_Kadanoff.pdf\"\u003ea Kadanoff-Wilson renormalization group flow\u003c/a\u003e, coarse-graining microscopic token representations into stable semantic attractors.\u003c/p\u003e\n\u003cp\u003eRenormalization group flow is \u003ca href=\"http://max2.physics.sunysb.edu/~rastelli/2019/WilsonKogut.pdf\"\u003ean advanced topic in non-equilibrium statistical mechanics\u003c/a\u003e, so the remainder of this essay may be of interest only to physicists. But the connection does provide some useful perspective for interpretability, alignment, and even scalability of the transformer architecture, which I will be exploring. Note that unlike standard block-spin RG, which decimates the lattice (reducing the number of sites), the transformer preserves the token count. The coarse-graining is of \u003cem\u003einformation\u003c/em\u003e, integrating out syntactic fluctuations to leave behind stable semantic operators.\u003c/p\u003e\n\u003cp\u003eTo make this more precise, let\u0026rsquo;s trace how a transformer generates \u0026ldquo;Paris\u0026rdquo; in response to \u0026ldquo;What is the capital of France?\u0026rdquo; The layer-by-layer account and the numbers in it are illustrative, not measured from a particular model.\u003c/p\u003e\n\u003ch2 id=\"the-setup-a-statistical-mechanical-system\"\u003eThe Setup: A Statistical Mechanical System\u003c/h2\u003e\n\u003cp\u003eThe prompt arrives as a string of approximately seven tokens, depending on tokenization: \u003ccode\u003e[What] [is] [the] [capital] [of] [France] [?]\u003c/code\u003e. We can view this as a one-dimensional lattice where each site carries a high-dimensional \u0026ldquo;spin\u0026rdquo; variable — the token embedding.\u003c/p\u003e\n\u003cp\u003eFrom the Bayesian perspective, the model begins with a prior over all possible next tokens. The vocabulary contains roughly 50,000 possibilities, and at this stage, none has been eliminated.\u003c/p\u003e\n\u003cp\u003eThe statistical mechanics framing treats each token embedding \u003cspan class=\"katex\"\u003e\u003cmath xmlns=\"http://www.w3.org/1998/Math/MathML\"\u003e\u003csemantics\u003e\u003cmrow\u003e\u003cmsubsup\u003e\u003cmi\u003ex\u003c/mi\u003e\u003cmi\u003ei\u003c/mi\u003e\u003cmrow\u003e\u003cmo stretchy=\"false\"\u003e(\u003c/mo\u003e\u003cmn\u003e0\u003c/mn\u003e\u003cmo stretchy=\"false\"\u003e)\u003c/mo\u003e\u003c/mrow\u003e\u003c/msubsup\u003e\u003cmo\u003e∈\u003c/mo\u003e\u003cmsup\u003e\u003cmi mathvariant=\"double-struck\"\u003eR\u003c/mi\u003e\u003cmi\u003ed\u003c/mi\u003e\u003c/msup\u003e\u003c/mrow\u003e\u003cannotation encoding=\"application/x-tex\"\u003ex_i^{(0)} \\in \\mathbb{R}^d\u003c/annotation\u003e\u003c/semantics\u003e\u003c/math\u003e\u003c/span\u003e as a spin configuration at lattice site \u003cspan class=\"katex\"\u003e\u003cmath xmlns=\"http://www.w3.org/1998/Math/MathML\"\u003e\u003csemantics\u003e\u003cmrow\u003e\u003cmi\u003ei\u003c/mi\u003e\u003c/mrow\u003e\u003cannotation encoding=\"application/x-tex\"\u003ei\u003c/annotation\u003e\u003c/semantics\u003e\u003c/math\u003e\u003c/span\u003e. The embedding dimension \u003cspan class=\"katex\"\u003e\u003cmath xmlns=\"http://www.w3.org/1998/Math/MathML\"\u003e\u003csemantics\u003e\u003cmrow\u003e\u003cmi\u003ed\u003c/mi\u003e\u003c/mrow\u003e\u003cannotation encoding=\"application/x-tex\"\u003ed\u003c/annotation\u003e\u003c/semantics\u003e\u003c/math\u003e\u003c/span\u003e counts the internal degrees of freedom per site. We\u0026rsquo;re working at the ultraviolet scale — the full microscopic description, before any coarse-graining.\u003c/p\u003e\n\u003ch2 id=\"positional-encoding-breaks-a-symmetry\"\u003ePositional Encoding Breaks a Symmetry\u003c/h2\u003e\n\u003cp\u003eBefore attention operates, positional encodings \u003cspan class=\"katex\"\u003e\u003cmath xmlns=\"http://www.w3.org/1998/Math/MathML\"\u003e\u003csemantics\u003e\u003cmrow\u003e\u003cmsub\u003e\u003cmi\u003ep\u003c/mi\u003e\u003cmi\u003ei\u003c/mi\u003e\u003c/msub\u003e\u003c/mrow\u003e\u003cannotation encoding=\"application/x-tex\"\u003ep_i\u003c/annotation\u003e\u003c/semantics\u003e\u003c/math\u003e\u003c/span\u003e modify each embedding: \u003cspan class=\"katex\"\u003e\u003cmath xmlns=\"http://www.w3.org/1998/Math/MathML\"\u003e\u003csemantics\u003e\u003cmrow\u003e\u003cmsubsup\u003e\u003cmi\u003ex\u003c/mi\u003e\u003cmi\u003ei\u003c/mi\u003e\u003cmrow\u003e\u003cmo stretchy=\"false\"\u003e(\u003c/mo\u003e\u003cmn\u003e0\u003c/mn\u003e\u003cmo stretchy=\"false\"\u003e)\u003c/mo\u003e\u003c/mrow\u003e\u003c/msubsup\u003e\u003cmo\u003e←\u003c/mo\u003e\u003cmsubsup\u003e\u003cmi\u003ex\u003c/mi\u003e\u003cmi\u003ei\u003c/mi\u003e\u003cmrow\u003e\u003cmo stretchy=\"false\"\u003e(\u003c/mo\u003e\u003cmn\u003e0\u003c/mn\u003e\u003cmo stretchy=\"false\"\u003e)\u003c/mo\u003e\u003c/mrow\u003e\u003c/msubsup\u003e\u003cmo\u003e+\u003c/mo\u003e\u003cmsub\u003e\u003cmi\u003ep\u003c/mi\u003e\u003cmi\u003ei\u003c/mi\u003e\u003c/msub\u003e\u003c/mrow\u003e\u003cannotation encoding=\"application/x-tex\"\u003ex_i^{(0)} \\leftarrow x_i^{(0)} + p_i\u003c/annotation\u003e\u003c/semantics\u003e\u003c/math\u003e\u003c/span\u003e. This breaks translational invariance on the lattice, introducing explicit spatial structure. Without positional encoding, the system would be permutation-invariant — every spin interacting equally with every other, like a Sherrington-Kirkpatrick (SK) spin glass model.\u003c/p\u003e\n\u003cp\u003eThe original transformer used sinusoidal encodings:\n\u003c/p\u003e\n\u003cspan class=\"katex\"\u003e\u003cmath xmlns=\"http://www.w3.org/1998/Math/MathML\" display=\"block\"\u003e\u003csemantics\u003e\u003cmrow\u003e\u003cmsub\u003e\u003cmi\u003ep\u003c/mi\u003e\u003cmrow\u003e\u003cmi\u003ei\u003c/mi\u003e\u003cmo separator=\"true\"\u003e,\u003c/mo\u003e\u003cmn\u003e2\u003c/mn\u003e\u003cmi\u003ek\u003c/mi\u003e\u003c/mrow\u003e\u003c/msub\u003e\u003cmo\u003e=\u003c/mo\u003e\u003cmi\u003esin\u003c/mi\u003e\u003cmo\u003e⁡\u003c/mo\u003e\u003cmo stretchy=\"false\"\u003e(\u003c/mo\u003e\u003cmi\u003ei\u003c/mi\u003e\u003cmi mathvariant=\"normal\"\u003e/\u003c/mi\u003e\u003cmsup\u003e\u003cmn\u003e10000\u003c/mn\u003e\u003cmrow\u003e\u003cmn\u003e2\u003c/mn\u003e\u003cmi\u003ek\u003c/mi\u003e\u003cmi mathvariant=\"normal\"\u003e/\u003c/mi\u003e\u003cmi\u003ed\u003c/mi\u003e\u003c/mrow\u003e\u003c/msup\u003e\u003cmo stretchy=\"false\"\u003e)\u003c/mo\u003e\u003cmo separator=\"true\"\u003e,\u003c/mo\u003e\u003cmspace width=\"1em\"/\u003e\u003cmsub\u003e\u003cmi\u003ep\u003c/mi\u003e\u003cmrow\u003e\u003cmi\u003ei\u003c/mi\u003e\u003cmo separator=\"true\"\u003e,\u003c/mo\u003e\u003cmn\u003e2\u003c/mn\u003e\u003cmi\u003ek\u003c/mi\u003e\u003cmo\u003e+\u003c/mo\u003e\u003cmn\u003e1\u003c/mn\u003e\u003c/mrow\u003e\u003c/msub\u003e\u003cmo\u003e=\u003c/mo\u003e\u003cmi\u003ecos\u003c/mi\u003e\u003cmo\u003e⁡\u003c/mo\u003e\u003cmo stretchy=\"false\"\u003e(\u003c/mo\u003e\u003cmi\u003ei\u003c/mi\u003e\u003cmi mathvariant=\"normal\"\u003e/\u003c/mi\u003e\u003cmsup\u003e\u003cmn\u003e10000\u003c/mn\u003e\u003cmrow\u003e\u003cmn\u003e2\u003c/mn\u003e\u003cmi\u003ek\u003c/mi\u003e\u003cmi mathvariant=\"normal\"\u003e/\u003c/mi\u003e\u003cmi\u003ed\u003c/mi\u003e\u003c/mrow\u003e\u003c/msup\u003e\u003cmo stretchy=\"false\"\u003e)\u003c/mo\u003e\u003c/mrow\u003e\u003cannotation encoding=\"application/x-tex\"\u003ep_{i,2k} = \\sin(i/10000^{2k/d}), \\quad p_{i,2k+1} = \\cos(i/10000^{2k/d})\u003c/annotation\u003e\u003c/semantics\u003e\u003c/math\u003e\u003c/span\u003e\u003cp\u003e\nThese Fourier modes encode position at multiple frequency scales. High-frequency components (small \u003cspan class=\"katex\"\u003e\u003cmath xmlns=\"http://www.w3.org/1998/Math/MathML\"\u003e\u003csemantics\u003e\u003cmrow\u003e\u003cmi\u003ek\u003c/mi\u003e\u003c/mrow\u003e\u003cannotation encoding=\"application/x-tex\"\u003ek\u003c/annotation\u003e\u003c/semantics\u003e\u003c/math\u003e\u003c/span\u003e) capture local structure; low-frequency components (large \u003cspan class=\"katex\"\u003e\u003cmath xmlns=\"http://www.w3.org/1998/Math/MathML\"\u003e\u003csemantics\u003e\u003cmrow\u003e\u003cmi\u003ek\u003c/mi\u003e\u003c/mrow\u003e\u003cannotation encoding=\"application/x-tex\"\u003ek\u003c/annotation\u003e\u003c/semantics\u003e\u003c/math\u003e\u003c/span\u003e) capture global ordering.\u003c/p\u003e\n\u003cp\u003eIn renormalization group terminology, positional encoding introduces \u003cem\u003erelevant operators\u003c/em\u003e — perturbations that grow under RG flow and dominate at long distances. Position matters more as we zoom out, which explains why low-frequency modes capture long-range dependencies.\u003c/p\u003e\n\u003ch2 id=\"attention-as-partition-function\"\u003eAttention as Partition Function\u003c/h2\u003e\n\u003cp\u003eThe attention mechanism implements the core statistical mechanical operation. For each token \u003cspan class=\"katex\"\u003e\u003cmath xmlns=\"http://www.w3.org/1998/Math/MathML\"\u003e\u003csemantics\u003e\u003cmrow\u003e\u003cmi\u003ei\u003c/mi\u003e\u003c/mrow\u003e\u003cannotation encoding=\"application/x-tex\"\u003ei\u003c/annotation\u003e\u003c/semantics\u003e\u003c/math\u003e\u003c/span\u003e, the transformer computes queries, keys, and values:\n\u003c/p\u003e\n\u003cspan class=\"katex\"\u003e\u003cmath xmlns=\"http://www.w3.org/1998/Math/MathML\" display=\"block\"\u003e\u003csemantics\u003e\u003cmrow\u003e\u003cmsub\u003e\u003cmi\u003eq\u003c/mi\u003e\u003cmi\u003ei\u003c/mi\u003e\u003c/msub\u003e\u003cmo\u003e=\u003c/mo\u003e\u003cmsub\u003e\u003cmi\u003eW\u003c/mi\u003e\u003cmi\u003eQ\u003c/mi\u003e\u003c/msub\u003e\u003cmsubsup\u003e\u003cmi\u003ex\u003c/mi\u003e\u003cmi\u003ei\u003c/mi\u003e\u003cmrow\u003e\u003cmo stretchy=\"false\"\u003e(\u003c/mo\u003e\u003cmn\u003e0\u003c/mn\u003e\u003cmo stretchy=\"false\"\u003e)\u003c/mo\u003e\u003c/mrow\u003e\u003c/msubsup\u003e\u003cmo separator=\"true\"\u003e,\u003c/mo\u003e\u003cmspace width=\"1em\"/\u003e\u003cmsub\u003e\u003cmi\u003ek\u003c/mi\u003e\u003cmi\u003ej\u003c/mi\u003e\u003c/msub\u003e\u003cmo\u003e=\u003c/mo\u003e\u003cmsub\u003e\u003cmi\u003eW\u003c/mi\u003e\u003cmi\u003eK\u003c/mi\u003e\u003c/msub\u003e\u003cmsubsup\u003e\u003cmi\u003ex\u003c/mi\u003e\u003cmi\u003ej\u003c/mi\u003e\u003cmrow\u003e\u003cmo stretchy=\"false\"\u003e(\u003c/mo\u003e\u003cmn\u003e0\u003c/mn\u003e\u003cmo stretchy=\"false\"\u003e)\u003c/mo\u003e\u003c/mrow\u003e\u003c/msubsup\u003e\u003cmo separator=\"true\"\u003e,\u003c/mo\u003e\u003cmspace width=\"1em\"/\u003e\u003cmsub\u003e\u003cmi\u003ev\u003c/mi\u003e\u003cmi\u003ej\u003c/mi\u003e\u003c/msub\u003e\u003cmo\u003e=\u003c/mo\u003e\u003cmsub\u003e\u003cmi\u003eW\u003c/mi\u003e\u003cmi\u003eV\u003c/mi\u003e\u003c/msub\u003e\u003cmsubsup\u003e\u003cmi\u003ex\u003c/mi\u003e\u003cmi\u003ej\u003c/mi\u003e\u003cmrow\u003e\u003cmo stretchy=\"false\"\u003e(\u003c/mo\u003e\u003cmn\u003e0\u003c/mn\u003e\u003cmo stretchy=\"false\"\u003e)\u003c/mo\u003e\u003c/mrow\u003e\u003c/msubsup\u003e\u003c/mrow\u003e\u003cannotation encoding=\"application/x-tex\"\u003eq_i = W_Q x_i^{(0)}, \\quad k_j = W_K x_j^{(0)}, \\quad v_j = W_V x_j^{(0)}\u003c/annotation\u003e\u003c/semantics\u003e\u003c/math\u003e\u003c/span\u003e\u003cp\u003e\nAttention scores measure the compatibility between query \u003cspan class=\"katex\"\u003e\u003cmath xmlns=\"http://www.w3.org/1998/Math/MathML\"\u003e\u003csemantics\u003e\u003cmrow\u003e\u003cmi\u003ei\u003c/mi\u003e\u003c/mrow\u003e\u003cannotation encoding=\"application/x-tex\"\u003ei\u003c/annotation\u003e\u003c/semantics\u003e\u003c/math\u003e\u003c/span\u003e and key \u003cspan class=\"katex\"\u003e\u003cmath xmlns=\"http://www.w3.org/1998/Math/MathML\"\u003e\u003csemantics\u003e\u003cmrow\u003e\u003cmi\u003ej\u003c/mi\u003e\u003c/mrow\u003e\u003cannotation encoding=\"application/x-tex\"\u003ej\u003c/annotation\u003e\u003c/semantics\u003e\u003c/math\u003e\u003c/span\u003e:\n\u003c/p\u003e\n\u003cspan class=\"katex\"\u003e\u003cmath xmlns=\"http://www.w3.org/1998/Math/MathML\" display=\"block\"\u003e\u003csemantics\u003e\u003cmrow\u003e\u003cmsub\u003e\u003cmi\u003es\u003c/mi\u003e\u003cmrow\u003e\u003cmi\u003ei\u003c/mi\u003e\u003cmi\u003ej\u003c/mi\u003e\u003c/mrow\u003e\u003c/msub\u003e\u003cmo\u003e=\u003c/mo\u003e\u003cmfrac\u003e\u003cmrow\u003e\u003cmsub\u003e\u003cmi\u003eq\u003c/mi\u003e\u003cmi\u003ei\u003c/mi\u003e\u003c/msub\u003e\u003cmo\u003e⋅\u003c/mo\u003e\u003cmsub\u003e\u003cmi\u003ek\u003c/mi\u003e\u003cmi\u003ej\u003c/mi\u003e\u003c/msub\u003e\u003c/mrow\u003e\u003cmsqrt\u003e\u003cmi\u003ed\u003c/mi\u003e\u003c/msqrt\u003e\u003c/mfrac\u003e\u003c/mrow\u003e\u003cannotation encoding=\"application/x-tex\"\u003es_{ij} = \\frac{q_i \\cdot k_j}{\\sqrt{d}}\u003c/annotation\u003e\u003c/semantics\u003e\u003c/math\u003e\u003c/span\u003e\u003cp\u003e\nThe softmax normalizes these scores into weights:\n\u003c/p\u003e\n\u003cspan class=\"katex\"\u003e\u003cmath xmlns=\"http://www.w3.org/1998/Math/MathML\" display=\"block\"\u003e\u003csemantics\u003e\u003cmrow\u003e\u003cmsub\u003e\u003cmi\u003eα\u003c/mi\u003e\u003cmrow\u003e\u003cmi\u003ei\u003c/mi\u003e\u003cmi\u003ej\u003c/mi\u003e\u003c/mrow\u003e\u003c/msub\u003e\u003cmo\u003e=\u003c/mo\u003e\u003cmfrac\u003e\u003cmrow\u003e\u003cmi\u003eexp\u003c/mi\u003e\u003cmo\u003e⁡\u003c/mo\u003e\u003cmo stretchy=\"false\"\u003e(\u003c/mo\u003e\u003cmsub\u003e\u003cmi\u003es\u003c/mi\u003e\u003cmrow\u003e\u003cmi\u003ei\u003c/mi\u003e\u003cmi\u003ej\u003c/mi\u003e\u003c/mrow\u003e\u003c/msub\u003e\u003cmo stretchy=\"false\"\u003e)\u003c/mo\u003e\u003c/mrow\u003e\u003cmrow\u003e\u003cmunder\u003e\u003cmo\u003e∑\u003c/mo\u003e\u003cmi\u003ek\u003c/mi\u003e\u003c/munder\u003e\u003cmi\u003eexp\u003c/mi\u003e\u003cmo\u003e⁡\u003c/mo\u003e\u003cmo stretchy=\"false\"\u003e(\u003c/mo\u003e\u003cmsub\u003e\u003cmi\u003es\u003c/mi\u003e\u003cmrow\u003e\u003cmi\u003ei\u003c/mi\u003e\u003cmi\u003ek\u003c/mi\u003e\u003c/mrow\u003e\u003c/msub\u003e\u003cmo stretchy=\"false\"\u003e)\u003c/mo\u003e\u003c/mrow\u003e\u003c/mfrac\u003e\u003c/mrow\u003e\u003cannotation encoding=\"application/x-tex\"\u003e\\alpha_{ij} = \\frac{\\exp(s_{ij})}{\\sum_k \\exp(s_{ik})}\u003c/annotation\u003e\u003c/semantics\u003e\u003c/math\u003e\u003c/span\u003e\u003cp\u003e\nFinally, the output aggregates values according to these weights:\n\u003c/p\u003e\n\u003cspan class=\"katex\"\u003e\u003cmath xmlns=\"http://www.w3.org/1998/Math/MathML\" display=\"block\"\u003e\u003csemantics\u003e\u003cmrow\u003e\u003cmsub\u003e\u003cmi\u003ez\u003c/mi\u003e\u003cmi\u003ei\u003c/mi\u003e\u003c/msub\u003e\u003cmo\u003e=\u003c/mo\u003e\u003cmunder\u003e\u003cmo\u003e∑\u003c/mo\u003e\u003cmi\u003ej\u003c/mi\u003e\u003c/munder\u003e\u003cmsub\u003e\u003cmi\u003eα\u003c/mi\u003e\u003cmrow\u003e\u003cmi\u003ei\u003c/mi\u003e\u003cmi\u003ej\u003c/mi\u003e\u003c/mrow\u003e\u003c/msub\u003e\u003cmsub\u003e\u003cmi\u003ev\u003c/mi\u003e\u003cmi\u003ej\u003c/mi\u003e\u003c/msub\u003e\u003c/mrow\u003e\u003cannotation encoding=\"application/x-tex\"\u003ez_i = \\sum_j \\alpha_{ij} v_j\u003c/annotation\u003e\u003c/semantics\u003e\u003c/math\u003e\u003c/span\u003e\u003cp\u003e\nThe mapping to statistical mechanics is exact. The attention score \u003cspan class=\"katex\"\u003e\u003cmath xmlns=\"http://www.w3.org/1998/Math/MathML\"\u003e\u003csemantics\u003e\u003cmrow\u003e\u003cmsub\u003e\u003cmi\u003es\u003c/mi\u003e\u003cmrow\u003e\u003cmi\u003ei\u003c/mi\u003e\u003cmi\u003ej\u003c/mi\u003e\u003c/mrow\u003e\u003c/msub\u003e\u003c/mrow\u003e\u003cannotation encoding=\"application/x-tex\"\u003es_{ij}\u003c/annotation\u003e\u003c/semantics\u003e\u003c/math\u003e\u003c/span\u003e plays the role of negative energy \u003cspan class=\"katex\"\u003e\u003cmath xmlns=\"http://www.w3.org/1998/Math/MathML\"\u003e\u003csemantics\u003e\u003cmrow\u003e\u003cmo\u003e−\u003c/mo\u003e\u003cmi\u003eβ\u003c/mi\u003e\u003cmsub\u003e\u003cmi\u003eE\u003c/mi\u003e\u003cmrow\u003e\u003cmi\u003ei\u003c/mi\u003e\u003cmi\u003ej\u003c/mi\u003e\u003c/mrow\u003e\u003c/msub\u003e\u003c/mrow\u003e\u003cannotation encoding=\"application/x-tex\"\u003e-\\beta E_{ij}\u003c/annotation\u003e\u003c/semantics\u003e\u003c/math\u003e\u003c/span\u003e, measuring interaction strength between sites. The denominator \u003cspan class=\"katex\"\u003e\u003cmath xmlns=\"http://www.w3.org/1998/Math/MathML\"\u003e\u003csemantics\u003e\u003cmrow\u003e\u003cmsqrt\u003e\u003cmi\u003ed\u003c/mi\u003e\u003c/msqrt\u003e\u003c/mrow\u003e\u003cannotation encoding=\"application/x-tex\"\u003e\\sqrt{d}\u003c/annotation\u003e\u003c/semantics\u003e\u003c/math\u003e\u003c/span\u003e regulates the temperature. Without it, high-dimensional dot products would diverge, driving the softmax into a frozen \u0026lsquo;argmax\u0026rsquo; state. The scaling ensures the system remains at a finite temperature, maintaining a soft, probabilistic ensemble. The normalization factor \u003cspan class=\"katex\"\u003e\u003cmath xmlns=\"http://www.w3.org/1998/Math/MathML\"\u003e\u003csemantics\u003e\u003cmrow\u003e\u003cmsub\u003e\u003cmi\u003eZ\u003c/mi\u003e\u003cmi\u003ei\u003c/mi\u003e\u003c/msub\u003e\u003cmo\u003e=\u003c/mo\u003e\u003cmsub\u003e\u003cmo\u003e∑\u003c/mo\u003e\u003cmi\u003ej\u003c/mi\u003e\u003c/msub\u003e\u003cmi\u003eexp\u003c/mi\u003e\u003cmo\u003e⁡\u003c/mo\u003e\u003cmo stretchy=\"false\"\u003e(\u003c/mo\u003e\u003cmsub\u003e\u003cmi\u003es\u003c/mi\u003e\u003cmrow\u003e\u003cmi\u003ei\u003c/mi\u003e\u003cmi\u003ej\u003c/mi\u003e\u003c/mrow\u003e\u003c/msub\u003e\u003cmo stretchy=\"false\"\u003e)\u003c/mo\u003e\u003c/mrow\u003e\u003cannotation encoding=\"application/x-tex\"\u003eZ_i = \\sum_j \\exp(s_{ij})\u003c/annotation\u003e\u003c/semantics\u003e\u003c/math\u003e\u003c/span\u003e is precisely a partition function, and the weights \u003cspan class=\"katex\"\u003e\u003cmath xmlns=\"http://www.w3.org/1998/Math/MathML\"\u003e\u003csemantics\u003e\u003cmrow\u003e\u003cmsub\u003e\u003cmi\u003eα\u003c/mi\u003e\u003cmrow\u003e\u003cmi\u003ei\u003c/mi\u003e\u003cmi\u003ej\u003c/mi\u003e\u003c/mrow\u003e\u003c/msub\u003e\u003c/mrow\u003e\u003cannotation encoding=\"application/x-tex\"\u003e\\alpha_{ij}\u003c/annotation\u003e\u003c/semantics\u003e\u003c/math\u003e\u003c/span\u003e are Boltzmann probabilities.\u003c/p\u003e\n\u003cp\u003eThe log-partition function gives the free energy at token \u003cspan class=\"katex\"\u003e\u003cmath xmlns=\"http://www.w3.org/1998/Math/MathML\"\u003e\u003csemantics\u003e\u003cmrow\u003e\u003cmi\u003ei\u003c/mi\u003e\u003c/mrow\u003e\u003cannotation encoding=\"application/x-tex\"\u003ei\u003c/annotation\u003e\u003c/semantics\u003e\u003c/math\u003e\u003c/span\u003e:\n\u003c/p\u003e\n\u003cspan class=\"katex\"\u003e\u003cmath xmlns=\"http://www.w3.org/1998/Math/MathML\" display=\"block\"\u003e\u003csemantics\u003e\u003cmrow\u003e\u003cmsub\u003e\u003cmi\u003eF\u003c/mi\u003e\u003cmi\u003ei\u003c/mi\u003e\u003c/msub\u003e\u003cmo\u003e=\u003c/mo\u003e\u003cmo\u003e−\u003c/mo\u003e\u003cmsqrt\u003e\u003cmi\u003ed\u003c/mi\u003e\u003c/msqrt\u003e\u003cmo\u003e⋅\u003c/mo\u003e\u003cmi\u003elog\u003c/mi\u003e\u003cmo\u003e⁡\u003c/mo\u003e\u003cmunder\u003e\u003cmo\u003e∑\u003c/mo\u003e\u003cmi\u003ej\u003c/mi\u003e\u003c/munder\u003e\u003cmi\u003eexp\u003c/mi\u003e\u003cmo\u003e⁡\u003c/mo\u003e\u003cmrow\u003e\u003cmo fence=\"true\"\u003e(\u003c/mo\u003e\u003cmfrac\u003e\u003cmrow\u003e\u003cmsub\u003e\u003cmi\u003eq\u003c/mi\u003e\u003cmi\u003ei\u003c/mi\u003e\u003c/msub\u003e\u003cmo\u003e⋅\u003c/mo\u003e\u003cmsub\u003e\u003cmi\u003ek\u003c/mi\u003e\u003cmi\u003ej\u003c/mi\u003e\u003c/msub\u003e\u003c/mrow\u003e\u003cmsqrt\u003e\u003cmi\u003ed\u003c/mi\u003e\u003c/msqrt\u003e\u003c/mfrac\u003e\u003cmo fence=\"true\"\u003e)\u003c/mo\u003e\u003c/mrow\u003e\u003c/mrow\u003e\u003cannotation encoding=\"application/x-tex\"\u003eF_i = -\\sqrt{d} \\cdot \\log \\sum_j \\exp\\left(\\frac{q_i \\cdot k_j}{\\sqrt{d}}\\right)\u003c/annotation\u003e\u003c/semantics\u003e\u003c/math\u003e\u003c/span\u003e\u003cp\u003e\nThis log-sum-exp \u003ca href=\"https://arxiv.org/abs/2008.02217\"\u003eappears in the literature on Modern Hopfield Networks (or Dense Associative Memories) as the continuous relaxation\u003c/a\u003e. (See equation 1 at 3.) The KL divergence between the attention distribution \u003cspan class=\"katex\"\u003e\u003cmath xmlns=\"http://www.w3.org/1998/Math/MathML\"\u003e\u003csemantics\u003e\u003cmrow\u003e\u003cmsub\u003e\u003cmi\u003eα\u003c/mi\u003e\u003cmrow\u003e\u003cmi\u003ei\u003c/mi\u003e\u003cmi\u003ej\u003c/mi\u003e\u003c/mrow\u003e\u003c/msub\u003e\u003c/mrow\u003e\u003cannotation encoding=\"application/x-tex\"\u003e\\alpha_{ij}\u003c/annotation\u003e\u003c/semantics\u003e\u003c/math\u003e\u003c/span\u003e and the uniform distribution over tokens measures how much information this attention head extracts — how far the system has moved from maximum ignorance. Each attention computation produces relative entropy; each layer pays a \u003ca href=\"https://www.symmetrybroken.com/the-synchronization-tax/\"\u003esynchronization tax\u003c/a\u003e to establish which tokens matter to which. Each attention computation \u003cem\u003eis\u003c/em\u003e a relative entropy — the divergence between \u0026ldquo;attend to everything equally\u0026rdquo; and \u0026ldquo;attend to what matters.\u0026rdquo;\u003c/p\u003e\n\u003cp\u003eThe output \u003cspan class=\"katex\"\u003e\u003cmath xmlns=\"http://www.w3.org/1998/Math/MathML\"\u003e\u003csemantics\u003e\u003cmrow\u003e\u003cmsub\u003e\u003cmi\u003ez\u003c/mi\u003e\u003cmi\u003ei\u003c/mi\u003e\u003c/msub\u003e\u003cmo\u003e=\u003c/mo\u003e\u003cmsub\u003e\u003cmo\u003e∑\u003c/mo\u003e\u003cmi\u003ej\u003c/mi\u003e\u003c/msub\u003e\u003cmsub\u003e\u003cmi\u003eα\u003c/mi\u003e\u003cmrow\u003e\u003cmi\u003ei\u003c/mi\u003e\u003cmi\u003ej\u003c/mi\u003e\u003c/mrow\u003e\u003c/msub\u003e\u003cmsub\u003e\u003cmi\u003ev\u003c/mi\u003e\u003cmi\u003ej\u003c/mi\u003e\u003c/msub\u003e\u003c/mrow\u003e\u003cannotation encoding=\"application/x-tex\"\u003ez_i = \\sum_j \\alpha_{ij} v_j\u003c/annotation\u003e\u003c/semantics\u003e\u003c/math\u003e\u003c/span\u003e computes a thermal expectation — an observable averaged over the Boltzmann ensemble. In Misra\u0026rsquo;s Bayesian framing, the query encodes \u0026ldquo;what information is token \u003cspan class=\"katex\"\u003e\u003cmath xmlns=\"http://www.w3.org/1998/Math/MathML\"\u003e\u003csemantics\u003e\u003cmrow\u003e\u003cmi\u003ei\u003c/mi\u003e\u003c/mrow\u003e\u003cannotation encoding=\"application/x-tex\"\u003ei\u003c/annotation\u003e\u003c/semantics\u003e\u003c/math\u003e\u003c/span\u003e seeking?\u0026rdquo; while keys advertise \u0026ldquo;what information does token \u003cspan class=\"katex\"\u003e\u003cmath xmlns=\"http://www.w3.org/1998/Math/MathML\"\u003e\u003csemantics\u003e\u003cmrow\u003e\u003cmi\u003ej\u003c/mi\u003e\u003c/mrow\u003e\u003cannotation encoding=\"application/x-tex\"\u003ej\u003c/annotation\u003e\u003c/semantics\u003e\u003c/math\u003e\u003c/span\u003e offer?\u0026rdquo; The dot product measures evidential compatibility. For our prompt, the query from position 7 asks about capital cities; the key from \u0026ldquo;France\u0026rdquo; advertises European nationhood; their large dot product reflects compatibility.\u003c/p\u003e\n\u003ch2 id=\"the-first-rg-step-coarse-graining\"\u003eThe First RG Step: Coarse-Graining\u003c/h2\u003e\n\u003cp\u003eAttention implements the first renormalization group transformation. Before attention, each site carries an independent spin \u003cspan class=\"katex\"\u003e\u003cmath xmlns=\"http://www.w3.org/1998/Math/MathML\"\u003e\u003csemantics\u003e\u003cmrow\u003e\u003cmsubsup\u003e\u003cmi\u003ex\u003c/mi\u003e\u003cmi\u003ej\u003c/mi\u003e\u003cmrow\u003e\u003cmo stretchy=\"false\"\u003e(\u003c/mo\u003e\u003cmn\u003e0\u003c/mn\u003e\u003cmo stretchy=\"false\"\u003e)\u003c/mo\u003e\u003c/mrow\u003e\u003c/msubsup\u003e\u003c/mrow\u003e\u003cannotation encoding=\"application/x-tex\"\u003ex_j^{(0)}\u003c/annotation\u003e\u003c/semantics\u003e\u003c/math\u003e\u003c/span\u003e. After attention, each site carries an effective spin \u003cspan class=\"katex\"\u003e\u003cmath xmlns=\"http://www.w3.org/1998/Math/MathML\"\u003e\u003csemantics\u003e\u003cmrow\u003e\u003cmsub\u003e\u003cmi\u003ez\u003c/mi\u003e\u003cmi\u003ei\u003c/mi\u003e\u003c/msub\u003e\u003cmo\u003e=\u003c/mo\u003e\u003cmsub\u003e\u003cmo\u003e∑\u003c/mo\u003e\u003cmi\u003ej\u003c/mi\u003e\u003c/msub\u003e\u003cmsub\u003e\u003cmi\u003eα\u003c/mi\u003e\u003cmrow\u003e\u003cmi\u003ei\u003c/mi\u003e\u003cmi\u003ej\u003c/mi\u003e\u003c/mrow\u003e\u003c/msub\u003e\u003cmsub\u003e\u003cmi\u003ev\u003c/mi\u003e\u003cmi\u003ej\u003c/mi\u003e\u003c/msub\u003e\u003c/mrow\u003e\u003cannotation encoding=\"application/x-tex\"\u003ez_i = \\sum_j \\alpha_{ij} v_j\u003c/annotation\u003e\u003c/semantics\u003e\u003c/math\u003e\u003c/span\u003e that incorporates information from the entire lattice.\u003c/p\u003e\n\u003cp\u003eIn Kadanoff\u0026rsquo;s block-spin renormalization, which Wilson made quantitative, one averages spins within fixed spatial blocks. Attention performs something more sophisticated: an \u003cem\u003eadaptive\u003c/em\u003e blocking where the attention pattern defines which sites contribute to each effective spin. High attention weights pull sites together; low weights decouple them.\u003c/p\u003e\n\u003cp\u003eThe operation integrates out individual token identities and produces a coarse-grained description. The original microscopic structure — raw embeddings — gives way to an effective theory encoding correlations. New couplings emerge from this integration, captured in the transformed representations.\u003c/p\u003e\n\u003ch2 id=\"feedforward-networks-local-field-renormalization\"\u003eFeedforward Networks: Local Field Renormalization\u003c/h2\u003e\n\u003cp\u003eAfter attention, each token passes through a feedforward network:\n\u003c/p\u003e\n\u003cspan class=\"katex\"\u003e\u003cmath xmlns=\"http://www.w3.org/1998/Math/MathML\" display=\"block\"\u003e\u003csemantics\u003e\u003cmrow\u003e\u003cmtext\u003eFFN\u003c/mtext\u003e\u003cmo stretchy=\"false\"\u003e(\u003c/mo\u003e\u003cmi\u003ex\u003c/mi\u003e\u003cmo stretchy=\"false\"\u003e)\u003c/mo\u003e\u003cmo\u003e=\u003c/mo\u003e\u003cmsub\u003e\u003cmi\u003eW\u003c/mi\u003e\u003cmn\u003e2\u003c/mn\u003e\u003c/msub\u003e\u003cmo\u003e⋅\u003c/mo\u003e\u003cmtext\u003eReLU\u003c/mtext\u003e\u003cmo stretchy=\"false\"\u003e(\u003c/mo\u003e\u003cmsub\u003e\u003cmi\u003eW\u003c/mi\u003e\u003cmn\u003e1\u003c/mn\u003e\u003c/msub\u003e\u003cmi\u003ex\u003c/mi\u003e\u003cmo\u003e+\u003c/mo\u003e\u003cmsub\u003e\u003cmi\u003eb\u003c/mi\u003e\u003cmn\u003e1\u003c/mn\u003e\u003c/msub\u003e\u003cmo stretchy=\"false\"\u003e)\u003c/mo\u003e\u003cmo\u003e+\u003c/mo\u003e\u003cmsub\u003e\u003cmi\u003eb\u003c/mi\u003e\u003cmn\u003e2\u003c/mn\u003e\u003c/msub\u003e\u003c/mrow\u003e\u003cannotation encoding=\"application/x-tex\"\u003e\\text{FFN}(x) = W_2 \\cdot \\text{ReLU}(W_1 x + b_1) + b_2\u003c/annotation\u003e\u003c/semantics\u003e\u003c/math\u003e\u003c/span\u003e\u003cp\u003e\nwith residual connections and layer normalization. Unlike attention, which mixes information across tokens, the feedforward network transforms each site independently—a local, site-wise operation.\u003c/p\u003e\n\u003cp\u003eThe ReLU nonlinearity zeros negative activations, creating sparsity. This decimates degrees of freedom: only some modes survive, and irrelevant fluctuations are integrated out. The expansion to 4× width followed by projection back resembles a temporary introduction of auxiliary variables that are then marginalized.\u003c/p\u003e\n\u003cp\u003eThe residual connection ensures information preservation. We don\u0026rsquo;t discard UV data entirely; we augment it with coarse-grained correlations. Each layer refines rather than replaces. We acknowledge that this is a little different than standard RG: we don\u0026rsquo;t discard UV data entirely; we maintain a superposition of the specific token identity (UV) and the emerging coarse-grained correlations (IR). The attention mechanism acts as a filter, pushing irrelevant syntactic details into the background while amplifying the relevant semantic signal.\u003c/p\u003e\n\u003ch2 id=\"deeper-layers-flow-toward-the-infrared\"\u003eDeeper Layers: Flow Toward the Infrared\u003c/h2\u003e\n\u003cp\u003eLayers 2 through \u003cspan class=\"katex\"\u003e\u003cmath xmlns=\"http://www.w3.org/1998/Math/MathML\"\u003e\u003csemantics\u003e\u003cmrow\u003e\u003cmi\u003eL\u003c/mi\u003e\u003c/mrow\u003e\u003cannotation encoding=\"application/x-tex\"\u003eL\u003c/annotation\u003e\u003c/semantics\u003e\u003c/math\u003e\u003c/span\u003e repeat this structure, and the Wilson RG picture becomes most powerful here. Each layer defines a new effective Hamiltonian:\n\u003c/p\u003e\n\u003cspan class=\"katex\"\u003e\u003cmath xmlns=\"http://www.w3.org/1998/Math/MathML\" display=\"block\"\u003e\u003csemantics\u003e\u003cmrow\u003e\u003cmsubsup\u003e\u003cmi\u003eH\u003c/mi\u003e\u003cmtext\u003eeff\u003c/mtext\u003e\u003cmrow\u003e\u003cmo stretchy=\"false\"\u003e(\u003c/mo\u003e\u003cmi mathvariant=\"normal\"\u003eℓ\u003c/mi\u003e\u003cmo\u003e+\u003c/mo\u003e\u003cmn\u003e1\u003c/mn\u003e\u003cmo stretchy=\"false\"\u003e)\u003c/mo\u003e\u003c/mrow\u003e\u003c/msubsup\u003e\u003cmo\u003e=\u003c/mo\u003e\u003cmi mathvariant=\"script\"\u003eR\u003c/mi\u003e\u003cmo stretchy=\"false\"\u003e[\u003c/mo\u003e\u003cmsubsup\u003e\u003cmi\u003eH\u003c/mi\u003e\u003cmtext\u003eeff\u003c/mtext\u003e\u003cmrow\u003e\u003cmo stretchy=\"false\"\u003e(\u003c/mo\u003e\u003cmi mathvariant=\"normal\"\u003eℓ\u003c/mi\u003e\u003cmo stretchy=\"false\"\u003e)\u003c/mo\u003e\u003c/mrow\u003e\u003c/msubsup\u003e\u003cmo stretchy=\"false\"\u003e]\u003c/mo\u003e\u003c/mrow\u003e\u003cannotation encoding=\"application/x-tex\"\u003eH_{\\text{eff}}^{(\\ell+1)} = \\mathcal{R}[H_{\\text{eff}}^{(\\ell)}]\u003c/annotation\u003e\u003c/semantics\u003e\u003c/math\u003e\u003c/span\u003e\u003cp\u003e\nwhere \u003cspan class=\"katex\"\u003e\u003cmath xmlns=\"http://www.w3.org/1998/Math/MathML\"\u003e\u003csemantics\u003e\u003cmrow\u003e\u003cmi mathvariant=\"script\"\u003eR\u003c/mi\u003e\u003c/mrow\u003e\u003cannotation encoding=\"application/x-tex\"\u003e\\mathcal{R}\u003c/annotation\u003e\u003c/semantics\u003e\u003c/math\u003e\u003c/span\u003e denotes the RG transformation implemented by layer \u003cspan class=\"katex\"\u003e\u003cmath xmlns=\"http://www.w3.org/1998/Math/MathML\"\u003e\u003csemantics\u003e\u003cmrow\u003e\u003cmi mathvariant=\"normal\"\u003eℓ\u003c/mi\u003e\u003c/mrow\u003e\u003cannotation encoding=\"application/x-tex\"\u003e\\ell\u003c/annotation\u003e\u003c/semantics\u003e\u003c/math\u003e\u003c/span\u003e.\u003c/p\u003e\n\u003cp\u003eEarly layers operate at the ultraviolet scale, capturing short-range correlations—syntactic patterns, local dependencies. Middle layers extend the correlation length, picking up grammatical and phrasal structure. Deep layers approach the infrared, where long-range semantic content dominates.\u003c/p\u003e\n\u003cp\u003eWilson classified operators by their behavior under RG flow. \u003cem\u003eRelevant\u003c/em\u003e operators grow and dominate at long distances. \u003cem\u003eIrrelevant\u003c/em\u003e operators shrink and wash out. \u003cem\u003eMarginal\u003c/em\u003e operators remain constant.\u003c/p\u003e\n\u003cp\u003eFor our prompt, \u0026ldquo;France → European country → has a capital\u0026rdquo; exemplifies relevant information: it grows in importance as we coarse-grain. The exact word order—\u0026ldquo;What is the capital of\u0026rdquo; versus \u0026ldquo;The capital of what is\u0026rdquo;—behaves as irrelevant: the semantic content survives while surface details fade.\u003c/p\u003e\n\u003cp\u003eMisra\u0026rsquo;s Bayesian interpretation tracks this flow as progressive hypothesis elimination. Early layers establish \u0026ldquo;this is a question,\u0026rdquo; eliminating statement continuations. Middle layers narrow to \u0026ldquo;about a capital city,\u0026rdquo; then \u0026ldquo;of a European country.\u0026rdquo; By layer 8, \u0026ldquo;Paris\u0026rdquo; dominates the posterior. Later layers refine and verify consistency.\u003c/p\u003e\n\u003cp\u003eEach layer asks a question and gathers evidence through attention. Like a game of twenty questions, each round eliminates roughly half the remaining hypotheses.\u003c/p\u003e\n\u003ch2 id=\"the-fixed-point-semantic-attractor\"\u003eThe Fixed Point: Semantic Attractor\u003c/h2\u003e\n\u003cp\u003eAt layer \u003cspan class=\"katex\"\u003e\u003cmath xmlns=\"http://www.w3.org/1998/Math/MathML\"\u003e\u003csemantics\u003e\u003cmrow\u003e\u003cmi\u003eL\u003c/mi\u003e\u003c/mrow\u003e\u003cannotation encoding=\"application/x-tex\"\u003eL\u003c/annotation\u003e\u003c/semantics\u003e\u003c/math\u003e\u003c/span\u003e, the RG flow has approximately reached a fixed point — a self-similar configuration where further coarse-graining changes nothing:\u003c/p\u003e\n\u003cspan class=\"katex\"\u003e\u003cmath xmlns=\"http://www.w3.org/1998/Math/MathML\" display=\"block\"\u003e\u003csemantics\u003e\u003cmrow\u003e\u003cmsup\u003e\u003cmi\u003eH\u003c/mi\u003e\u003cmo\u003e∗\u003c/mo\u003e\u003c/msup\u003e\u003cmo\u003e=\u003c/mo\u003e\u003cmi mathvariant=\"script\"\u003eR\u003c/mi\u003e\u003cmo stretchy=\"false\"\u003e[\u003c/mo\u003e\u003cmsup\u003e\u003cmi\u003eH\u003c/mi\u003e\u003cmo\u003e∗\u003c/mo\u003e\u003c/msup\u003e\u003cmo stretchy=\"false\"\u003e]\u003c/mo\u003e\u003c/mrow\u003e\u003cannotation encoding=\"application/x-tex\"\u003eH^* = \\mathcal{R}[H^*]\u003c/annotation\u003e\u003c/semantics\u003e\u003c/math\u003e\u003c/span\u003e\u003cp\u003eThe final representation at the last token position encodes all processed information. From the Hopfield perspective, the system has converged to an attractor state — a local minimum in the energy landscape where dynamics settle.\u003c/p\u003e\n\u003cp\u003eUniversality emerges here. Different prompts asking the same question — \u0026ldquo;What\u0026rsquo;s France\u0026rsquo;s capital?\u0026rdquo;, \u0026ldquo;The capital of France is\u0026rdquo;, \u0026ldquo;France\u0026rsquo;s seat of government is\u0026rdquo; — flow to the \u003cem\u003esame\u003c/em\u003e infrared fixed point despite their different UV initializations. Microscopic details wash out; only universal semantic content remains.\u003c/p\u003e\n\u003cp\u003eIn Misra\u0026rsquo;s terms, the posterior has collapsed. From a uniform prior over 50,000 tokens, elimination has left a handful of candidates with \u0026ldquo;Paris\u0026rdquo; dominating.\u003c/p\u003e\n\u003ch2 id=\"output-measurement-at-the-ir-scale\"\u003eOutput: Measurement at the IR Scale\u003c/h2\u003e\n\u003cp\u003eThe final hidden state projects to vocabulary size:\u003c/p\u003e\n\u003cspan class=\"katex\"\u003e\u003cmath xmlns=\"http://www.w3.org/1998/Math/MathML\" display=\"block\"\u003e\u003csemantics\u003e\u003cmrow\u003e\u003cmtext\u003elogits\u003c/mtext\u003e\u003cmo\u003e=\u003c/mo\u003e\u003cmsub\u003e\u003cmi\u003eW\u003c/mi\u003e\u003cmtext\u003eout\u003c/mtext\u003e\u003c/msub\u003e\u003cmo\u003e⋅\u003c/mo\u003e\u003cmsubsup\u003e\u003cmi\u003ex\u003c/mi\u003e\u003cmtext\u003efinal\u003c/mtext\u003e\u003cmrow\u003e\u003cmo stretchy=\"false\"\u003e(\u003c/mo\u003e\u003cmi\u003eL\u003c/mi\u003e\u003cmo stretchy=\"false\"\u003e)\u003c/mo\u003e\u003c/mrow\u003e\u003c/msubsup\u003e\u003cmo\u003e∈\u003c/mo\u003e\u003cmsup\u003e\u003cmi mathvariant=\"double-struck\"\u003eR\u003c/mi\u003e\u003cmi\u003eV\u003c/mi\u003e\u003c/msup\u003e\u003c/mrow\u003e\u003cannotation encoding=\"application/x-tex\"\u003e\\text{logits} = W_{\\text{out}} \\cdot x_{\\text{final}}^{(L)} \\in \\mathbb{R}^V\u003c/annotation\u003e\u003c/semantics\u003e\u003c/math\u003e\u003c/span\u003e\u003cp\u003eThe softmax converts logits to probabilities:\u003c/p\u003e\n\u003cspan class=\"katex\"\u003e\u003cmath xmlns=\"http://www.w3.org/1998/Math/MathML\" display=\"block\"\u003e\u003csemantics\u003e\u003cmrow\u003e\u003cmi\u003eP\u003c/mi\u003e\u003cmo stretchy=\"false\"\u003e(\u003c/mo\u003e\u003cmtext\u003etoken\u003c/mtext\u003e\u003cmo\u003e=\u003c/mo\u003e\u003cmi\u003ew\u003c/mi\u003e\u003cmo stretchy=\"false\"\u003e)\u003c/mo\u003e\u003cmo\u003e=\u003c/mo\u003e\u003cmfrac\u003e\u003cmrow\u003e\u003cmi\u003eexp\u003c/mi\u003e\u003cmo\u003e⁡\u003c/mo\u003e\u003cmo stretchy=\"false\"\u003e(\u003c/mo\u003e\u003cmsub\u003e\u003cmtext\u003elogits\u003c/mtext\u003e\u003cmi\u003ew\u003c/mi\u003e\u003c/msub\u003e\u003cmi mathvariant=\"normal\"\u003e/\u003c/mi\u003e\u003cmi\u003eT\u003c/mi\u003e\u003cmo stretchy=\"false\"\u003e)\u003c/mo\u003e\u003c/mrow\u003e\u003cmrow\u003e\u003cmunder\u003e\u003cmo\u003e∑\u003c/mo\u003e\u003cmsup\u003e\u003cmi\u003ew\u003c/mi\u003e\u003cmo mathvariant=\"normal\"\u003e′\u003c/mo\u003e\u003c/msup\u003e\u003c/munder\u003e\u003cmi\u003eexp\u003c/mi\u003e\u003cmo\u003e⁡\u003c/mo\u003e\u003cmo stretchy=\"false\"\u003e(\u003c/mo\u003e\u003cmsub\u003e\u003cmtext\u003elogits\u003c/mtext\u003e\u003cmsup\u003e\u003cmi\u003ew\u003c/mi\u003e\u003cmo mathvariant=\"normal\"\u003e′\u003c/mo\u003e\u003c/msup\u003e\u003c/msub\u003e\u003cmi mathvariant=\"normal\"\u003e/\u003c/mi\u003e\u003cmi\u003eT\u003c/mi\u003e\u003cmo stretchy=\"false\"\u003e)\u003c/mo\u003e\u003c/mrow\u003e\u003c/mfrac\u003e\u003c/mrow\u003e\u003cannotation encoding=\"application/x-tex\"\u003e\nP(\\text{token} = w) = \\frac{\\exp(\\text{logits}_w / T)}{\\sum_{w^\\prime} \\exp(\\text{logits}_{w^\\prime} / T)}\n\u003c/annotation\u003e\u003c/semantics\u003e\u003c/math\u003e\u003c/span\u003e\u003cp\u003eThe output projection defines an observable—the operator we measure. Logits are energies (with sign flipped), and softmax computes the Boltzmann distribution over vocabulary states under the final effective theory.\u003c/p\u003e\n\u003cp\u003eFor \u0026ldquo;Paris,\u0026rdquo; the logit towers above competitors. At temperature \u003cspan class=\"katex\"\u003e\u003cmath xmlns=\"http://www.w3.org/1998/Math/MathML\"\u003e\u003csemantics\u003e\u003cmrow\u003e\u003cmi\u003eT\u003c/mi\u003e\u003cmo\u003e=\u003c/mo\u003e\u003cmn\u003e1\u003c/mn\u003e\u003c/mrow\u003e\u003cannotation encoding=\"application/x-tex\"\u003eT=1\u003c/annotation\u003e\u003c/semantics\u003e\u003c/math\u003e\u003c/span\u003e:\u003c/p\u003e\n\u003cspan class=\"katex\"\u003e\u003cmath xmlns=\"http://www.w3.org/1998/Math/MathML\" display=\"block\"\u003e\u003csemantics\u003e\u003cmrow\u003e\u003cmi\u003eP\u003c/mi\u003e\u003cmo stretchy=\"false\"\u003e(\u003c/mo\u003e\u003cmtext\u003eParis\u003c/mtext\u003e\u003cmo stretchy=\"false\"\u003e)\u003c/mo\u003e\u003cmo\u003e≈\u003c/mo\u003e\u003cmfrac\u003e\u003cmsup\u003e\u003cmi\u003ee\u003c/mi\u003e\u003cmn\u003e15\u003c/mn\u003e\u003c/msup\u003e\u003cmrow\u003e\u003cmsup\u003e\u003cmi\u003ee\u003c/mi\u003e\u003cmn\u003e15\u003c/mn\u003e\u003c/msup\u003e\u003cmo\u003e+\u003c/mo\u003e\u003cmsup\u003e\u003cmi\u003ee\u003c/mi\u003e\u003cmn\u003e10\u003c/mn\u003e\u003c/msup\u003e\u003cmo\u003e+\u003c/mo\u003e\u003cmsup\u003e\u003cmi\u003ee\u003c/mi\u003e\u003cmn\u003e8\u003c/mn\u003e\u003c/msup\u003e\u003cmo\u003e+\u003c/mo\u003e\u003cmo\u003e…\u003c/mo\u003e\u003c/mrow\u003e\u003c/mfrac\u003e\u003cmo\u003e≈\u003c/mo\u003e\u003cmn\u003e0.99\u003c/mn\u003e\u003c/mrow\u003e\u003cannotation encoding=\"application/x-tex\"\u003e\nP(\\text{Paris}) \\approx \\frac{e^{15}}{e^{15} + e^{10} + e^{8} + \\ldots} \\approx 0.99\n\u003c/annotation\u003e\u003c/semantics\u003e\u003c/math\u003e\u003c/span\u003e\u003cp\u003eThe system sits deep in an ordered phase — one configuration dominates. Sampling from this distribution yields \u0026ldquo;Paris\u0026rdquo; with near certainty.\u003c/p\u003e\n\u003ch2 id=\"the-energy-landscape-evolves\"\u003eThe Energy Landscape Evolves\u003c/h2\u003e\n\u003cp\u003eThe transformation across layers reshapes the energy landscape:\u003c/p\u003e\n\u003cpre tabindex=\"0\"\u003e\u003ccode\u003eLayer 0:     ∿∿∿∿∿∿∿∿∿∿    (rough, many shallow minima)\n              \nLayer 4:     ∿∿∿╲  ╱∿∿∿    (structure emerging)\n                 ╲╱\n                 \nLayer 8:       ╲    ╱       (dominant basin forming)\n                ╲  ╱\n                 ╲╱\n                 \nLayer 12:        ╲╱         (single deep attractor)\n                  V\n                Paris\n\u003c/code\u003e\u003c/pre\u003e\u003ch2 id=\"why-the-mapping-works\"\u003eWhy the Mapping Works\u003c/h2\u003e\n\u003cp\u003eThe deep reason these frameworks align: they solve the same problem. Renormalization group extracts stable, low-dimensional descriptions from high-dimensional, noisy data. Bayesian inference updates beliefs to concentrate probability on hypotheses consistent with evidence. Attention mechanisms weight information sources by relevance and aggregate them into coherent representations.\u003c/p\u003e\n\u003cp\u003eThe principle connecting them — stationary action yields stable information — explains why transformers find good answers. \u0026ldquo;Paris\u0026rdquo; wins not by accident but because it sits at an attractor where informational consensus forms. Neighboring attention patterns, slightly different hypothesis configurations, all flow to the same answer. The paths agree.\u003c/p\u003e\n\u003cp\u003eThe RG flow adds precision. We don\u0026rsquo;t just find any stable point; we find the \u003cem\u003euniversality class\u003c/em\u003e — the set of UV configurations flowing to the same IR fixed point. Every way of asking about France\u0026rsquo;s capital belongs to this class and converges to \u0026ldquo;Paris.\u0026rdquo;\u003c/p\u003e\n\u003cp\u003eTransformers implement the logic of statistical physics. The attention mechanism computes partition functions. Layer depth traces RG scale. Training sculpts an energy landscape where semantic attractors form at the right places. When we ask \u0026ldquo;What is the capital of France?\u0026rdquo;, we initialize a statistical mechanical system that flows, layer by layer, toward its equilibrium configuration.\u003c/p\u003e\n\u003cp\u003eThe answer was always there, encoded in the trained weights. The forward pass simply lets the system relax into its ground state.\u003c/p\u003e\n\u003cdiv class=\"footnotes\" role=\"doc-endnotes\"\u003e\n\u003chr\u003e\n\u003col\u003e\n\u003cli id=\"fn:1\"\u003e\n\u003cp\u003eQuantum mechanics and statistical mechanics are connected through a \u003ca href=\"https://en.wikipedia.org/wiki/Wick_rotation\"\u003eWick rotation\u003c/a\u003e of \u003ca href=\"https://en.wikipedia.org/wiki/Path_integral_formulation#Feynman's_interpretation\"\u003ethe quantum amplitudes\u003c/a\u003e, which is how I recognized the connection in this case, but that connection is not relevant to the remainder of this particular essay.\u0026#160;\u003ca href=\"https://www.symmetrybroken.com/transformer-as-renormalization-group-flow/#fnref:1\" class=\"footnote-backref\" role=\"doc-backlink\"\u003e\u0026#x21a9;\u0026#xfe0e;\u003c/a\u003e\u003c/p\u003e\n\u003c/li\u003e\n\u003c/ol\u003e\n\u003c/div\u003e\n",
  "content_md": "# The Transformer as Renormalization Group Flow\n\nA recent essay and linked papers by Vishal Misra and colleagues, [\"Attention is Bayesian Inference,\"](https://medium.com/@vishalmisra/attention-is-bayesian-inference-578c25db4501) argues that the transformer attention mechanism implements a form of probabilistic inference — each layer asks questions and eliminates hypotheses until the posterior collapses onto the correct answer. The framework connects naturally to ideas I explored in [A Stationary Action is Stable Information](/a-stationary-action-is-stable-information/): systems find stable configurations where neighboring paths agree, where informational consensus forms.[^1] \n\nThe connection runs deeper than analogy. The mathematics of attention — softmax over dot products, weighted sums of values — maps precisely onto the partition functions and thermal expectations of statistical mechanics. When we stack these operations into deep networks, something remarkable emerges: the forward pass through a transformer implements [a Kadanoff-Wilson renormalization group flow](https://faculty.kfupm.edu.sa/phys/imnasser/Phase_transition/Maris_Kadanoff.pdf), coarse-graining microscopic token representations into stable semantic attractors.\n\nRenormalization group flow is [an advanced topic in non-equilibrium statistical mechanics](http://max2.physics.sunysb.edu/~rastelli/2019/WilsonKogut.pdf), so the remainder of this essay may be of interest only to physicists. But the connection does provide some useful perspective for interpretability, alignment, and even scalability of the transformer architecture, which I will be exploring. Note that unlike standard block-spin RG, which decimates the lattice (reducing the number of sites), the transformer preserves the token count. The coarse-graining is of *information*, integrating out syntactic fluctuations to leave behind stable semantic operators.\n\nTo make this more precise, let's trace how a transformer generates \"Paris\" in response to \"What is the capital of France?\" The layer-by-layer account and the numbers in it are illustrative, not measured from a particular model.\n\n## The Setup: A Statistical Mechanical System\n\nThe prompt arrives as a string of approximately seven tokens, depending on tokenization: `[What] [is] [the] [capital] [of] [France] [?]`. We can view this as a one-dimensional lattice where each site carries a high-dimensional \"spin\" variable — the token embedding.\n\nFrom the Bayesian perspective, the model begins with a prior over all possible next tokens. The vocabulary contains roughly 50,000 possibilities, and at this stage, none has been eliminated.\n\nThe statistical mechanics framing treats each token embedding \\(x_i^{(0)} \\in \\mathbb{R}^d\\) as a spin configuration at lattice site \\(i\\). The embedding dimension \\(d\\) counts the internal degrees of freedom per site. We're working at the ultraviolet scale — the full microscopic description, before any coarse-graining.\n\n## Positional Encoding Breaks a Symmetry\n\nBefore attention operates, positional encodings \\(p_i\\) modify each embedding: \\(x_i^{(0)} \\leftarrow x_i^{(0)} + p_i\\). This breaks translational invariance on the lattice, introducing explicit spatial structure. Without positional encoding, the system would be permutation-invariant — every spin interacting equally with every other, like a Sherrington-Kirkpatrick (SK) spin glass model.\n\nThe original transformer used sinusoidal encodings:\n$$p_{i,2k} = \\sin(i/10000^{2k/d}), \\quad p_{i,2k+1} = \\cos(i/10000^{2k/d})$$\nThese Fourier modes encode position at multiple frequency scales. High-frequency components (small \\(k\\)) capture local structure; low-frequency components (large \\(k\\)) capture global ordering.\n\nIn renormalization group terminology, positional encoding introduces *relevant operators* — perturbations that grow under RG flow and dominate at long distances. Position matters more as we zoom out, which explains why low-frequency modes capture long-range dependencies.\n\n## Attention as Partition Function\n\nThe attention mechanism implements the core statistical mechanical operation. For each token \\(i\\), the transformer computes queries, keys, and values:\n$$q_i = W_Q x_i^{(0)}, \\quad k_j = W_K x_j^{(0)}, \\quad v_j = W_V x_j^{(0)}$$\nAttention scores measure the compatibility between query \\(i\\) and key \\(j\\):\n$$s_{ij} = \\frac{q_i \\cdot k_j}{\\sqrt{d}}$$\nThe softmax normalizes these scores into weights:\n$$\\alpha_{ij} = \\frac{\\exp(s_{ij})}{\\sum_k \\exp(s_{ik})}$$\nFinally, the output aggregates values according to these weights:\n$$z_i = \\sum_j \\alpha_{ij} v_j$$\nThe mapping to statistical mechanics is exact. The attention score \\(s_{ij}\\) plays the role of negative energy \\(-\\beta E_{ij}\\), measuring interaction strength between sites. The denominator \\(\\sqrt{d}\\) regulates the temperature. Without it, high-dimensional dot products would diverge, driving the softmax into a frozen 'argmax' state. The scaling ensures the system remains at a finite temperature, maintaining a soft, probabilistic ensemble. The normalization factor \\(Z_i = \\sum_j \\exp(s_{ij})\\) is precisely a partition function, and the weights \\(\\alpha_{ij}\\) are Boltzmann probabilities.\n\nThe log-partition function gives the free energy at token \\(i\\):\n$$F_i = -\\sqrt{d} \\cdot \\log \\sum_j \\exp\\left(\\frac{q_i \\cdot k_j}{\\sqrt{d}}\\right)$$\nThis log-sum-exp [appears in the literature on Modern Hopfield Networks (or Dense Associative Memories) as the continuous relaxation](https://arxiv.org/abs/2008.02217). (See equation 1 at 3.) The KL divergence between the attention distribution \\(\\alpha_{ij}\\) and the uniform distribution over tokens measures how much information this attention head extracts — how far the system has moved from maximum ignorance. Each attention computation produces relative entropy; each layer pays a [synchronization tax](/the-synchronization-tax/) to establish which tokens matter to which. Each attention computation *is* a relative entropy — the divergence between \"attend to everything equally\" and \"attend to what matters.\"\n\nThe output \\(z_i = \\sum_j \\alpha_{ij} v_j\\) computes a thermal expectation — an observable averaged over the Boltzmann ensemble. In Misra's Bayesian framing, the query encodes \"what information is token \\(i\\) seeking?\" while keys advertise \"what information does token \\(j\\) offer?\" The dot product measures evidential compatibility. For our prompt, the query from position 7 asks about capital cities; the key from \"France\" advertises European nationhood; their large dot product reflects compatibility.\n\n## The First RG Step: Coarse-Graining\n\nAttention implements the first renormalization group transformation. Before attention, each site carries an independent spin \\(x_j^{(0)}\\). After attention, each site carries an effective spin \\(z_i = \\sum_j \\alpha_{ij} v_j\\) that incorporates information from the entire lattice.\n\nIn Kadanoff's block-spin renormalization, which Wilson made quantitative, one averages spins within fixed spatial blocks. Attention performs something more sophisticated: an *adaptive* blocking where the attention pattern defines which sites contribute to each effective spin. High attention weights pull sites together; low weights decouple them.\n\nThe operation integrates out individual token identities and produces a coarse-grained description. The original microscopic structure — raw embeddings — gives way to an effective theory encoding correlations. New couplings emerge from this integration, captured in the transformed representations.\n\n## Feedforward Networks: Local Field Renormalization\n\nAfter attention, each token passes through a feedforward network:\n$$\\text{FFN}(x) = W_2 \\cdot \\text{ReLU}(W_1 x + b_1) + b_2$$\nwith residual connections and layer normalization. Unlike attention, which mixes information across tokens, the feedforward network transforms each site independently—a local, site-wise operation.\n\nThe ReLU nonlinearity zeros negative activations, creating sparsity. This decimates degrees of freedom: only some modes survive, and irrelevant fluctuations are integrated out. The expansion to 4× width followed by projection back resembles a temporary introduction of auxiliary variables that are then marginalized.\n\nThe residual connection ensures information preservation. We don't discard UV data entirely; we augment it with coarse-grained correlations. Each layer refines rather than replaces. We acknowledge that this is a little different than standard RG: we don't discard UV data entirely; we maintain a superposition of the specific token identity (UV) and the emerging coarse-grained correlations (IR). The attention mechanism acts as a filter, pushing irrelevant syntactic details into the background while amplifying the relevant semantic signal.\n\n## Deeper Layers: Flow Toward the Infrared\n\nLayers 2 through \\(L\\) repeat this structure, and the Wilson RG picture becomes most powerful here. Each layer defines a new effective Hamiltonian:\n$$H_{\\text{eff}}^{(\\ell+1)} = \\mathcal{R}[H_{\\text{eff}}^{(\\ell)}]$$\nwhere \\(\\mathcal{R}\\) denotes the RG transformation implemented by layer \\(\\ell\\).\n\nEarly layers operate at the ultraviolet scale, capturing short-range correlations—syntactic patterns, local dependencies. Middle layers extend the correlation length, picking up grammatical and phrasal structure. Deep layers approach the infrared, where long-range semantic content dominates.\n\nWilson classified operators by their behavior under RG flow. *Relevant* operators grow and dominate at long distances. *Irrelevant* operators shrink and wash out. *Marginal* operators remain constant.\n\nFor our prompt, \"France → European country → has a capital\" exemplifies relevant information: it grows in importance as we coarse-grain. The exact word order—\"What is the capital of\" versus \"The capital of what is\"—behaves as irrelevant: the semantic content survives while surface details fade.\n\nMisra's Bayesian interpretation tracks this flow as progressive hypothesis elimination. Early layers establish \"this is a question,\" eliminating statement continuations. Middle layers narrow to \"about a capital city,\" then \"of a European country.\" By layer 8, \"Paris\" dominates the posterior. Later layers refine and verify consistency.\n\nEach layer asks a question and gathers evidence through attention. Like a game of twenty questions, each round eliminates roughly half the remaining hypotheses.\n\n## The Fixed Point: Semantic Attractor\n\nAt layer \\(L\\), the RG flow has approximately reached a fixed point — a self-similar configuration where further coarse-graining changes nothing:\n\n$$H^* = \\mathcal{R}[H^*]$$\n\nThe final representation at the last token position encodes all processed information. From the Hopfield perspective, the system has converged to an attractor state — a local minimum in the energy landscape where dynamics settle.\n\nUniversality emerges here. Different prompts asking the same question — \"What's France's capital?\", \"The capital of France is\", \"France's seat of government is\" — flow to the *same* infrared fixed point despite their different UV initializations. Microscopic details wash out; only universal semantic content remains.\n\nIn Misra's terms, the posterior has collapsed. From a uniform prior over 50,000 tokens, elimination has left a handful of candidates with \"Paris\" dominating.\n\n## Output: Measurement at the IR Scale\n\nThe final hidden state projects to vocabulary size:\n\n$$\\text{logits} = W_{\\text{out}} \\cdot x_{\\text{final}}^{(L)} \\in \\mathbb{R}^V$$\n\nThe softmax converts logits to probabilities:\n\n$$\nP(\\text{token} = w) = \\frac{\\exp(\\text{logits}_w / T)}{\\sum_{w^\\prime} \\exp(\\text{logits}_{w^\\prime} / T)}\n$$\n\nThe output projection defines an observable—the operator we measure. Logits are energies (with sign flipped), and softmax computes the Boltzmann distribution over vocabulary states under the final effective theory.\n\nFor \"Paris,\" the logit towers above competitors. At temperature \\(T=1\\):\n\n$$\nP(\\text{Paris}) \\approx \\frac{e^{15}}{e^{15} + e^{10} + e^{8} + \\ldots} \\approx 0.99\n$$\n\nThe system sits deep in an ordered phase — one configuration dominates. Sampling from this distribution yields \"Paris\" with near certainty.\n\n## The Energy Landscape Evolves\n\nThe transformation across layers reshapes the energy landscape:\n```\nLayer 0:     ∿∿∿∿∿∿∿∿∿∿    (rough, many shallow minima)\n              \nLayer 4:     ∿∿∿╲  ╱∿∿∿    (structure emerging)\n                 ╲╱\n                 \nLayer 8:       ╲    ╱       (dominant basin forming)\n                ╲  ╱\n                 ╲╱\n                 \nLayer 12:        ╲╱         (single deep attractor)\n                  V\n                Paris\n```\n\n## Why the Mapping Works\n\nThe deep reason these frameworks align: they solve the same problem. Renormalization group extracts stable, low-dimensional descriptions from high-dimensional, noisy data. Bayesian inference updates beliefs to concentrate probability on hypotheses consistent with evidence. Attention mechanisms weight information sources by relevance and aggregate them into coherent representations.\n\nThe principle connecting them — stationary action yields stable information — explains why transformers find good answers. \"Paris\" wins not by accident but because it sits at an attractor where informational consensus forms. Neighboring attention patterns, slightly different hypothesis configurations, all flow to the same answer. The paths agree.\n\nThe RG flow adds precision. We don't just find any stable point; we find the *universality class* — the set of UV configurations flowing to the same IR fixed point. Every way of asking about France's capital belongs to this class and converges to \"Paris.\"\n\nTransformers implement the logic of statistical physics. The attention mechanism computes partition functions. Layer depth traces RG scale. Training sculpts an energy landscape where semantic attractors form at the right places. When we ask \"What is the capital of France?\", we initialize a statistical mechanical system that flows, layer by layer, toward its equilibrium configuration.\n\nThe answer was always there, encoded in the trained weights. The forward pass simply lets the system relax into its ground state.\n\n[^1]: Quantum mechanics and statistical mechanics are connected through a [Wick rotation](https://en.wikipedia.org/wiki/Wick_rotation) of [the quantum amplitudes](https://en.wikipedia.org/wiki/Path_integral_formulation#Feynman's_interpretation), which is how I recognized the connection in this case, but that connection is not relevant to the remainder of this particular essay.\n",
  "created": "2026-01-03T16:05:25Z",
  "id": "4g6ayy4pqk77jza5w2mzg864ng",
  "kind": "thread",
  "media": [],
  "origin": "https://www.symmetrybroken.com/",
  "page": "https://www.symmetrybroken.com/transformer-as-renormalization-group-flow/",
  "transclusions": [],
  "updated": "2026-11-19T00:17:44Z",
  "version": 2
}