[{"content":"Notes taken while learning, not after. Rough edges left in on purpose: the draft is the point. Large language models for now; whatever else I can\u0026rsquo;t get past, eventually.\n","date":"28 August 2026","externalUrl":null,"permalink":"/learning/","section":"Learning","summary":"","title":"Learning","type":"learning"},{"content":"","date":"28 August 2026","externalUrl":null,"permalink":"/series/learning-llms/","section":"Series","summary":"","title":"Learning LLMs","type":"series"},{"content":"What I\u0026rsquo;ve had to work through to stop hand-waving. Filed here before I forget why I bothered.\n","date":"28 August 2026","externalUrl":null,"permalink":"/learning/llms/","section":"Learning","summary":"Notes on large language models: the working through, and the parts I keep having to re-derive.","title":"LLMs","type":"learning"},{"content":"","date":"28 August 2026","externalUrl":null,"permalink":"/tags/meta/","section":"Tags","summary":"","title":"Meta","type":"tags"},{"content":"","date":"28 August 2026","externalUrl":null,"permalink":"/","section":"Mostly Second Thoughts","summary":"","title":"Mostly Second Thoughts","type":"page"},{"content":" Before writing anything worth reading, it seems prudent to check that the site can display it. If inline math typeset by KaTeX, a centred display equation, a syntax-highlighted Python snippet, and a captioned figure all render cleanly below, the plumbing is in order and future posts can concern themselves with their contents rather than their appearance.\nInline math inside prose # Scaled dot-product attention computes a soft lookup from queries \\(Q\\), keys \\(K\\), and values \\(V\\). Written inline, it is \\(\\text{Attention}(Q, K, V) = \\text{softmax}\\!\\left(\\tfrac{QK^{\\top}}{\\sqrt{d_k}}\\right) V\\), where the \\(\\sqrt{d_k}\\) divisor keeps the pre-softmax logits from growing too large as the key dimension \\(d_k\\) increases.\nA display equation # Language models are trained to minimize the token-level cross-entropy loss. For a sequence of tokens \\(x_1, \\dots, x_T\\), this is\n$$ \\mathcal{L}(\\theta) = -\\frac{1}{T} \\sum_{t=1}^{T} \\log p_\\theta\\!\\left(x_t \\mid x_{\\lt t}\\right). $$Each term \\(\\log p_\\theta(x_t \\mid x_{\\lt t})\\) is the log-probability the model assigns to the true next token given everything that came before.\nA code block # Here is a compact PyTorch-flavoured implementation of the attention scores from the equation above. Nothing fancy, just enough to see syntax highlighting work.\nimport math import torch import torch.nn.functional as F def attention(Q, K, V, mask=None): \u0026#34;\u0026#34;\u0026#34;Scaled dot-product attention.\u0026#34;\u0026#34;\u0026#34; d_k = Q.size(-1) scores = Q @ K.transpose(-2, -1) / math.sqrt(d_k) if mask is not None: scores = scores.masked_fill(mask == 0, float(\u0026#34;-inf\u0026#34;)) weights = F.softmax(scores, dim=-1) return weights @ V, weights A figure with caption # Below is a 4-token causal attention pattern after softmax. The lower-triangular structure reflects the mask: token \\(t\\) can only attend to tokens \\(\\le t\\).\nA 4x4 causal attention matrix: each row sums to 1, upper triangle is zero because of the causal mask. That\u0026rsquo;s it. If everything above reads cleanly in both light and dark mode, the site is calibrated correctly.\n","date":"28 August 2026","externalUrl":null,"permalink":"/learning/llms/hello-math-test/","section":"Learning","summary":"A load-bearing sanity check for the site: does the maths render, and does the code sit next to it without complaint.","title":"Proof that the math renders","type":"learning"},{"content":"","date":"28 August 2026","externalUrl":null,"permalink":"/series/","section":"Series","summary":"","title":"Series","type":"series"},{"content":"","date":"28 August 2026","externalUrl":null,"permalink":"/tags/","section":"Tags","summary":"","title":"Tags","type":"tags"},{"content":"I\u0026rsquo;m Clément Bosquet, a student at ESCP Business School. Outside class I\u0026rsquo;m teaching myself modern language models end-to-end: the maths and code that make them run, and the habits of using them day-to-day without losing the thread. Alongside that, I write the occasional essay on whatever else refuses to leave me alone.\nThis site is called Mostly Second Thoughts on purpose. I publish notes before I\u0026rsquo;m sure of them because writing in public is the fastest way I\u0026rsquo;ve found to notice what I don\u0026rsquo;t actually understand. Nothing here is final; a few posts will age into embarrassments, and I\u0026rsquo;ll edit them or leave them standing as evidence.\nIf a post is up, it means I could no longer hand-wave through it. If it quietly vanishes a month later, or turns out to argue against an earlier one, consider that part of the exercise.\n","externalUrl":null,"permalink":"/about/","section":"About","summary":"","title":"About","type":"about"},{"content":" Suggest something I should learn about. I keep a list. I even read it sometimes. Website Your suggestion Who are you? (optional) Send Received. Your suggestion joins a queue I take more seriously than it looks. That didn\u0026rsquo;t send. Try again in a bit; the server is usually more cooperative the second time. ","externalUrl":null,"permalink":"/suggest/","section":"Mostly Second Thoughts","summary":"","title":"Suggest a topic","type":"page"},{"content":"Longer arguments on subjects I have opinions about and, occasionally, grounds for them. Fewer than the notes, and slower to arrive.\n","externalUrl":null,"permalink":"/thoughtpieces/","section":"Thoughtpieces","summary":"","title":"Thoughtpieces","type":"thoughtpieces"}]