Ask a language model to finish “king is to queen as man is to ___” and it gets it right instantly. Ask it to work out that Paris relates to France the way Tokyo relates to Japan, and it handles that too, without ever having been told a rule about capitals. There’s no lookup table of royal titles or geography facts making this happen. Something else is going on underneath, and it starts with turning words into numbers.

Turning words into addresses

Every word, or fragment of a word, that a model works with gets converted into a long list of numbers called a vector, typically a few hundred to a few thousand of them. That list is not a code or a cipher standing in for the word. It’s a coordinate, a precise address in a space with hundreds or thousands of dimensions, one axis for each number in the list.

The useful way to picture this is a city grid, one big enough that every address encodes something real about how that location relates to every other one. In an actual city, two addresses close together are physically close, and the direction from one to the other (three blocks north, two east) tells you something consistent, the kind of consistency that lets you apply the same move (three blocks north, two east) from a different starting point and land somewhere meaningful. Embeddings work the same way, just with far more dimensions than a street grid has. “King” and “queen” sit near each other because they show up in similar contexts across huge amounts of text. “Paris” and “Tokyo” cluster in a neighborhood of capital cities. And the step you take to get from “man” to “king” turns out to be almost the same step, the same direction and rough distance, as the one from “woman” to “queen.” Nobody programmed that relationship in. It fell out of the geometry once enough text was compressed into that space, because words used in parallel ways end up positioned in parallel ways.

Why this changes how you read AI behavior

Once you know meaning is being handled as position in a space, a lot of otherwise strange AI behavior stops being mysterious. A model can connect “couch” and “sofa” not because it looked up a synonym list, but because training pushed them into nearly the same coordinates. It can also get tripped up by two unrelated things that happen to sit close together for shallow statistical reasons, producing a confident answer that’s confidently wrong. That’s not a glitch layered on top of understanding. It’s the same mechanism, geometry doing what geometry does, landing in the wrong neighborhood instead of the right one. It also explains why these systems handle metaphor, analogy, and loose synonyms so fluidly: those are exactly the kinds of relationships that show up as consistent directions in a space built from how words actually get used together, not as relationships defined by any dictionary.

Position is not comprehension

Here’s the part worth sitting with. For a model, meaning is not something it grasps, weighs, or reflects on. Meaning is a location, a set of coordinates fixed by training, and “related” means “nearby,” full stop. When the system completes that king and queen analogy, it isn’t reasoning about monarchy or gender. It’s doing arithmetic on addresses in a space where those relationships happen to be encoded as directions. That this works as well as it does, well enough to look like comprehension from the outside, is precisely what makes it disorienting. Useful, fluent, often correct behavior is coming out of a process that has no concept of a concept, only a very well-organized map of where things sit relative to everything else. The output can be indistinguishable from understanding without there being any understanding underneath it, and that gap between “behaves as if it knows” and “knows” is one worth holding onto every time the behavior looks impressive.

This piece builds on Tokens: How a Model “Sees” Text, which covers the step before any of this geometry exists, breaking text into fragments in the first place. From here, the natural next question is how a model decides which of those positioned concepts to pay attention to at any given moment, which is exactly what Attention: What the Model Looks At covers next.