Scaling Hides What Attention Cannot Solve

Attention mechanisms have become the assumed engine of modern artificial intelligence. Researchers have begun collapsing the engineering solution with something larger. They propose that attention itself is how cognition works.

This is not a new mistake. In 1956, Marvin Minsky and John McCarthy presented symbolic logic as the foundation of human reasoning at the Dartmouth Conference.

Logic was clean, formalizable. It worked on toy problems—but it took nearly two decades before the field collided with the binding problem. Symbols could be manipulated, yet nobody could explain how individual symbols assembled into unified percepts. How does "red" bind to "square" to create the experience of seeing a red square rather than two separate facts about the world?

Attention mechanisms face an identical vulnerability. They are exquisite at filtering and reweighting information once features already exist in a coherent form. Humans bind features into unified objects before attention engages. You see a face as a face, not as a collection of features that attention later learns to prioritize. The mechanism works beautifully at scale because transformer models train on enough data that statistical associations approximate the binding that evolution and development solve. Approximation is not explanation. Scaling makes the problem invisible and does not solve it.

What changed between 1956 and now is that we know what to watch for. When someone claims their system has discovered the essence of something, ask what binding problem it cannot solve. In your own field, when a methodology scales dramatically, separate what it does well from what it does instead of addressing the hard problem.