Borrowed Bodies: How human sensorimotor experience could ground the next generation of AI

Page

20261004

The Problem Nobody Is Naming

Tesla’s Full Self-Driving system has driven hundreds of millions of miles. It has seen more road than any human driver alive. It responds faster, never blinks, and does not get distracted by a conversation or the radio. And yet it still fails in ways that feel absurd to any human who has ever driven a car.

It hesitates at a construction zone where the cones are arranged unusually. It misreads a shadow as an obstacle. It slows for a plastic bag that has drifted into the lane. These are not failures of processing power or data volume. They are failures of a different kind - the kind that comes from never having had a body.

When a human driver approaches an unusual construction zone, something happens before any conscious assessment. The body tenses slightly. The grip on the wheel changes. The gaze narrows. These are not decisions; they are the sensorimotor substrate of understanding - a pre-linguistic layer of world-engagement that exists before language, before explicit reasoning, before any symbolic representation of the situation. The driver is not consulting a mental model of construction zones. The driver is inhabiting the situation.

FSD cannot inhabit a situation. It can only observe one. And that distinction - between observation and inhabitation - may be the deepest unsolved problem in autonomous AI.

What We Actually Mean by Embodiment

The word “embodied” has been overused to the point of meaninglessness in AI discourse. It needs to be restored to precision.

Embodiment is not the presence of sensors. A thermostat has sensors. Embodiment is the coupling of a sensory apparatus to a system of action possibilities - what ecological psychologist James Gibson called affordances - in a way that is continuous, reactive, and meaningful without requiring symbolic translation.

When you walk through a narrow gap between two people, you do not calculate the width of your shoulders and compare it to the available space. You perceive your passage-ability directly. The geometry of the gap is legible to your body before it is legible to your mind. This is what the philosopher Albert Michotte demonstrated with his experiments on causality perception in the 1960s: humans perceive causal relations directly, through temporal and spatial patterns, without requiring symbolic encoding of those relations. The world reveals itself as a field of action possibilities, and the reading of that field operates below the threshold of language.

This sensorimotor layer is not a luxury added on top of intelligence. It is the substrate from which intelligence emerges. Language is not the foundation of thought; it is something that grows out of a prior layer of embodied sense-making that is in principle measurable, quantifiable, and - crucially - separable from the conscious experience of the agent doing the sensing.

This last point matters enormously. We do not need to solve the hard problem of consciousness to make use of the pre-linguistic sensorimotor layer. We only need to measure it.

Tesla Already Knows This, Implicitly

Here is what is remarkable: Tesla’s approach to FSD already contains the seed of this idea.

The Data Engine - the system by which Tesla collects driving data from its human-operated fleet to train FSD - is already using humans as an embodiment proxy. Every time a human driver performs a shadow detection, navigates an ambiguous merge, or makes a micro-adjustment around a cyclist, that sensorimotor event enters the training pipeline. Tesla has understood, correctly, that the richest training signal available is a human being navigating the physical world in real time.

But there is a gap in the signal. The Data Engine captures what the human did. It does not capture what the human was experiencing when they did it.

It captures the steering input. It does not capture the change in grip pressure that preceded it. It captures the braking event. It does not capture the postural shift and the narrowing of gaze that signalled, 400 milliseconds before the brake was applied, that something uncertain had entered the driver’s perceptual field. It captures the trajectory. It does not capture the pre-linguistic assessment of the situation that made that trajectory feel correct.

The human driver is already the training ground. The question is whether we are reading the full signal.

The Framework: Human as Embodiment Proxy

The proposal is this: treat the human user as the AI agent’s borrowed body, and use continuous sensorimotor data from that human - physiological, attentional, postural, environmental - as the non-verbal signal stream that grounds the agent’s understanding of the world.

This is not a metaphor. It is an architecture.

In standard agent design, the unit of interaction is the agent alone, operating on a model of the world constructed from sensor data. The agent observes; the world is its input. In the framework proposed here, the unit of interaction is the human-agent hybrid system. The agent does not model the world abstractly. It participates in the world through the human’s sensorimotor engagement with it.

The signal stream feeding the agent would include, at minimum:

Eye tracking and gaze dynamics - not just where the human is looking, but how their gaze moves, what it lingers on, what it skips over. Gaze behaviour is among the richest pre-linguistic signals available. It encodes uncertainty, interest, threat assessment, and spatial understanding without any of those states being explicitly labelled.

Physiological state - heart rate variability, galvanic skin response, respiration rate. These are the biological signatures of stress, calm, alertness, and arousal. They are not perfectly reliable indicators of subjective experience, but they are measurable correlates of the pre-linguistic appraisal system that runs continuously beneath conscious thought.

Postural and micro-motor signals - the shift in seat position before a difficult manoeuvre, the grip change on the wheel, the involuntary muscle tension that precedes a decision. These are the body’s own predictive signals, generated by a sensorimotor system that has been solving the physical world for decades.

Environmental context - not just what the sensors see, but what the human’s body is responding to in that environment. The conjunction of physiological state and environmental sensor data produces something neither alone can provide: a signal that tells the system not just what is present in the world, but what is salient.

This is the grounding signal that FSD currently lacks. It is not more visual data. It is the layer of pre-linguistic appraisal that tells an intelligent system which parts of the visual field matter, and why.

From DELCA to FSD: A Twenty-Year Arc

This framework does not emerge from nowhere. It has a research lineage.

In 2005, research into Disembodied Location-specific Conversational Agents (DELCA) at the IT University of Copenhagen confronted a version of the same problem. The DELCA agents were software entities that needed to operate in physical hybrid spaces - environments where digital and physical information were combined. The central finding was that such agents could not rely on symbolic encoding to communicate in dynamic environments. The world changes faster than symbols can be updated. Instead, the research proposed interaction design based on causal relations: direct, perceivable connections between agent behaviour and environmental events, in the tradition of Michotte’s causality perception research.

The DELCA work identified the problem: disembodied agents in physical space lack any genuine causal anchor to the world they are operating in. It proposed a design solution - causal, non-encoded interaction - but it did not resolve the deeper question of how the agent itself could acquire the kind of world-understanding that would make such interaction genuinely intelligent rather than merely responsive.

The framework proposed here is the resolution of that question. The human body is the causal anchor. The sensorimotor proxy is what connects the disembodied agent to the pre-linguistic layer of world-engagement that DELCA identified but could not provide.

The Transfer Problem: From Driver to Car to Robot

The most direct application of this framework to Tesla’s mission is also the most practically significant.

FSD currently learns from what humans do. The proposed framework adds: FSD should also learn from what humans experience when they do it. The enriched training signal would include not only the driving event but the full sensorimotor context of the human agent who performed it - the pre-linguistic appraisal, the attentional dynamics, the physiological state. This would allow the system to learn not only the correct action in a given situation, but the correct weighting of perceptual signals that led a competent human to that action.

This has an important implication for the hardest cases. The situations where FSD currently struggles are precisely the situations where human drivers rely most heavily on pre-linguistic appraisal rather than explicit rule-following. Novel construction configurations. Ambiguous road markings. Social negotiation between vehicles at uncontrolled intersections. These are situations where the body knows before the mind does. A training signal that includes the sensorimotor context of a skilled human navigating these situations is a fundamentally different kind of data than a training signal that captures only the resulting action.

The transfer to robotics - specifically to a platform like Optimus - follows the same logic. The challenge of deploying a humanoid robot in unstructured human environments is identical in structure to the challenge of deploying FSD in unstructured road environments. The robot needs to perceive affordances directly, to navigate situations that were never in its training distribution, to inhabit a physical environment rather than merely observe it.

If the robot’s sensorimotor representations have been learned through a human proxy - trained on the continuous conjunction of human physiological state, attentional dynamics, and environmental context - those representations carry a quality of grounding that representations learned from observation alone cannot possess. The robot would have learned, through a borrowed body, what it feels like for a human sensorimotor system to encounter a narrow passage, an unstable surface, an unexpected social intrusion. That is a different kind of knowledge than what a camera and a classifier can produce.

The Interactivist Account: Why This Is Not Just More Data

It is important to be precise about why this approach differs from simply adding more sensor modalities to existing architectures.

Multimodal AI systems - systems that process visual, auditory, and other input streams simultaneously - do not solve the grounding problem. They extend the observation surface. But the signals are still processed as representations: transformed into latent space, correlated with other representations, and decoded into outputs. The world is still observed, not inhabited.

The difference in the framework proposed here is the feedback loop. The agent’s outputs influence the human’s behaviour; the human’s sensorimotor response to those outputs is captured and returned to the agent as signal. This creates an interactive loop of the kind that philosopher and cognitive scientist Mark Bickhard identifies as the basis of genuine meaning-formation: not representation but interaction. Not encoding but participation.

The agent does not need to know what stress is in order to learn that a particular conjunction of physiological signals, gaze dynamics, and environmental features predicts a class of situations that require slowing down. It learns this through interaction with a system - the human body - that already knows, pre-linguistically, what stress means in the context of driving. The grounding is not synthetic. It is borrowed, and borrowing it is legitimate because the sensorimotor layer it borrows from operates below the threshold of subjective experience, in the domain of measurable, quantifiable biological response.

What This Requires

Implementing this framework at the level of a production system requires three things that are not yet assembled in combination, though each exists in some form.

First, a continuous, low-latency sensorimotor data pipeline from the human driver to the training system. The required sensors - eye tracking, physiological monitoring, postural sensing - are individually mature technologies. Their integration into a vehicle cabin data system, synchronized with the driving event data already captured by the fleet, is an engineering challenge rather than a research problem.

Second, a training architecture that accepts the enriched signal. This means treating the sensorimotor data not as a label applied to a driving event after the fact, but as a continuous co-signal that is processed in parallel with the environmental sensor data. The human’s pre-linguistic appraisal should be temporally aligned with the environmental context that produced it, so that the system can learn the mapping between environmental features and the sensorimotor response they evoke in a skilled human agent.

Third, a transfer mechanism for robot deployment. The representations learned through the human proxy should be extracted and adapted for a different morphology. This is the most research-intensive step, but it is not without precedent. Work in imitation learning and domain adaptation in robotics provides the technical starting points. The specific contribution here is the quality of the source representations: grounded in pre-linguistic human sensorimotor experience rather than in observation alone.

The Larger Claim

Behind the practical proposal is a larger claim about the direction of AI development.

The current path - scale linguistic models, add modalities, expand training data - produces increasingly sophisticated systems that remain, in a fundamental sense, observational. They have never gripped anything. They have never hesitated at a construction zone. They have never felt the pre-linguistic tightening of attention that tells a body something unusual is ahead.

The path proposed here is different. It does not bypass embodiment; it borrows it. It takes the richest embodiment signal available - the continuous sensorimotor experience of a skilled human agent navigating the physical world - and makes it available as a training substrate for a disembodied system. The result is not a system that has a body. It is a system that has learned through a body.

That is not the same as genuine embodiment. But it is the closest approximation currently achievable without waiting for hardware that does not yet exist. And for the specific challenge of FSD and humanoid robotics, it may be close enough to matter - to produce the qualitative difference between a system that observes the world and one that, in a meaningful and measurable sense, has begun to understand it.

The human driver is already in the loop. The question is how much of what they know we are willing to read.

This paper draws on research in ambient intelligence and interface design (Hartmann, 2005), the philosophy of embodied cognition and interactivism (Bickhard, 1983; Gibson, 1979; Brooks, 1991), and the analysis of large language models and the body problem (Hartmann, 2025). It proposes a framework that connects these foundations to the practical challenges of autonomous vehicle development and humanoid robotics.