Insights

What is Facial Activity Intelligence?

16 October 2026 · Charles Nduka · 5 min read

There’s tremendous interest in world models, Physical Intelligence and robotics. However, the human aspect of AI, and the preference data that drives behavior is rarely considered. Facial Activity Intelligence (FAI) is AI that learns from how people respond to the world. It reads the activity of the face, head and eyes as people work, eat, watch, and interact, then pairs that activity with the moment that caused it. The result is a continuous, in-context record of the inputs and outputs of daily life.

AI is only as good as the feedback it learns from

AI labs tune today’s most capable models on human feedback: rankings, ratings, a thumbs up or down. That feedback is sparse and deliberate. People give it after the fact, often for outputs meant for someone else.

Every person already carries a richer feedback channel. The face registers effort, surprise, error and enjoyment moment to moment, in context, whether anyone asks or not. Almost all of it goes unrecorded. Your AI agent speaks, but is tone-deaf to the meaning of your silent response.

How Facial Activity Intelligence is different

The signals a model learns from shape what it can do. Language models learn from what people have written. Preference-tuned models learn from which answer people pick when asked to compare two. Facial Activity Intelligence learns from how people respond while an experience unfolds.

Language AI

Learns from

What people write

When the signal arrives

Long before the model answers

Where it comes from

Text on the web

Preference AI

Learns from

What other people rate

When the signal arrives

After the output, when asked

Where it comes from

An average rater at a desk

Facial Activity Intelligence

Learns from

How people respond

When the signal arrives

During the experience

Where it comes from

The person’s own day

Facial Activity Intelligence is not facial coding

Facial Activity Intelligence goes far beyond mere facial coding via a webcam, or smartphone. Those methods assume a universal model of expression. Also, the face is judged in isolation, ignoring the person’s context. FAI learns from face, head and eye activity in real time, in the places people actually live and work, in everyday life.

Sense glasses do this with inward-facing sensors in the frame. The sensors point only at the wearer, and save no facial images or audio, so bystanders keep their privacy. The sensors also pick up what a camera cannot: a clenched jaw, a micro-expression too subtle to show, a face turned away. Critically, it incorporates the context of what the wearer was doing when their response happened.

The paper that taught AI to learn from people

In 2017, researchers from OpenAI and DeepMind published Deep Reinforcement Learning from Human Preferences. They tackled an old problem. Writing down exactly what you want from a complex system is hard, and a goal that is slightly wrong produces the wrong behavior.

Their answer was to let people show the goal. Raters watched pairs of short clips of an agent and picked the better one.

A reward model learned the goal from those choices, and the agent learned to satisfy it. With about an hour of human time, agents learned behaviors more complex than any previously learned from human feedback.

Five years later, OpenAI applied the same recipe to language in Training Language Models to Follow Instructions with Human Feedback. Labelers ranked a model’s answers, a reward model learned from the rankings, and reinforcement learning tuned GPT-3 against it. People preferred the 1.3-billion-parameter InstructGPT to the 175-billion-parameter GPT-3, a model more than 100 times its size.

That method, reinforcement learning from human feedback (RLHF), became the standard way to turn a capable model into a helpful assistant.

What asked-for preference misses

RLHF made human preference the target. But the target can only be as good as the preference data, and teams collect most of that data one way: a rater at a desk, comparing finished answers to prompts someone else wrote.

Asked-for preference also carries known biases. Towards Understanding Sycophancy in Language Models found that preference judgments favor answers that match the reader’s views. People and preference models sometimes picked a convincing sycophantic answer over a correct one.

The more commonly known biases are the memory bias (the Peak-End rule) described by Daniel Kahneman, and social masking: you smiled at the waiter and said the meal was delicious, but your face during the meal said otherwise.

Stated preference, gathered away from the moment of use, is a noisy guide to what actually helps a person, or what they like.

The face supplies the missing preference data

Facial Activity Intelligence captures preference as response, at the moment of use, in the setting where it occurred. So the AI’s sentence that confused, the instruction that took effort, or the answer that surprised are judged in context. Every interaction becomes labeled preference data, with no one stopping to tap a rating. Importantly, even a neutral face in response to an experience might be speaking volumes depending on the context.

Sense takes biometric sensing out of the lab and into glasses worn through the day. Here is what in-context preference data adds to RLHF:

  • Continuous. Response runs across the whole output, so a model learns which part worked.

  • Specific. Each response carries its task, time, setting and the person’s state. A tired reader at 4 pm and a fresh one at 9 am give different evidence.

  • Personal. Per-person baselines build over weeks, so a model can learn the preferences of the person it serves.

  • Effortless. People give the signal by using the product: no survey, no interruption, no extra task.

  • Cleaner labels. Where explicit ratings still matter, FAI flags when a rater is fatigued or drifting, so teams know which labels to trust.

From response to action: the AI agent opportunity

Facial Activity Intelligence will power agents that adapt to the person using them. Intelligence moves out of the prompt box and into the moment of use. An AI agent makes contextually tuned recommendations based on current bodily signals rather than last year’s chats:

  • An assistant that learns which responses are confusing or frustrating

  • A notification that is snoozed when your face shows deep focus

  • A food or travel recommendation based on what your behavior says you actually like

  • Entertainment recommendations based on measures of real people in the real world

Introducing Sense, the platform for Facial Activity Intelligence

Sense glasses carry inward-facing optical flow sensors and a motion sensor in the frame. They measure face, head and eye activity continuously and encode it as facial activity markers. The Sense API exposes those token-like markers as interpretable measures of effort, error, valence and engagement, with no single black-box score.

Put Facial Activity Intelligence to work

Talk to our team about adding in-context preference data to your models, products and research.

Contact us →

Sources

  1. Kiprijanovska, I., Stankoski, S., Broulidakis, M. J., Archer, J., Fatoorechi, M., Gjoreski, M., Nduka, C. and Gjoreski, H. (2023). Towards smart glasses for facial expression recognition using OMG and machine learning. Scientific Reports, 13.

  2. Christiano, P., Leike, J., Brown, T. B., Martic, M., Legg, S. and Amodei, D. (2017). Deep reinforcement learning from human preferences. NeurIPS 2017.

  3. Ouyang, L. et al. (2022). Training language models to follow instructions with human feedback. NeurIPS 2022.

  4. Sharma, M. et al. (2024). Towards understanding sycophancy in language models. ICLR 2024.

  5. Kahneman, D., Fredrickson, B. L., Schreiber, C. A. and Redelmeier, D. A. (1993). When more pain is preferred to less: Adding a better end. Psychological Science, 4(6), 401-405.

← All insights