Shaking presents for science

After watching people rattle boxes, study participants could accurately predict the shaker’s goals.

Laura Baisas

Published Dec 24, 2023 4:00 PM EST

Understanding how people guess another person’s actions could have future implications for AI. Deposit Photos

This time of year, it’s not unusual to see a family member or a friend get impatient and try to figure out what is inside a wrapped present by shaking it. But what are they trying to figure out? Are they attempting to find out the shape of the present inside or how many objects are in there? A recent study published in Proceedings of the National Academy of Sciences found that it only took observers of the present-shaker a few seconds to tell which information they’re looking for. This research into human cognition and perception could have implications for artificial intelligence in the future.

[Related: Can exercising the mind improve our abilities, or is it just another self-improvement fantasy?]

“Just by looking at how someone’s body is moving, you can tell what they are trying to learn about their environment,” study co-author and Johns Hopkins University cognitive scientist Chaz Firestone said in a statement. “We do this all the time, but there has been very little research on it.”

Pragmatic vs. epistemic actions

Without even realizing it, our brains recognize and analyze another person’s actions multiple times a day. Pragmatic actions include anything that moves a person towards a goal. Our brains analyze these actions to guess which way someone is walking down a street or determine what they’re reaching for. Earlier studies have shown that people can quickly and accurately guess the goal of another person’s pragmatic actions just by observation.

The new study investigates a different kind of behavior consisting of epistemic actions. These kinds of actions are performed when a person is trying to learn something about their surroundings. Epistemic action is dipping your toes into a pool to test out the water temperature or sampling a soup to see if it needs more seasoning.

While pragmatic and epistemic actions are similar, there are some subtle differences. Firestone and the team were curious to see if participants could detect another person’s epistemic goals just by watching them and designed a series of experiments to find out.

What’s in the box?

Researchers asked 500 participants to watch two videos of a person picking up a box full of objects and shaking it. One video showed a person shaking a box to determine the numbers of objects that are inside of it. The other video showed someone shaking the box in order to decipher the shape of the objects inside.

CREDIT: Johns Hopkins University

Almost every participant in the study could tell who was shaking the box to figure out the number of objects and who was shaking to figure out the content’s shape.

“What is surprising to me is how intuitive this is,” study co-author and Johns Hopkins graduate student Sholei Croom said in a statement. “People really can suss out what others are trying to figure out, which shows how we can make these judgments even though what we’re looking at is very noisy and changes from person to person.”

[Related: How you see these shapes may depend on your culture.]

More research into epistemic actions could help engineers develop more anticipatory AI systems that are designed to interact with humans better. In future studies, the team is curious if it is possible to observe epistemic intent versus their pragmatic intent and decipher what is going on in their brain when someone performs an action like sticking your hand out of a window to test the air temperature. They’re also curious it’s possible to build models that detail exactly how observed physical actions reveal epistemic intent.

“When you think about all the mental calculations someone must make to understand what someone else is trying to learn, it’s a remarkably complicated process,” said Firestone. “But our findings show it’s something people do easily. It’s one thing to know where someone is headed or what product they are reaching for, but it’s another thing to infer whether someone is lost or what kind of information they are seeking.”