Japanese Supermarkets Are Using AI to Monitor Employees’ Smiles Because They Want Customers to Be Completely Satisfied

Imagine having one of those blue days when few things make you smile. Well, you still have to go to work that day and try to make the best of it. You don't have to smile unless you work in an AEON supermarket in Japan. The reason? The owners have installed an artificial intelligence system to rate their employees' smiles. Not only that, but it also encourages them to compete.

This “initiative” obviously reopens the debate about the treatment and harassment of workers in Japan.

Gamifying a grimace. According to the South China Morning Post, AEON announced on July 1 that it would become the first company in the world to promote an artificial intelligence system called “Mr. Smile” to measure the smiles of employees at its 240 stores nationwide.

The software uses AI to monitor employees’ smiles, facial expressions, voice volume, and tone when greeting and interacting with customers. The system has more than 450 parameters and scores the employee. But AEON goes further: By using Mr. Smile and its gamification elements, it encourages employees to compete against each other and with their records.

Harassment controversy. According to a test conducted in eight of its stores, the company claims that employee attitude improved by as much as 1.6 times in three months. Its goal is to “standardize employee smiles and maximize customer satisfaction.” Of course, that has repercussions in a country that has had to mediate the treatment of workers by some customers. In a survey of 30,000 workers affiliated with one of Japan’s largest unions this year, nearly half reported being harassed by customers.

However, critics of the smile system claim that “forcing service workers to smile according to a standard is another form of customer harassment.” Another critic said, “A smile should be beautiful and sincere, not treated as a commodity.” A third one echoed this sentiment, noting that “using a machine to standardize people's attitude sounds cold and silly.”

Baristaeye. Mr. Smile isn’t a new system. In May 2023, the company NeuroSpot presented Baristaeye, a tool for monitoring employees and customers in coffee shops. It uses artificial intelligence to monitor everything that happens in a coffee shop. According to the product description, it allows owners to “to monitor your baristas’ actions in real-time, ensuring they adhere to proper procedures and prevent potential fraud cases.”

It also analyzes customer behavior, including dwell time, satisfaction levels, and popular visit times. It does all of this with instant alerts when “customer dissatisfaction is detected, allowing the management team to quickly address issues and maintain a positive customer experience.”

Smiles at zero yen. The smiles delivery service isn’t a new initiative. In Japan, people have been able to request a free smile at McDonald’s for decades. The fast-food chain has 3,000 locations nationwide with an estimated 100 million customers. As part of the menu, consumers can request a smile. The price is zero yen, but the employees will smile at you because they have to. Over the years, however, some customers have abused the service, causing more stress for the employees. Some demanded the smiles disrespectfully or even asked the staff to smile repeatedly.

Such behavior became so notorious that McDonald’s launched the “No Smiles” campaign. It even created a song that reflected the employees’ feelings, but it was just part of the marketing campaign to attract more workers. Why would it do that? Well, Generation Z represents 60% of McDonald’s workforce, and they don’t want to work in a company where customers harass employees for things like smiling. It worked: The song went viral, and the company saw a 115% increase in applications.

Smiles at home. Customers could even order smiles at home. The SoraNews outlet confirmed it with an experiment. It ordered two rounds of hamburgers and discovered something interesting: Whether the team asked for a smile or not, the delivery girl smiled at them when she delivered the package. The difference was that she had drawn a smile emoji and messages like “thank you” or “enjoy your meal.”

This article was written by Alejandro Alcoea and originally published in Spanish on Xataka.

Image | Abasaa日本語: あばさ

Related | In Japan, People Can’t Stop Shopping, But There’s No One to Transport the Orders. The Solution Is an Epic 310-Mile Conveyor Belt