#5: Why Mandatory Employee Smile Testing Will Backfire
Requiring employees to smile before they work will backfire, and makes these three incorrect assumptions about humans
Welcome to the latest edition of Etcetera. Last month I took a break to focus on my impending PhD deadline! Welcome back. This month I take a look at a new phenomenon in the workplace - mandatory smiling to enter the office, as fuelled by AI facial expression analysis. Here is my critique as to why the decision to implement this technology is based on three incorrect assumptions, and why it will backfire.
Canon China, a subsidiary of the Japanese camera and electronics giant, launched a new initiative to “create a positive [working] atmosphere”. The initiative monitors employees through cameras as they enter the building, and uses facial expression analysis algorithms to permit them to pass security and book meeting rooms, only once they have smiled.
My research extensively uses facial expression analysis AI, so I am familiar with the benefits — and shortcomings — of utilising this technology within the context of organisations. That’s why I am very sceptical when I read about Canon’s use of this AI. How can an organisation go so far wrong in their attempt to create a better working environment?
A spokesperson for Canon China detailed their reasoning to The Financial Times: “We have been wanting to encourage employees to create a positive atmosphere by utilising this system with the smile detection setting ‘on’. Mostly, people are just too shy to smile, but once they get used to smiles in the office, they just keep their smiles without the system which created a positive and lively atmosphere.”
You could take their justification for the technology at face value. But I see three potential justifications for implementing this ‘smile AI’. First, the most sinister justification is that Canon are not interested in increasing employee wellbeing at all, and this software is part of a wider trend of mass surveillance of employees. Worker surveillance has only increased throughout covid, with more employers reporting using software to monitor remote workers.
A second justification is that Canon thinks that this smile AI will make them appear to prioritise wellbeing. If so, this is an exact example of a phenomenon I wrote about in the last newsletter called “wellwashing”. Wellwashing occurs when companies spend more resources on creating tokenistic initiatives that make them appear to prioritise wellbeing, rather than creating systemic changes to practices and cultures that ensure employee wellbeing is prioritised.
Third, maybe the HR department and leaders at Canon simply do not understand human behaviour or psychology at all. An in their misunderstanding, they have made three flawed assumptions about humans, and the implications of this technology on a human workforce.
The three incorrect assumptions Canon made about their workers
Assumption one: Our emotions can be changed after changing our facial expressions
I’m sure we have all remember that one study which showed that holding a pencil in our mouths (thus forcing a smile) will make us happier. I remember trying it in high school when I was particularly bored. The only problem with that study is that now it has been replicated, scientists have realised it doesn’t hold up. The first study had methodological issues which rendered it erroneous.
The unfortunate reality for Canon is that forcing someone to smile simply does not change their emotions. Science backs this up. Take for example the phenomenon known as surface acting. Surface acting is when you feel one emotion and attempt to express another.
Surface acting has a negative effect on employees, increasing harmful behaviour towards coworkers, and increasing emotional exhaustion. In other words, not only does Canon’s new initiative have no proven efficacy in creating a more positive work environment, it actually threatens the wellbeing of their employees.
Surface acting is a known form of emotional labour. Emotional labour is “the process of managing feelings and expressions to fulfil the emotional requirements of a job”. Increasing the emotional labour required to do a job increases the stress and decreases the enjoyment, leading to higher rates of burnout. Many workers are already noting these effects: “So now the companies are not only manipulating our time but also our emotions,” one user said on Weibo.
The assumption that making workers smile more will make them happier is very unproductive in increasing well-being in general.
Assumption two: We can improve employee wellbeing through technology only
As I noted in a recent article about how AI alone won’t fix complicated real-world problems, leaders in both business and politics suffer from a shared common bias. They are too quick to dream of using AI to fix a range of problems. They assume the why of a problem, without doing any user research.
E.g. Canon has assumed their workers aren’t happy and would be happier if they smile more. Rather, the leaders should invest in proper employee engagement research and actually ask:
1) Are our employees happy?
2) If not, why not?
3) What can we do to foster a positive work environment.
Of course, there are technological tools that can help to improve employee wellbeing. But moreover, company-funded smile cameras don’t substitute a living wage, dignified working conditions, meaningful work, adequate sick & holiday pay. If a company is serious about creating a positive working environment, then more autonomy, shorter working hours and higher wages should do the trick. Although that is harder than implementing smile technology.
Assumption three: Facial expressions reliably show our emotions
A core question within the scientific community is ‘do facial expressions reliably communicate emotions?’. An influential psychologist called Paul Ekman developed a coding system for facial expressions in the 1960s and 1970’s, called the Facial Action Coding System. This coding system, shown below is the basis for all facial expression analysis algorithms.
An embedded assumption in Ekman’s coding system was that facial expressions are universal. While the questions of the validity of his coding system are widespread, it hasn’t stopped companies implementing that facial expression analysis AI to ‘see emotions’ in job applicants, consumers, and now employees. Yet, the obvious problem is that people can fake emotions, and equally, they can experience emotions without moving their faces. Facial expressions don’t always tell the whole story. So to believe that we always emote what we truly feel is a flawed assumption. Hence, asking employees to smile because we think they are happy to be at work is not an effective way of making a happy work culture.
Ultimately, it is a tempting proposition to believe there is a quick AI fix to all of our human problems. But our emotions are not a problem, especially not at work. If employees in our companies are profoundly unhappy, it is worth asking the big questions to establish why. Making workers smile to get into work, and surface act throughout their day to hide their low levels of motivation and discontent will not create a better working environment. If people are unhappy, yet forced to hide it, they will find other ways to level the playing field like working slower and reducing their productivity. We are powered by emotions, and instead of seeing them as an inconvenience to be managed — let’s understand that when the working conditions are right, our emotions can increase our creativity, productivity and dedication to our work. Sadly for Canon, smile technology won’t do the trick.
Thank you for being one of the readers who read my writing in July and August. Last week I was selected to be part of the Medium writer’s fellowship. This is a great opportunity to expand my readership and receive a guaranteed minimum monthly income. If you have any suggestions of places to donate part of the profit made from my readers, please let me know. If you’d like to read more (or less) of something, comment!
Some things I consumed in July:
A lot of academic articles! Hopefully, my reading habits will get more interesting post thesis submission!
If you are new here and haven’t subscribed, you can receive next month’s newsletter straight to your inbox.