As part of the research project "Use of humanoid robots in customer service", the University of Ulm, in cooperation with Optik Mersmann, developed an innovative, technology-driven addition to the personal sales talk. The aim is to demonstrate that artificial intelligence, humanoid robots and human product specialists complement and support each other perfectly.

Traditional sales processes often reach their limits in terms of data availability and analysis and processing capabilities. In stationary retail, employees usually cannot have all product and customer details at hand by heart. Even if this information were available to service experts (e.g., via a traditional IT system), processing would be almost impossible to manage manually due to the large amount of relevant data and its complexity or would be distorted by subjective personal views. Similarly, online stores reach their limits because they cannot fully access crucial information directly from the physical world (e.g., about customers' physical characteristics or emotional reactions to recommended products). In this context, humanoid robots could combine the benefits of both the digital and the physical world. Robots like Pepper have the ability to perceive and interact with customers in the physical world, and also have the computing power to process the resulting data more objectively, quickly and accurately than humans.

In a specific use case, Pepper took on the task of welcoming hundreds of customers at Optik Mersmann in Ulmer Hafengasse. He made contact with the customers, greeted them and then provided an initial consultation with regard to a suitable shape for the glasses. To do this, he analyzed the individual facial geometry using Deep Learning and determined a more modern or more traditional suitable spectacle shape depending on the customer's wishes. In addition to the general shape of the glasses, pictures of various celebrities wearing glasses that look similar to the customer were also displayed as inspiration. Pepper's ability to interpret basic emotions of the human face and tone and to distinguish between "like and dislike" came in handy during this task. This allowed him to make further improved suggestions to customers if they did not like them. Pepper then passed his customer on to a member of the Optik Mersmann team for spectacle selection and further advice.

Cooperation partner: Optik Mersmann, Ulm, Germany

Project period: May 2020 - July 2020