RePERO is a smartphone app that allows the user to take a picture of his or her tongue with a smartphone camera and then employs AI to analyze the image and assess the user’s risk of bad breath on a 1-5 scale. Because bad breath is a delicate topic, Lion utilized a SNS service to elicit people’s true concerns. People wrote that even after using breath-care products, they still worried about whether their breath had actually improved. Lion realized that an opportunity for providing new value lay in being able to remove that anxiety by showing that the user’s breath really had improved.
There is a halitosis-check product already on the market, but it is inconvenient to carry around and, furthermore, only describes the halitosis risk without offering specific information for addressing it. Lion conceived of a novel approach to determining halitosis risk by using images of the tongue and ultimately decided to develop a smartphone app for this approach, first because people always have their smartphones with them, and second because an app would be able to provide care information specific to the issues revealed by the halitosis check.
The application’s development was strongly supported by Lion's knowledge that "the coating on the tongue causes bad breath”. The coating on the tongue is a kind of grime that derives from a combination of exfoliated mucosal epithelial cells and food debris, and it causes bad breath when metabolized by bacteria in the oral cavity. Working from the assumption that there must be a correlation between the specific state of the tongue coating (adhesion amount, thickness, color, etc.) and the level of bad breath, the Lion researcher who initiated the idea began developing a smartphone app that would “determine the risk of bad breath from photographs of the tongue taken with a smartphone.”
He began by photographing his own tongue at various points during the day, for example after eating or brushing his teeth, and created a data set of images and halitosis-level values for each of these times. When the color of the tongue was analyzed with image editing software, the results correlated with expected halitosis levels. He continued to collect and analyze more data sets until he was able to predict halitosis levels from the tongue images to a high degree of accuracy. He devised an algorithm for determining the risk of bad breath and completed a simplified version of the bad-breath risk-check application.
At the test stage of the simplified application, the developer encountered an obstacle, namely that the color of the photo depended on which model of smartphone was used. However, an AI workshop he happened to attend offered a hint to a possible solution, suggesting that AI would be able to discern hidden, subliminal features in the photographs and, accordingly, to assess the risk of bad breath without being confused by the differences in color between smartphone models. He immediately reached out to the host of the seminar, Fujitsu Cloud Technologies, and the collaboration between the two began.
Since thousands of data sets are required for AI image learning, he collected data by photographing the same tongue using different smartphone models and levels of ambient brightness, or in other words, in different environments. This taught the AI that even the same tongue could look different depending on the conditions in which it was photographed. The AI was able to resolve the differences between models within about a week, and, three months later, Lion developed the halitosis-care support app RePERO. It has received evaluations like “easy and good” and “innovative and interesting” by the companies that have introduced it. Comments from their staff members include, “I have less anxiety about halitosis” and “I am now more aware of and interested in taking care of bad breath.”
The combination of the AI technology of Fujitsu Cloud Technologies and Lion’s data and knowledge base produced a smartphone app that can ascertain the risk of bad breath through photographed images regardless of variations in smartphone model or photographic environment.
Several factors contributed to the success of the development. First, the person in charge of development, feeling that it would take years to become expert in the necessary AI, did not hesitate to seek the cooperation of a company that specializes in AI. Second, he had already delved into the technology and knowledge base enough to have created a simple version of the app. This allowed him to participate as an equal in discussions with the person in charge of the AI development and to convey accurately what Lion was seeking. This was not simply a case of outsourcing. During the process of developing the app, which was new to Lion, he also paid close attention to protecting intellectual property rights when collaborating with other companies. Furthermore, early on he built systems for close cooperation with the company’s internal offices related to intellectual property rights and contracts.
The project was a success both because of the openness with which it dealt with customers in collaboration with other companies and because of the close internal collaboration that enabled in-house development of the app thanks to robust cooperation from other parts of the company. Lion will use the experience it has gained here in its future development of new businesses.
Back to Science Journal Menu