P&G - Pearls on film – how AI uses cameras in the mouth

Artificial intelligence is all over the news, and seems to be pervading everything. Our dental and periodontal professions must surely be looking at how it will affect oral health.

In this series of interviews, the European Federation of Periodontology is exploring what our partners are doing, or perhaps NOT doing with AI.

In this interview, we talk with Ralf Adam, a Senior Director, Research Fellow at P&G.

What have an electronic toothbrush and putting on a golf course got in common?  Read on and you’ll find out!

Ralf absolutely loves electronic toothbrushes.  His pristine white smile gleams when he starts talking about them with an almost childlike enthusiasm.  However, his message is deadly serious.  AI has really helped P&G develop their Oral B iO toothbrush to help people brush properly and effectively. How? “Dentists typically recommend what we call Bass technique with this kind of 45 degree angle, and then this vibrating movement towards the tip of the tooth” he says, however video analysis research in Germany has found that “basically 0.0% of people are doing that.” It is just too complicated and people don’t do it and this is where AI comes in.  

An Oral B iO toothbrush, depending on the model, comes with motion and pressure sensors and, at the top end, the brush will know where in the mouth the brush is!  “We used AI to enable reliable and precise position detection of the brush in the mouth via monitoring people with video analysis but also another analysis which we call motion tracking. Within the video analysis we monitored more than 4000 people”, explains Ralf Adam. 

So what did this video analysis look at? “Things like pincer grip versus full hand grip and changing left and right hand, which makes it difficult to reliably follow the exact position of the brush in the mouth to say the least. And the other thing is the sizes and the anatomy of our mouths are very much different, so we need to teach the toothbrush where you are at a given point in time and how much time do you spend there and how you evenly distribute that. Such videos had been used to label brushing events and develop algorithms accordingly.  We then developed a method called Motion Tracking, which we validated against video analysis. We published a very nice paper on that one specifically with the new IO brush which we developed.” Ralf’s enthusiasm is infectious! "What is interesting is that pure old fashioned video analysis would take about two hours to observe a two minute brushing. With motion tracking it is instant and very accurate. And this enabled the AI for  very precise position detection, which we use in our App. Your own personal toothbrush with these sensors in it, feeds all that information about YOUR brushing technique to your own app in real time if you opt in, which then feeds back how well you are doing."

Given each mouth is unique, a missing tooth here, a twisted tooth there, this is very clever! Also, PG asks people to volunteer their own app information onto a server which then uses that information anonymously to continually further improve the algorithms that the AI generates. You brush, AI learns! Indeed it is so good that the EFP’s Maurizio Tonetti, former JCP editor recently published a paper in the Journal of Clinical Periodontology where he uses the iO system for his periodontitis maintenance patients.

Ralf Adam says that from this paper “you can clearly see how such AI enabled App Assistant systems can help patients really overcome their shortcomings and really get to a much better treatment outcome as well.” So AI in your domestic toothbrush (albeit a top end model) can help the profession look after patients better in the clinic! What about folk on a more limited budget? 

“What you will see is that we are working heavily on a democratisation of oral health. So we are easing down technologies to a lower level and with time you will have this flow down of technologies,” claims Adam, meaning that this AI innovation will become cheaper as it develops. Where does golf, and putting in particular, come in? Adam explains! “There was a German company showcasing AI and a camera assisted and training system for pro golfers in improving their putting technique, and they had sensors attached to the club as well as to the arm and then showcase how they could enable this movement.

And a guy was smart enough in order to translate that and say this could be an opportunity for us to monitor toothbrushing. So we developed the system with nine different cameras covering the 3D space around people.” He adds with the bit between his teeth, to excuse the pun! “So we very quickly have a individualized and easy going kind of system where we can see the absolute position of the brush head in the mouth throughout the brushing system and then we can determine the XYZ of what they are doing absolutely and can see where their shortcomings are.”

The big question is can brushing your teeth well help with your putting technique?  Probably not!