You are, virtually, what your smartphone says you eat

If you have a wearable fitness tracker – and there are a lot of people who do (Fitbit, one of the market leaders, had over 50 million registered users at the end of 2016) – then you’ll know that the application associated with the tracker itself also enables logging and analysis of other parameters such as what you weigh, your body fat index, and what you eat and drink (so that your calorie intake can be tracked as well as you calorie usage through exercise).

But it is really awkward to record details of food consumption. There’s the issue of portion size, of course, and the difficulty of having to enter separate details for the components of what’s on your plate. I can’t find numbers, but I strongly suspect that very, very few people make use of the apps’ food logging features.

Artificial intelligence for food recognition

Could AI, visual recognition, miniaturised sensors and the cloud change this? The idea of food recognition has been around for a long time. I remember seeing a demo at least 15 years ago of an image-processing webapp that promised to analyse what was in a photograph of your dinner plate. But advances in key technologies may deliver a step change in usability and accuracy of such approaches.

Two companies currently active in this area were demonstrating their work in Barcelona in February, at MWC and Pepcom’s start-up show. The heavyweight contender was ARM – the chip developer that is now part of Japan’s giant telecoms company SoftBank. It has developed a library of computer vision and machine learning software for its CPUs and GPUs that would help developers of food recognition apps. ARM demonstrated just such an application, alongside developer ThunderSoft. An app running on a Huawei P9 smartphone was successfully able to differentiate between a number of discrete foods (pumpkin seeds, sliced apple, etc), and provide an estimate of the calorie count of the specific plateful.

At the other end of the corporate weighing scale is Consumer Physics, an Israel-headquartered sensor developer, which was demonstrating a similar food recognition application using a smartphone from Chinese OEM Changhong, and silicon from Analog Devices. But while the application is similar to that of ARM, and both use a smartphone to provide computing power and the human interface, the principles are quite different. Consumer Physics doesn’t use the smartphone’s camera or cameras, but employs instead its SCiO ‘molecular sensor’ – a device that emits and analyses near infra-red (NIR) light using a miniaturised spectrometer to determine the chemical composition of a scanned object, and hence its nutritional value. The spectrometer output is processed and matched in the cloud against a database of known substances.

Consumer applications help technology develop

Both approaches are, of course, capable of delivering much more than simply a way to track your food intake quickly, but the reason for choosing to demonstrate the technologies with this application is clear: consumer use cases help drive volume, and higher volumes reduce costs and open up new markets. This is particularly important for a relatively new business like Consumer Physics.  By persuading smartphone makers to embed it in their products, it could support a large number of health and wellbeing applications ranging from identification of medications to body fat measurement.

But this is a tough challenge to overcome. Space on smartphones for new sensors is extremely limited, and makers are reluctant to commit unless a real market need – and a business model – can be demonstrated.

Even the lower-risk, software-only approach being pursued by ARM and its partners is not guaranteed to work, even though ARM intends to makes its libraries open source. Software consumes resources on a smartphone, if not physical space, and there may be royalties to pay to switch on enabling technologies on the chips. My view is that we won’t be seeing these technologies becoming widespread on phones any time soon.

[Hamburger photo credit: ericd/Wikimedia Commons, CC BY-SA 3.0 license]

Hot topics:

Big Innovator content type:

Add this: