It can be difficult for blind and visually impaired people to tell whether food is still good or if mold is beginning to form on it. The new bMyVision v2.0 app is designed to help them; it uses a smartphone’s camera to recognize objects and their characteristics. One of its features is the detection of mold on food. Mendel University in Brno is collaborating with the Technical University of Liberec and TyfloCentrum Liberec on the development of the system and the creation of the dataset.
MENDELU is involved in the app’s development through the Department of Food Technology, which assists with the expert evaluation of food photographs and the expansion of the database used by the system to learn how to recognize mold. Using the app is simple: the user places the food in a suitable spot and takes a photo using a smartphone with a speaker. The app then analyzes the image and uses voice feedback to indicate whether the food shows signs of mold or is safe to eat. The feature is designed for use in the home and is intended for blind and visually impaired users.
“For a person with a visual impairment, it can be very difficult to assess the condition of food, especially in the case of early-stage mold. Our goal is therefore to use our expertise to help develop a tool that will serve as a practical aid in everyday life. It is in projects like these that we see the purpose of science—when research results do not remain confined to academic publications but find concrete practical applications and can truly help people,” said Andrea Roztočilová from the Department of Food Technology at MENDELU.
Currently, the development team led by Josef Chaloupka has approximately 1,400 photographs at its disposal. However, to achieve more reliable results, the database would need to be significantly expanded. The developers estimate that approximately 10,000 photographs would be needed to ensure robust results from the recognition model. The more diverse the set of images available, the better the system will be able to handle the various forms and stages of food spoilage.
The public can also participate in the photo collection. People can take photos from home of food items showing signs of mold and send the images via email to poradna.potraviny@mendelu.cz or bmyvision@seznam.cz. According to experts, the diversity of the photos is particularly valuable. This is because the system must learn to recognize mold on different foods, in different environments, and at various stages of its development.
Experts from the Department of Food Technology at MENDELU want to focus primarily on baked goods in the first phase of the collaboration. With baked goods, it can be difficult for the system to distinguish emerging mold from other surface structures, such as sugar. In the future, however, it will be possible to expand detection to other types of food.
BMyVision also offers other features designed for people with visual impairments. For example, it can recognize expiration dates on food, the value of coins and bills, human faces, or objects in the vicinity. For outdoor use, the developers are also working on detecting obstacles, crosswalks, and dog feces, which can be useful for blind people who own a service dog.
The system will also include a special control ring and a camera mounted on the glasses. The ring will allow the user to discreetly switch between different functions depending on what information they need at any given moment. The goal is to prevent the user from being overwhelmed by information that is not relevant to them at that moment. Furthermore, the entire program is being developed to be multilingual; in the system settings, users can choose whether the system should communicate with a blind person in Czech or English. In the future, it will be possible to add additional languages.
An important part of the project is the involvement of the future users themselves. Through Ivana Valecká, TyfloCentrum Liberec provides developers with feedback from blind and visually impaired people and helps assess the clarity and intuitiveness of the controls. The organization also collaborates with instructors who teach blind and visually impaired people how to use modern technologies.
“We greatly appreciate the opportunity to draw on the expertise of the Institute of Food Technology in developing an app that addresses the specific needs of people with visual impairments. We consider it exceptionally valuable that the Czech development team is creating the app in close contact with its future users and listening to their experiences and needs. Thanks to this, the solution is not based solely on assumptions, but rather a tool is being created that can have a real impact on everyday life and will be accessible to people throughout the Czech Republic. For our institute, this collaboration also represents an opportunity to address a topic we would like to focus on over the long term through the MENDELU Food Advisory Service—making the field of food and nutrition accessible to people with visual impairments. In the future, we would therefore like to build on this collaboration with additional projects that can help this target group navigate the selection and purchase of food, assess its quality and safety, and ensure proper storage and subsequent preparation. This is precisely where we see an area where science and its findings can provide solutions with a direct impact on the quality of everyday life in practice,” added Roztočilová.
A working prototype of the app currently exists and will undergo practical testing starting this September. The developers plan to first make the system available for iPhones via the App Store, and later for Android phones as well. The app is expected to launch on the App Store in late 2026. The app performs recognition directly on the device and does not send captured images to third parties over the internet during this process. The goal of the project is to create a practical tool that will be available to people with visual impairments throughout the Czech Republic and that will be based on their actual needs.
More information about the BMyVision project is available on the project’s website.
The entire development of the app is being carried out thanks to the project: Challenges for the 21st Century at TUL (CZ.02.02.XX/00/23_022/0008721)