Educational and Interactive Plant Recognition for Smartphone Software

Informations

Funding country

France

Acronym

ReVeRIES

URL

-

Start date

1/1/2016

End date

-

Budget

643,464 EUR

Fundings

Abstract

The urbanization of society has gradually separated humans from the plant world. Most people have forgotten the names of plants and their potential uses. Yet there is a growing awareness that biodiversity is a treasure we must preserve and transmit to future generations. The identification of plant species is a necessary step to understand our environment. However, for most people, botany remains difficult to understand and to learn. It is not easy to decrypt botanical literature because it requires a solid theoretical background. In the ReVeRIES project (French acronym that means “dreams” and that stands for Interactive, Fun and Educational Plant Recognition on Smartphones), we propose to use mobile technologies in order to help humans recognize plants that surround them. We believe that a promising way to recreate the relationship between modern human beings and their natural environment is to provide smartphone applications that help them recognize and learn about plants. The ReVeRIES project relies on a mobile application called Folia and developed during the ANR ReVeS project. This application is capable of recognizing species of trees and shrubs (taller than 1m20 and originating from France) by analyzing photos of their leaves. This prototype simulates the behavior of a botanist when trying to determine the plant species, which makes it different from all the other tools available on the market. In the context of ReVeRIES, we propose to go much further by developing the following aspects: game-based mobile learning, multimodal images recognition and citizen sciences. First of all, we intend to design mobile learning games that will help users learn about plant characteristics and especially learn the methods, used by expert botanists, to recognize plant families, genera and species. In order to motivate children and botanical neophytes to learn about plants and explore their natural environment, we also intend to use game mechanics for creating fun activities based on plant recognition. The users will be able to improve their skills by comparing their results to those found by the recognition algorithm. Concerning the image recognition process, we intend to extend the previous prototype to the main exotic woody trees and shrubs. Moreover, we aim to take into account various organs of the plant. This multimodality is essential if we want users to learn and practice the correct recognition method, for which botanists use a variety of organs (i.e. leaf, bark, size of plant, flower, fruit, etc.). In addition, the use of organs should greatly improve the algorithm’s accuracy. In terms of image processing, the work done on the leaves cannot be extended directly to flowers, fruits and barks. This will greatly increase the complexity of the data fusion process. Finally, we intend to explore ways in order to enhance social awareness of our natural resources and to support citizen science. The geolocated photos and information taken with the application and validated by experts, could be transferred to specialized networks, such as Tela Botanica, integrated into the OpenStreetMap geographic information system and mobilized by local institutions to support actions and projects involving citizens. This addresses problems related to the field of Volunteered Geographical Information. The project raises many scientific challenges in TEL (Technology Enhanced Learning), Serious Game, image analysis, data fusion, HCI, and also in the field of collaborative environmental inventories. The possible impacts are numerous: teaching of botany at different levels and with various learning audiences, collective intelligence, citizen sciences, nature preservation and environmental collaborative games. In addition to citizens interested in nature, this system could be very useful for teachers and their students, botanists and also nature parks.