How your phone recognizes your home: An investigation of mobile object recognition

Thanh Minh Vu


Often what is effortless for a human brain challenges machines the most. Visual recognition, a fairly easy task for humans, can be surprisingly difficult for machines due to variations in angle, size, and lighting. The challenge is amplified on mobile platforms because of computational constraints. There have been a number of studies on image recognition, but few focus on algorithms that run completely on portable devices. With this work, we present an improved image retrieval method that can run on mobile devices in real time without the need to access a remote server, targeting building and poster recognition. One of its possible applications is an electronic tour guide, where users instantly gain detailed information on buildings or posters by taking pictures of them with their phones or tablets. In this work, we designed a fast and robust image matching technique using binary object descriptors. First, since no well-structured database was publicly available, we built new datasets comprised of hundreds of photographs of college buildings and academic posters, taken from a combination of distances and angles. These datasets will be made publicly available. Then, the speed and accuracy of different known keypoint detectors and descriptors were studied to select the best one. Finally, we further optimized the results by filtering best matches, exploiting the user’s location, and extending a grayscale descriptor to include colors. The experiment was done on a real mobile device. The program successfully matched various objects to locally stored sample images with an improved accuracy of 98.5% in less than a second. The extended color descriptor in particular boosted efficiency for poster recognition significantly. Although in this work we focused on buildings and posters, our algorithm could potentially be used in other image recognition algorithms.

Full Text: PDF


  • There are currently no refbacks.