FPGA-Based Real Time Hand Gesture and AR Marker Recognition and Tracking for Multi Augmented Reality Applications

Abstract

Most Human Computer Interaction (HCI) systems, generally, and Augmented Reality (AR) systems, specifically, are designed based on general purpose processors. Consequently, the power consumption is considerably high as systems work at Gigahertz rates. In this paper, the recognition and tracking processes of hand gestures, and marker based interactive multi applications AR system, are implemented on a low power FPGA to reduce the overall power consumption, by working at lower operating frequencies. Recognition is performed based on shape features, whereas the depth feature, of gestures and markers, was estimated using an ordinary 2D webcam to reduce the power consumption and cost. The most suitable five hand gestures for 3 to 10 year-olds were determined and the FPGA implemented system was, practically, applied on a 100 children. Implementation results revealed that the system can work at up to 102.8 MHz, whereas only 25 MHz are sufficient to achieve real time performance at 30 fps. This, significantly, reduces the power consumption of the implemented system that was compared to other systems. The recognition rate achieved 93.2 %, on average.