HUMAN ROBOT INTERACTION WITH CLOUD ASSISTED VOICE CONTROL AND VISION SYSTEM

Abstract

The objective of this research is to investigate a way of interaction between humans and robots, which is through voice or speech commands. A Biomimetic Artificial Hand (BAH) is used as a platform to perform grasping tasks using human voice as an interacting and instructing medium between humans and robots. It is a hands-free approach of issuing commands to the BAH since it does not require the user to wear any specialized equipment. Previous research has shown difficulties in recognizing more than one word, database management for stored voice, and requirement of sufficient computing power. National Instruments software LabVIEW and hardware myRIO are used as the interface between the user and BAH. The concept of using cloud application services is applied, which is based on using the speech recognition Application Program Interface (API) by Microsoft which accepts a verbal command, transfers to the cloud for further processing and returns the command in string (text) form. This approach reduces the use of local computing power requirements and yields fast and accurate speech recognition (SR). A vision system is also incorporated as a a safety feature to verify the presence of the correct object in the workspace. The string results returned from the API is further locally processed to identify the action to perform, object, object identifiers (number, color, size) and grasping pattern of object from the existing database. Voice command evaluation performed on the hardware platform with a biomimetic artificial hand indicates that the proposed interaction modality could be advantageously employed for successfully instructing or interacting with a robotic device.