SlideShare a Scribd company logo

published paper

•
•
K
Khushboo Vatsal Desai

This document describes a voicebox system for mute people that uses flex sensors, an accelerometer, and wireless transmission to recognize hand gestures and provide emergency messaging. The system includes a transmitter glove fitted with flex sensors to detect gestures. An accelerometer detects falls and triggers an emergency SMS. The gestures and signals are sent wirelessly and received by a microcontroller connected to a voice module that plays pre-recorded messages corresponding to the gestures and signals. The system aims to help mute people communicate through translated hand gestures.

1 of 7
Download to read offline
International Conference on: “Engineering: Issues, opportunities and Challenges for Development” ISBN: 978-81-929339-1-7 11th April, 2015, S.N. Patel Institute of Technology & Research Centre, Umrakh, Bardoli 1 VOICEBOX FOR THE MUTE PEOPLE WITH EMERGENCY MESSAGE SERVICE Khushbu Parikh1, Khushboo Desai2, Avani Arya3, Kaushika Patel4 Final year BE Student, electronics Department, BVM Engineering College, V.V.Nagar, Gujarat, India1 Final year BE Student, electronics Department, BVM Engineering College, V.V.Nagar, Gujarat, India2 Final year BE Student, electronics Department, BVM Engineering College, V.V.Nagar, Gujarat, India3 Assistant Professor, electronics Department, BVM Engineering College, V.V.Nagar, Gujarat, India4 kparikh06@gmail.com1 khushboodesai.56@gmail.com2 avaniaryastar@gmail.com3 kdpatel@bvmengineering.ac.in4 Abstract—Embedded systems now-a-days are very popular with emerging technologies that reduces the communication gap and gives a space where one can design and develop a device where communication between ordinary people and dumb people are assisted[3]. This proposed system is mainly designed to assist dumb people to communicate with ordinary people. There are mainly four units which are covered in the work, they are voice module unit, sensing unit, wireless transmission and processing unit. A digital glove is used which is normal glove, fitted with flex sensors. Gesture is recognized and the process is done in LPC2148 ARM controller, the corresponding signal is sent via zigbee module which is received by 89V51 microcontroller. According to the received signal, pre-recorded message which is stored already in APR9600 voice module is played. Flex sensors and APR9600 are the key modules which are going to be used in designing. Accelerometer is used to sense the fall detection and if it is beyond the threshold, then a predefined message is sent via GSM module to a predefined number. Mute people can use these gloves to perform hand gesture and it will be converted into speech so that normal people can understand their expression. Fall detection mechanism is implemented as an additional feature in this work. Keywords— ARM, Accelerometer, APR9600, Flex sensor, GSM module, LPC2148, Zigbee, 89V51 microcontroller I. INTRODUCTION Embedded system has changed the lifestyle of people in a very smart way [3]. Communication becomes effective when all are using same language for communicating. Sign language is a natural way to communicate between mute and normal people [1]. In order to improvise a life style of mute people the system is developed. There are two ways of conveying message by a dumb person; they are physical and non-physical [4]. Non-physical includes movement of head, facial expressions and physical communication includes the hand movement which is normally a sign language. Sign language is different in different countries as all are having different languages, and every sign has its own different meaning.
International Conference on: “Engineering: Issues, opportunities and Challenges for Development” ISBN: 978-81-929339-1-7 11th April, 2015, S.N. Patel Institute of Technology & Research Centre, Umrakh, Bardoli 2 The proposed system is a translator which includes two sides, transmitter and receiver. In transmitter side, flex sensors are attached with a glove. Data are obtained directly from each sensor’s degree of bending. Controller senses the gesture and decides which signal is to be sent to the receiver side. Signals are transmitted wirelessly via zigbee module. Accelerometer is attached to the transmitter device which is going to be with the disable person. Accelerometer is used to sense the movement. When a person falls abruptly, GSM module will send SMS to the receiver mobile handset. At the receiver side, Signals which are fetched are processed and the corresponding command is given to the voice module. Voice module will play the prerecorded message as per the command received. II. LITERATURE SURVEY Priyanka R. Potdar has developed a system which uses ARM7 microcontroller which senses the gesture and plays the voice according to it and text is also displayed on LCD module. A SD card is used which stores the data. An audio amplifier is there as a voice generating unit. The work is useful for mute and deaf both. The system is very compact and flexible to user [1]. Naveen kumar and Chandni have developed a system where a camera reads a movement of a human body and sends data to computer. Gesture recognition system is used to assist impaired people to interact with computer. Keyboard and joystick allows the body to give signals to computer through gestures such as pointing to screen [2]. Gunasekaran and Manikandan have developed a system where they have used PIC controller at both transmitter and receiver side. RF module is used as a wireless medium. Flex sensors are used to sense finger movement and APR9600 as a voice module which plays voice according to received signals[3]. III. SYSTEM DESCRIPTION The design starts after the analysis of the requirements for the system development. Hardware at the transmitter side is an LPC2148 ARM7 microcontroller which has two UARTs on which GSM and ZIGBEE Transmitter has been connected, two inbuilt 10-bit ADCs (In all, 14 channels are available to which 14 analogue devices can be connected), High speed USB 2.0 utilised for loading the HEX File of the program code into the controller and up to nine external interrupts [6]. Flex sensors are variable analogue resistors that generally works as variable voltage dividers. On the transmitter side, 3 flex sensors have been used for gesture recognition connected to 3 ADC channels of ARM7. They are usually in the form of a thin strip with length of 1”-5” and width of 0.25”and thickness up to 0.19”. Resistance varies as the size changes [1].
International Conference on: “Engineering: Issues, opportunities and Challenges for Development” ISBN: 978-81-929339-1-7 11th April, 2015, S.N. Patel Institute of Technology & Research Centre, Umrakh, Bardoli 3 Accelerometer (having three axes) is interfaced with 3 ADC channels for fall detection which will be sensed by ARM7. If that is detected to be above threshold, command is given to GSM module. GSM module works on AT commands given by controller. Figure 1: Block diagram of transmitter Zigbee transceiver works on 2.4 GHz radio frequency and requires low power to operate. The transmitter will send signals to the receiver about which gesture was recognized. Figure 2: Block diagram of receiver Hardware at the receiver side is an 89V51 microcontroller board prepared taking into consideration the power supply circuit, reset circuit, GPIO pins, LCD and other peripherals required as per the proposed system. It has four 8-bit I/O ports and enhanced USART[5]. A MAX232 IC has been used to serially transfer the data from PC which is then compatible with RS232 level at the controller side. Zigbee receiver receives the signal sent by the transmitter and gives command to APR9600 Voice module having 16-bit digital audio processor. It has built-in audio recording microphone amplifier so it is user friendly and easy to operate. This will play voice according to the command received. The basic general flow diagram shown below that if the gesture movement is detected then the voice is played accordingly.
International Conference on: “Engineering: Issues, opportunities and Challenges for Development” ISBN: 978-81-929339-1-7 11th April, 2015, S.N. Patel Institute of Technology & Research Centre, Umrakh, Bardoli 4 Figure 3: Flow diagram of proposed system If no sign detected then it senses the ADC pins continuously. Similarly, an accelerometer is sensed continuously, and if abrupt fall detection is found then message is sent to the mobile handset. IV. KEY MODULES A. Flex sensors B. Accelerometer C. GSM D. Zigbee Transceiver E. Voice module 1) The Flex Sensors For exact detection of signals the flex sensors can be mounted on each finger and thumb of the glove. But for ease of operation and to make it cost effective three flex sensors are used, which are fitted on the fingers by using threads and needle. Flex sensors are basically carbon resistive elements, having greater form factor on a thin flexible substrate; more the carbon means less the resistance. When the substrate is bent, the sensor yields a resistance output correlated to the bend radius—the smaller the bend radius, the higher the resistance value. In this way, flex sensors offer variable resistance readings. Flex sensors operate on 5-volt input voltage, output ranging from 0 - 5 V, resistivity varying as per the degree of bend of sensors and output voltage altering accordingly. The sensors are connected to the device via 3 pin connectors. The resistance increases to 35- 45 kilo ohms at 90 degree bend. Flex sensor basically is a voltage divider circuit. The flex sensors may be used as a switch without going through a microcontroller, the outputs of which can be inputted into op-amps (comparator) to amplify their voltage. Greater the degree of bending, lower the output voltage. Using the voltage divider concept, the output voltage is determined which ranges from 1.30v to 2.6v.
International Conference on: “Engineering: Issues, opportunities and Challenges for Development” ISBN: 978-81-929339-1-7 11th April, 2015, S.N. Patel Institute of Technology & Research Centre, Umrakh, Bardoli 5 Figure 4: Basic Flex sensor Circuit1 A potentiometer can also be added to the Circuit to adjust the sensitivity range. 2) Zigbee Transceiver The zigbee module is used as a transceiver for wireless communication in this system:- : - To transmit the digital code for corresponding hand gesture sensed by the flex sensors : - To receive the digital code which was transmitted. The receiver controller then notifies the voice module to play the corresponding voice saved for that particular digital code. 3) Accelerometer To detect a sudden, abrupt fall of a person, an Accelerometer sensor has been used. The three axes have a default range of coordinates that define the normalcy of movements of the person being taken care of. But as soon as a person falls and there’s a sudden alteration in his coordinates from the expected range, an emergency message is sent to a predefined mobile number via the GSM module. Figure 5: Receiver hardware snapshot
International Conference on: “Engineering: Issues, opportunities and Challenges for Development” ISBN: 978-81-929339-1-7 11th April, 2015, S.N. Patel Institute of Technology & Research Centre, Umrakh, Bardoli 6 The complete TRANSCEIVER is to be made handy and portable to serve the following purpose:- 1. one-to-one communication wherein the mute person can adjust the volume audible enough for the other person to hear and communicate properly. 2. Distant communication in range of zigbee wherein the mute person can have the transmitter while the other person can have the receiver. E.g. if a mute person goes to 2nd floor or away (but within the zigbee range, he can take the transmitter while keeping the receiver on the ground floor and communication can be carried out. 3. Fall detection is an additional feature. V. CONCLUSION  The paper describes the design and working of an electronic system that converts the sign language into speech form useful for dumb people.  With the proposed system, their dream of speaking becomes true [3].  Zigbee module is effectively used for a long distance communication. With the use of ARM controller, efficiency is improvised. Hence, modern technology can be effectively utilized for a social cause.  GPS system can also be included with GSM module to locate the user position. That feature can be included as a future expansion. ACKNOWLEDGMENT The authors are thankfully acknowledge to Mr. J.N.Patel, Chairmain Vidyabharti Trust, Mr. K.N.Patel, Hon. Secretary, Vidyabharti Trust, Dr. H.R.Patel, Director, Dr.J.A.Shah, Principal, S.N.P.I.T.&R.C.,Umrakh, Bardoli, Gujarat,India for their motivational & infrastructural supports to carry out this research. REFERENCES [1] Priyanka R. Potdar,” Innovative Approach for Gesture to Voice Conversion”, International journel of innovative research and development, Vol 3 Issue 6, june-2014 [2] Naveen Kumar Toshawar, Chandni Jain,” Intelligent Control System using Gesture Sensing”, National Conference on Synergetic Trends in engineering and Technology: Impact of Industry & Society (STET-2014) International Journal of Engineering and Technical Research (IJETR) ISSN: 2321-0869, Special Issue [3] Gunasekaran. K1, Manikandan. R2,” Sign Language to Speech Translation. System Using PIC Microcontroller”, ISSN : 0975-4024 Vol 5 No 2 Apr-May 2013
International Conference on: “Engineering: Issues, opportunities and Challenges for Development” ISBN: 978-81-929339-1-7 11th April, 2015, S.N. Patel Institute of Technology & Research Centre, Umrakh, Bardoli 7 [4] Oya Aran.B.S and in CmpE.M.S, in CmpE.”VISION BASED SIGN LANGUAGE RECOGNITION:MODELING AND RECOGNIZING SOLATED SIGNS WITH MANUAL AND NON-MANUAL COMPONENTS”.Graduate Program in ComputerEngineering.Bo¸gazi»ci University.1-169,2008 [5] 89v51 © Koninklijke Philips Electronics N.V. 2004. [6] LPC214X user manual Rev.4-23rd April 2012 by NXP semiconductors.

More Related Content

published paper

  • 1. International Conference on: “Engineering: Issues, opportunities and Challenges for Development” ISBN: 978-81-929339-1-7 11th April, 2015, S.N. Patel Institute of Technology & Research Centre, Umrakh, Bardoli 1 VOICEBOX FOR THE MUTE PEOPLE WITH EMERGENCY MESSAGE SERVICE Khushbu Parikh1, Khushboo Desai2, Avani Arya3, Kaushika Patel4 Final year BE Student, electronics Department, BVM Engineering College, V.V.Nagar, Gujarat, India1 Final year BE Student, electronics Department, BVM Engineering College, V.V.Nagar, Gujarat, India2 Final year BE Student, electronics Department, BVM Engineering College, V.V.Nagar, Gujarat, India3 Assistant Professor, electronics Department, BVM Engineering College, V.V.Nagar, Gujarat, India4 kparikh06@gmail.com1 khushboodesai.56@gmail.com2 avaniaryastar@gmail.com3 kdpatel@bvmengineering.ac.in4 Abstract—Embedded systems now-a-days are very popular with emerging technologies that reduces the communication gap and gives a space where one can design and develop a device where communication between ordinary people and dumb people are assisted[3]. This proposed system is mainly designed to assist dumb people to communicate with ordinary people. There are mainly four units which are covered in the work, they are voice module unit, sensing unit, wireless transmission and processing unit. A digital glove is used which is normal glove, fitted with flex sensors. Gesture is recognized and the process is done in LPC2148 ARM controller, the corresponding signal is sent via zigbee module which is received by 89V51 microcontroller. According to the received signal, pre-recorded message which is stored already in APR9600 voice module is played. Flex sensors and APR9600 are the key modules which are going to be used in designing. Accelerometer is used to sense the fall detection and if it is beyond the threshold, then a predefined message is sent via GSM module to a predefined number. Mute people can use these gloves to perform hand gesture and it will be converted into speech so that normal people can understand their expression. Fall detection mechanism is implemented as an additional feature in this work. Keywords— ARM, Accelerometer, APR9600, Flex sensor, GSM module, LPC2148, Zigbee, 89V51 microcontroller I. INTRODUCTION Embedded system has changed the lifestyle of people in a very smart way [3]. Communication becomes effective when all are using same language for communicating. Sign language is a natural way to communicate between mute and normal people [1]. In order to improvise a life style of mute people the system is developed. There are two ways of conveying message by a dumb person; they are physical and non-physical [4]. Non-physical includes movement of head, facial expressions and physical communication includes the hand movement which is normally a sign language. Sign language is different in different countries as all are having different languages, and every sign has its own different meaning.
  • 2. International Conference on: “Engineering: Issues, opportunities and Challenges for Development” ISBN: 978-81-929339-1-7 11th April, 2015, S.N. Patel Institute of Technology & Research Centre, Umrakh, Bardoli 2 The proposed system is a translator which includes two sides, transmitter and receiver. In transmitter side, flex sensors are attached with a glove. Data are obtained directly from each sensor’s degree of bending. Controller senses the gesture and decides which signal is to be sent to the receiver side. Signals are transmitted wirelessly via zigbee module. Accelerometer is attached to the transmitter device which is going to be with the disable person. Accelerometer is used to sense the movement. When a person falls abruptly, GSM module will send SMS to the receiver mobile handset. At the receiver side, Signals which are fetched are processed and the corresponding command is given to the voice module. Voice module will play the prerecorded message as per the command received. II. LITERATURE SURVEY Priyanka R. Potdar has developed a system which uses ARM7 microcontroller which senses the gesture and plays the voice according to it and text is also displayed on LCD module. A SD card is used which stores the data. An audio amplifier is there as a voice generating unit. The work is useful for mute and deaf both. The system is very compact and flexible to user [1]. Naveen kumar and Chandni have developed a system where a camera reads a movement of a human body and sends data to computer. Gesture recognition system is used to assist impaired people to interact with computer. Keyboard and joystick allows the body to give signals to computer through gestures such as pointing to screen [2]. Gunasekaran and Manikandan have developed a system where they have used PIC controller at both transmitter and receiver side. RF module is used as a wireless medium. Flex sensors are used to sense finger movement and APR9600 as a voice module which plays voice according to received signals[3]. III. SYSTEM DESCRIPTION The design starts after the analysis of the requirements for the system development. Hardware at the transmitter side is an LPC2148 ARM7 microcontroller which has two UARTs on which GSM and ZIGBEE Transmitter has been connected, two inbuilt 10-bit ADCs (In all, 14 channels are available to which 14 analogue devices can be connected), High speed USB 2.0 utilised for loading the HEX File of the program code into the controller and up to nine external interrupts [6]. Flex sensors are variable analogue resistors that generally works as variable voltage dividers. On the transmitter side, 3 flex sensors have been used for gesture recognition connected to 3 ADC channels of ARM7. They are usually in the form of a thin strip with length of 1”-5” and width of 0.25”and thickness up to 0.19”. Resistance varies as the size changes [1].
  • 3. International Conference on: “Engineering: Issues, opportunities and Challenges for Development” ISBN: 978-81-929339-1-7 11th April, 2015, S.N. Patel Institute of Technology & Research Centre, Umrakh, Bardoli 3 Accelerometer (having three axes) is interfaced with 3 ADC channels for fall detection which will be sensed by ARM7. If that is detected to be above threshold, command is given to GSM module. GSM module works on AT commands given by controller. Figure 1: Block diagram of transmitter Zigbee transceiver works on 2.4 GHz radio frequency and requires low power to operate. The transmitter will send signals to the receiver about which gesture was recognized. Figure 2: Block diagram of receiver Hardware at the receiver side is an 89V51 microcontroller board prepared taking into consideration the power supply circuit, reset circuit, GPIO pins, LCD and other peripherals required as per the proposed system. It has four 8-bit I/O ports and enhanced USART[5]. A MAX232 IC has been used to serially transfer the data from PC which is then compatible with RS232 level at the controller side. Zigbee receiver receives the signal sent by the transmitter and gives command to APR9600 Voice module having 16-bit digital audio processor. It has built-in audio recording microphone amplifier so it is user friendly and easy to operate. This will play voice according to the command received. The basic general flow diagram shown below that if the gesture movement is detected then the voice is played accordingly.
  • 4. International Conference on: “Engineering: Issues, opportunities and Challenges for Development” ISBN: 978-81-929339-1-7 11th April, 2015, S.N. Patel Institute of Technology & Research Centre, Umrakh, Bardoli 4 Figure 3: Flow diagram of proposed system If no sign detected then it senses the ADC pins continuously. Similarly, an accelerometer is sensed continuously, and if abrupt fall detection is found then message is sent to the mobile handset. IV. KEY MODULES A. Flex sensors B. Accelerometer C. GSM D. Zigbee Transceiver E. Voice module 1) The Flex Sensors For exact detection of signals the flex sensors can be mounted on each finger and thumb of the glove. But for ease of operation and to make it cost effective three flex sensors are used, which are fitted on the fingers by using threads and needle. Flex sensors are basically carbon resistive elements, having greater form factor on a thin flexible substrate; more the carbon means less the resistance. When the substrate is bent, the sensor yields a resistance output correlated to the bend radius—the smaller the bend radius, the higher the resistance value. In this way, flex sensors offer variable resistance readings. Flex sensors operate on 5-volt input voltage, output ranging from 0 - 5 V, resistivity varying as per the degree of bend of sensors and output voltage altering accordingly. The sensors are connected to the device via 3 pin connectors. The resistance increases to 35- 45 kilo ohms at 90 degree bend. Flex sensor basically is a voltage divider circuit. The flex sensors may be used as a switch without going through a microcontroller, the outputs of which can be inputted into op-amps (comparator) to amplify their voltage. Greater the degree of bending, lower the output voltage. Using the voltage divider concept, the output voltage is determined which ranges from 1.30v to 2.6v.
  • 5. International Conference on: “Engineering: Issues, opportunities and Challenges for Development” ISBN: 978-81-929339-1-7 11th April, 2015, S.N. Patel Institute of Technology & Research Centre, Umrakh, Bardoli 5 Figure 4: Basic Flex sensor Circuit1 A potentiometer can also be added to the Circuit to adjust the sensitivity range. 2) Zigbee Transceiver The zigbee module is used as a transceiver for wireless communication in this system:- : - To transmit the digital code for corresponding hand gesture sensed by the flex sensors : - To receive the digital code which was transmitted. The receiver controller then notifies the voice module to play the corresponding voice saved for that particular digital code. 3) Accelerometer To detect a sudden, abrupt fall of a person, an Accelerometer sensor has been used. The three axes have a default range of coordinates that define the normalcy of movements of the person being taken care of. But as soon as a person falls and there’s a sudden alteration in his coordinates from the expected range, an emergency message is sent to a predefined mobile number via the GSM module. Figure 5: Receiver hardware snapshot
  • 6. International Conference on: “Engineering: Issues, opportunities and Challenges for Development” ISBN: 978-81-929339-1-7 11th April, 2015, S.N. Patel Institute of Technology & Research Centre, Umrakh, Bardoli 6 The complete TRANSCEIVER is to be made handy and portable to serve the following purpose:- 1. one-to-one communication wherein the mute person can adjust the volume audible enough for the other person to hear and communicate properly. 2. Distant communication in range of zigbee wherein the mute person can have the transmitter while the other person can have the receiver. E.g. if a mute person goes to 2nd floor or away (but within the zigbee range, he can take the transmitter while keeping the receiver on the ground floor and communication can be carried out. 3. Fall detection is an additional feature. V. CONCLUSION  The paper describes the design and working of an electronic system that converts the sign language into speech form useful for dumb people.  With the proposed system, their dream of speaking becomes true [3].  Zigbee module is effectively used for a long distance communication. With the use of ARM controller, efficiency is improvised. Hence, modern technology can be effectively utilized for a social cause.  GPS system can also be included with GSM module to locate the user position. That feature can be included as a future expansion. ACKNOWLEDGMENT The authors are thankfully acknowledge to Mr. J.N.Patel, Chairmain Vidyabharti Trust, Mr. K.N.Patel, Hon. Secretary, Vidyabharti Trust, Dr. H.R.Patel, Director, Dr.J.A.Shah, Principal, S.N.P.I.T.&R.C.,Umrakh, Bardoli, Gujarat,India for their motivational & infrastructural supports to carry out this research. REFERENCES [1] Priyanka R. Potdar,” Innovative Approach for Gesture to Voice Conversion”, International journel of innovative research and development, Vol 3 Issue 6, june-2014 [2] Naveen Kumar Toshawar, Chandni Jain,” Intelligent Control System using Gesture Sensing”, National Conference on Synergetic Trends in engineering and Technology: Impact of Industry & Society (STET-2014) International Journal of Engineering and Technical Research (IJETR) ISSN: 2321-0869, Special Issue [3] Gunasekaran. K1, Manikandan. R2,” Sign Language to Speech Translation. System Using PIC Microcontroller”, ISSN : 0975-4024 Vol 5 No 2 Apr-May 2013
  • 7. International Conference on: “Engineering: Issues, opportunities and Challenges for Development” ISBN: 978-81-929339-1-7 11th April, 2015, S.N. Patel Institute of Technology & Research Centre, Umrakh, Bardoli 7 [4] Oya Aran.B.S and in CmpE.M.S, in CmpE.”VISION BASED SIGN LANGUAGE RECOGNITION:MODELING AND RECOGNIZING SOLATED SIGNS WITH MANUAL AND NON-MANUAL COMPONENTS”.Graduate Program in ComputerEngineering.Bo¸gazi»ci University.1-169,2008 [5] 89v51 © Koninklijke Philips Electronics N.V. 2004. [6] LPC214X user manual Rev.4-23rd April 2012 by NXP semiconductors.