Intelligent travelling and home automation aid, for visually impaired

IJRET: International Journal of Research in Engineering and Technology eISSN: 2319-1163 | pISSN: 2321-7308
_______________________________________________________________________________________
Volume: 04 Issue: 11 | Nov-2015, Available @ http://www.ijret.org 341
DESIGN AND DEVELOPMENT OF INTELLIGENT ELECTRONICS
TRAVELLING AID FOR VISUALLY IMPAIRED (IETA-VI)
Umar Shuaibu1
, H. K. Verma2
, Rabiu Aliyu Abdulkadir3
, C. Mohan4
1&3
PG Student Dept. of Electrical & Electronics Eng. Sharda University, Greater Noida, UP, India
2
Distinguish Prof. Dept. of Electrical & Electronics Eng. Sharda University, Greater Noida, UP, India
4
Assistant Prof. Dept. of Electrical & Electronics Eng. Sharda University, Greater Noida, UP, India
Abstract
The dynamic nature of the environment poses a big challenge to the visually impaired in navigation. This compels most of the
visually impaired persons in the developing countries, who generally do not get any technology support, to depend on the visual
sense of others, thus, undermining their independence.
The intelligent electronic travelling aid for the visually impaired (IETA-VI) designed and developed by the authors and reported
here, intends to provide solution to the navigational challenges of the visually impaired individuals. The system utilizes the
ultrasonic detection technology for detecting any obstacle in his/her path, and then converts the distance to the obstacle into voice
using voice synthesis technology so as to inform the visually impaired user. The device uses GPS and GSM technologies to
determine the location of the user and to send this location to his/her care givers on mobile phone, respectively. The device also
recognizes the voice signals of the visually impaired when in distress, by making use of voice recognition technology and send
SMS to his care givers giving his location and asking them to help him.
In a moment of emergency, when the visually impaired requires the attention of his/her care givers, the device provides three
alternatives: The first alternative it provides is an emergency key, which when pressed will automatically send the location of the
visually impaired to the care givers. The second alternative is that the visually impaired shouts (gives a voice command), the
device responds by sending the location of the visually impaired to the care givers. The third alternatives take care of the situation
when the user is unconscious or cannot even speak. This is provided by the display device, which displays the names and contact
details of his/her care givers, so that someone can offer a helping hand.
KEYWORDS: Ultrasonic detection, Voice recognition, Voice synthesis, GPS, GSM.
--------------------------------------------------------------------***----------------------------------------------------------------------
1 INTRODUCTION
Human eye is like a camera that receives and focuses light
into the retina, which then stimulates the cells that carry the
signal to the brain; without retina, eye cannot see, hence the
visual impairment. The visual impairment is the severe
reduction of vision that cannot be corrected with the
standard glasses or contact lenses, thereby reducing the
person’s ability to function at certain level [1].
The ever growing number of the people with visual
disability, and their continues reliance on the visual sense of
others especially in the developing countries serve as an
impetus to this research work report in this paper; which
intends to design and develop the intelligent electronic
travelling aid to aid the navigation of the visually impaired
in both the indoor and the outdoor environment.
2 RELATED WORKS
Over the years, there has been an evolution of various
techniques of guiding visually impaired persons, thus,
toward attaining their self-independent by freely moving
around their environment without guidance from others;
some of these are:
i. Smart walking stick by Mohammed H. Rana and
Sayemil ( 2013):
This is based on ping sensor for detecting obstacle, wet
electrode, vibration motor and the buzzer. The obstacle is
detects by the ping sensor and the obstacle distance is
communicates to the visually impaired by the vibration of
the motor [2].
ii. The electronic travelling aid for blind navigation
and monitoring by Mohan M.S Madulika S. et al
(2013): This is arm7 controller based that used
ultrasonic technology for detecting the obstacle and
inform the obstacle distance to the visually impaired,
and also used the GPS and GSM technologies for
localization of the visually impaired [3].
iii. New indoor navigation for the visually impaired
using visible light communication by Madoka
Nakajima and Shinichiro Haruyama (2013):
This is a study that used light fidelity (LI-FI), in this study,
the authors propose using visible light communication to
help guide the visually impaired in the indoor environment.
The visible light communication technology utilises the
capability of the led lamps to be probe at a rate faster than
the human eye can detect, as such data transmission can be
achieved while providing the illumination [4].
iv. Multi-dimensional walking aid by Olakanmi. O.
Oladayo (2014): This used ultrasonic detection
technology and the voice module, the obstacle is
detects by the ultrasonic sensor and the direction of the

_______________________________________________________________________________________
obstacle is communicates to the user through voice
output [5].
v. 3D ultrasonic stick for the blind by the Osama
Bader Al-Barm (2014):
The system uses ultrasonic sensor for detecting the obstacle
in three directions (i.e front, left and the right sides of the
visually impaired), and the vibration motor which vibrate
with the intensity depending on the obstacle’s distance. It
also uses GPS and GSM for localization of the visually
impaired [6].
In this paper, design and development of intelligent
electronic travelling aid for visually impaired is presented.
The device employs ultrasonic detection, GSM, GPS, voice
recognition and voice synthesis technologies. The design
process comprises of two parts; the first part is the obstacle
detection and voice generation unit design using ultrasonic
detection technology and voice synthesis technology
respectively, and the second part is the localization and
monitoring unit design using GPS technology GSM
technology, as well as the voice recognition technology. The
two units are then combines to form the complete device.
3 TECHNOLOGIES USED
For the development of intelligent electronic travelling aid
for visually impaired (IETA-VI) reports in this paper, the
following technologies are use:
3.1 ULTRASONIC DETECTION TECHNOLOGY
Ultrasonic detection mimics the innate echolocation
capability of bat and dolphin; bat and dolphin make high
frequency sound (in ultrasonic range) that bounce off
whatever is in front of them. They receive these sounds and
can sense the size, shape, density, distance and even texture
of an object, this enable them to recognize their target, their
prey and predators.
The ultrasonic sensor detects objects by emitting a short
ultrasonic burst and then listening for an echo under the
control of a host microcontroller; it emits a short eight 40
kHz burst of ultrasound energy. This burst travels through
the air, hits an object and then bounces back to the sensor.
The sensor provides an output pulse to the host controller
that terminate when the echo is detect; hence the width of
this pulse represent the distance to the target. Fig 1 is
obstacle detection diagram by the ultrasonic sensor [7].
Fig-1: Obstacle detection by the ultrasonic module [8]
3.2 GLOBAL POSITIONING SYSTEM (GPS)
TECHNOLOGY
Global positioning system is the satellite base navigation
consisting of a network of twenty four satellites placed in an
orbit; these satellites circle the earth twice a day,
transmitting the information to the earth GPS receiver.
Basically the GSP receiver determine the four variables
which are: longitude, latitude, height and time, additional
information can be drive from these four components [9].
Fig 2 is the diagram of the GPS receiver used in this
research.
Fig-2: GPS module used
3.3 GLOBAL SYSTEM FOR MOBILE (GSM)
COMMUNICATION TECHNOLOGY
In the early 1980s, there was analogue communication
technique, where by each country developed its own system,
this was problematic because the system work only within
the boundary of each country. To this end, global system for
mobile communication was developed by the conference of
the European posts and telecommunication, prior to which
exist a number of incompatible systems throughout Europe.
The GSM was developed with the main aim of improving
efficiency, international roaming, and compatibility with the
international subscriber dialing and other telephone
company [10]. Fig 3 is the diagram showing GSM module
used in this research work which uses GSM technology for
communication between the IETA-VI and the care givers.
Fig-3: GSM module used
3.4 VOICE RECOGNITION TECHNOLOGY
The ability of a machine or software program to identify
word or phrases, and convert them into a format that is
readable to the computer or other machine is what is termed

_______________________________________________________________________________________
as voice recognition. The voice recognition processes start
with the pre-processing of the voice signal; and then follows
by feature extraction of the voice signal.
First the speech analog signal is converts into digital form;
this then follows by the silence removal where by the
silences in the various location of the speech signal are
remove. When the silences in the signal are removed, the
signal is then boost so as to amplify its low frequency
content, this is called pre emphasis. The speech signal then
is block into frames of N samples using windowing
technique and then the features of each samples of the signal
are then extracted. The last stage is the feature matching
where by the system unmatched the unknown user from the
already known user; and matched the already known user,
when this happen, the voice is said to be recognized or not
recognized [11].
3.5 VOICE SYNTHESIS TECHNOLOGY
The most importance quality of speech synthesis is the
naturalness and intelligibility, the naturalness refers to how
closeness the generated speech is to the actual human voice;
and the intelligibility is the ease to which the synthesized
speech is understood. The speech generation or the speech
synthesis is the artificial production of human speech;
speech can be synthesized by different techniques, but the
primary methods are:
a) Articulatory synthesis that attempt to mimic the human
articulatory system directly.
b) Concatenate synthesis that uses pre-recorded samples of
voice drive from human speech as well as the
c) Formant synthesis that uses the pole frequency of the
speech signal, or the transfer functions of the vocal tract
[12].
4 DESIGN AND BLOCK DIAGRAM
In the development of this system, the Arduino mega2560
development board is selected because of it flexibility,
simplicity and most importantly its four serial
communication ports.
Fig-4: Arduino mega2560 development board
Considering the requirement at hand, Atmega8 is also uses
to handle the obstacle detection and the voice generation
part of the system. This is necessary due to the complexity
of the system; and the amount of delay requires in producing
the voice output (around 3-6 seconds); within which many
events may happen that the controller cannot notice. Fig 5
below shows the block diagram of the system.
Fig-5: Block diagram
The design of this system is divided into two parts which are
obstacle detection and voice generation unit design; as well
as the monitoring and localization unit design.
4.1 OBSTACLE DETECTION AND VOICE
GENERATION UNIT DESIGN
The obstacle detection is handles by the ultrasonic module
under the control of the host controller. First the start pulse
of 10 micro-seconds duration is given by the controller to
the ultrasonic module; this is the obstacle detection
initialization process. The ultrasonic module then transmit
eight consecutive ultrasonic bursts each of 40 kHz
frequency; these bursts travel in search of the obstacle, if
detected, the transmitted energy is reflect back to the sensor
as an echo.
Fig-6: Obstacle detection process [13]
The host controller then listens to this echo; the width of the
echo represents the distance to the obstacle; the controller
applies the following technique to determine the obstacle’s
distance from the width of the echo.

_______________________________________________________________________________________
𝑠𝑝𝑒𝑒𝑑 𝑜𝑓 𝑠𝑜𝑢𝑛𝑑 𝑖𝑛 𝑎𝑖𝑟
=
𝑡𝑜𝑡𝑎𝑙 𝑑𝑖𝑠𝑡𝑎𝑛𝑐𝑒 𝑑𝑖𝑠𝑡𝑎𝑛𝑐𝑒 𝑡𝑟𝑎𝑣𝑒𝑙𝑙𝑒𝑑 𝑏𝑦 𝑡ℎ𝑒 𝑝𝑢𝑙𝑠𝑒𝑠
𝑡𝑖𝑚𝑒 𝑡𝑎𝑘𝑒𝑛
𝑡𝑖𝑚𝑒 𝑡𝑎𝑘𝑒 = 𝑤𝑖𝑑𝑡ℎ 𝑜𝑓 𝑡ℎ𝑒 𝑒𝑐ℎ𝑜 𝑝𝑢𝑙𝑠𝑒 (𝑖𝑛 µ𝑠)
𝑡𝑜𝑡𝑎𝑙 𝑑𝑖𝑠𝑡𝑎𝑛𝑐𝑒 𝑡𝑟𝑎𝑣𝑒𝑙𝑙𝑒𝑑
= 𝑡𝑜 𝑎𝑛𝑑 𝑓𝑟𝑜 𝑑𝑖𝑠𝑡𝑎𝑛𝑐𝑒 𝑡𝑟𝑎𝑣𝑒𝑙 𝑏𝑦 𝑡ℎ𝑒 𝑝𝑢𝑙𝑠𝑒𝑠
𝑡ℎ𝑒𝑟𝑒𝑓𝑜𝑟𝑒, 𝑑𝑖𝑠𝑡𝑎𝑐𝑒 𝑡𝑜 𝑡ℎ𝑒 𝑛𝑒𝑎𝑟𝑒𝑠𝑡 𝑜𝑏𝑠𝑡𝑎𝑐𝑙𝑒
=
𝑡𝑜𝑡𝑎𝑙 𝑑𝑖𝑠𝑡𝑎𝑛𝑐𝑒 𝑡𝑟𝑎𝑣𝑒𝑙𝑙𝑒𝑑 𝑏𝑦 𝑝𝑢𝑙𝑠𝑒𝑠
2
𝑑𝑖𝑠𝑡𝑎𝑛𝑐𝑒
=
𝑠𝑝𝑒𝑒𝑑 𝑜𝑓 𝑠𝑜𝑢𝑛𝑑 𝑖𝑛 𝑎𝑖𝑟 × 𝑤𝑖𝑑𝑡ℎ 𝑜𝑓 𝑒𝑐ℎ𝑜 𝑝𝑢𝑙𝑠𝑒
2
.
=
𝑡𝑖𝑚𝑒 × 10−6
2 × 340
=
𝑡𝑖𝑚𝑒 𝑖𝑛 𝜇𝑠
58
𝑐𝑚 [14]
In this research, the voice output is produce only if an
obstacle is detect within the range of ten centimeters to three
meters; here, different voices are generate using text to
speech conversion process, the differences between the
actual distance of the detected obstacle to the voice output
distance announced is within ±5cm range. Fig 7 below is an
example of text to speech conversion process of one of the
generated voice.
Fig-7: Example of text to speech conversion
Fig 8 is the waveform of the speech signal generated from
the text above, which is then amplified with a gain of
400dB.
Fig-8: Speech waveform generated
The speech is sample at 44.1 KHz and store as
uncompressed 8-bit mono wav-file in the micro SD card.
Fig9 below shows wav audio player which play the wav
files generate using text to speech conversion technique
shown above.
Fig-9: Wav audio player from the SD card
4.2 MONITORING AND LOCALIZATION UNIT
DESIGN
This unit comprises of voice recognition module, GSM
module, emergency key as well as the GPS module. The
emergency key is connected to an interrupt pin of the
controller which serve to interrupt the controller when
pressed, thereby making the controller to service the
interrupt (i.e sending the location of the visually impaired to
the care givers and calling them to help).
The voice recognition module receives the configuration
command and response through the serial interface.
Fig-10: Voice recognition module used
The process for the operation of the voice module is divided
into two phases.
The first is the training phase in which the module is train
with the voice instruction, it can store up to fifteen
instruction that are divide into three groups, each group can
be select through the serial interface, in this research only
group one instructions are use.
The second phase is the recognition phase, where by the
module is made to recognize the voice instruction it is train
for, each group has to be import for the system to recognize
the instruction in that group. During the training phase, the
module configuration is first perform; where by a serial
commands are send to the voice recognition module,
informing it about the communication baud rate, module
response mode, as well as the group to train for voice
recognition. Fig 11 shows the arduino and voice recognition
module setup for training and recognition phase.
Fig-11: Voice training and recognition setup with the
arduino

_______________________________________________________________________________________
After configuration, the first voice instruction is given to the
module, same instruction is repeats again for the second
time, after which the module response and gives information
about whether the two command matches, if it doesn’t, the
module ask for the repeat of same command until it
matches; when the commands matches, the next voice
instructionwithin that group is then proceed. This process
continues until all the voice instructions in that group are
train; the voice recognition is language independent, which
means the module can be train to recognizes instructions
irrespective of the language use; as testify by some of the
instructions which are in native Hausa language of the
northern part of Nigeria, as well as the Hindi language of
India.
After the training, then the recognition session begins; first
the serial command is send to the module informing it that
we are into recognition phase. For any group to be
recognize, that group has to be import; thus group one is
imports for recognition using serial command; in the
recognition phase, the voice is feed into the voice
recognition module and the module response by giving the
sequence of strings corresponding to that instruction; table1
shows all the five instructions in group one.
Table-1: Voice instruction table
SN INSTRUCTIONS
1 Help!
2 Help me Dad!
3 Some body help me!
4 Wayyo ataimakamin!5
5 मेरी मदद करो6
Mērīmadadkarō!
5
Hausa language (native language of northern part of
Nigeria) which means oh! Help me
6
Hindi language (widely spoken language in India) which
mean help me.
The GPS module is then program and integrated into the
system to receive the GPS data.
GSM module uses the GSM network for communication
between the IETA-VI and the care giver/s of the visually
impaired through short messages. The arduino controller
communicate with the GSM module using AT command,
the module is test with the AT command using terminal
software to confirm it condition this is shown in figure 12.
The module is then program to send and receive messages
from the mobile phone and route these messages to the
microcontroller for further processing.
Fig-12: Checking the GSM module working condition
5. CIRCUIT DIAGRAM
Fig-13: Circuit diagram of the system
6. FLOW CHART OF THE SYSTEM

_______________________________________________________________________________________
Fig-14: System flow chart
7. OPERATION OF IETA-VI
The intelligent electronic travelling aid for visually impaired
(IETA-VI) is a device consisting of multiple technologies
intends to help toward the independent living of the visually
impaired persons. Its operation can be explains in two parts
as follows:
a) The first part is the obstacle detection and voice
generation unit; this part of the system is responsible for
detecting the obstacle in the path of the visually
impaired, and informing the visually impaired person
about the obstacle and the distance to that obstacle
through the voice output.
b) The second part of the system consisted of GPS
module, GSM module, emergency key as well as the
voice recognition module. When the visually impaired
is not at home, and the care givers may like to know
his/her location; to this end, the care giver/s can send a
request to the IETA through their mobile phone. The
IETA is program to recognizes only two requests from
the care givers which are:
i. Location, in which the care givers will send a
request by texting “*LOCATION#” from their
mobile phone to the IETA and
ii. Get location, where by the care givers will send a
request by texting “GET_LOCATION#” to the
IETA.
When the correct request code is receive by the IETA, it will
responds by getting the current location of the visually
impaired from the GPS; and send the GPS data to the care
givers using GSM. When the visually impaired person is in
a situation that requires the urgent attention from his/her
care givers; to this end, there are two options available to
him/her which are:
1) The emergency key: here the visually impaired will
request for the help from the care givers; by simply
pressing the emergency key which will automatically
send the visually impaired location to the mobile
phone/s of the care givers.
The second is the voice recognition system: here, the
visually impaired will give the voice command to the
IETA-VI, the system will recognizes his/her voice
instruction; and will automatically send his/her location to
the care givers.
8. PERFORMANCE EVALUATION OF IETA-VI
The performance evaluation of this system starts by
evaluating the performance of the components forming the
system. First ultrasonic sensor performance is determine
and is obtain to be quite amazing as given in fig 15
considering the three meters detection range that the system
intends to work with. Different measurements are taken
using the ultrasonic sensor and the result are display in the
arduino serial monitor (in centimeters); each measure
distance is then compare with the distance measure using
scale and the result is find to be the same.
The voice generation module is then integrate, and the
appropriate code for playing the obstacles’ distances detect
is upload into the controller; and it is observe thatthe
obstacle detection and voice generation unit work as
expected with only ±5 centimeters deviation between the
actual distance of the obstacle and the voice output distance
announce.
Fig-15: Ultrasonic sensor output
GPS module performance is then test by getting the GPS
data of my location and displaying it on the serial monitor;
the latitude and longitude obtain is then compare with the
actual latitude and longitude of that location and the result
is find to be correct as shown in fig 16
Fig -16: GPS data obtained during performance testing

_______________________________________________________________________________________
Voice recognition module is then tested after it is train; the
group one instructions is imports and then test by feeding
the voice instruction for that group. When the module
recognizes the instruction, it gives an output representing
that instruction.Fig17 is the result obtain from voice
recognition module for all the five instructions in group one.
Fig-17: Simulation for the recognition of the voice
instructions
Fig-17: Simulation for the recognition of the voice
instructions.
Table-2: Meaning of the result obtain in fig17 and the
recognize instructions
Sn Result Meaning of the result Recognized
instruction
1 0x11 First instruction of
group one is recognized
Help!
2 0x12 Second instruction of
Help me Dad!
3 0x13 Third instruction of
Somebody help
me!
4 0x14 Fourth instruction of
Wayyo-
ataimakamin!5
5 0x15 Fifth instruction of
मेरी मदद करो6
Mērīmadadkarō!
GSM module performance is lastly evaluate by first using it
to send and receive message through the terminal window;
and then programming it with the Arduino to send and
receive messages, all the result work satisfactorily as shown
in fig 18 below.
Fig-18: Using terminal window and GSM module for
messaging
8. CONCLUSIONS
The intelligent electronic travelling aid for visually impaired
developed by the authors and reported here intended to
improve the ability of the visually impaired to live
independently, and to perform his day to day activities
easily, and more safely by providing the following
functionalities:
a) Providing the voice based information about the distance
of any obstacle in the path of the visually impaired.
b) Providing localization and monitoring through GPS,
GSM as well as the voice recognition. The device makes use
of the following five technologies interactively:
i. Ultrasonic detection technology
ii. Global positioning system (GPS) technology
iii. Global system for mobile (GSM) communication
technology
iv. Voice synthesis technology and
v. Voice recognition technology
To further extend the IETA-VI, the following are the
recommendations:
i. Addition of the RFID technology to provide the RFID
mapping for the indoor environment so as to provide the
voice based guidance and localization to the visually
impaired in his/her home.
ii. It can also be extended for automation of the home of
visually impaired using voice recognition, which is already
available in the system, so that the visually impaired can
have control over various lights, fans, HVAC system and
other appliances in home. This will further enhance his
independence inside the home.
REFFERENCES
[1] Visual impairment: At
www.who.int/mediacentre/factsheets accessed on 2015-05-
24
[2] Hazaz Mahmud, RanaSaha et al 2013. Smart walking
stick.International journal of scientific and engineering
research 4(1), 111-114
[3] Mohan M.S, Madulika S. Arm7 based electronic travel
aid for the blind people,An International Journal of
Research in Computer and Communication Technology
2(12) 1345-1350
[4] Nakajima M. and Haruyana S. (2013): new indoor
navigation system for the visually impaired people using
visible light communication, EURASIP journal on wireless
communication and networking. 1-10.
[5] Olakanmi O. Oladayo: Multidimensional walking aid.
International journal of Intelligent Systems and
Applications, 2014, 08, 53-59
[6] Al-Barm O.B and Vinouth.(2014) 3D Ultrasonic
walking stick for the blind. International Journal of Latest
Trends in Engineering and Technology3(3), 108-114
[7]SR04 Ultrasonic sensor datasheet
[8] Ultrasonic sensor: at www.robotronicspro.blogspot.in
accessed on 24/05/2015
[9] Ignacio GmesFernadez, GPS space segment monitoring:
at www.fenixtechnico.ulisboa.pt .accessed on 25/07/2015

_______________________________________________________________________________________
[10] GSM-architecture: at
www.tutorialspoint.com/gsm/gsmarchitectureacessed on
27/07/2015
[11] Abd-Ghaffar H. Fatma A.M et al (2011) Smart blind
stick: Speech recognition; Mansoura University department
of electronic and communication.
[12] Wikipedia:Speech synthesis accessed on 20/07/2015
[13] Hc-sr04 sensor: at www.electrosome.com accessed on
20/07/2015
[14] How to measure distance with SR04 ultrasonic sensor:
at www.Robotica.webs.upv.es accessed on 20/06/2015
BIOGRAPHIES
Mr Umar Shuaibu
PG Student, Instrumentation and
control engineering, School of
engineering and technology, Sharda
University Greater Noida, UP, India.
Dr H.K Verma
Distinguish Professor Department
Electrical and Electronics Engineering,
School of engineering and technology
Sharda University Greater Noida, UP
India.
Mr RabiuAliyuAbdulkadir
PG Student, Instrumentation and
control engineering, School of
engineering and technology, Sharda
University Greater Noida, UP India.
C. Mohan
Assistant Professor Department
Electrical and Electronics Engineering,
School of engineering and technology
Sharda University Greater Noida, UP
India.

Intelligent travelling and home automation aid, for visually impaired

Related slideshows

More Related Content

Intelligent travelling and home automation aid, for visually impaired