Development of Hybrid System for Combating Human Kidnapping Using Smart Objects and Internet of Things
Osagie Kingsley Omoruyi1, Oluwole Charles Akinyokun1*, Babatude Gabriel Iwasokun1, Kolawole Gabriel Akintola1 and Abejide Michael Adegoke2
1Federal University of Technology, Akure, Nigeria
2Bells University of Technology, Ota, Nigeria
Submission: October 14, 2022; Published: August 07, 2023
*Corresponding author: Oluwole Charles Akinyokun, Federal University of Technology, Akure, Nigeria
How to cite this article: Osagie Kingsley O, Oluwole Charles A, Babatude Gabriel I, Kolawole Gabriel A, Abejide M A. Development of Hybrid System for Combating Human Kidnapping Using Smart Objects and Internet of Things. Trends Tech Sci Res. 2023; 6(4): 555694. DOI: 10.19080/TTSR.2023.06.555694
Abstract
The paper developed a system for combating human kidnapping using Smart Objects and Internet of Things with a view to improving the security awareness of the society and providing data for informative, educative, efficient and effective investigation of combating human kidnapping. The model comprises of modules for positioning, tracking and monitoring of kidnappers and kidnapped victims. The output from the implementation of the model as well as its evaluation and comparative study established its practical function in addition to its superior performances to some of the existing and similar models.
Keywords: RFID; RTLS; GPS; Smart Object; Internet of Things; Positioning; Tracking; Monitoring; Kidnapping; Microcontroller; Testbed; Satellite; Coordinate
Abbreviations: RTLS: Real Time Locating System; IOT: Internet-of-Things; GPS: Global Positioning System; RFID: Radio Frequency Identification; IRID: Infrared Identification; US-ID: Ultrasound Identification; WLAN: Wireless Local Area Network; SPS: Sensor Processing Stations; DSR: Digital Station Room; MSS: Media Server Station; SES: Smart Engine Server; NVM: Non-Visual Module; VCS: Video Capture Station; ADC: Analogue to Digital Converter; SMS: Security Management Station; IPv6: Internet Protocol version 6; VM: Visual Module; NMEA: National Marine Electronics Association; CCTV: Closed-Circuit TV; SMM: Scene Monitoring Module; VGG: Visual Geometry Group; ToA: Time of Arrival; RSSI: Received Signal Strength Indicator; LOS: Line of Sight; CPU: Central Processing Unit; ADC: Analog to Digital Converter; ADC: Analog to Digital Converter; HLE: Hybrid Location Engine
Introduction
Human Kidnapping has been argued to have originated during the era of slavery between the eighteenth century and nineteenth century when human beings were kidnapped and sold into slavery. In 2002, the 8th United Nations survey on crime trends and operation of criminal justice system was conducted, there was no case reported for Nigeria [1]. Human kidnapping is an unlawfully detaining a person against his or her will including the use of force, threat, fraud or enticement for the purpose of demanding for liberation and illicit gain, economic gain or material benefit in order to oblige someone [2]. Protus & John [3] opined that inadequate information available to security agents, lack of operation intelligence and absence of modern tracking equipment are critical challenges militating against the growth trends of human kidnapping. The lack of informative and educative investigative procedure is a major factor militating against combating the menace.
The magnitude at which technology has evolved in recent decades and applied to several sectors from manufacturing, fleet management, car tracking technology to retail management has been impressive. Unfortunately, the gains in technology advancement have not truly been felt in the area of positioning, tracking and monitoring of human kidnapping. Several traditional and manual approaches which includes random and limited-informed search of the human kidnapping, waiting for a phone call from the human kidnappers, payment of ransom to the kidnappers for the release of the victims and so many other methods are susceptible to delays, corruption, huge loss of funds, high casualties and many other things. The need for a more efficient system which will adequately address the limitations and vulnerability of the traditional methods gave birth to the development of tracking technology and Real Time Locating System (RTLS).
RTLS has made a positive impact in several industries like retail services, objects tracking, hospital management and so on. The ISO standards have defined RTLS as wireless systems with the ability to locate the position of an item anywhere in a defined space at a point in time that is close to real time. Position is derived by measurements of the physical properties of the radio link in Gladysz & Santarek [4]. The emergence of Smart Objects and Internet-of-Things (IoT) has further expanded the application domain of RTLS. Several technologies that are usually combined to provide RTLS solution are: Global Positioning System (GPS), active and passive Radio Frequency Identification (RFID), Infrared Identification (IRID), Ultrasound Identification (US-ID), Bluetooth, Wireless Local Area Network WLAN, Wi-Fi, and many others.
Smart objects are independent physical and/or digital objects that in addition to making application logic, they can also sense, process and network with other objects. They make sense of their local environment and interact with human users. They sense, log and interpret what is occurring within themselves and the world, act on their own, intercommunicate with each other and exchange information with people [5]. Cisco has defined Internet-of-Things as the network of physical objects accessed through the Internet. The objects contain embedded technology to interact with internal states or external environments. Basically, IoT brings together smart objects and wireless connectivity.
The GPS is arguably the most reliable solution in outdoor tracking. It is a satellite navigation system that is made up of satellites launched into orbit by the USA Department of Defense. Originally, the GPS technology was meant to be used by the military, but it was later approved by the USA government to be available for civilian use. The GPS signal contains satellite identity code, current date, current time and other information sufficient to determine the position of any object in real time. The downside of GPS technology is that it is not efficient in places like underground tunnels, inside of buildings and many other areas where there is no GPS signal from the satellites. Other technologies like RFID can suffice in enclosed regions where GPS signal is limited. RFID can be defined as non-contact automatic identification technique via which an object can be identified using radio frequency and obtain data autonomously. There are basically three components that make up this technology namely: RFID readers, RFID tags (both passive and active), hardware and software infrastructures. RFID technology is more effective for indoor positioning.
Some of the limitations of the existing works pose a research gap, which include bulky and lack of capacity for real time operation, lack of experimental study, poor handling of obscure scene, high-cost intensity, susceptibility to signal loss and performance failure due to energy exhaustion, loss of connectivity and unfavorable weather. A cutting-edge model that fills the research gap created by these limitations towards achieving optimal and more satisfactory combat of human kidnapping is presented in this paper. First, the method and materials employed for combating human kidnapping using Smart Objects and Internet of Things is presented. Second, the architecture of the proposed system is presented. Third, the implementation of the proposed system which covers the technical details of the hardware and software requirements, the setting of the experimental testbed, the implementation procedures and user interfaces of the application. Finally, a case study of the application for monitoring a presumed kidnapped victims between Ibadan, Oyo State and Ilishan Remo, Ogun State, in Nigeria was carried out to test the adequacy and practical functions of the system. The basis of evaluation and comparative analysis of the system is based on coverage area, cost incurred in deployment, positioning accuracy and signal sensitivity.
Literature Review of Some Existing Related Research Work
Chaudhari et al., [6] developed GPS/GSM enabled person tracking system. The research was motivated by the high cost of tracking using on GPS. The objectives of the research were to reduce the overall cost of tracking system using GPS which is a satellite system-based service and develop a low-cost GPS/GSM tracking system. Some literature reviews were carried out by the authors revealing the following three main methods used for tracking:
A. Automotive navigation system which is used for tracking automobiles.
B. GPSylon which is used to enable maps which can be downloaded from Expedia Maps Server.
C. Open GTS which provides web-based GPS tracking system.
In the research, GPS was used to monitor a person location anywhere on earth. The GPS consists of GPS transmitter and GPS receiver. The GPS unit measures the travel time of the signals transmitted from the satellites, multiplies them by the speed of light to determine exactly how far the unit was from every satellite it was sampling. By locking onto the signals from a minimum of three different satellites, the GPS receiver could calculate a two-dimensional positional fix, consisting of latitude and longitude. The GPS receiver was interfaced with an ARM through RS232 converter. RS232 converter was used to convert RS232 logic to TTL logic and vice versa because the GPS receiver used is on RS232 logic while the ARM-7 is on the TTL logic. The receiver sent the received signal to ARM-7 which is the flash type programmable ARM that was earlier programmed by the authors. The ARM-7 displays the person location on an LCD display. This position information was transmitted through the GSM network whereby the person could be remotely tracked. The system developed was very bulky and there was no real time tracking of persons.
Akinyokun et al., [7] proposed a framework for combating human trafficking and human kidnapping using Smart Objects and Internet of things. The research was motivated by the following:
A. High rate of human kidnapping and human trafficking which has attained global dimension and has defied several measures adopted by governments and organizations.
B. The advancement and complexity of present-day society which has made most of the crimes to go online and high tech in nature, hence the need for matching strategies.
C. The use of human personnel for surveillance has been ineffective and susceptible.
The general objective is to develop a system for combating human kidnapping in Ondo State of Nigeria. The specific objectives are to:
A. Carry out baseline study of human kidnapping in Nigeria and develop its national dataset.
B. Design a Smart Objects, Internet-of Things and whistle blower model for combating human kidnapping in On-do State.
C. Design all-inclusive national dataset on human kidnapping in Nigeria
D. Implement the models designed in (b) and (c)
E. Perform experimental study of the models based on the results in (d).
In order to accomplish the objectives of the research, the authors carried out an extensive review of previous research works on RFID, GPS, Smart Objects and IoT. The proposed framework has two phases namely: hardware operational requirements phase and software analytic framework phase.
The authors have developed adaptive video analytic framework for real-time human monitoring in workplace, but with very limited operational range. They have also developed a fingerprint recognition system which only operates in offline mode and a hybrid system for human resource monitoring of the activities and movements in the workplace which gives no considerations to individuals with unknown identities. Though, there is evidence of substantial preliminary works on the research domain, the issue of identifying and tracking kidnappers has not been addressed. The aforementioned limitations imply that the reported works are not suitable for providing lasting solution to the prevailing problem of human kidnapping. Further challenges to addressing the problem of human kidnapping include lack of national dataset as well as absence of scientific and standard techniques for combating it.
The system employs Smart Objects and Internet of Things wearable devices which are made up of Arduino Uno microcontroller, wireless RFID tags/readers and GPS modules. Hardware Operation Requirements Phase which has four stations. First, is the Sensor Processing Stations (SPS) which have workstations equipped with video camera sensors, RFID tags, GPS receivers and body worn sensors. Video camera sensors capture the video data of the area being monitored. A non-digital camera embedded into a broach type model attached to buttons or human hair for taking and forwarding real time pictures to the Digital Station Room (DSR). RFID tags monitor objects indoor while GPS handle global monitoring. The SPS has ID cards carrying chips embedded in the skin, wrist bands, button or hair of individuals for checkpoints identification of carriers. Second, the captured data are stored in the cloud-based Media Server Station (MSS).
Third, the Smart Engine Server (SES) located in the cloud handles motion detection, face recognition, position tracking and activities recognition. The Digital Situation Room (DSR) monitors events in real time and on demand using RFID tags and embedded communication protocols version chips. The Internet protocols version is selected for addressing scheme for data transfer on the web. The Non-Visual Module (NVM) comprises IoT objects, RFID sensors and GPS that are for signal and location, The Video Capture Station (VCS) has digital cameras and software for sensing, digitizing and compressing video cameras.
There is a visual monitoring subsystem, which make use of Internet as communication medium. Video data captured in monitored area is sent to video captured adapter board attached to the system. On the video capture adapter card, an Analogue to Digital Converter (ADC) chip converts the wavering analogue video and audio signals to a pattern of 0s and 1s. The Internet will be used as the communication framework for real time video transmission to remote locations. The Digital Situation Room (DSR) will be used to view the raw video files after they have been stored on the server. Each video playback station will have playback client software based on Microsoft media player Figure 1.
Software analytic framework phase which consists of the following modules for object location, detection and tracking using RFID analyzer, face recognition and activity recognition:
A. Video Analyzer: The system cameras will capture images of the scene and will record it in a video storage device or passed to the next modules for analysis. Identity recognition and intrusion detection will then be carried out using a video analysis module. After a kidnapping scenario is detected by the system, human detection, classification and tracking operations will follow, features will then be extracted from the face and the whole body. The extracted facial features will be passed to the face recognition module while the features from the whole body will be passed to the activity recognition module. The extracted set of features will also be used to identify intruders. The activity recognition module is responsible for action segmentation and recognition. These modules generate different alerts which are stored in the database for subsequent retrieval or viewing. It is noted that face and activity analysis modules can be executed in parallel.
B. GPS Analyzer: GPS is for global location and tracking. For outdoor surveillance, human beings will be equipped with GPS receivers and the location of the user will be tracked continuously. Obtained information will be sent to the DSR and in case the location is outrageous; an alert will be generated as acknowledgement to the remote user for onward transmission to the law enforcement agents for necessary action. In case of removal of the devices from the user, an instantaneous alert will also be sent.
C. RFID Analyzer: The RFID will be used for indoor localization. For indoor surveillance, human beings will be equipped with wireless wearable sensors and RFID tags. While indoor and with the help of wireless sensor networks, the location of the user in the environment will be tracked continuously. This information will be sent through GSM to remote stations that are monitoring the objects. If the allowed radius of movement is exceeded, an alert will be generated and sent to the remote user while a short video stream of the previous durations will be sent to the remote user for transmission to the law enforcement agents for necessary action.
D. Sensor Analyzer: This will be used to analyze and connect other sensors such as infrared sensors, motion sensors, ultrasonic sensors, laser sensors and buzzers to the tracking system.

Method And Materials of the Proposed System
The method and materials employed for combating human kidnapping using Smart Objects and Internet of Things is presented in this Section which consists of the proposed system architecture shown in Figure 2,3. The architecture has two phases namely: hardware operational requirements phase and software analytic framework phase. The hardware operational environment of the integrated sensor framework has four stations namely: Sensor Processing Station (SPS), Media Server Station (MSS), Smart Engine Server (SES) and Security Management Station (SMS). The SPS has hardware and software for signal sensing and comprises workstations equipped with video camera sensors, RFID tags, GPS receivers and body worn sensors. Video camera sensors are required for capturing the video data of the area being monitored. A nano-digital camera is also embedded into a broach-type model attached/hidden to/in buttons or human hair for taking and forwarding real time pictures to a Digital Server Room (DSR). The hidden wearable device worn by the kidnapped victim, consisting of a nano-digital camera and sensors, will be able to capture the picture of the kidnapper alongside the GPS coordinates of the location and transmit to a remote server.


RFID tags are required for monitoring objects’ location in indoor environment while GPS provides global monitoring. The SPS requires ID cards carrying chips embedded tools for effective checkpoint identification of carriers. These chips could be embedded, injected beneath the skin, placed into wristbands, placed into button or hair of an individual for monitoring and safety. The captured data from the sensor processing unit are stored on the cloud-based MSS. The SES is responsible for logical reasoning of the system. Internet Protocol version 6 (IPv6) was selected as the communication protocol and for provisioning of addressing scheme for data transfer on the web. The hardware components of the Non-Visual Module (NVM) comprise of IoT objects and RFID sensors and Geographical Positioning Systems that are responsible for signals and location. The Visual Module (VM) on the other hand comprises of camera network that consistently captures snapshots of an area and sends it to a remote server as shown in Figure 4.

Idachaba et al. [8] developed an IoT based smart campus wide shuttle tracking system. The authors designed and implemented an integrated, embedded GPS-GSM vehicle tracking system. The application enables passengers to view the location of bus shuttles in real time. The authors also reported that the device allows bus managers to ascertain the driver behavior and observe the past and present locations of the bus shuttle. Users can access the location of the buses either using the SMS request option or utilizing the mobile application developed as an alternative. The tracking system sends feedback which includes both the latitude and longitude coordinates of the vehicle. The authors reported that the coordinates can also be logged to a remote server for further trail analysis on Google Map. The is no fail-safe alternative in scenarios when the GPS receiver runs out of battery or in instances where there is no network reception on the single SIM card used on the device.
Tiwarkar et al., [9] proposed an android-based solution to aid parents to track their children in real time. The aim of the research was to develop a safety system which provides details of entry and exit of the student from school using RFID and GPS technology. The authors utilized Android studio and Arduino IDE for the software development environment and Node MCU, RC522 Sensor, ZX303 PCB and ESP8266 as the hardware devices. The position notification to parents is primarily via SMS and this is a single point of failure as it is strongly reliant on the network availability on the network carrier for the SIM card.
Khutar et al., [10] developed a smart system for school children tracking. The authors were motivated by the rising cases of missing children in the country. The tracking system used Global System for Mobile Communications GSM and Global Positioning System (GPS) as the main tools to carry out the mission. The designed device can be used easily by carrying it manually, placing it inside the child’s clothes, attaching it to his belt or wearing it like a wristwatch. When the child exceeds the limits of the area specified for him, the tracking device is activated, and an alert message is sent to the person responsible for tracking him to confirm that he is informed of the child’s new location. The tracker can be triggered using a Short Messaging System (SMS) where the user is able to communicate remotely to the GSM using a mobile phone. The system is designed with a GPS antenna, GPS receiver, Microcontroller Arduino Mobile or GSM modem and a personal computer or cellular phone. The solution has the drawback of power drain and eventually shutdown as it is operated by a battery and also the SMS delivery is only guaranteed only when signal is available on the sole SIM card utilized in the device.
Kamalraj & Pooja [11] carried out a survey on RFID child monitoring system using IoT. The authors proposed the development of child monitoring systems that utilize a combination of GSM and GPS technology. When the child presses an SOS button, the location of the child is sent to the parents’ phone number that has been preconfigured on the device, via SMS. Little to no information is given about the architecture of the solution. The proposed system is also prone to a single point of failure has it is heavily reliant on the battery-operated device. Also, there is a high likelihood of message delivery failure as a result of loss of network signal.
Shylesh et al., [12] proposed an IoT based smart security and safety system for women and children. Microcontroller, GSM and GPS module were used to send notification as and current location of women to various mobile numbers on their contact. The author reported that the proposed solution is a self-defense device via which on occasion of emergency, the woman or child have to press a button to trigger distress notification. The challenges with this solution are delay of notification when there is no signal on the GSM network, the victim may not have the chance to trigger the notification by pressing the SOS button due to panic or hands being tied up by her abductors.
Database of Victims, Devices and Landmarks
The first step in the design of the system is the development of the database infrastructure. The database includes the kidnap victims, wearable devices, electronic devices and landmarks. The database is conceptualized as follows:
A. Personal details {per-id, per-name, per-other name, per-gender, per-marital-status, per-dob, per-photo location, per-email-add, per-phone, per-passport-number, per-wearable-id, per-photo location, per-wd}
B. Fixed Device details {dev-id, dev-name, dev-location, dev-macadd, dev-longitude, dev-latitude, dev-sim no}
C. Wearable Device details {wd-id, wd-macid, wd-simno}
RFID Readers and Mounted Cameras Location
The physical landmarks where the fixed devices like RFID readers and cameras are mounted and the database of all necessary parameters will be created. The following are required namely: Landmark name, GPS location coordinates and Landmark sensor/device ID.
Electronic Devices
This is the database of GPS receivers, RFID readers and IP cameras. The information needed includes device name, device type, physical location of electronic devices, media access control address, serial number, deployment information and assigned identification code.
Localization and Positioning Data Generation
The localization and positioning data generation sub-system handles the detection of signals by the individual readers, carries out the position calculation and final estimation of position. Due to the nature of human kidnapping, search and rescue efforts are basically done outdoors. Hence, open-air outdoor and signal-denied areas are crucial factors to be considered in this research.
GPS Positioning Estimation
GPS comprises a network of 24 satellites which are in six different 12-hour orbital paths that are strategically spaced out so that at least there are five in view from every point on the globe, as depicted in Figure 5. There are five monitor stations and four ground antennas as at the time of this research, located in various places around the world that gather range data on each satellite exact position. The information is relayed to a master control station whose responsibility is to provide overall coordination of the network and transmit correction data to the satellites.
By measuring the distance from a group of satellites in space, an object position on earth is calculated. This calculation is carried out by measuring the time it takes for a radio signal from a particular satellite to reach the object of interest. The radio signal travels at 186,000 miles per second, which is the speed of light, and the distance is computed using the formula:
Distance = Velocity x Time ………………………… Equation 3.1
GPS satellite ranging allows a receiver to determine its three dimensions position namely latitude, longitude and height. In a GPS system, the user whose position is to be determined must have access to at least four satellites in orbit at the same time [13]. The subsequent calculation of the position of the user is obtained by solving a set of equations based on Pythagoras theorem in three dimensions and a time offset from the GPS reference time as shown in Equations 3.2, 3.3, 3.4 and 3.5. Working in three dimensions, there is a need for information from four satellites. Call these S1,S2,S3
, and S4; and suppose that Si is located at (Xi,Yi,Zi ) when it transmits a signal at time Ti . If the signals are received at times , according to the clock in the receiver, let Δti = Ti − Ti ,and let ε represent any error in the clock’s time. The receiver allows for the mean effects of passage through the earth’s atmosphere and computes distances that indicates how far observers are from each of the satellites [14].ss


The position (xo, yo, zo) is located on each of four huge spheres. In most situations, there will be only one sensible value of ε that allows the spheres to have a point in common. The location is determined by solving the system of equations. When a numerical solution is found, the rectangular coordinates (xo, yo, zo) are converted into essentially spherical coordinates of latitude, longitude, and altitude above sea level.

GPS NMEA Format
The GPS readings are expressed as the National Marine Electronics Association (NMEA). NMEA format gives equipment users the ability to mix and match hardware and software. NMEA-formatted GPS aids software developers in developing software for a wide variety of GPS receivers instead of having to write a custom interface for each GPS receiver [15].
GPS Accuracy
In the GPS measurement system, Line of Sight (LOS) access to at least four satellites are required as shown in Figure 6, refers to the ith. satellite (1, 2, 3, 4) coordinates,
indicates the GPS receiver coordinates and Ri represents satellite range as
.
Kalman filter is used to reduce GPS errors and thus increases the accuracy of the localization system. It is an algorithm that uses a series of measurements observed over time, containing statistical noise and other inaccuracies and produces estimates of unknown variables that tend to be more accurate than those based on a single measurement alone, by estimating a joint probability distribution over the variables for each time frame. The states of the system are defined to model the system dynamics. Also, a measurement model is defined to characterize the relationship between the state vector and any measurement. The Kalman filter procedure shown in Figure 5 is initiated by the assumption of the initial estimate of states, and its error covariance,
. The optimal Kalman gain
is utilized to achieve the update estimate of the pseudo range measurements
and its error covariance
is then calculated. The next state
and error covariance
ρ is then calculated based on the current state estimate. The GPS accuracy is measured using the Twice Distance Root Mean Squared, ρ , as shown in Equation 3.5:
where and are the standard deviations of latitude and longitude respectively for the estimated coordinates by Kalman filter. Kalman filter is used to reduce GPS errors and thus increases the accuracy of the localization system. It is an algorithm that uses a series of measurements observed over time, containing statistical noise and other inaccuracies and produces estimates of unknown variables that tend to be more accurate than those based on a single measurement alone, by estimating a joint probability distribution over the variables for each time frame. The states of the system are defined to model the system dynamics. Also, a measurement model is defined to characterize the relationship between the state vector and any measurement. The Kalman filter procedure shown in Figure 7 is initiated by the assumption of the initial estimate of states,
and its error covariance,
. The optimal Kalman Kk gain is utilized to achieve the update estimate of the pseudo range measurements
and its error covariance
is then calculated. The next state
and error covariance
is then calculated based on the current state estimate. The GPS accuracy is measured using the Twice Distance Root Mean Squared, ρ, as shown in Equation 3.5 [16-20].


RFID Positioning Estimation
The progressive advancement in RFID technology has brought about advantages related to optimization of resources, increase in efficiency with business processes, enhanced customer care. RFID tags store unique identification information of objects and communicate with the tags in order to allow remote retrieval of their unique identifier.
Face Identification and Object Recognition
So many times, a kidnapping event would have occurred hours before it is known to friends and family of the victim, simply because there is an absence of a real-time mechanism to identify the pointers or common objects of a kidnapping scene. Some of these common objects are facemasks, guns, cutlasses and so on. Traditional Closed-Circuit TV (CCTV) can record the scene but will need a replay or continuous human monitoring of the scene which is very unreliable, expensive and delayed solution. The proposed solution will include a Scene Monitoring Module (SMM) that will harness the features of computer vision technology to automatically in real-time identify a pre-defined list of notable objects at a crime scene to further trigger a possible notification of human kidnapping event to a remote server, also including the geolocation of the said scene. Furthermore, there have been instances of kidnapped victims seen in a different state of abduction and not recognized by occupants of the new location. The proposed solution will feature a facial recognition module that upon identification of a face of a previously enlisted missing victim, it will send a snapshot of the victim and the geolocation coordinates to a remote server so an instant search and recovery effort can be carried out by the nearest authorities [21,22].
There are notable objects that are prevalent in crime scenes like guns, facemask, cutlass, knife and so on. A model will be trained to recognize these objects both on a LIVE video field or snapshot and be able to send a distress notification to the DSR module or a pre-configured hotline. This will consequently improve the response time of authorities to a crime scene as against watching a non-smart recording of an event that could have been nipped in the bud.
As depicted in Section 3.1.1b, every camera on the architecture is annotated with a location coordinate of the scene of installation. Hence, any distress notification trigger sent to the control room or authorities carries a place-of-origin signature that will enable the easy identification of the place of interest. For example, a camera installed at Ring Road Junction, Ibadan, Nigeria, is annotated as [Cam001, longitude, latitude].
Similarly, a pre-trained model harnessing the VGG (Visual Geometry Group) model, which is a very deep convolutional network, is used for recognizing faces of already profiled images of victims on the architecture. Upon recognition of the face of the victim within the LIVE video-stream scanning, by finding a match, a notification is automatically sent to the control room with a location signature of the camera via which the triggered notification came through. VGG Face is renowned for better recognition accuracy than other models, hence its adoption in our experimentation [23].
GPS Signal Denied Areas
One of the major challenges with GPS technology is the areas where there is no reliable link to the satellites in orbit. Examples of such areas are tunnels, inside buildings, basements, heavy forest areas and so on. The proposed alternative to address this is the use of RFID tags to identify these places as landmarks and to carry out a location estimation calculation based on the closest GPS reliable zone [24].
The estimation of positioning data in indoor environment is based on hybridization of Time of Arrival (ToA) and Received Signal Strength Indicator (RSSI) distance estimation techniques. In ToA technique, the distance α between the HR and the reader in a direct Line of Sight (LOS) environment is obtained based on the formula

where v is the velocity of the signal, t is the time spent by the signal travelling from the reader to the tag worn by the victim. In a non-LOS environment, RSSI estimation technique was used based on the signal strength of the transmission between the reader and the tag which is converted to distance by using the Euclidean equation as follows:

where d represents the distance between the reference tag and unknown tag, qi is the signal strength of the reference tag, pi is the signal strength of the unknown tag received on the reader and n is the number of times measurement is taken [25-28]. The physical location of the tag was obtained from the measurement of distances by using Trilateration method which is expressed as follows:

r1, r2 and r3 are the distances of the unknown tag from the reference positions respectively.
Microcontroller
The majority of the devices in this research are meant to be autonomous and wearable. One important characteristic of the wearable devices for a tracking application is covertness and miniaturization, that is, it can be made in a very small form factor and secretly worn and hidden out of sight from adopters. Therefore, a microcontroller will meet these requirements. A microcontroller is a compact integrated circuit designed to govern a specific operation in an embedded system. A typical microcontroller includes a processor, memory and input/output (I/O) peripherals on a single chip. The structure of the adopted microcontroller is shown in Figure 8 [29,30].


The Central Processing Unit (CPU) is the brain of a microcontroller, and is responsible for fetching the instruction, decodes it and finally executes the code. It connects every part of a microcontroller into a single system. The function of memory is the same as a microprocessor. It stores data and program, and it has a certain amount of RAM and ROM (EEPROM, EPROM and so on) or flash memories for storing program source codes. The Parallel input/output ports are mainly used to drive/interface various devices such as LCDs, LEDs, printers, memories and so on while the Serial ports provide various interfaces between a microcontroller and other peripherals like parallel ports. The Timers/Counters provide timing and counting based on clock functions, odulations, pulse generations, frequency measuring, making oscillations and so on. The Analog to Digital Converter (ADC) are used for converting the analog signal to digital form which can be used for various digital applications. The Digital to Analog Converter (DAC) performs the opposite operation of ADC conversion. The interrupt control provides interrupt (delay) for a working program, and it may be activated by using interrupt pin or by using interrupt instruction during programming. The special functioning block featuring space systems and robotics are used only for some special applications [31-36].
Hybrid Location Engine
The hybrid location engine serves as a centralization unit by bringing together all captured signals by the various heterogeneous sensors on the positioning and tracking system. The algorithm that runs on this engine periodically calculates and estimates the position of the human being tracked. In order to make an inferential conclusion out of all the signals from the various components and modules, the Hybrid Location Engine (HLE) will take these feedbacks as input, carry out necessary computations and analysis to produce a well-informed output that sufficiently gives a reliable position of the tracked individual [37,38].
Implementation of the Proposed System
This section presents the implementation of the proposed system. It covers the technical details of the hardware and software requirements, the setting of the experimental testbed, the implementation procedures and user interfaces of the application. A case study of the application for monitoring a presumed kidnapped victims between Ibadan, Oyo State and Ilishan Remo, Ogun State, in Nigeria was carried out to test the adequacy and practical functions of the system. The evaluation and comparative analysis of the system is based on coverage area, cost incurred in deployment, positioning accuracy and signal sensitivity [39-41].
Hardware Requirements
The hardware requirements are GPS modules, GSM modules, RFID tags, antennas and readers, IP cameras, Arduino-based microcontrollers and cloud servers. The research utilized ESP32 SIM800L T-call GPRS Microcontroller and NEO 6M GPS modules for the GPS tracking. The components are shown in Plate 4.1. A combination of these two modules allows for a seamless tracking of an object at any location and at any time if it is within the signal coverage of the underlying GSM network. The ESP32 SIM800L T-Call GPRS features very low power consumption when in sleep mode especially when integrated with batteries having a long standby time. The NEO 6m GPS has reliable signal reception sensitivity. The module is controlled by AT commands via UART and supports both 3.3V and 5V logical level. The ESP32 SIM800L offers the advantage of portability. These are depicted in Figure 9 (a,b,c).
The hardware specifications of both the ESP32 and GPS Module are shown in Appendix A and Appendix B respectively. The complementary RFID tracking sub-system made use of wearable RFID tags, antennas and readers strategically stationed in designated landmarks. The RFID sub-system also supports multi-detection of tags and offers a high baud rate that ranges from 2400 bps to 19200 bps. The chosen RFID reader used in the implementation has a tag read range of 50 meters. It is equipped with an antenna to extend the signal coverage. These are depicted in Figures 10(a,b,c). Figure 10 (d) shows one of the volunteers for the experiment wearing an RFID trackable wristband on his left hand
Software Requirements
The system is developed using Python programming language, MySQL database management system, XAMPP as the local server and Google Map API as the mapping and geo-coding platform. Recognition AWS Cloud services are also utilized in the development of the system.
Experimental Test-bed Setup
The research was conducted on the Housing Estate at North Gate, Sasa, Ojoo, Federal School of Statistics zone, Ibadan, Oyo State, Nigeria. It is in the Southern part of Oyo State. The aerial view of the housing estate landscape is shown in Figure 11. It is located within the following satellite coordinates (7.477847) and (3.907565) to the north of the estate and (7.478070) and (3.908681) to the south of the estate. The position tracking of an assumed kidnapped victim was used as a case study for the research. The arial capture on Figure 11 was captured using Google Maps platform. The coordinate of the area was utilized in the geolocating. The red marker on the center of the map indicates the current position of the simulated assumed victim [41].
System Implementation
The application is activated by clicking on a web browser and entering the web address of the application in the address bar, which will take the user to the home page as depicted in Figure 12. When the “Read Me” button is clicked, it opens the page showing a brief description of the system as shown in Figure 13. The user is taken to the login page when he or she clicks on the “Login” button. On this page, the user registers as the System Administrator to use the system for the first time as shown in Figure 14.





Password Reset Screen to Help Users Recover their Forgotten Password
On successful registration, the user is expected to sign into the system through the view shown in Figure 14. There is a “Password Reminder” module to assist the users in cases where they cannot recall their correct username or password. On successful login, the user is taken to the configuration page, where the following entities can be set up:\
A. Profile
B. Mobile Electronic Devices
C. Immobile Electronic Devices
D. Landmarks
E. Geo-locations
After a successful configuration of the system, an enrolled user can be tracked anywhere in the world if he or she is putting on the wearable device. Due to the unpredictability of location in kidnapping cases, no perimeter restriction is set up on the system. This is to enable the possibility of a missing victim to be tracked from any location.
Case Study of a Simulated Victim
An RFID reader and IP camera were installed at a strategic location within the premises. The RFID reader scans and monitors the environment to read tags within its range that have been registered on the database. Similarly, the IP camera, which is linked with a pre-trained facial recognition and identification model monitors the environment to compare the faces within its field coverage to faces on the database that has been flagged for surveillance search. The RFID reader serves as a fail-safe alternative to the GPS receiver which is battery operated and renowned for making it impossible to track when the battery drains out. When this is the case, any closest long-range RFID reader can pick up a signal from the wearable RFID tag on the victim’s dress accessories and trigger a notification message to the server room or web portal Figure 15(a,b,c), 16.




The captured data stream was connected to the latest edition of Google Earth Pro, version 7.1 to view and monitor the progression of the movement of the kidnapped victim. Figure 17 shows the test bed area where the simulated kidnapping tracking was carried out. The land area spans from Ibadan through Ibadan-Lagos express way and terminating the tracking at Babcock University, Ilisan Remo, Ogun State. Figure 18 shows the timeline of the victim’s movement. It is depicted with a dynamic red marker on the map. The plot was generated at a rate of one record per second, hence the crowed marker of the victim’s location on the map. This was done to make sure we can track the location of the victim in near real time as much as possible, specifically per seconds in this instance.
On Figure 19, the map plot rate was set at a record per minute, hence an enhanced timeline was achieved. On Figures 20, 21, an info tip marker was generated for each point. This is useful in a worst-case scenario when the wearable GPS receiver has run out of power supply due to a drained battery. The platform provides in addition to historical coordinates, the timestamp of any point of interest. Figure 22 shows a no-signal zone between two close cities, Ode Remo and Ilishan Remo in Ogun State. This is a potential landmark for RFID Reader installation to serve as a fail-safe alternative. Figures 23,24 show the crowed position markers depicting the slow movement of the victim as the course has moved off the express way to an intra-city road. The end of the tracking terminates at Bethel Splendor Male Hall at Babcock University, Ilisan Remo, Ogun State.








Image Recognition Module for Weapons
An image recognition module was developed by training a model using 500 images of different kinds of guns. An instance of the data set is shown in Figure 25(a). The AWS Recognition engine was utilized in this training. The dataset was divided into a training set and test set on an 80:20 ratio. The final model produced a recognition accuracy of 99%. Figure 25(a) shows the model’s ability to recognize weapon on a scenery. The recognition accuracy for Gun is 99.6%, Weapon is 99.6%, riffle is 75.2% and so on. This accuracy score can be used as trigger to initiate a distress notification to the server room [42-45].
Figure 25(b) shows the ability of the trained model to recognize weaponry in various shapes. This was necessary so that the model is not narrowed to recognizing just one time of weapon shape. An accuracy average of 90% was still maintained. Figure 25(c) shows the ability of the model to recognize a weapon on a scene when carried by a person. This is a common object at a crime scene hence of valuable importance to serve as a trigger for a distress notification to the remote server upon recognition on a live stream video which is one of the limitations of a regular CCTV that does not notify in real time.





Image Recognition Module for Cutlasses
An image recognition module for cutlasses was developed by training a model using 400 images of different kinds of cutlasses. An instance of the data set is shown in Figure 11. The AWS Recognition engine was utilized in this training. The dataset was divided into a training set and test set on an 80:20 ratio. The final model produced a recognition accuracy of 80%. Figure 26(a,b) are subsets of the images used for the training of the cutlass. Gathering quality dataset for cutlass on a crime scene was a challenge. The closest labels it could be identified with is Weapon at 81.5% accuracy, Sword at 81.5%, blade at 81.5% accuracy and weaponry at also 81.5%. The model was trained with this synonym in mind and the average accuracy is 81% as shown on Plate 4.17.
Facial Recognition and Identification Module
Two datasets were utilized in training the facial recognition and identification model. First, the Flickr-Faces-HD Dataset (FFHQ) public facial image dataset, which consist of human faces which were crawled from Flickr, was utilized. The dataset consists of 70,000 high quality PNG images at 1024 by 1024 resolution. A bespoke facial dataset was generated by taking pictures of 10 volunteers. An image recognition module was developed by training a model using 500 facial images. An instance of the data set is shown in Figure 27(a). Figure 27(b) shows the resultant ability of the trained model to be able to predict the age range, gender and other important features that can match a reported kidnapped victim’s features.
Figure 27 (c) shows the ability of the model to identify a person with a similarity score of 99.9% irrespective of the difference in facial expression. Figures 27 (d,e) show the buildup of the dataset from unlabeled data to a label dataset (Victim 1) respectively. The AWS Recognition engine was utilized in the training. The dataset was divided into a training set and test set on an 80:20 ratio. The final model produced a recognition accuracy of above 91%.





Comparative Analysis with Existing Systems
The results obtained in this research work were compared with other existing systems. The comparisons were based on the availability to a fail-safe alternative to GPS signal, victim identification, environment where they were deployed, that is, either indoor or outdoor and the metrics considered were technologies used, algorithm, accuracy, complexity, scalability, robustness and cost. The comparative analysis of the system with other works Table 1.

Conclusions
Smart Objects and Internet of Things (IoT), RFID, GPS, Microcontrollers, GSM module, Smart cameras were used in the research. A web application portal was developed for easy profiling of devices, victims, landmarks and real-time tracking of object and facial recognition. The research developed a system for combating human kidnapping using Smart Objects and IoT with a view to improving the security awareness of a community and providing data for informative investigation and efficient tracking of kidnapped victims. The study successfully tracks, in real-time, a victim wearing covert GPS device and recognize simulated victims at landmarks using facial recognition technology at 90% accuracy. Scene analysis was also successfully carried out where key objects like cutlasses, guns and facemasks are automatically detected by the object recognition algorithm. RFID has been used to implement fail-safe alternative in areas where there is no signal or weak GPS signal or there is loss of power on the GPS device.
References
- Graham I (2005) Kidnapping by Country: Definition, Graph and Map.
- UN-CTS M5.5. United Nations Surveys on Crime Trends and the Operations of Criminal Justice Systems (UN-CTS). M5.5.
- Protus NU, John NB (2014) Challenges of Hostage Taking and Kidnapping in the Southeastern Nigeria. International Journal of Research in Humanities, Arts and Literature 2(6): 131-142.
- Gladysz B, Santarek K (2017) An Approach to RTLS Selection. Proceedings of 24th International Conference on Production Research (ICPR 2017).
- Kortuem G, Kawsar F, Fitton D, Sundramoorthy V (2010) Smart Objects as Building Blocks for the Internet of Things. Internet of Things Track, IEEE Internet Computing, IEEE Computer Society 14(1): 44-51.
- Chaudhari A, Bohra S, Karma H, Dhupadale A (2015) GPS/GSM Enabled Person Tracking System. International Journal of Innovative Research in Science, Engineering and Technology 4(3).
- Akinyokun OC, Akintola KG, Iwasokun GB, Angaye CO, Arekete SA (2018) Design of a Framework for Combating Human Trafficking and Kidnapping using Smart Objects and Internet-of-Things. International Journal of Artificial Intelligence Research 7(1): 45-52.
- Idachaba FE, Odufowokan A, Uyi Aiyudubie (2020) Development of an IoT based Smart Campus Wide Shuttle Tracking System. Proceedings of the Future Technologies Conference FTC2020 3: 610-621.
- Tiwarkar SA, Bhumannavar SS, Kshirsagar GK, Ghare AD (2020) School Child Tracker System using IoT. International Journal of Innovative Science and Research Technology 5(1).
- Dawood Zahi Khutar, Omar Hashim Yahya, Haider Th Salim Alrikabi (2021) Design and Implementation of a Smart System for School Children Tracking. IOP Conference Series: Materials Science and Engineering.
- Kamalraj R, Pooja R (2022) RFID Child Monitoring System using IoT: A Survey. International Journal of Computational Engineering Research (IJCER) 12(1).
- Shylesh S, Shahan K, Reddy YAK, Naveen KK, Manasa K (2022) IoT Based Smart Security and Safety System for Women and Children. Journal of Emerging Technologies and Innovative Research (JETIR) 9(5): f830-f834.
- Adewole AB (2019) Development of a Hybrid Human Resources Location Monitoring System. PhD Thesis, Department of Computer Science, Federal University of Technology, Akure, Nigeria.
- Matthews VO, Adetiba E, Ike DN (2013) A Covert Kidnapping Alert and Location Identifier (CKALI)”. International Journal of Innovative Technology and Creative Engineering 3(12): 1-11.
- Pankaj V, Bhatia JS (2013) Design and Development of GPS-GSM Based Tracking System with Google Map Based Monitoring. International Journal of Computer Science, Engineering and Applications 3(3).
- Hadi NHB, Saleh HBM, Asi SBM (2015) Prevention System for Child Abduction by using Image Processing Algorithm on Intel Galileo Board. Journal of Electrical and Electronics Engineering (JEEE) 5(2): 1-8.
- Adewole DB, Akinyokun OC, Iwasokun GB (2019) Hybrid Human Resources Localization and Tracking System. International Journal of Artificial Intelligence Research 8(1): 1-13.
- Poonkodi R, Devaki E, Karthika M (2019) Internet of Things Based Smart Object Tracking System. International Research Journal of Engineering and Technology 6(2): 929-932.
- Akinode JL, Alawode AJ, Ojuawo OO (2010) Improving National Security using GPS Tracking System Technology. Proceedings of First International Conference on Technology, Education and Environment Conference, African Society for Scientific Research 2(5): 634-645.
- Kosmatos E, Tselikas N, Boucouvalas A (2011) Integrating RFIDs and Smart Objects into a Unified Internet of Things Architecture. Advances in Internet of Things 1(1): 5-12.
- Akintola KG, Boyinbode OK (2011) The Place of Emerging RFID Technology in National Security and Development. International Journal of Smart Home 5(2): 37-44.
- Rozyyev A, Hasbullah H, Subhan F (2011) Indoor Child Tracking in Wireless Sensor Network using Fuzzy Logic Technique Research. Journal of Information Technology 3(2): 81-92.
- Okengwu K (2011) Kidnapping in Nigeria: Issues and Common-Sense Ways of Surviving. Global Journal of Educational Research 1(1): 1-8.
- Ning Y, Zhong W, Malekian R, Ru-chuan W, Adbullah AH (2013) Design of Accurate Vehicle System using RFID. Journal of Elektronika IR Electrotechnika 19(8): 105-110.
- Pankaj V, Bhatia JS (2013) Design and Development of GPS-GSM Based Tracking System with Google Map Based Monitoring. International Journal of Computer Science, Engineering and Applications 3(3).
- Ayman MA (2013) Using of Tracking Systems for Devices Designing to Face Children Kidnapping. Phenomenon (GSM-GPRS-GPS). International Journal of Scientific and Research Publications 3(10).
- Hernández MEP, Reiff-Marganiec S (2014) Classifying Smart Objects using Capabilities. Proceedings of International Conference on Smart Computing.
- Atzori L, Iera A, Morabito G (2014) From Smart Objects to Social Objects: The Next Evolutionary Step of the Internet of Things. IEEE Communications Magazine 52(1): 97-105.
- Sungur C, Gokgunduz HB, Altun AA (2014) Road Vehicles Identification and Positioning System. Proceedings of the 2014 Federated Conference on Computer Science and Information Systems, Pp. 1353-1359.
- Fortino G, Guerrieri A, Russo W, Savaglio C (2014) Middlewares for Smart Objects and Smart Environments: Overview and Comparison, in Internet of Things based on Smart Objects: Technology, Middleware and Applications, Springer Series on the Internet of Things. Journal of Technology, Communications and Computing, USA.
- Xing L, Orlando B (2016) A comparison of the Definitions for Smart Sensors, Smart Objects, and Things in IoT. IEEE Annual Information Technology, Electronics and Mobile Communication Conference Proceedings, Pp. 1-4.
- Patel KK, Patel SM (2016) Internet of Things-IOT: Definition, Characteristics, Architecture, Enabling Technologies, Application and Future Challenges. International Journal of Engineering Science 6(5).
- Dukare SS, Patil DA, Rane KP (2015) Vehicle Tracking, Monitoring and Alerting System: A Review. Journal of Computer Applications 119(10).
- Awolusi AB (2015) Development of Online System for Monitoring of Personnel in Public Service. MTech Thesis, Department of Computer Science, Federal University of Technology, Akure, Nigeria.
- Ahmad A, Rathore M, Paul A, Hong W, Seo H (2016) Context-Aware Mobile Sensors for Sensing Discrete Events in Smart Environment. Journal of Sensors 2016: 1-10.
- Du M, Jing C, Du M (2016) Tag Location Method Integrating GNSS and RFID Technology. The Journal of Global Positioning Systems 14(2).
- Awolusi AB, Akinyokun OC, Iwasokun GB (2016) RFID and RTLS-Based Human Resource Monitoring System. British Journal of Mathematics and Computer Science 14(4): 1-14.
- Akinyokun OC, Akintola KG, Olabode O (2016) Real-Time Whole-Body Action Recognition in Videos using Threshold Hidden Markov Model. Journal of Mathematical Sciences of the National Mathematical Centre 4(1).
- Ibrahim B, Mukhtar JI (2016) Changing Pattern of Prostitution: An Assessment of Transnational Commercial sex by Nigerian Women. European Scientific Journal 12(2): 81-95.
- Abdulkabir OS (2017) Causes and Incisive Solutions to the Widespread of Kidnapping in Nigeria Current Administration: Under Scholastic Scrutiny. Journal of Political Science 5(2): 1-6.
- Akinyokun OC, Akintola KG, Iwasokun GB (2017) A Framework for Combating Human Trafficking and Kidnapping using Smart Objects and Internet-of-Things. Proceedings of the Second International Conference on Applied Information Technology (AIT 17), Abeokuta, Nigeria, Pp. 9-15.
- Akinyokun OC, Adewole DB (2018) Monitoring of Human Resources in the Workplace. International Journal of Current Advanced Research 7(7): 14215-14222.
- Munshi N (2019) Threat of Abduction Makes Nigeria a High-Risk Business, Financial Times.
- Adewole DB, Akinyokun OC (2020) Workplace and Human Resource Safety Monitoring using Internet of Things, Journal of Artificial Intelligence Research, Canada.
- Punch (2021) Source: https://punchng.com/2371-people-abducted-in-nigeria-in-first-half-of-2021-report.