key: cord-0058771-wcaxdprr authors: Buele, Jorge; Yánez-Arcos, Estefanía; Moscoso, María Elena; Huilca, Jordan S.; Jordán, Edisson P.; Urrutia-Urrutia, Pilar; Salazar, Franklin W. title: Telemetry and Video Surveillance System in a UAV for the Control and Monitoring of Long-Distance Missions date: 2020-08-24 journal: Computational Science and Its Applications - ICCSA 2020 DOI: 10.1007/978-3-030-58817-5_48 sha: 1c645dca00a4826f11911dd54dd1941897bff4ff doc_id: 58771 cord_uid: wcaxdprr Unmanned aerial vehicles (UAVs) are multidisciplinary technological tools, which are used in the military area for surveillance, reconnaissance and intelligence tasks in conflict zones. Due to its versatility and performance, this manuscript describes the implementation of a telemetry and video surveillance system to develop long-range missions in a UAV prototype. Communications subsystems and a ground station are integrated, with which an unmanned aerial system (UAS) is formed, carrying out the necessary calibrations and configurations. After a set of tests and the necessary adjustments, a long-distance mission is carried out and its energy consumption, height reached and trajectory tracking are analyzed. In addition, the video signal level obtained and the percentage of telemetry signal in each of the tests are established, which managed to be 90% with a distance greater than 4 km. They have good benefits, with a low economic investment, representing an interesting proposal for developing countries with a limited budget for their improvement. system that includes unmanned aerial vehicles (UAVs), ground operator stations, launch mechanisms, etc. [6] The UAS must consider within its design a command, a control and communication (C3) system and the personnel necessary to control the unmanned aircraft [7] . UAS systems have been integrated into applications in a wide variety of disciplines, such as geography, agriculture, archeology, planimetry, etc., due to their dynamism and flexibility [8] However, there has been a particular growth in the number of applications aimed at remote sensing, surveillance and monitoring, for data collection. Sometimes, the data obtained has a limited reliability which produces uncertainty in the effectiveness of their implementation, which has represented a limitation for their widespread use. The main part of the system is the unmanned aerial vehicle (UAV), an aircraft which performs its function remotely [9] [10] [11] . Today they are mainly used for entertainment, although they have also played an important role in the military field [12, 13] . They represent one of the most promising tools for intelligence, surveillance and reconnaissance missions, due to their capabilities to achieve these objectives [14] . The need to develop these aircraft arises from the existence of illegal activities such as smuggling, cultivation, production, and drug trafficking that commonly affect various regions of the globe. In this way, air traffic control and the contribution of real-time information can be made for decision-making, both in military operations and for risk management and natural disasters [15] . The emergence of UAVs as a tool for surveillance and a means for providing services in an emergency, raises the need to establish ethical regulations. For this reason, the manuscript of [16] presents a study of how drones have become devices that require regulation. To carry out this procedure, 4 steps have been carried out that are detailed in order to formulate arguments regarding Latin American airspace. Previous studies showing the implementation of surveillance systems in UAVs are of great relevance, as shown in [17] . In this work an electro-optical/infrared sensor is designed, developed and adapted in a mid-range UAV, for traffic detection, surveillance and recognition. Tests carried out at close range show the operation of the system through the acquisition of video in real time and its position in Google Earth. Similarly, in [18] the coupling of an electro-optical sensor in the Twin Otter DHC-6 aircraft is presented, for surveillance, reconnaissance and intelligence activities. To carry out this non-invasive and easy-to-assemble procedure, a previous study of affectation, aerodynamic, structural analysis, loads, weight and balance is considered. On the other hand, in the proposal of [19] , the design and implementation of an ETL system (extraction, transformation and loading) is described, through which information is extracted, processed and shared from the UAV systems to the operator. The ETL system is developed on an integrated platform with the ability to perform multiple tasks, integrating signals with different communication protocols and high-speed information processing in real time. The development of this system facilitates the operation, control, monitoring and execution of information registration tasks of a midrange UAV. In response to the needs of the Ecuadorian Air Force, the initial study presented in [20] has been complemented. Using advanced technology, it is proposed to implement a UAS that allows long-range monitoring and surveillance tasks. From the ground station, you can control and receive video in real time, based on how the UAV performs the desired displacement. This prototype has good features and long-distance communication (range of kilometers) and with the possibility of storing this information. Through this research project, military control and patrolling activities can be carried out in tropical environments, where there is difficult access for the police personnel by land. Despite being a low-cost proposal, results obtained have demonstrated its efficient operation. This document is distributed as follows: Sect. 1 describes a brief introduction and the respective state of the art. In Sect. 2 the selection of the elements that make up the Unmanned Aerial System (UAS) is described and Sect. 3 contains the software configuration and programming process for said equipment. The results of the experimental tests and the conclusions can be seen in Sect. 4 and Sect. 5 respectively. The UAS segments used for the project are shown in Fig. 1 . The diagram consists of: 1) The air segment corresponds to the hummingbird aircraft, with all its instrumentation to develop manual and automatic flights; 2) The communications segment, which corresponds to the telemetry and video links; and 3) The land segment, which is made up of mechanisms for observing the mission and executing control processes. Table 1 details each of the mechanical characteristics with which the aircraft has been designed and built. These characteristics are of great importance since from them the proposed system is designed. For location of each of the components, certain restrictions must be considered, including the adequate distribution of the payload to achieve correct stability and control of the aircraft. Also, it should be considered that the physical structure with which the aircraft is built is made of carbon fiber, taking into account that this is a conductive material. In the air segment, the elements described in Table 2 are implemented, based on the mechanical specifications of the UAV. In the ground segment by means of a computer (personal computer) that has an interactive interface, the user can execute different processes, such as the programming of missions to the micropilot board (automatic navigation) and monitoring, where he/she can observe all data from telemetry as well as video on an auxiliary monitor. There is also the remote control (RC), which in conjunction with communication interfaces can take manual control of the aircraft at any time, either to take off, land or perform certain maneuvers. The communications segment is made up of two systems: one for data and the other for video, which works independently and is in charge of sending data from the flight controller to the ground station. The software used in the earth station will be Mission Planner (MP) due to the ease of use it offers. It is worth mentioning that the video kit has a mission planner installed by default, which is QGround Control (QGC), where only the flight variables will be displayed; and additional mission information is obtained by doing added programming. The MP software version 1.3.64 is installed on the PC that acts as an earth station, its compatibility with the recommended versions of the micropilot used is verified and a wireless link is established between them, using the MAVLINK protocol. The MP software works with various types of UAV platforms, so it is necessary to install the firmware (software necessary to manage the information received from the outside, process it and transmit orders to the aircraft actuators) on the micropilot board, specifically for type platforms fixed wing. For communication, the parameters of the transmission (TX) and reception (RX) modules of the RFD 900 radio modem are configured, with a minimum and maximum frequency of 915000 kHz and 928000 kHz respectively. The air transmission speed is established at 64 Kbps, considering that the higher the speed, the shorter the range and vice versa. However, the use of extremely low speeds due to loss of information by interference is not recommended in this application. As a security parameter against possible interference from other equipment of the same frequency that is operating in the area, the network identifier or Net ID is set to a value of 25 (in a range from 0 to 499). The transmission power is 20 dBm, denoting that all the modified parameters must be the same in both the TX and RX modules. Calibration of the aircraft sensors is performed using the MP and QGC software, in order to store these settings in the memory of the micropilot. Using MP, the pitot tube and battery monitor are generally calibrated on the Optional Hardware tab and the remaining sensors on the Initial Setup and Mandatory Hardware tabs. To calibrate the internal micropilot accelerometer, choose the Accel Calibration option, then place the aircraft on a horizontal plane and using the recommended configuration manager, follow the procedure indicated in Fig. 2 (left) . For external GPS sensor, with the option Compas will open the recommended configuration manager, follow the movements in Fig. 2 (right) and when finished, restart the Pixhawk board. In the case of the external Pitot Tube speed sensor, MP does not have the capacity to calibrate it, so QGC is used. To do this, start by covering the sensor, so that it measures a value of 0 and as the calibration process progresses, introduce air progressively until completing the procedure; battery capacity is 5000 mAh. The physical control joysticks need to be calibrated to capture the maximum and minimum ranges of PWM that will be obtained by executing a movement with these levers. The channels programmed in the physical control must be selected, as well as the Pitch, Yaw, Roll and Throttle axes and move the joysticks in all their directions (horizontal, vertical and circle). In this way the program captures the maximum and minimum ranges of PWM that are generated with the movements, i.e. the ranges in which the servos and the motor will operate. Figure 3 shows the results obtained from said process. Flight modes establish how the aircraft should interact based on the needs of the operating missions and the degree of autonomy required. These modes are selected by the pilot through radio transmitter switches or buttons or commands from the earth station. The following flight modes were programmed for this project: Manual, Return to Launch (RTL), Auto, Autotune and Loiter, whose function is described in Table 3 . The video system is a totally independent segment from the rest of the aircraft, because due to its high-power consumption, the pilot needs to have priority in telemetry data over video. The video kit is made up of a radio, called an aerial unit, which has two omnidirectional antennas with pin-type connections. One controller for the ground unit with an omnidirectional antenna and one directional with screw-type connector. A power supply to provide with energy the entire system and a GoPro camera. It is worth mentioning that the air unit and the controller have Full-Duplex communication. The controller only supports QGC mission management software, since it can be installed on computers with an Android operating system, as is the case with this system. For the operation of the camera it is connected to one of the HDMI ports of the air unit and its resolution is modified to 1080p and a speed of 60fps. To view the video that the camera transmits, you have the screen of the controller, which through a video stream manager determines its source and the type of input (HDMI). In case you want to record in the internal memory of the controller or you want to stop recording, you have two buttons to carry it out. An auxiliary monitor (PC or smartphone) can also be used through the RTSP protocol, connecting to an enabled Hotspot network and thus running the player. After carrying out six operation tests at short distances as part of the initial phase of this project (one of them in mountainous locations), an overflight is carried out in the town of the South American Amazon order to verify the equipment operating efficiency over long distances. For this, a mission was carried out at a speed of 15 m/s, a maximum height of 350 m, a maximum distance of 5.02 km and a total perimeter traveled of 9.36 km. At the end of the mission, the information obtained is saved in a.log extension document for later analysis. The results obtained are presented below. The waypoints (WP) programmed for this mission with their respective heights are displayed in Fig. 4 , where from a height of 100 m in RTL mode, the aircraft must climb to a maximum height of 350 m and descend again to 100 m during its flight in auto mode. During RTL mode it is required to maintain a constant height of 100 m. In Fig. 5 the behavior of the UAV is observed throughout the mission, where it is verified that it reaches the values defined in each WP, as well as maintaining its height in a stable and constant way in the RTL mode (100 m). The summary of reached heights is detailed in Table 4 ; to reach the desired height between nearby points (WP1-WP2) it takes 1:24 min and for distant points (WP4-WP5) 2:27 min. For this flight a constant speed of 15 m/s is required, since it allows the aircraft to capture the video adequately. This value has been used for all missions in Auto, RTL and Loiter mode. In Fig. 6 the speed behavior is analyzed, where a constant of 15 m/s is verified during all modes. However, at 15:15:37 a small acceleration peak of 20 m/s is observed, indicating mode change from Auto to RTL, when descending from 300 m to 100 m. The variables analyzed are amps discharged per hour (mAh) and the voltage; the flight starts with a fully charged battery (16.5 V) and it is estimated that for 20:13 min the maximum consumption is 1700 mAh and a voltage discharge of less than 20%. It is worth mentioning that it is estimated a lower consumption than calculated by flying in a clear area with less height than the one reached in the mountain region. The graph in Fig. 7 describes the current consumption that aircraft has had throughout the flight, where maximum value reached is 860mAh, being less than expected. Similarly, it is identified that in Auto mode there is a higher consumption compared to RTL mode where it remains permanent. Through Fig. 8 , battery behavior (voltage discharge) during flight can be evidenced. It started with a voltage of 16.5 V and ends with an average of 14.8 V, which represents a discharge of 10.3%. The discharge is adequate throughout the flight, in small areas low voltage spikes are displayed, indicating high consumption on certain occasions, such as takeoff. It is analyzed if the aircraft complied with the points programmed in an appropriate way, for this, in Fig. 9 the desired path is observed (black plane) and the executed path is shown (yellow plane). An optimal compliance of the trajectory without deviations is observed, satisfying each established WP. Overall, in all tests carried out, it has been shown that the aerodynamics of the aircraft and its configurations were optimal considering the environmental conditions of the South American Amazon and the mountain region (in a previous test), registering a lower energy consumption than estimated. Due to the favorable conditions being a flight a few meters above sea level, it is identified that the battery consumption is 1.6 V for 20 min, an adequate proportion. The telemetry link for a maximum height of 350 m was on average 94% of the signal, indicating that the aircraft can be sent over distances of 5 km without losing its monitoring and being able to execute remote control orders through of telemetry interfaces. In the video link at a distance of 3.58 km, a loss of information was obtained due to the controller equipment heating up and generating a restart. This is because proper precautions were not taken regarding the temperature to which the controller was exposed, with which this exceeded the manufacturer's recommended temperature. However, this design flaw was corrected. At a distance of 2.82 km, the link is recovered, obtaining a loss of information of 2:12 min. Table 5 shows the signal levels obtained when performing the seven tests, although they are not all presented, as they are not of interest. The signal level of the link is displayed on the monitoring interface as a percentage as shown in Fig. 10 , from which the data for each of the tests has been obtained according to the required distance. Tests 2 and 5 show considerable signal losses at distances not greater than 1 km. This caused the data transmission to intermittent at certain points while in test 6 when antennas changed its position, the signal level increased considerably reaching a distance of close to 3 km. Finally, it can be seen that the levels in the operational test (Test 7) are efficient, after making certain corrections, since only at 2 km there was a percentage of less than 90%, recovering in a short time. These data confirm that, for environments with high vegetation and humidity, the telemetry transmission system works effectively and the navigation distance of the aircraft can be increased. The signal spectrum is monitored in the video controller. Similar to the case of telemetry, the signal to noise ratio (SNR) data have been taken from the tests that were interest. At a distance of 200 m from the base station, video transmission is good in all tests, although the quality of the captured image is better in tests 6 and 7 and can be confirmed with SNR levels that was obtained in each test. By setting a distance of 600 m from the base station, low SNR levels can be detected, and especially in Test 5 the image quality is lower because there are notable pixelations. For tests 6 and 7 you have high SNR levels and good image quality. Figure 11 demonstrates all said above. At a distance of 1000 m from the base station, tests 2 and 5 show extremely low SNR levels, so the images transmitted are of poor quality compared to the images in tests 6 and 7. These losses are attributed to a bad antennas location which allowed the absorption of the signal. Results can be seen in Fig. 12 . Starting at 2000 m from the base station, only tests 6 and 7 are compared, since in the previous two missions the distance greater than 1 km were not met. Both tests have a stable video transmission, but when reaching a distance of 2.5 km in test 6 there is a link loss. The video recording of each test can be seen in Fig. 13 . At a distance of 3500 m from the base station, we only have the analysis in test 7, where it can be perceived that the SNR falls down completely as well as the video quality. The only test that was successful at this distance is presented in Fig. 14. With each of the results obtained, it has been determined that the transmission quality of the images is superior in tests 6 and 7 due to the modifications in the antennas positioning. In addition to these tests, the aircraft flew over to a maximum height of 3000 m, which did not happen in tests 2 and 5, this being another important factor to consider in the video link. As seen in references presented, there are several investigations where low-cost video surveillance systems (prototypes) are implemented in aircraft. In [17] and [18] the electro-optical sensor is presented as the tool to capture video in real time. However, in this proposal an own system based on a GO Pro camera is used. For its part [19] , it presents a ETL system in a mid-range UAV but now, this work has now been done on a low-end aircraft. This represents a lower resource investment that can be exploited by developing countries, since long-distance results are highly efficient. The UAV used for this project is medium-range, having a structure suitable for developing video surveillance missions at the borderline or in areas with difficult access. Despite having an efficient mechanical design and capable of executing stable flights, it has been detected that the systems for both mission control and monitoring are deficient, as they present a slow response and a maximum range of 800 m. For the hummingbird aircraft and its area of operation, specific requirements were established for each of the systems. A fast response control system with redundancy and a power system suitable for greater autonomy and long-range communication systems. For the latter, two different frequency bands were selected to avoid interferences between telemetry and video data; these were 915 MHz and 2.4 GHz respectively, since their characteristics are excellent for obtaining a greater range and penetration in areas whose climate is humid with high vegetation. The hummingbird aircraft was converted into an operational aircraft as it performs long-range missions with constant monitoring of all telemetry or flight variables and live streaming of high-definition video. Through tests carried out in the operation area, communication ranges of 5 km for telemetry and 4 km for video have been reached but, it was detected by the levels of the telemetry signal that greater distances could be reached since only it shows a loss of 10%. Another aspect to note is that the aircraft was subjected to other tests with atmospheric difficulties where the same data quality was obtained, but with a slight decrease in distance. As future work, it is presented to expand this research, carrying out further tests in other geographical areas, with the possibility of carrying out a new prototype system. Analysis of the static behavior of a new landing gear model based on a four-bar linkage mechanism Interactive system for monitoring and control of a flow station using LabVIEW Managing artificial intelligence technology for added value Application of robotics in offshore oil and gas industry-a review Part II Analyzing the threat of unmanned aerial vehicles (UAV) to nuclear facilities Counter UAS solutions through UAV swarm environments A review of Small Unmanned Aircraft System (UAS) advantages as a tool in condition survey works A meta-analysis and review of unmanned aircraft system (UAS) imagery for terrestrial applications Analysis and determination of minimum requirements of an autopilot for the control of unmanned aerial vehicles (UAV) Geometric accuracy evaluation of geospatial data using low-cost sensors on small UAVs UAV survey of bridges and viaduct: workflow and application Using hands as an easy UAV joystick for entertainment applications Self-organization as a supporting paradigm for military UAV relay networks Multi-UAV routing for persistent intelligence surveillance & reconnaissance missions Comparison of a head-mounted display and flat screen display during a micro-UAV target detection Task Revisitando el espacio aéreo latinoamericano: una exploración de los drones como sujetos de regulación Interface of optimal electro-optical/infrared for unmanned aerial vehicles Implementation of an electro-optical sensor in the twin Otter FAE plane for the strengthening of the strategic surveillance capacity Analysis and implementation of ETL system for unmanned aerial vehicles (UAV) Structural design of a telemetry and video surveillance system implemented in a UAV Acknowledgments. Special thanks to the support provided by CIDFAE Research Center of FAE, DIDE and SISAu Research Group.