key: cord-0060941-4pcchkjc
authors: Wang, Xiaoyuan; Huang, Weijie; Zhao, Qinjun
title: Image Processing-Based Electronic Fence: A Review
date: 2020-06-13
journal: Multimedia Technology and Enhanced Learning
DOI: 10.1007/978-3-030-51103-6_29
sha: 2a00a46ad1dd6e225c45b9db65dc567787f5fdec
doc_id: 60941
cord_uid: 4pcchkjc

With the development of science and technology, using electronic fence to replace traditional physical fence becomes a trend. Research in the image processing-based electronic fence is gaining more popularity due to its low cost, low power consumption, and intelligence. In this paper, we are interested in the study of three types of scenarios with an image processing-based electronic fence. The scenarios contains national border protection, safety management in manufacturing field and vehicles safety plans. The state of the art frameworks of the three scenarios are reviewed, and the trend of these fields is also discussed.

Nowadays, physical fence has the following problems:

-The physical fence height is not high enough so that it is easily to be crossed by intruder. -Physical fences do not have monitoring character so that it is impossible to record which intruders are cross the fences. -Physical fences have no automatic alarm character so that the offender can not stop the behavior of intrusion.

These problems have led to safety precautions that can not meet its requirements.

With the development of electronic technology, sensing technology and computer technology, security precaution has gradually developed into a specialized public security technology discipline [48] . Among them, image processing-based electronic fence is a high-tech way that can not be ignored in security technology. Intrusion detection and alarm systems based on electronic fence have been widely used in national border, banks, financial departments, museums, office buildings, manufacturing field, important workshops, office buildings, hotels and other areas [21] . Video surveillance system [22] has been widely used in electronic fence, and the core of video surveillance is image processing which can perform intelligent analysis based on images so that a large number of manpower is reduced. Image processing-based electronic fence has five main advantages compared with traditional physical fence:

-First, the key technology of image processing-based electronic fence is to analyze the images in the scene of the monitoring area. This analysis can be used to analyze the specific area to achieve regional intrusion detection based on the algorithm without changing the physical device. -Second, image processing-based electronic fence does not need significant intrusion detection devices to establish monitoring points while it only needs to analyze the collected images. The camera can be installed in a secret place so that the risk of equipment damage is reduced and remote detection is achieved. -Third, image processing-based electronic fence analyses images captured in target area. No matter which direction the intruder enters the monitoring area, the alarm rings when the system detects the intrusion. This way can eliminate any form of bypassing the detector and achieve no omission alarm. -Fourth, the monitoring area can be arbitrarily specified in image processingbased electronic fence so that it is easy to be modified without any additional cost. -Finally, image processing-based electronic fence can not only detect the intrusion but also implement tracing objects compared with traditional physical fence.

Nowadays, image processing-based electronic fence with flexible settings, less false alarms, strong anti-interference ability and compatibility with each interface has become one of the hottest research areas.

The most important part of image processing-based electronic fence is objects detection [8] . Objects detection is to find all the objects (objects) of interest in the image and to determine their position and size. Objects detection has always been the most challenging problem in machine vision because of the different appearances, shapes, poses, and the interference of illumination and occlusion during processing. Objects detection focus on the following problems: a) objects may appear anywhere in the image; b) objects are available in a large number of different sizes from an image; c) objects may express a variety of different shapes in the image. Some research works on object detection based on Convolution Neural Network (CNN) [26] are recorded in Table 1 . Based on the results of objects detection, the intruders are detected whether they exceeds the limits specified by the electronic fence. Next section we are interested in the study of the framworks of image processing-based electronic fence in national border protection [18] , safety management in manufacturing field [36] and vehicles safety plans [1] . 

Border security refers to the protection of national borders from the illegal movement of goods, drugs, weapons and humans. It makes that trade and legal travel can be maintained and anti-terrorism protection can be provided globally. National border protection system [45] is employed to monitor the intrusion events around the fence and determine whether suspicious activities are carried out. If any suspicious event occurs, a set of scheduled tasks, such as warning or combat systems, are performed. Image processing-based electronic fence is an integral part of national border protection system. It aims to monitor hostilities, detect and track intruders, analyse their behaviors based on serial image sequences. This electronic fence insteads watching and monitoring 24 × 7 surveillance task by humans and the framework can be proved very useful to generate Intruder Detection System (IDS) [38] automatic alarms. Vittal et al. [46] propose an intrusion detection and automatic shooting device controlled by computer to improve border security. Camera images are transfered from Video to computer in real-time through the USB bus, and then color images are convert to gray images. The current gray image is compared with the reference image to find out the changes between object and background so that the intruders are searched out based on these changes. It also intergrates an automatic shooting mechanism for automatic positioning and shooting targets. This framework has strong ability to suit to worse environment and saves a lot of manpower. KeremIrgan et al. [18] propose a cost-effective approach for dynamic (on-the-fly) prioritization of image macro-blocks. Data blocks are labeled as "important" and "not-important" to use impel encoding scheme at the source node. Various prioritization measurements and labeling method based on threshold can successfully improve efficiency and save time when the framework performs JPEG encoding. Raheja et al. [31] propose an approach of detecting instruders in hilly region. The instrusion can be noted no matter daytime or in night because the kinect sensor integrates infrared camera and color camera. The framework can easily distinguish animal and human with skeletal labeling approach provided by Kinect. Standing, walking, crawling, and bending etc. can be recognized based on the HMM. Deshmukh et al. [6] propose a three layer image processing-based electronic fence which each layer runs parallelly and the results of all layers are merged by a module. Two cameras are embedded into between first and second layer and provided image information for the framework. This electronic fence can effectively monitoring evil activities. Kim et al. [19] propose an image processing-based electronic fence with optical and thermal cameras. The electronic fence is generated by curve fitting and human-computer interactive processing. A convolutional neural network (CNN) is used to distinguish humans and animals and the results of classification is accurate. A long-term recurrent convolutional network (LRCN) model is also used in this framework to record five-types-behavior. The proposed method can be considered as an automatic protection system.

Intrusion is an unauthorized entrance into dangerous areas without being aware of the potential hazards. Intrusion not only causes unauthorized workers to be suffered from accidents, but also disturbs or even injures other authorized workers in dangerous areas [4] . Traditional safety management relies mainly on the inspection of security officials and it is very time-consuming [28] . Image processing-based electronic fence solves this problem by integrating and processing information about the behaviors of workers and the scene environment [50] . The main aspects of image processing-based electronic fence building safety monitoring include: (1) on-site work behavior monitoring; (2) on-site environmental monitoring, and (3) information integration, analysis and early warning [40] .

Naticchia et al. [29] propose an intelligent video surveillance system to address conflict issues that may arise among teams operating on building-sites. Actions in scene is tracked by image processing and the analyzing result is transformed by ZigBee. Real-time site state visualization and remote interaction with security inspectors are also designed. In this approach, the framework is considered to be cover a wide range of areas while maintaining its non-intrusive functions because a new low-power method is implemented. Revaud et al. [36] propose a novel approach for detecting workers or other equipments with significant occlusions. The optical flow field is estimated by motion model. Edgepreserving interpolation and variational energy minimization are combined to obtain geodesic distance. This approach used in electronic fence can deal with the occlusion problem effectively. Fang et al. [9] develops a cognitive model of construction workers' unsafe behaviors (CM-CWUB). This module contains five steps: getting information, understanding information, perceiving response, choosing response and taking action. Works and fences are located by Building Information Modeling (BIM) approach. The framework can provide early warning and active protection. Tang et al. [42] propose that indoor industrial production space is divided into three different semantics (structure, connectivity and volume). Semantic information is generated by image segmentation, and then the unsafety areas are defined. Combining with action recognition, the framework can easily find the intruders.

There are two main types of uses of Image processing-based electronic fence in the field of vehicles safety plans. One is dockless bike-sharing services, the other is high-speed railway instrusion detection.

Bike-sharing is becoming a more popular form of transport in the world. With the increasing popularity of various types of shared bicycles, the problem of random parking has become more and more serious. These uncivilized actions have some adverse effects on the social environment and atmosphere [7] . It is impossible to manage 24 h a day by human resources testing alone. Despite a series of policies, the sharing of bicycles has not been alleviated. Image processing-based electronic fence can push reliable detection and warning information to traffic management, which greatly reduces the manual workload. Lee et al. [23] propose a static objects tracking and detection approach based on foreground model. The time for a pixel to remain foreground in this foreground model is determined by the brightness of the image. The longer the time means the brighter the position of the current pixel in the model. The framework also use 1−D transformation and image projection to reduce the dimensionality of image data. Thus bicycles and parking areas are detected in real-time. Xie et al. [49] also propose a foreground bicycles tracking approach based on CNN. The vehicle in the input image is detected by SSD+ROI (Region of Interest), and all vehicles in the ROI are tracked and counted once they stop moving. However, this method has higher algorithm complexity and poor real-time performance. Bock et al. [3] propose an objects detection approach based on static corner matching. The basic idea is to select the Harris corner according to the characteristics of the bicycles on each frame in the video. Then the static corner points is extracted according to the motion state of the Harris corners. However, this method requires a large number of training sets to learn in advance to achieve the interference of removing the static corners in the image and the calculation amount is too large.

Electronic fence used in high-speed railway instrusion detection needs to provide many features, which contains day and night surveillance, high-definition, full coverage, system linkage, foreign object intrusion monitoring, intelligent identification and analysis, effective discovery, drive-out, prevention of climbing over the invasion [7] . It is the most direct, effective and credible party for railway perimeter prevention. There are many researches in various environmental effects and some are summarized in Table 2 . After removing the impact of the special environment, Electronic fence detecting instruders can be handled. Guo et al. [15] propose an accurate and effective intrusion detection. They improve the average accuracy of Single Shot multibox Detector (SSD) with the fusion between high-level features and low-level features. Semantic information are enhanced based on the fusion results. The network are also clipped using the L1 norm so that the real-time performance is improved. Wang et al. [47] propose an adaptive segmentation algorithm to implement the function of the electronic fence on the high-speed railway. They first use hough-transform to generate adaptive direction of each Gaussian kernel. Then, according to the boundary weight and size of the region, a new clustering approach is used to merge small fragments to candidate regions. Last, the boundary of electronic fence is classified by a deep network. The framework can execute seamlessly on memory-constrained, tiny embedded devices.

Image processing-based electronic fence must deal with massive video surveillance data, which brings huge challenges to intelligent system applications. These challenges can be summarized as cross-scenario, cross-media, and cross-space. The trends of image processing-based electronic fence contains high efficiency, high coordination and high initiative.

High Efficiency. The new electronic fence can be combined and adjusted automatically at any time. An orderly docking is formed between the dynamic monitoring tasks and the ubiquitous camera resources so the framework need to use distributed monitoring system. On the one hand, it can enhance the scalability of the monitoring network and can increase the monitoring terminal of different functions, on the other hand, it can improve the working efficiency and response speed.

High Coordination. The data from the physical space and the network space are fused. A three-dimensional perception and early warning mode of dual spatial cooperative sensing and active modulation are established. This can help the new electronic fence to better detect intruders.

High Initiative. Extracting various characteristics of public security events from different media data, including time, place, people, timing, relationship, etc., improves the integrity of semantic expression, and it is more reliable for event detection. The new electronic fence can learn massive cross-media data through robust structure, and the related information can express the multifaceted features of the same event, thus this fence improves the reliability of monitoring network early warning.

This paper reviews some works of image processing-based electronic fence. We discuss research status of objects detection, the framworks in three scenarios and some technologies for eliminating the effects of bad weather. The development trend of electronic fence is forecasted and some advices are put forward with summarizing the characteristics of these studies.

Safety verification of autonomous vehicles for coordinated evasive maneuvers

Inside-outside net: detecting objects in context with skip pooling and recurrent neural networks

Learning on-street parking maps from position information of parked vehicles

Video surveillance systems to enhance occupational safety: a case study

R-FCN: object detection via region-based fully convolutional networks

Intelligent surveillance system using energy efficient intrusion detection and tracking techniques

Detection method for vehicles in tunnels based on surveillance images

A survey of deep learning methods and software tools for image classification and object detection

A cognitive model of construction workers' unsafe behaviors

Real-time human detection and tracking in infrared video feed

DSSD: deconvolutional single shot detector

On-line vehicle detection at nighttime-based tail-light pairing with saliency detection in the multi-lane intersection

Fast R-CNN

Rich feature hierarchies for accurate object detection and semantic segmentation

High-speed railway clearance intrusion detection with improved ssd network

Single image haze removal using dark channel prior

Spatial pyramid pooling in deep convolutional networks for visual recognition

Low-cost prioritization of image blocks in wireless sensor networks for border surveillance

Intelligent intrusion detection system featuring a virtual fence, active intruder detection, classification, tracking, and action recognition

Restoration of video by removing rain

Image based 3d surveillance for flexible man-robot-cooperation

Automatic video-based human motion analyzer for consumer surveillance system

Real-time detection of illegally parked vehicles using 1-D transformation

D3R-NET: dynamic routing residue recurrent network for video rain removal

SSD: single shot multibox detector

A survey of deep neural network architectures and their applications

Adaptive object detection using adjacency and zoom prediction

Model-based construction work analysis considering process-related hazards

A monitoring system for real-time interference control on large construction sites

Removing rain and snow in a single image using saturation and visibility features

Cross border intruder detection in hilly terrain in dark environment

You only look once: unified, real-time object detection

YOLO9000: better, faster, stronger

YOLOV3: an incremental improvement

Faster R-CNN: towards real-time object detection with region proposal networks

EpicFlow: edge-preserving interpolation of correspondences for optical flow

OverFeat: integrated recognition, localization and detection using convolutional networks

Survey of intrusion detection techniques and architectures in wireless sensor networks

Training region-based object detectors with online hard example mining

Information technology applications in construction safety assurance

Self-organized night video enhancement for surveillance systems

A full level-of-detail specification for 3D building models combining indoor and outdoor scenes

DeNet: scalable real-time object detection with directed sparse sampling

FACE: fully automated context enhancement for night-time video sequences

Design and fabrication of solar powered unmanned aerial vehicle for border surveillance

Computer controlled intrusion-detector and automatic firing-unit for border security

An adaptive track segmentation algorithm for a railway intrusion detection system

A design of alarm system for substation perimeter based on laser fence and wireless communication

Real-time illegal parking detection system based on deep learning

Applying sensor-based technology to improve construction safety management

Single-shot refinement neural network for object detection

CoupleNet: Ccupling global structure with local parts for object detection