key: cord-0939785-grs9eqql authors: Rumiński, Jacek title: Analysis of the parameters of respiration patterns extracted from thermal image sequences date: 2016-08-20 journal: Biocybern Biomed Eng DOI: 10.1016/j.bbe.2016.07.006 sha: 26f876b0e407cc3a54b59ff2b61180e5bf6e92a8 doc_id: 939785 cord_uid: grs9eqql Remote estimation of vital signs is an important and active area of research. The goal of this work was to analyze the feasibility of estimating respiration parameters from video sequences of faces recorded using a mobile thermal camera. Different estimators were analyzed and experimentally verified. It was demonstrated that the respiration rate, periodicity of respiration, and presence and length of apnea periods could be reliably estimated from signals recorded using a portable thermal camera. The size of the camera and efficiency of the methods allow the implementation of this method in smart glasses. The three main vital signs often monitored by healthcare professionals are body temperature, heart rate, and respiration rate. Respiratory rate (RR) is defined as the number of breaths per minute. A typical goal of respiratory rate measurement is to evaluate whether the respirations are normal, too fast (tachypnea), too slow (bradypnea), or nonexistent (apnea). Apnea is defined by the cessation of respiratory airflow and is especially dangerous during sleep. The length of time necessary to qualify a true apneic event (i.e., considered clinically significant in sleep apnea syndromes) is measured in seconds [1] , e.g., >10 s for central sleep apnea syndrome [2] . The threshold values for sleep apnea syndromes are defined as a result of cumulative clinical experience and are different for adults (e.g., >9 s [3] or >10 s [2] ) and children (>19 s [1] ). In this paper, the term ''apnea'' is used in wide sense, i.e., as a ''cessation of respiration airflow''. Respiratory rate is an important physiological parameter that is indicative of potential health risks. For example, a RR value greater than 27 bpm is an important predictor of cardiac arrest [4] and is also used in prediction of pneumonia [5] or lower respiratory tract infection [6] . It has also been shown that respiratory rate is more discriminatory between stable and unstable patients than pulse rate [4] . In [7] , the authors specified many recommendations for measurement of RR in hospitals and in admission to intensive care units, including ''the respiratory rate should be measured and documented accurately in all hospital patients at least once a day, and should always be documented when other vital signs are measured''. In basic epidemiology, the WHO guidelines recommend that pneumonia case detection can be based on clinical signs alone, primarily respiratory rate [8] . In clinical observations, the respiratory rate is often determined by counting the number of times that the chest rises or falls per minute [9] . More quantitative methods use inductive plethysmographs or thoracic impedance systems [10] , oxygen masks [11] , bioacoustic sensors [12] , accelerometers or gyroscope sensors [13] , among others. Respiration-related parameters are typically analyzed in sleep studies. For example, the American Academy of Sleep Medicine (AASM) recommends, ''Sleep apnea must first be diagnosed at a sleep center or lab during an overnight sleep study, or 'polysomnogram''' [14] . The International Classification of Sleep Disorders [15] underlines that ''Polysomnographic monitoring of obstructive sleep apnea syndrome should consist of monitoring of sleep by electroencephalography, electrooculography, electromyography, airflow, and respiratory muscle effort, and should also include measures of electrocardiographic rhythm and blood oxygen saturation''. Other parameters are also measured to analyze respiration-related context, e.g., body position, snore acoustic signals, etc. [16, 17] . Remote measurement of RR could be especially useful for medical screening (e.g., severe acute respiratory syndrome (SARS), pandemic influenza, etc.). Therefore, the use of nearinfrared and thermal imaging was found to be a useful method for evaluation of chest respiration movements or nasal heat flow. In [18] , the authors used an active-stereo depth-sensing system composed of a near-infrared (NIR) illuminator and a camera. After calculation of the depth map, the region of interest (ROI) containing the rib cage and the abdominal cavity was detected. Finally, from the processed set of frames, a temporal sequence of volumes was calculated that represented breathing signals, which could be further processed to calculate the respiration rate and patterns. The disadvantage of this method is that the patient must wear a tight elastic top during measurement. Extraction of signals from a series of thermal images has been found valuable for specifying diagnostic parameters [19, 20] . In [21, 22] , thermal imaging was used to estimate the human respiration rate from a sequence of facial images. The side view of the subject to the camera was used to visualize the breathing-jet dynamics. In [21] , the CO 2 content of exhaled air was captured using a narrow band-pass filter in the CO 2 absorption band (4.3 mm). Experiments were performed on 9 subjects at distances ranging from 6-8 ft. Groundtruth measurements were obtained via a traditional contact device (an abdominal transducer). The results showed a high correlation between the imaged breath rate and the ground-truth breath rate. The measurement methods presented in [23, 24] used a thermal camera that was positioned collinear to the subject's face. The methodology was based on capture of the modulation of nasal tissue radiance during inhalation and exhalation of air. In [24] , the authors described the method and lab validation with the participation of 20 healthy volunteers. In [23] , the authors reported application of the method in a clinical study with the participation of healthy subjects and pathological subjects (suffering from sleep apnea). In the described studies, the respiration rate was typically estimated using statistical methods or analysis of dominant peaks in the frequency spectrum. A similar method was proposed in [25] in which the temperature gradient distribution throughout the nasal cavity (temperature difference between the inspiration and expiration phases) was used to extract the respiration waveforms. The temperature difference is the product of radiative and convective heat transfer components during the breathing cycle. The authors estimated the values of respiration rate and compared the results with manually registered values from the reference bedside monitor. Validation performed on 5 subjects showed small differences between the estimated values and reference values (mean = 1.18 bpm, std. dev. =0.80). In [26] , using a portable thermal camera, the authors presented a preliminary study focused on evaluation of respiration rate for subjects during silence or speech. The results showed highly accurate RR estimation with respect to reference measurements. The measured thermo-physiological signal for respiration analysis represents the modulated nasal tissue radiance, and signal extraction requires location of the source of thermal changes in the area of nostrils or mouth. The related source points or a region of interest (ROI) can be specified manually or obtained using automatic detection (in a frame) and tracking (between frames). For example, in [27] , the authors presented a particle filter tracker driven by a probabilistic template function, which is capable of adapting to abrupt positional and physiological changes. First, the user manually inputs the initial template (rectangular ROI) in the first frame (using the computer mouse). For the following frames, the method automatically performs the template updating process by first extracting stable and unstable seeds and subsequently computing the spatiotemporal fuzzy mask. The method was tested on a dataset consisting of 25 thermal sequences for various conditions (e.g., movements of subjects). The method demonstrated robustness and accuracy in thermal facial imaging. Other tracking methods have also been proposed (e.g., [28, 29] ). A single thermo-physiological signal is usually constructed by calculating a mean temperature in the ROI of each video frame. Such a methodology is simple and effective but does not allow visualization and localization of subtle pathological patterns, as proposed in [30] . The miniaturization of thermal imaging sensors (e.g., TAMARISK cameras, Flir Lepton sensors) allows embedding of such sensors in mobile portable devices, e.g., in smart glasses. In recent years, many smart glasses have been proposed, including Google Glass, Epson Moverio BT-200, and so on, and many applications of smart glasses in healthcare have been presented. For example, Evena Medical reported the use of smart glasses to improve visualization of skin veins with IR illumination [31] . Under the eGlasses project (www.eglasses.eu), our group is developing a smart glasses platform that can use different sensors, including visible and thermal cameras [32, 33] . The user of these dedicated smart glasses can identify a person (e.g., using graphical markers [34] or facial features [33] ), attempt to estimate vital sign parameters (using a thermal camera, visual camera, etc.) and store the information in a healthcare information system. Therefore, it is important to analyze the reliability of the RR estimation for short image sequences recorded using a mobile thermal camera. Additionally, the goal of the work reported in this paper is to analyze the types of respiration patterns that can be accurately evaluated and described using such mobile system. In particular, three parameters of respiration patterns are evaluated: the respiration rate for normal and abnormal breathing patterns, the presence of breathing pauses or apnea events (cessation of respiration) and the relationship between respiration effort and respiration activity (respiration depth). A potential practical application is remote monitoring of respiration rate and patterns during screening activities or during classical interviews with patients. The method proposed in this paper is based on the measurement of frames (with a given frame rate) from the face region using a thermal camera (Fig. 1a) . It is assumed that the temperature changes in the region of the nose or/and mouth due to respiration activities. Therefore, a temperature gradient should be observed if the body temperature and ambient temperature are different. Healthy volunteers were asked to participate in experiments to collect video sequences for verification of the proposed method. Two experiments were performed. A total of 16 healthy volunteers (avg. age = 34.75 y AE 13.16) participated in the first experiment, and 12 healthy volunteers participated in the second experiment 12 (avg. age = 37.15 y AE 9.16). In all experiments, a Tamarisk 320 long-wave thermal camera module was used (resolution of 320  240, sensitivity <50 mK, size of 3 cm  2.6 cm  4 cm). A portable thermal camera (that can be easily connected to smart glasses frame; see Fig. 2a ) was fixed using a special handle mounted on a table and positioned collinear to the subject's face. Each subject was asked to sit in an armchair (fixed position) in front of the camera. Using the camera preview mode, the position of each subject was fixed to fulfill the following conditions: nostrils visible and nose within approximately 1/5 of the frame width. This condition was one of the assumptions of this study. Because the subjects were different, the distance between the camera face and subject's face was in the range of 40-70 cm. Such distances could be easily used in practice, e.g., at the beginning of every routine medical examination when the nurse takes the patient's vital signs. For each subject, the session was divided into three phases: (1) subject sitting on the armchair with position fixed as described above, (2) subject asked to rest (calmly) for several seconds, and (3) subject asked to performed breathing exercises, which were different for both experiments, each for 1 min. During the last phase (3rd), thermal video sequences were recorded together with the reference measurements of respiration activities collected using a pressure belt (Vernier RMB). The sampling frequency for both devices (belt and thermal camera) was set to 25 Hz. To synchronize both respiration signals, subjects were asked to hold a breath and begin normal breathing after the acquisition was started. Each data acquisition phase lasted 1 min (1-min recordings; see Fig. 2b ). During the first experiment, subjects were asked to perform a simple breathing exercise, i.e., to breath naturally, without speaking or moving. Data from the first experiment were analyzed to verify different RR estimators. After this step, the second experiment was performed. In the second experiment, a subset of subjects who participated in the first experiment was available (12 out of 16). Each subject was asked to repeat the respiration patterns [35] presented in Fig. 3 . Each exercise lasted 1 min with breaks (rest) between each exercise. The patterns that appear in typical respiration problems were chosen and additionally allowed for later quantitative analysis of the thermal-based respiration waveforms for possible detection of apnea events (or respiration pauses) and evaluation of frequency (rate) changes and amplitude changes. The subjects were asked to follow the patterns but could individually decide how (e.g., they could decide when to hold the breath: during inhalation, exhalation, in the middle, etc.). For the P5 pattern, participants were asked to generate respiration actions chaotically with irregular apnea periods. Informed consent was obtained for both experiments. In total, 76 signals (16 + 60), each with a length of 1 min, were extracted and used in further analysis. In both experiments, a respiration belt (Vernier RMB) was used to simultaneously measure the reference respiration signals from the chest/abdomen region. Measurement of thoracoabdominal movements was used in this study as a reference method, as in many other similar studies. In this study, it was also important to compare the respiratory effort (measured by the respiratory belt) with the respiratory airflow (measured by the thermal camera). This comparison was made possible using signals obtained for the P1 pattern for which modulation of amplitudes (respiration depths) should be observed. The region of interest (ROI) was manually extracted (e.g., nose/ nostrils area) for each recorded video frame, and the corresponding values were averaged (one value for a frame). As a result, a digital, thermo-physiological signal was extracted (Fig. 1b) . For the purposes of this study, the nose ROI was selected manually on the first frame using software developed in the Java programming language (Fig. 2b) . The rectangular ROI was drawn to cover the nostrils. The width of the ROI was equal to the maximum width of the nose. The bottom portion of the ROI was set as half the distance between the bottom of the nose and the upper lip. The ROI indicated in the first frame was subsequently copied to all successive frames, and its location in reference to the nose location was visually verified by reviewing all frames with the developed software. It is important to underline that the experiments were performed under controlled conditions, i.e., subjects were asked to remain still, and an averaging operation performed on the pixels in the ROI compensated for potential small movements. For each ROI of each frame, the average value is calculated as: where N ROI is the number of pixels in the ROI, r s and c s are the respective first (start) row and column of the ROI rectangle, r e and c e are the respective last (end) row and column of the ROI rectangle, and C(i, j) is the pixel value of the data matrix. After calculation of average values, the samples of the digital signals were obtained (Fig. 1b) : where K is the number of frames in the processed video (image sequence). In the next step, the digital signals were normalized by removing the mean value: The extracted signals were presented using the developed software and could be visually verified. The normalized signals were optionally filtered using the moving average operation in the time domain. Two lengths of windows size were considered, i.e., L = 5 and L = 7, for which the À3 dB cut-off frequencies (low-pass filters) are approximately f c = 2.26 Hz and f c = 1.598 Hz (for f s = 25). We tested both window lengths, and the results of the respiration rate estimates were practically the same. We chose N = 5 because it introduces fewer changes to the original signal. Post-processing Baseline removal was performed using a 4th-order high-pass digital Butterworth filter with the cutoff frequency set to 0.1 Hz. The filter was dynamically designed using the butter function from the Octave signal processing toolkit. The respiration waveforms were analyzed to extract different descriptors that represent properties of the respiration patterns recorded using thermal imaging. The goal of the analysis was to answer the question of whether it is possible to reliably estimate the respiratory rate, periodicity of respiration patterns, presence and properties of apnea events, and relative change in respiration waveform amplitudes using a mobile thermal camera. First, the respiration rate estimation was investigated. The most popular estimator is based on detection of the highest peak in the frequency spectrum (the method is later referenced as eRR_sp). This approach assumes that the respiration frequency f SP is the frequency value for the highest peak in the frequency spectrum (e.g., amplitude spectrum) and also assumes (as in other methods) that the respiration signal is dominant in the analyzed signal. This method has two disadvantages. First, for short signals, this method has a low frequency resolution (e.g., for T = 30 s, N = 750, f s = 25 Hz, the spectral resolution is (25/750)  60, i.e., approximately 2 bpm). Second, this method almost always returns a result, even for noise signals, and thus ''false'' respiration rate values could be returned for the filtered signals. Therefore, additional respiration rate estimators were investigated, i.e., eRR_ac is an estimator based on periodicity of peak locations for the autocorrelation function in the time domain, eRR_zc is the estimator based on the number of zero-crossings, and eRR_pk is the estimator based on the number of detected peaks. Because the autocorrelation sequence of a periodic signal has the same cyclic characteristics as the signal itself, for the eRR_ac estimator, the autocorrelation for different time lags is calculated, and the period is determined by calculating an average time differences between the detected peaks. The xcorr Octave function was used. As a peak detector, we used a method that searches for a local minimum and a local maximum for which the difference is greater than the threshold value T: where s fn ðt j Þ is the filtered signal value of the local minimum at j, and s fn ðt jþl Þ is the filtered signal value of the local maximum at j + l. The threshold value T was calculated in two phases. In the first phase (T= T 1 ): where T K1 is the scaling value set to 0.33. The median of the peak-to-peak gradient values was calculated and used to obtain the T = T 2 threshold value. This scaled threshold (T K1 = 0.25) was applied in the second pass of the algorithm to detect the final peaks of each signal. The respiration frequency f AC was estimated as the inverse of the average time period between the peaks of the xcorr for f s time lags. The third respiratory rate estimator was based on the total number of zero-crossings (nZC) of the filtered signal and estimates the frequency as: The last respiratory rate estimator is based on the peaks detected using (4) and (5), calculates the number of inspiration peaks in the filtered signal and estimates the frequency as: where nPK i is the number of inspiration peaks, and N i is the total number of samples between the start of the first detected inspiration event and the last one. The calculated frequencies were multiplied by 60 to obtain the results in units of beats per minute (bpm). Additionally, the reference signals measured using the belt were visually inspected to manually calculate the respiration rate using the standard definition (number of events in time). The time period of one respiration event was assumed as the time between two successive inspiration starts. The number of respiration events was counted (N_RE), and the total time of all respiration events was calculated (T_RE). Finally, the respiration rate was calculated as (N_RE*60)/T_RE and used as a reference. The second goal of the data analysis was to evaluate the periodicity of the respiration waveforms extracted using thermal imaging. Additional signal descriptors were investigated to analyze the differences between the true respiratory signals and other signals (possibly not periodical) that can be extracted from thermal sequences (e.g., from other face regions). Based on prior knowledge, it can be assumed that the filtered (smoothed) regular respiratory signal resembles a sine function, i.e., the result of periodical inhalation/exhalation activities. Based on the previous results obtained for pulse rate analysis in videoplethysmography, two signal parameters defined using spectral moments have been found to be valuable: the ''complexity'' measure (Hijorth parameter) and the ''spectral purity index''. Both of these signal parameters can be efficiently calculated in the time domain. If the signal is more similar to the sine signal, the ''complexity'' parameter has lower values (≥0), whereas the SPI parameter has higher values (≥1). Theoretically, the sine-like periodicity of the signal can be represented by simple parameters. The spectral moments can be defined with reference to the mean power of a continuous time signal s(t) and its derivatives [36] : where E[] is the expectation value. Considering the sampled, filtered and normalized signal s fn ðt i Þ of the continuous time signal s(t) (sampling period T s , t i = i  T s ), the two derivatives can be approximated as And the spectral moments can be estimated as The ''complexity'' and the spectral purity measure (SPI) can be defined as: During post-processing, signals for 4 additional ROIs were extracted (cheek and forehead and left and right side of the face) to investigate the differences of both descriptors for thermal-based respiration waveforms and other signals. The third goal of the data analysis was to investigate whether it is possible to reliably detect apnea events from respiration waveforms recorded using a portable thermal camera. During the experiments, volunteers were asked to hold their breath to simulate apnea periods in the P1, P2, P4, and P5 patterns. However, the volunteers could decide when to hold the breath (e.g., at the beginning of next inspiration event or expiration event or in the middle) and how long the apnea event should last. The problem is that for thermal recordings of apnea periods, the temperature inside the analyzed ROI slowly changes due to many internal (e.g., blood flow) and external (heat flow due to ambient temperature changes) thermal conditions. This change can be positive (trend with positive slope), negative (trend with negative slope) or neutral (without a slope, normalized mean near 0). However, the rate of change is typically smaller than that of the respiration rate. Additionally, the observed temperature gradient is also significantly smaller than that of respiration activity. Therefore, the apnea event detection algorithm (Algorithm 1) was proposed based on the first derivative of the filtered signal. The absolute values of the first derivative signal are normalized in reference to the maximum signal value. The algorithm counts all successive samples for which the values are smaller than the threshold value. The threshold value is calculated as the weighted (K) value of the interquartile range (IQR) for the processed signal. An apnea event is detected if the number of samples (or time after scaling using the sampling frequency value f s ) is greater than the assumed parameter value Tapnea (e.g., >10 s). Finally, the relative amplitude values of respiration signals were analyzed. The change in amplitudes for the respiration waveforms recorded with the reference belt and the thermal camera was investigated. The amplitudes were compared using a qualitative method with comparison of the signal plots in the time domain. Table 1 presents the results obtained from all investigated respiration rate estimators for thermal-based respiration signals and for the reference signals recorded using the pressure belt. Table 2 presents the mean square error (MSE), the mean and standard deviation of the absolute differences between the respiration rate estimated using the given estimator for thermal-based respiratory signals and the manually calculated respiration rate (for belt-based signals). The last 4 columns of Table 2 present the results for belt-based signals. The charts in Fig. 4 present the correlation of RR between values calculated using the reference method (manually for belt-based signals) and for two estimators for thermal-based signals. The values of the two signal descriptors (i.e., ''complexity'' and SPI) were calculated for thermal-based signals (experiment 1, no apnea events) from the nose ROI and from the forehead and cheeks ROIs. For the nose ROI, the mean values and standard deviations were ''complexity'' = 0.12 AE 0.01 (max value = 0.14; min value = 0.09) and SPI = 0.30 AE 0.09 In Fig. 5 , the aligned, thermal-based (black) and belt-based (green) respiration signals measured for the subject S01 are presented for all patterns used in experiment 2. The 60 recorded pattern signals were analyzed using the methods described in Section 2. Selected results are presented. Table 3 lists the results of the apnea event detection algorithm for the P2pattern signals (apnea time period to detect > Tapnea = 4 s). The mean, standard deviation and values for the coefficient of determination for the linear regression model were calculated and are shown in Table 3 . b i o c y b e r n e t i c s a n d b i o m e d i c a l e n g i n e e r i n g 3 6 ( 2 0 1 6 ) 7 3 1 -7 4 1 Fig. 6 illustrates selected examples of the apnea event detection results for two special cases: slow respiration rate and increasing temperature values during apnea (subject S04, left chart) and fast respiration rate and decreasing temperature values during apnea (subject S05, right chart). The respiration rate was also estimated for signals from experiment 2 (60 signals, P1-P5 patterns). The previously used RR estimators were used. In the absence of apnea events (P3), the results were practically the same as those for the data in experiment 1 (i.e., accurate RR estimation, especially using the eRR_ac estimator). In the presence of apnea events (P1, P2, P4, and P5), differences were observed due to the different definitions of the estimators. The eRR_sp and eRR_ac estimate the dominated signal's frequency such that apnea periods did not highly influence the result. Table 4 lists the estimated respiration rate values for signals with apnea events (P2 pattern). The reference values were calculated for the belt signals using manual indication of points in time for inspiration starts and counting the number of breath events (between beginning of inspirations). Two sets of reference values were calculated: RV1 = values calculated including apnea periods ((no. of events * 60)/(total time)) and RV2 = values calculated excluding apnea periods ((no. of events * 60)/ (sum of all periods of breath events)). The values of respiration rates for RV1 are smaller than those for RV2. The results presented in Table 4 for the eRR_sp and eRR_ac estimators were calculated with reference to the RV2 values, whereas the results for eRR_pk were calculated using the RV1 values (i.e., with apnea periods, according to the definition of the respiration rate). Table 5 displays the mean values and standard deviations of the respiration rate differences between estimated values and reference values. Additionally, values for the coefficient of determination for the linear regression model were calculated. For example, comparing the respiration rates obtained for the eRR_sp and eRR_ac estimators and using RV1 as a reference leads to much higher (than for RV2) differences between the estimated and reference respiration rates (mean and std. dev.), i.e., 4.677 AE 3.99 for eRR_sp and 5.611 AE 3.214 for eRR_ac. The ''complexity'' and SPI descriptors were also calculated for the pattern signals. The highest mean complexity value (lowest similarity to sine signal) was found for the P5 patterns, i.e., 0.163 AE 0.023, whereas the smallest mean value was found for the P3 patterns, i.e., 0.128 AE 0.050. Similarly, for the SPI descriptor, the highest mean value (more spectrally ''pure'' signal) was found for the P3 patterns, i.e., 0.489 AE 0.191, and the smallest mean value was noted for the P5 patterns, i.e., 0.308 AE 0.095. The recorded signals for the P1 and P5 patterns (irregular amplitudes) were also analyzed to verify whether the relative change of amplitudes observed for thermal recordings corresponds to the similar change observed for belt recordings. No such correlation was observed. Amplitude modulation for thermal recordings was not related to belt-based signals. Therefore, no quantitative analysis was performed. Fig. 7 shows examples of respiratory signals recorded simultaneously for the belt and the thermal camera (subject S01). The spectrogram is additionally presented (short time Fourier transform, window size = 250). The goal of this paper was to investigate the reliability of estimation of the respiration rate from short thermal image sequences obtained using a mobile thermal camera and to determine which types of respiration patterns can be described. Four RR estimators were investigated. The best results were obtained for the estimator based on the periodicity of the autocorrelation as a function of time lags (eRR_ac). A slightly worse accuracy was obtained for the eRR_pk and eRR_sp estimators. The mean differences between the estimated values and reference values were small even for the estimator based on zero-crossing counting. It can be concluded that it is possible to reliably estimate the respiration rate from shorttime thermal sequences acquired for faces. Similar studies were previously presented for an estimator based on frequency detection for the maximum peak in the amplitude or PSD spectrum after FFT transform [21] or with classification of frames [22] . However, it was proved in this study that other estimators might produce notably good or even better results. It was also shown that the introduction of signal descriptors that represent the ''periodicity'' of the signal could aid in differentiation between highly periodical signals (potential respiration waveforms) and other signals (background signals). This approach could be important in automatic evaluation of source ROIs that contain respiration-related information during data acquisition using a thermal camera mounted on smart glasses. In such cases, the face in the thermal image should be automatically detected (e.g., [37, 38] ) as well as the nose or mouth ROIs. The dominant frequency of the signal can be detected for almost all signals and also for signals that are not respiration waveforms. Therefore, additional measures such as ''complexity'' or SPI can be used to aid in differentiation between less or more probable respiration signals or to describe the periodicity of the respiration waveforms. The analysis performed for signals that represent different respiration patterns showed rather interesting results. First, it was demonstrated that it is possible to reliably detect apnea periods from thermal-based signals using a relatively simple and fast algorithm. It is also possible to reliably estimate the respiration rate for signals that contain apnea periods. For example, the spectrograms presented in Fig. 6 show highly similar time-frequency characteristics between thermalbased signals and reference signals. However, in the presence of apnea events, the investigated RR estimators produce different results, which is highly related to the definitions of the estimators. The respiration rate is defined as the number of respiration events in a given time window. Therefore, in the presence of apnea events, the estimated respiration rate values should be and were observed to be higher (in reference to the RR definition) for the estimators based on dominant frequencies (eRR_ac, eRR_sp). The methods based on signal changes in the time domain, such as eRR_pk, produced more reliable results. However, different values of the respiration rates estimated using either the dominant frequency or the number of respiration events in the time domain could be used as an additional measure for discrimination between respiration patterns with apnea periods or without apnea. In the absence of apnea events, such estimators should produce similar values (e.g., eRR_ac and eRR_pk). If the values of both estimators are significantly different, the algorithm for detection of apnea periods could be used to verify the presence of cessation of respiratory airflow. With the performed qualitative analysis, we verified that it is not possible to directly correlate the amplitude values between thermal-based signals and signals obtained from the reference pressure belt (P1 and P5 patterns). Therefore, it would be difficult to directly differentiate between, e.g., Cheyne-Stokes respirations (P1) and Biot's breathing (P2). However, it can be observed that deeper breaths in the series of respiration events are typically longer due to the physiological mechanisms of breathing. This topic requires further research. In conclusion, it was proven that using thermal-based signals captured by mobile thermal camera, it is possible to reliably extract many important respiration parameters, including the RR, periodicity of respiration, presence and length of apnea periods. The results obtained from thermalbased signals are quite similar to and highly correlate with the results obtained for reference measurements (belt). In this study, selected assumptions were applied to focus on a comparison of different RR estimators and extraction of selected parameters of respiration waveforms. It was assumed that subjects do not move (controlled experiment), and thus, the averaging operation in the manually specified ROI should represent the local temperature change due to breathing activity. In real applications, automatic detection and tracking of the source would be much more practical, e.g., using methods described in the introduction. The fixed ROI size was also assumed as being defined in reference to the nose size. Further studies should be performed to analyze the role of the ROI size in reference to the visibility of nasal cavities, especially to eliminate possible amplitude modulation. This last point is especially important when the visibility of nasal cavities changes due to larger head movements. The proposed methodology can be used in practice for unobtrusive respiratory monitoring during routine interviews with patients or for screening purposes (e.g., at airports). For example, it is possible to detect tachypnea (RR > 20 bpm) that might indicate hypoxemia, hypercapnia, or anxiety [39] . It is also possible to detect bradypnea (RR < 12 bpm) that might indicate head injuries, drug overdoses or other problems related to the central nervous system. The thermal-based method has theoretical limits related to the required thermal gradient between the human body and the environment. In this study, measurements were performed indoors for two different ambient temperatures: 22 8C and 27 8C. No significant differences were observed in the dynamics of the recorded signals. However, the influence of ambient environment and other measurement limits should be further investigated. Members of task force on prolonged apnea. Reports of the task force on prolonged apnea of the American Academy of Pediatrics Central sleep apnea, pathophysiology and treatment Practice parameters for the indications for polysomnography and related procedures: an update for Respiratory rate measurement: a comparison of methods Prediction of pneumonia in a pediatric emergency department Derivation and validation of age and temperature specific reference values and centile charts to predict lower respiratory tract infection in children with fever: prospective observational study Respiratory rate: the neglected vital sign Basic epidemiology Webster's new world medical dictionary Comparison of respiratory inductive plethysmograpy and thoracic impedance for apnea Accuracy and tolerance of a novel bioacoustic respiratory sensor in pediatric patients Biowatch: estimation of heart and breathing rates from wrist motions. 9th Pervasive Health Conference International classification of sleep disorders, revised: diagnostic and coding manual Standards of Practice Committee of the American Sleep Disorders Association. Practice parameters for the indications for polysomnography and related procedures A multisensor detector of a sleep apnea for using at home. The 6th HSI Conference Non contact monitoring of respiratory function via depth sensing. Biomedical and Health Informatics Conference (BHI) Model-based parametric images in dynamic thermography Thermal parametric imaging in the evaluation of skin burn depth Imaging breathing rate in the CO 2 absorption band Non-contact monitoring of respiratory function using infrared imaging Thermal infrared imaging: a novel method to monitor airflow during polysomnography Thermistor at a distance: unobtrusive measurement of breathing Evaluation of the respiration rate and pattern using a portable thermal camera Spatio-temporal smoothing as a basis for facial tissue tracking in thermal imaging Tracking human face features in thermal images for respiration monitoring Contact-free respiration rate monitoring using a pan-tilt thermal camera for stationary bike telerehabilitation sessions Spatiotemporal reconstruction of the breathing function Interactions with recognized objects Interactions with recognized patients using smart glasses Interaction with medical data using QR-codes A peculiar type of dyspnea: Kussmaul, Cheynestokes, and biot respiration Bioelectrical signal processing in cardiac and neurological applications Detection of selected face areas on thermograms with elimination of typical problems Face detection using a portable thermal camera This work has been partially supported by NCBiR, FWF, SNSF, ANR and FNR in the framework of the ERA-NET CHIST-ERA II, European project eGLASSES -The interactive eyeglasses for mobile, perceptual computing and by Statutory Funds of Electronics, Telecommunications and Informatics Faculty, Gdansk University of Technology. The authors declare that there is no conflict of interest regarding the publication of this paper.