key: cord-0196329-2v4rnelh
authors: El-Kebir, Hamza; Bentsman, Joseph
title: Physics-based Learning of Parameterized Thermodynamics from Real-time Thermography
date: 2022-03-24
journal: nan
DOI: nan
sha: 0c2a6da11ad783a350e4da3450cecae1fc9de533
doc_id: 196329
cord_uid: 2v4rnelh

Progress in automatic control of thermal processes has long been limited by the difficulty of obtaining high-fidelity thermodynamic models. Traditionally, in complex thermodynamic systems, it is often infeasible to estimate the thermophysical parameters of spatiotemporally varying processes, forcing the adoption of model-free control architectures. This comes at the cost of losing any robustness guarantees, and implies a need for extensive real-life testing. In recent years, however, infrared cameras and other thermographic equipment have become readily applicable to these processes, allowing for a real-time, non-invasive means of sensing the thermal state of a process. In this work, we present a novel physics-based approach to learning a thermal process's dynamics directly from such real-time thermographic data, while focusing attention on regions with high thermal activity. We call this process, which applies to any higher-dimensional scalar field, attention-based noise robust averaging (ANRA). Given a partial-differential equation model structure, we show that our approach is robust against noise, and can be used to initialize optimization routines to further refine parameter estimates. We demonstrate our method on several simulation examples, as well as by applying it to electrosurgical thermal response data on in vivo porcine skin tissue.

Boundary control of thermodynamic processes has increasingly been gaining attention over the past two decades, spanning applications such as control of continuous steel casting [1, 2] , melting sea-ice [3, 4] , and more recently, electrosurgical processes [5, 6] . Most of these approaches are based on a model-reference controller architecture, requiring some internal model of the thermodynamic process. These models are often based on partial differential equations (PDEs), * Research reported in this publication was supported by the National Institute of Biomedical Imaging and Bioengineering of the National Institutes of Health under award number R01EB029766. starting with a Fourier-type diffusion equation, with appropriate extensions when dealing with additional dynamics. Some examples are the introduction of a Stefan condition for phase-change processes [1, 7] , reaction-diffusion equations for chemical state-change processes [8, Ch. 11] , and the introduction of a wave-term for relativistic heat propagation of the Maxwell-Cattaneo-type [9] . Naturally, all of these models must be fit to the actual physical process before they can be employed for simulation or controller synthesis. This process of model or parameter estimation is the focus of this work. In particular, we highlight the utility of our parameter estimation approach as it relates to electrosurgical processes [5, 6] . Infrared (IR) thermography (IRT) allows one to measure surface temperature fields at high frequencies and resolutions, simply by capturing infrared radiation in the long-infrared band (9-14 µm) without any active external excitation [10] . In [5] , it was for the first time proposed to use non-collocated point-wise temperature feedback (as is seen in thermographic cameras) for control of electrosurgical processes. This idea was first put to practice in [6] , showing that real-time feedback control based on thermography of the tissue is indeed possible. However, as mentioned previously, most of the control laws in use today employ some internal model, which is based on thermophysical parameters of the thermodynamic process to be controlled. For this reason, some form of thermodynamic modeling is required, often prior to deploying the controller.

Modern approaches to thermodynamic modeling try to employ thermographic data, but often resort to computationally intensive optimization routines to fit the parameters based on this data. In recent years, genetic algorithms (GAs) have seen extensive use [11] , mainly due to the fact that the Jacobian of the error system depending on the partial differential equation need not be known, and therefore is hard to obtain in practice. Even when using these Jacobian-free optimization algorithms, convergence to the global optimum may not be guaranteed, and there is a possibility that optimization results are contaminated by sensor noise. Finally, GAs require a prohibitive number of model error evaluations, especially when the parameter space is of high dimension, and the numerical PDE solver uses a fine spatiotemporal resolution. In [12] , a Laplace transform of the original PDE was used to fit on instead, allowing the use of an efficient least-squares algorithm. This is only possible when dealing with linear dynamics, and becomes analytically prohibitive when the problem has higher spatial dimension, and also poses questions regarding noiserobustness.

Given that there are many variations in the governing coefficients on a patient-by-patient basis [13] , in order to conduct safe autonomous electrosurgery, it is therefore required that a good estimate of the thermal properties is obtained in real-time, without the need for any invasive laboratory testing. Ideally, this learning procedure should be carried out for the entirety of an electrosurgical process, without disturbing the process itself, such that any changes in thermophysical parameters may be identified and acted upon.

When high-resolution spatiotemporal data of a thermodynamic process is available, one would like to directly extract the process dynamics, rather than indirectly computing solutions to PDEs and optimizing their parameters. We propose, to the best of our knowledge, the first direct approach in which the spatial and time derivatives of the thermographic readings are estimated, and a PDE model is directly fitted to the readings. Since this is susceptible to high frequency sensor noise, we propose the use of noise-robust convolutional gradient operators. In addition, we introduce a novel weighting approach based on the rate of thermal change, so as to fa-vor regions in observed thermal field with high thermal activity. We call our approach attention-based noise robust averaging (ANRA). The proposed model estimation architecture is described in Fig. 1 .

This paper is organized as follows. In Sec. 2, the model structure that will be considered is introduced, and the model identification problem is identified. Sec. 3 presents the model architecture that underpins our learning method. In addition, simulation results demonstrating the efficacy of our method compared to other possible approaches are shown. In Sec. 4, we apply our method to real-life thermographic data from in vivo electrosurgery on porcine epidermal tissue. Conclusions and future research directions are presented in Sec. 5.

We consider the classical Fourier heat transfer equation (FHT) in two dimensions:

In this work, we aim to solve the following problem: 

such that the error eâ := N k=0

Classically, Problem 1 would be solved using some optimization routine, but this is untenable in real-time, since this requires repeated solving of the model PDE. Hence, we aim for real-time-obtainable suboptimal solutions to both problems. A key standing requirement is therefore that there be no evaluation of any PDE solutions in our method.

We now proceed to present our learning framework. There are three main components to be discussed: (i) numerical gradient computation from thermal fields; (ii) attention layer definition; (iii) direct parameter estimation. We start with our approach to numerical gradient computation based on finite differencing and noise-robust filtering.

Since our learning architecture depends on direct computation of the spatial and temporal temperature gradients, we wish to suppress the effects of high frequency noise, which is observed both spatially and temporally in thermographic data. For this reason, a noise-robust filter, i.e., one that subdues high frequency noise while leaving low frequency content unattenuated, is desirable [14] . Pavel Holoborodko introduced noise-robust gradient operators, in backward differencing form [14] , as well as linear filters for spatial gradients [15] ; this family of operators is instrumental in our work.

In this work, we introduce a rate of thermal change (RTC) field which indicates which areas of the temperature field exhibit most activity. For this reason, we propose to draw the attention of a learning algorithm to areas of high RTC, using a data-driven attention layer. This latter focus on areas of high thermal activity serves a triple purpose: first, it actively incentivizes learning new dynamics, second, it draws attention away from areas that are likely to be dominated by noise, and third, it maximizes the learning rate by focusing on the areas supporting the shortest learning time for capturing the parameters of interest.

LetT [k] ∈ R N,M be the observed temperature field at time index k ∈ N 0 := {0} ∪ N. We assume that the temperature fields are sampled at a fixed period ∆t > 0. Let∂ t be some finite-difference based time derivative operator, and let ∂ ij =∂ j∂i be the product of two spatial differential operators, where i, j ∈ {1, 2}.

We define the rate of thermal change (RTC) as follows:

The RTC layer serves to disregard points that are likely to be noise-driven, and instead favors regions with high thermal activity. This gives rise to a weighted averaging step, as shown next.

In this work, we consider the Fourier heat transfer (FHT) equation of Problem 1. There are two main approaches to fitting the thermal diffusivity a in (1); a global approach, as well as a windowed approach. Given the space limitations, we present only the global approach; a description of the windowed approach is left for future work. Central to both ideas is the notion of the rate of temperature change (RTC) presented in the previous section. We estimate a in (1) directly as follows:

We now proceed to show this approach in action on a simulated example. We also compare performance when using non-noise-robust gradient operators.

We take a = 0.2, and simulate (1) based on a Gaussian initial temperature field with a peak temperature of 3.8 units, of diameter 50 pixels, similar to what one would capture on a thermographer (see, e.g., Fig. 3(a) ). We have added a zeromean Gaussian noise, with standard deviation 0.01 to all simulation results prior to parameter estimation. Our observation data spans 1 second, and is sampled at 27 Hz. The spatial resolution is 224 pixels squared, with a physical length of 10 by 10 units.

One can compare the performance of our method to approaches using classical image processing techniques. These include using non-noise-robust filters and smoothing. Both (f) Predicted temp. field after 5 seconds. approaches attenuate the dynamics by amplifying noise or introducing artificial diffusion. The effect of these choices can clearly be seen in the kernel density estimate of the final thermal diffusivity error, as shown in Fig. 2 .

Our method produced an estimate ofâ = 0.189 (recall that the ground truth is 0.2), while backward differencing combined with a Laplacian of Gaussian filter produced a = −7.9, and adding additional Gaussian smoothing resulted inâ = −41.9, with the latter two estimates being physically infeasible.

We apply our ANRA method to estimate the thermal diffusivity of in vivo porcine epidermis (skin tissue), after exposure to 10 seconds of 15 watts of pure cutting monopolar electrosurgical action using a Covidien Force FX-3 electrosurgical generator. We used a dual Optris Xi 400 microscopic thermographer setup, position about 11 centimeters from the tissue, and logging at 27 Hz.

Using our approach, we find a thermal diffusivity ofâ ≈ 0.065 mm 2 /s (see Fig. 3(d) ), which agrees very closely with data from the literature. Andrews et al. [16] report a thermal diffusivity of 0.03 ± 0.02 mm 2 /s, which closely aligns with our result. In Fig. 3(e) , we show the temperature deviation between the observed and predicted (using the initial condition in Fig. 3 (a) andâ in (1)). Figs. 3(c) and 3(f) show that the parameter estimate obtained is very likely to be close to optimal for the choice of model. The RTC-based attention layer of Fig. 3 (b) could also be used, e.g., for tissue damage classification.

In this work, we have presented a novel approach to realtime learning of thermodynamics using thermographic data. Unlike past methods which require solution of complex PDE systems, our approach, attention-based noise robust averaging (ANRA), operates directly on thermographic data and requires only the PDE structure to be given. We have applied ANRA to a theoretical test case involving Fourier heat transfer, as well as real-life data from an in vivo porcine specimen subject to electrosurgical action. In both cases, our approach clearly outperforms naive real-time estimation methods, suggesting that ANRA presents a robust real-time alternative to offline optimization methods.

The future work may extend this approach to other thermodynamic models, such as the reaction-diffusion equation. The same approach may also be extended to applications in fluid dynamics.

Enthalpy-based feedback control algorithms for the Stefan problem

Optimal Control of Free Boundary of a Stefan Problem for Metallurgical Length Maintenance in Continuous Steel Casting

Arctic sea ice temperature profile estimation via backstepping observer design

Materials Phase Change PDE Control & Estimation: From Additive Manufacturing to Polar Ice

PDEbased modeling and non-collocated feedback control of electrosurgical-probe/tissue interaction

Online hypermodel-based path planning for feedback control of tissue denaturation in electrosurgical cutting

Control and State Estimation of the One-Phase Stefan Problem via Backstepping Design

Mathematical Biology: I. An Introduction

Heat conduction in porcine muscle and blood: Experiments and time-fractional telegraph equation model

Infrared Thermography

Blood Perfusion Parameter Estimation in Tumors by means of a Genetic Algorithm

Inverse investigation of non-Fourier heat conduction in tissue

Heat stress induced histopathology and pathophysiology of the central nervous system

Smooth Noise Robust Differentiators

Noise Robust Gradient Operators

Quantifying the role of burn temperature, burn duration and skin thickness in an in vivo animal skin model of heat conduction