key: cord-0757155-dxf2ni0l
authors: Gurgitano, Martina; Angileri, Salvatore Alessio; Rodà, Giovanni Maria; Liguori, Alessandro; Pandolfi, Marco; Ierardi, Anna Maria; Wood, Bradford J.; Carrafiello, Gianpaolo
title: Interventional Radiology ex-machina: impact of Artificial Intelligence on practice
date: 2021-04-16
journal: Radiol Med
DOI: 10.1007/s11547-021-01351-x
sha: f24f38628413f18c1a46709b66774adfa22ba43d
doc_id: 757155
cord_uid: dxf2ni0l

Artificial intelligence (AI) is a branch of Informatics that uses algorithms to tirelessly process data, understand its meaning and provide the desired outcome, continuously redefining its logic. AI was mainly introduced via artificial neural networks, developed in the early 1950s, and with its evolution into "computational learning models." Machine Learning analyzes and extracts features in larger data after exposure to examples; Deep Learning uses neural networks in order to extract meaningful patterns from imaging data, even deciphering that which would otherwise be beyond human perception. Thus, AI has the potential to revolutionize the healthcare systems and clinical practice of doctors all over the world. This is especially true for radiologists, who are integral to diagnostic medicine, helping to customize treatments and triage resources with maximum effectiveness. Related in spirit to Artificial intelligence are Augmented Reality, mixed reality, or Virtual Reality, which are able to enhance accuracy of minimally invasive treatments in image guided therapies by Interventional Radiologists. The potential applications of AI in IR go beyond computer vision and diagnosis, to include screening and modeling of patient selection, predictive tools for treatment planning and navigation, and training tools. Although no new technology is widely embraced, AI may provide opportunities to enhance radiology service and improve patient care, if studied, validated, and applied appropriately.

Artificial intelligence (AI) is a methodology of computer engineering that uses algorithms to tirelessly process data, understand its meaning and provide the desired outcome, continuously redefining its logic and selecting the optimal pathway to address the applied question.

Although its official birth is referred to as in 1955, when John McCarthy first defined it as "the science of making intelligent machines" [1] , AI may have its roots with the invention of robots, whose first mention can be traced back to the third century in China. Over a millennium later, it was Leonardo Da Vinci, during the Renaissance, who made a detailed study of human anatomy and designed his humanoid robot. His sketches, rediscovered only in the 1950s, were in fact, a source of inspiration, even leading to the invention of the surgical system carrying the Da Vinci name, that facilitates complex surgery with a minimally invasive approach and can be controlled by a surgeon from a remote console.

Over the years, AI development has relied upon the introduction of artificial neural networks (RNA) in the early 1950s, and their subsequent further evolution into the "computational learning models," including Machine Learning (ML) and Deep Learning (DL).

ML is based on algorithms that can improve task performance and decision making or predict outcomes. Specifically, with a previous training period in which ML algorithms focus on specific or characteristic features through exposure to examples, they learn to extract desired meaningful patterns from data, even seeing features that are beyond human perception [2] .

DL is made of multilayered, "or deep," artificial neural networks: the numerous neural layers between input and output contribute to the plasticity of DL and offer the potential to define new patterns of intelligent classifications, mimicking the human brain mechanisms. Different from ML, DL can automatically discern the relevant features from data (often with an unknown mechanism or overall rationale) even if unlabeled, allowing to learn new patterns and to determinate more complex relationships, which is valuable if compared with a human reader who can only partially detect and use a fraction of the total big information content of digital images [3] .

Combining ML/DL image processing with clinical and pathology/histology data, to find direct correlation between intrinsic Computed Tomography (CT) and Magnetic Resonance Imaging (MRI) patterns and specific pathology and histology subtypes, is the purpose of a new amazing sector of research called "Radiomics" [4] [5] [6] .

Thus, AI is capable of globally revolutionizing healthcare and clinical practice, and will inform, influence, and transform radiologists more than most physician disciplines. AI will augment and support instead of replacing radiologists, via enhanced performance, both in terms of speed and precision, facilitating and improving the efficiency and accuracy of the diagnosis process. This will also help to customize treatments and use resources with maximum effectiveness, which are important priorities for interventional radiologists, who often diagnose and treat in real time and at the same time. Moreover, the utilization of novel navigational techniques that superimpose virtual pre-procedural 3D anatomic data onto real-world 2D visual images in real time, such as Augmented Reality (AR), multi-modality fusion, or Virtual Reality (VR), enables improved accuracy for minimally invasive treatments and reduced risks, complications and radiation, which are central goals for Interventional Radiology (IR) [7] .

The purpose of this article is to emphasize how AI is relevant not only for Diagnostic Radiology, but also for IR. The potential applications of AI in IR go beyond computer vision, to include screening and modeling of patient selection, predictive and supportive tools for treatment planning and execution, training and intra-procedural registration, segmentation, navigation and treatment planning. Their real-time utilization in the angiography suite, with the help of augmented/virtual reality, could improve patient selection, operator's expertise, drug discovery, and multidisciplinary pursuit of delivery of tailor-made precision medicine.

Workflow solutions: patient scheduling, screening and counseling AI may directly optimize the interventional radiologist's daily practice with intelligent patient scheduling, beforehand identifying patients at high risk and taking steps to avoid the risk, as well as reducing the likelihood of missing care, via intelligent scheduling and patient selection [8] .

In addition, AI algorithms can generate a relevant summary from the available patient records, including patient problem lists, clinical notes, laboratory data, pathology reports, vital signs, prior treatments, and prior imaging reports to give the radiologist the most pertinent contextual information during the evaluation of a patient for diagnosis or therapy [9, 10] .

In addition, intelligent algorithms are proposed for patient safety screening or reports, which have the potential for applications in radiology practice (for example, MRI safety screening or administration of contrast material) [11] .

Moreover, the AR/VR can improve patient's compliance through the possibility to show a simulation of the experience in angiography suite, allowing the patient to arrive calmer and more aware of what awaits at the time of the procedure [12, 13] .

One of the hardest challenges for IR is the identification of an accurate way to predict the success rate of a specific treatment in a specific patient, and to estimate/forecast the outcomes and benefits of a treatment in advance. Similarly, big data science has attempted to investigate how a patient's demographic and pathologic characteristics before the treatment can influence its efficacy [14] .

ML-or DL-based predictive models can try to classify patients as responders and non-responders to specific treatment algorithms. Such AI models based on DL or ML may help interventional radiologists to overcome this patient selection challenge, especially in the field of interventional oncology. Such tools hold the potential to reduce unnecessary and useless procedures and interventions, healthcare costs, and decreasing the risk for the patient [15] .

From this perspective, both clinical and imaging decision support tools have been created. Among clinical ones, ML has been showcased in the evaluation of the relative prognostic significance of the Child-Turcotte-Pugh score and albumin-bilirubin grade, along with other risk factors, within a cohort of patients undergoing trans-arterial chemoembolization [16] . In addition, a ML algorithm based on serum creatinine in electronic health records was capable of predicting Acute Kidney Insufficiency (AKI) up to 72 h before onset, in specific patients, potentially allowing clinicians to intervene before kidney damage manifests, in advance of diagnostic and interventional radiology procedures and treatments [17] .

Among imaging decision support tools, image texture analysis has made it possible to extrapolate "radiological biomarkers" that allow a prognostic pre-treatment evaluation.

For instance, the performance of pre-ablation CT texture features may predict post-treatment local progression and survival in patients who undergo tumor ablation, using ML to identifying specific CT texture patterns, as demonstrated by Daye et al. for adrenal metastases. During pre-procedural assessment, in fact, when CT-derived texture features were included in addition to clinical variables, there was an increase in accuracy to more than 95% [18] . Similarly, using MRI texture features and the presence or absence of cirrhosis, Abajian et al. assessed a ML algorithm able to predict chemoembolization outcomes in patients with hepatocellular carcinoma, obtaining a very good negative predictive value (88.5%) [19] .

Additionally, in Neuroradiology, AI algorithms have been used to assist not only the diagnosis, but especially the treatment decisions in acute ischemic stroke, from estimating time of onset (that is one of the most relevant clinical criteria to decide if a patient is eligible for lytic treatment) [20, 21] , lesion estimation [22] , and salvageable tissue (essential in decision making before endovascular treatment) [23, 24] . This aided in analysis of cerebral edema [25, 26] , and predicting complications, such as hemorrhagic transformation after treatment [27] [28] [29] and patient outcomes, as well as identifying factors that will contribute to neurological deterioration and increased morbidity or predict motor deficit in stroke patients after treatment. [30] [31] [32] [33] . However, AI is not able to provide an explanation of the underlying cause and pathophysiology: more complex interpretation problems like these require deductive reasoning, medical knowledge and mechanisms to integrate the findings within a clinical, personal, and societal context. This is why the holistic human intelligence and the medical judgment of interventional radiologists will be augmented and cannot be completely replaced [34] .

First of all, during pre-procedural assessment in a procedure room setting such as the angiography suite, it is critical to evaluate anatomy and its pathophysiologic changes. In this regard, Augmented Reality (AR) or Virtual Reality (VR) may find its best application. Through advanced 3D rendering and manipulation of imaging in space, AR and VR allow operators to conceptualize difficult anatomy, to increases realism in procedural planning (when compared with standard 2D images), and to improve procedural skills in a previous simulated environment, with no risk to patients [12] .

Furthermore, by typical AI tasks may be applied to preoperative imaging to semi-automatically detect or identify specific patterns or organ or lesion segmentation. This is already used in breast, lung, liver and prostate cancer diagnosis, but is also valuable for interventional radiologists, such as with the biologic evaluation of an atherosclerotic plaque, which may allow adaptation of treatment technique [35] .

For intra-procedural guidance, AI finds application in a wide range of settings:

• Automation of protocols, based on information gathered from the electronic health records and tailored to optimize radiation exposure in a quicker and automatic way [36] . • Real-time evaluation for proper catheter, sheath, or implant selection, such as stent or prothesis to treat stenotic vascular lesions or aortic aneurisms [37] . • Prediction of ischemia-related lesions, from quantitative CT angiography, for instance in myocardial or diabetic foot treatment [38, 39] . • Real-time estimation of fractional flow reserve, based on fluoroscopic imaging, for identifying ischemia-related stenosis in coronary artery or venous disease [40] . • Automatic vessels analysis, by use of intravascular ultrasound (IVUS): in complex cases of peripheral vascular diseases, it benefits by aiding with analysis of vessel size, lesion characteristic and potential post-treatment effects. • Fusion of pre-procedure 3D images onto intraoperative 2D images: synchronizing MRI, CT or Cone Beam CT cross-sectional preoperative imaging with intraoperative real-time fluoroscopic or ultrasound images, allows for more precise guidance for biopsies [41] [42] [43] [44] [45] and local image guided therapies [46, 47] (Figs. 1, 2 , 3) and improves problem solving during procedures [48] [49] [50] . Moreover, it is possible to generate, from CT images alone or in combination with MRI data, a virtual angiography [51] or "angioscopy" that could be a valu-able procedure planning tool for interventional radiologists such as for aortic endovascular repair (Figs. 4, 5 ). • Augmented Reality and AI-based navigation systems: by superimposing virtual 3D anatomic data, acquired using CT, CBCT or MR onto real-world 2D visual images in real-time, planning software generates a virtual device trajectory that is overlaid onto visual surface anatomy, facilitating accurate visual navigation, theoretically without need for fluoroscopy [7, [52] [53] [54] . Via integrated matching AI software, automatic landmark recognition and motion compensation was enabled using fiducial markers linked by a computer algorithm. This system is already applied in a range of minimally invasive surgery procedures including lesion targeting/localization, spinal/para-spinal injections, arthrograms, tumor ablation, bone biopsies and more recently in vertebroplasty, with reduction in patient radiation exposure compared with fluoroscopy alone [7, 47] . • Automatic tools for multi-modality registration and segmentation, as an enhanced and more ergonomic approach for fusion guided procedures, such as biopsy and ablation. This would enhance standardization and normalization, potentially enabling lesser experienced IRs to perform more advanced procedures, reproducing the skill or experience or techniques of experts. with the C-arm and overlaid on live fluoroscopy, during the intervention, to facilitate catheter navigation to the damaged vessel. d Once the target arterial culprit was engaged, a selective angiogram is performed to confirm correct targeting of the bleeding site • Touchless interaction devices directly correlate preprocedural, patient-specific and literature-derived information, which reduces distraction, errors, and proce-dure time. These include eye-tracking systems, inertial sensors, cameras or webcams, and voice-driven smart assistants. In particular, the possibility of interrogating Preoperative assessment on CT images; c Creation of virtual angioscopy to evaluate the approach for dissection treatment a previously instructed intelligent assistant to obtain suggestions on which device is most appropriate during a specific intervention procedure, before removing it from the sterile container, or what is the availability of the same in hospital stock or make a cost analysis with respect to another device, is even being studied by a group of researchers from the University of California. Its application could allow the operator to choose between two devices not only according to the appropriateness for the treatment but also in relation to the outcome data or patient-specific anatomy, optimizing results in terms of time and resource savings [55] .

Other hypothetical examples could be ML models that analyze the relationships between catheter or needle position, therapeutic effect and patient outcomes; particularly in the field of ablative therapies, this would allow for the estimation of ablation margins, optimal probe placement and selection of energy settings, potentially even minimizing risk of damage to nearby structures [1, 56] .

Machine learning combined with augmented reality systems could give trainees a new platform for honing their procedural skills, leading to novel ways of assessing performance and improving their competency [37] . In particular, AR and VR applications have been shown to improve upon current training methods in motivation, interactivity and learning of material; advanced 3D rendering and manipulation of imaging in space allows trainees to conceptualize anatomy and to improve procedural skills in a simulated environment, with no risk to patients. Moreover, VR increases realism for trainees in procedural planning when compared with standard 2D images and improve multidisciplinary communication with remote colleagues [12] .

Despite continuous progress over the past decades, AI has not and may never reach the holistic intelligence of humans; AI algorithms are intensely data-hungry, which can be difficult to obtain, and training can be time-consuming and expensive and require the clinically relevant question to be posed. Deep learning annotations for training may require bulk of human ground truth, and data curation may be a hurdle.

Whereas human intelligence can be acquired through even one simple observation and without feedback, AI software must see the same object potentially hundreds of thousands of times to recognize it, needing constant iterative feedback, to indicate whether it has guessed correctly or not.

One of the main weaknesses of AI systems is the lack of experience in real life, which is necessary to make it reliable for realistic questions, and capable of abstraction and generalization. Moreover, clinical practice is studded with atypical presentations of diseases that current AI systems might misdiagnose, because no finished training set can fully represent the variety of obscure or rare cases that can be seen in real clinical practice. Clearly, rare findings or features present possible weaknesses due to lack of large volume of particular features.

In addition, more complex problems of interpretation of radiology typically require deductive reasoning, using knowledge of disease processes and selective integration of information from previous examinations or patient records, and currently there is no learning system capable of achieving the most complex integration of thinking. [3] . In fact, some clinical problems, which may seem solvable in theory, are unsolvable in practice and are intractable for computers. Beyond the pre-treatment decision-making process, interventional radiologists perform sophisticated procedures that require technical expertise and instant intraprocedural decisions and in these situations, and even the fastest computer can get stuck in an infinite cycle or take an unimaginably long time to produce an accurate decision or result [34] .

A further limitation of AI, in particular of DL algorithms, is that they are relatively opaque: it remains difficult to clarify mechanisms or what the different parts of a large network do. This makes it difficult to delineate the limits of the network or debugging errors in image interpretation.

Ethical issues, privacy, and data sharing on the use of patient data for training the AI models are timely, relevant, and variable by geography. Although AI might improve reproducibility or standardization of specific medical practices, physicians must take responsibility for the medical diagnosis and treatment provided to patients. The clinical advice provided by AI software should always be reviewed by an experienced healthcare professional, who may or may not approve the recommendation provided.

In fact, AI applications could produce significant ethical, legal and governance problems in the healthcare sector if they cause abrupt disruptions to its contextual integrity and relational dynamics.

Actually, the current regulation of AI in healthcare is subject to a miscellaneous guidelines and subjectivity of the authorities; the EU policy observes the following three Directives on medical devices: 93/42/EEC Council Directive concerning Medical Devices [57] , 90/385/EEC Council Directive on the approximation of the laws of the Member States relating to active implantable medical devices [58] , Directive 98/79/EC of the European Parliament and of the Council on in vitro diagnostic medical devices [59] . Many diverse and different data sharing, and privacy policies govern how different geographies interact. Federated learning might overcome some of these challenges, whereby the AI model weights are exchanged instead of the actual private or identifiable patient imaging data.

With many of these limitations, today's AI systems are produced as an auxiliary tool for radiologists rather than a replacement; decision making within the IR is likely to remain a matter of natural intelligence for radiologists, or a multidisciplinary team (supported by the AI), but with the physicians taking full responsibility, considering the input from the AI but not necessarily following it in a prescriptive manner [60] .

Technology remains neutral and is only as useful or strong as the human guidance used to annotate or ask the appropriate questions of the correct data. Since the data used to train ML and DL algorithms are generated by humans, the utility all depends on how it is designed and implemented.

Although concerns have been expressed about the impact of AI on the radiology professional community, AI is not actually a threat, but a great opportunity. Used correctly, it could augment conventional practice of Diagnostic and also Interventional Radiology. Although futuristic and speculative, AI combined with Augmented Reality/Virtual Reality could help the interventional radiologist make faster, correct and more accurate and efficient, cost-effective decisions in terms of diagnosis, planning and treatment management, by allowing earlier disease identification, earlier treatments at an earlier stage and with less invasiveness, thus improving patient care and satisfaction.

Human and medical judgment, and even more empathy and compassion in communicating critical findings (upon which millions of patients rely in their medical care), are difficult to quantify and even more difficult, or even impossible, to simulate. Therefore, modern radiologists need to be aware of the basic principles of AI; thanks to their ability to adapt and innovate, they will remain essential for medical practice and take a more decisive role in the process of digitization and personalization of medicine.

• AI can improve patient's compliance through the possibility to show a simulation of the experience in angiography suite through the use of AR/VR, allowing patient to aware of what awaits him/her at the time of the procedure. • Through the use of ML DL algorithms, AI models predict the success rate of a specific treatment in a specific patient, to estimate/forecast the outcomes and/or the cost efficiency and benefits of a treatment before performing it, perhaps via investigation of demographic and pathologic characteristics in advance. • During pre-procedural assessment in the IR/angiography suite, advanced 3D rendering and manipulation of imaging in space allow operators to conceptualize difficult anatomy, to increases realism in procedural planning when compared with standard 2D images, and to improve procedural skills in a previous simulated environment, with no risk to patient. • One of the main weaknesses of AI systems is the lack of clear mechanistic rationale and real-life experience, and the lack of the capability to abstract and generalize. Atypical presentations of diseases could be misdiagnosed by AI systems, because no finished training set can fully represent the variety of cases that can be seen in clinical practice. • Radiologists and IR should embrace AI tools that augment the practicality, quality, and efficiency of their practice and reject any AI model that does not.

Artificial intelligence in medicine

Machine learning for medical imaging

Deep learning: A primer for radiologists

The potential of radiomic-based phenotyping in precisionmedicine a review

Radiomics: extracting more information from medical images using advanced feature analysis

Augmented reality and artificial intelligence-based navigation during percutaneous vertebroplasty: a pilot randomised clinical trial

Socioeconomic and demographic predictors of missed opportunities to provide advanced imaging services

A review of existing and potential computer user interfaces for modern radiology

Radiology and enterprise medical imaging extensions (REMIX)

Current applications and future impact of machine learning in radiology

Implementing virtual and augmented reality tools for radiology education and training, communication, and clinical care

Virtual and augmented reality: new frontiers for clinical psychology

The role of artificial intelligence in interventional oncology: a primer

Artificial intelligence in interventional radiology: a literature review and future perspectives

Nomogram and artificial neural network for prognostic performance on the albumin-bilirubin grade for hepatocellular carcinoma undergoing transarterial chemoembolization

Prediction of acute kidney injury with a machine learning algorithm using electronic health record data

CT texture analysis and machine learning improve post-ablation prognostication in patients with adrenal metastases: a proof of concept

Predicting treatment response to intra-arterial therapies for hepatocellular carcinoma with the use of supervised machine learning-an artificial intelligence concept

MRI-Guided thrombolysis for stroke with unknown time of onset

Classifying acute ischemic stroke onset time using deep imaging features. AMIA

Fully automatic acute ischemic lesion segmentation in DWI using convolutional neural networks

Early identification of potentially salvageable tissue with MRI-based predictive algorithms after experimental ischemic stroke

Quantitative prediction of acute ischemic tissue fate using support vector machine

Automated quantification of cerebral edema following hemispheric infarction: application of a machine-learning algorithm to evaluate CSF shifts on serial head CTs

Application of machine learning to automated analysis of cerebral edema in large cohorts of ischemic stroke patients. Front Neurol

Prediction of stroke thrombolysis outcome using CT brain machine learning

Multi-center prediction of hemorrhagic transformation in acute ischemic stroke using permeability imaging features. MagnReson Imaging

Prediction of hemorrhagic transformation severity in acute stroke from source perfusion MRI

Prediction of tissue outcome and assessment of treatment effect in acute ischemic stroke using deep learning

Machine learning for outcome prediction of acute ischemic stroke post intra-arterial therapy

Multiclass support vector machine-based lesion mapping predicts functional outcome in ischemic stroke patients

Decoding post-stroke motor function from structural brain imaging

Myths and facts about artificial intelligence: why machine-and deep-learning will not replace interventional radiologists

Evaluation of carotid wall thickness by using computed tomography and semiautomatedultrasonographic software

Machine learning in radiology: applications beyond image interpretation

Machine learning for the interventional radiologist

Integrated prediction of lesion-specific ischaemia from quantitative coronary CT angiography using machine learning: a multicentre study

Use of perfusional CBCT imaging for intraprocedural evaluation of endovascular treatment in patients with diabetic foot: a concept paper

Angiography-based machine learning for predicting fractional flow reserve in intermediate coronary artery lesions

C-arm cone-beam computed tomography needle path overlay for percutaneous biopsy of pulmonary nodules

Softwareassisted US/MRI fusion-targeted biopsy for prostate cancer

Comparison of cone-beam CT-guided and CT fluoroscopy-guided transthoracic needle biopsy of lung nodules

Comparison between CBCT and fusion PET/CT-CBCT guidance for lung biopsies

Percutaneous lung tumor biopsy under CBCT guidance with PET-CT fusion imaging: preliminary experience

Prostatic artery embolization in patients with benign prostatic hyperplasia: perfusion cone-beam CT to evaluate planning and treatment response

Clinical impact of cone beam computed tomography on iterative treatment planning during ultrasound-guided percutaneous ablation of liver malignancies

Usefulness of cone-beam computed tomography and automatic vessel detection software in emergency transarterial embolization. CardiovascIn-terventRadiol

Unenhanced cone beam computed tomography and fusion imaging in direct percutaneous sac injection for treatment of type II endoleak: technical note

Fusion of CT angiography or MR angiography with unenhanced CBCT and fluoroscopy guidance in endovascular treatments of aortoiliac steno-occlusion: technical note on a preliminary experience

MR-based synthetic CT generation using a deep convolutional neural network method

A novel 3D guidance system using augmented reality for percutaneous vertebroplasty

Augmented reality visualization with image overlay for MRI-guided intervention: accuracy for lumbar spinal procedures with a 1.5-T MRI system

Cone beam computed tomography and its image guidance technology during percutaneous nucleoplasty procedures at L5/S1 lumbar level

03:45 PM Abstract No. 38 The development of a machine learning smart speaker application for device sizing in interventional radiology

Magnetoaerotactic bacteria deliver drug-containing nanoliposomes to tumour hypoxic regions

on the approximation of the laws of the Member States relating to active implantable medical devices. Off J Eur Commun 59. Parliament thee, of Thec, Union Thee (1998) Directive 98/79/Ec of The European Parliament and of the Council of 27 October 1998 on in vitro diagnostic medical devices

Artificial intelligence: Who is responsible for the diagnosis

Authors' contributions Each author has contributed to conception and design, analysis and interpretation of the data, drafting of the article, critical revision and final approval.Funding This study was not supported by any funding.

Ethical standards This article does not contain any studies with human participants or animals performed by any of the authors.