key: cord-0021911-jksxl90a authors: Saadi, Soroush Baseri; Ranjbarzadeh, Ramin; Ozeir kazemi,; Amirabadi, Amir; Ghoushchi, Saeid Jafarzadeh; Kazemi, Oveis; Azadikhah, Sonya; Bendechache, Malika title: Osteolysis: A Literature Review of Basic Science and Potential Computer-Based Image Processing Detection Methods date: 2021-10-04 journal: Comput Intell Neurosci DOI: 10.1155/2021/4196241 sha: 2707484cd41bd8e00db8c35aa692366ae4e472ea doc_id: 21911 cord_uid: jksxl90a Osteolysis is one of the most prominent reasons of revision surgeries in total joint arthroplasty. This biological phenomenon is induced by wear particles and corrosion products that stimulate inflammatory biological response of surrounding tissues. The eventual responses of osteolysis are the activation of macrophages leading to bone resorption and prosthesis failure. Various factors are involved in the initiation of osteolysis from biological issues, design, material specifications, and model of the prosthesis to the health condition of the patient. Nevertheless, the factors leading to osteolysis are sometimes preventable. Changes in implant design and polyethylene manufacturing are striving to improve overall wear. Osteolysis is clinically asymptomatic and can be diagnosed and analyzed during follow-up sessions through various imaging modalities and methods, such as serial radiographic, CT scan, MRI, and image processing-based methods, especially with the use of artificial neural network algorithms. Deep learning algorithms with a variety of neural network structures such as CNN, U-Net, and Seg-UNet have proved to be efficient algorithms for medical image processing specifically in the field of orthopedics for the detection and segmentation of tumors. These deep learning algorithms can effectively detect and analyze osteolytic lesions well in advance during follow-up sessions in order to administer proper treatments before reaching a critical point. Osteolysis can be treated surgically or nonsurgically with medications. However, revision surgeries are the only solution for the progressive osteolysis. In this literature review, the underlying causes, mechanisms, and treatments of osteolysis are discussed with the main focus on the possible computer-based methods and algorithms that can be effectively employed for the detection of osteolysis. Basically, osteolysis is a biological process initiated by induced particles at the interface of bone metal or bone cement of prosthetic implants which is radiographically manifested as linear endosteal radiolucencies or scalloped focal. is phenomenon in long-term results in bone loss, periprostatic fractures, and finally loosening of implants. From early observations, it was generally believed that, in cemented implants, osteolysis is due to acrylic cement and the term "cement disease" was introduced because of this belief. Nevertheless, osteolysis is now defined as "particle disease" with the demonstration of lytic lesions in implants without cement in the interface [1] [2] [3] . Osteolysis is one of the main causes for late reoperation in patients with total joint arthroplasty (TJA). is operation is the most effective therapeutic solution for patients enduring end-stage degenerative arthritis. It is believed that the demands for TJAs will gradually increase worldwide in subsequent years where total hip arthroplasty (THA) and total knee arthroplasty (TKA) are the most frequents types. Although the lifetime of THA has improved, such that approximately 90% of the implants function appropriately at 15 years [4] , osteolysis is the main cause of at least 50% of all THA revision surgeries based on the majority of national registries. Despite the fact that the effectiveness of surgery may depend on various variables including the level of physical activity of patients, type of implant material, time of THA in service, model of the implant and placement [5] , more than 100,000 patients for each million THA procedures might experience a revision surgery within a 15-year period of service. Osteolysis has been noted in both cemented and cementless implants, with 0% to 16% for cemented and 6% to 30% for cementless TJA. It is indicated by clinical experience that there is a strong correlation between the likelihood of osteolysis and the magnitude of polyethylene [6] . e majority of the wear particles are ultrahigh molecular weight polyethylene (UH-MWPE) which is mainly implicated in the development of lesions [7] [8] [9] . e appearance of osteolysis at shorter times of follow-up will become more apparent as the higher wear rates cause higher rates of osteolysis [10] (Figure 1 ). Considering a uniform distribution of wear particles through the periprosthetic tissue, the wear volume can be directly related to the number of particles per unit volume of tissue [11] . Nevertheless, it is improbable that the distribution of wear particles is uniform as the permeability of tissues varies, and there are also a small number of pathways for particle access. Local osteolysis can be produced as a result of local accumulation of wear debris even if the general density of particles is low. is makes it difficult to determine the relationship of appearance of osteolysis to wear. Osteolysis is a progressive medical condition. us, regular follow-up sessions are advised. During these sessions, orthopedists check the progress of the condition with the use of different imaging modalities depending on the severity of the condition. Nevertheless, regardless of imaging techniques, detection and analysis of osteolysis are difficult tasks to perform as osteolytic lesions are not easy to be distinguished especially at the first years after implantation. is is the main reason that researchers and biomedical engineers in the field of medical image processing are making a great effort to devise new methods and algorithms to overcome these barriers. In recent years, artificial intelligence (AI) has become highly popular and is developing into every aspect of the modern life by its advances in large-data retrieval and explicit evaluation of features that are ideal for medical image processing [12] [13] [14] [15] . In comparison to traditional image processing methods, deep learning is more efficient and reliable, as it can automatically extract features of the images instead of hand-crafted features [16, 17] . anks to the deep neural networks (DNNs), the methods of computation can allow an algorithm to self-programme through learning from a large set of examples that show the desired behavior, eliminating the need for specifying the rules explicitly [18, 19] . Deep learning (DL) techniques have developed fast and have been shown to exceed the performance of human beings. Deep convolutional neural networks (DCNNs) demonstrate a great benefit in image processing. CNN is a subclass of deep, feedforward neural networks, in which image data moves in a forward direction from the input to the output nodes. e advantageous feature of CNNs is that they can learn directly useful image features and other structured data, whereas the task of feature extraction has been carried out by machine learning models or by hand before CNNs. ese types of neural network architecture proved to be powerful deep learning models in the field of image analysis, judging from the existence of specific features in their structure [20] [21] [22] . A typical CNN is composed of one or several filters that are called filters, kernels, or convolutional layers, accompanied by several layers (an aggregation and pooling layer) that are employed for classification purposes. As the characteristics of CNN are similar to those of the standard artificial neural network (ANN), backpropagation and gradient descent are used for training tasks, while it is comprised of additional pooling layers along with kernels. e final results are derived from the vector that is sited at the end of the network architecture. CNNs can have various structures and methods. In medical image analysis, the most popular structures are U-Net [23] , SegNet [24] , and V-Net [25] as well as the conventional CNN structures. Additionally, the most common CNNs methods are the following: ZFNet (2013) [26] , VGGNet-16 (2014) [27] , AlexNet (2012) [28] , GoogLeNet [29] , DenseNet (2017) [30] , and ResNet (2015) [31] . ere are several studies that prove the effectiveness of CNN algorithms in medical image processing, such as breast cancer detection [30] , diagnosis of breast ultrasonography images [32] , liver cancer detection and segmentation [33] , differentiation of liver masses [34] , lung infection segmentation [35, 36] , and classification of interstitial lung disease [37] . However, one key drawback of most deep learning approaches is that they need a lot of training data to learn features properly. e intent of this study was not to investigate comprehensively the biological and medical aspect of osteolysis. An attempt was made, however, to give a guideline for the main factors and issues involved in this medical phenomenon and to introduce computer-based methods and algorithms especially in the field of artificial neural networks used successfully for similar purposes. Osteolysis is an active biological response to particulate wear debris [38] , primary bone tumors [39] , and metastatic diseases [40] . In fact, this phenomenon is a cell-mediated biological process leading to bone loss as a direct reaction of stimulation of macrophages by biologically active particles. It is noted that the mechanical wear of the articulating surface that releases wear debris begins this process. is key debris is phagocytosed, which activates osteoclasts and macrophages and results in resorption of the bones [3] ( Figure 2 ). e cells which are mainly involved in creating a response to the particles are fibroblasts, lymphocytes, macrophages, osteoclasts, osteoblasts, and foreign body giant cells. e main cellular mediator of osteolysis is macrophages that take part individually in the resorption of bones by transforming into osteoclasts leading to much faster resorption. ese cells secrete a variety of cytokines and are present in the pseudomembrane. Among them, the most important cells are interleukin 6, interleukin 1α, interleukin 1β, TNF-α, prostaglandin E2, RANKL, gelatinase, and collagenases [1, [41] [42] [43] . Generally, we can distinguish between nonwear-related and wear-related byproducts. At each stage, wear particles are released from the surface of the softer material by abrasion and adhesion of bearing surface [44] . During the time of service, it is recognized that all lower-limb arthroplasties generate wear particles [45] . Based on the biotribological experiments, materials which are hard similar to ceramics that create nanosized wear particles can cause inflammation [46] . Bone cement debris and metallic debris were also noted to have relation to hypersensitivity and inflammation [47] [48] [49] . Nevertheless, there is limited evidence on the influence of the prosthetic particle dimension, surface charge, shape, and osteoclast capacity. e experimental evidence also demonstrates that erosive bone resorption can cause prosthetic joint infection if not diagnosed and treated early [50] [51] [52] . It has also been recognized that local hydrodynamics factors can contribute to osteolysis. Local fluid pressure gradients around implants are considered to create a mechanism for the particle to transport and force fluid and to surround the bone. Very high intra-articular pressures, caused by fluid pressure, changes during motion can mount osteocyte death and subsequently induce osteoclast bone resorption [53] . Finally, it is also noted that the type of metal debris has a profound consequence for the extent of osteolysis. Titanium has a greater destructive stimulatory impact than cobaltchromium (CoCr). Early death of the macrophages caused by CoCr particles reduces the inflammatory mediators, leading to osteolysis. On the other hand, titanium debris is less toxic to the macrophage, thus triggering upper levels of the inflammatory mediation [54] . In addition, experiments have demonstrated that the mean particle dimension of less than 1.7 ± 0.7 um of wear particulate debris will increase the rate of osteolysis after THA [55] . Generally, there is no agreement on the classification of osteolysis as different methods of measurements are used in experiments and analysis. Although many studies have been conducted on this phenomenon, normally the incidence is reported. In fact, for the classification of osteolysis, lesion volume is measured. is is carried out by measuring the area of the osteolytic regions on lateral and anteroposterior radiographs. In some cases, only one view is used. On plain radiographs, normally femoral lesions can be easily viewed, whereas this is not the case for acetabulum lesions. Revision surgeries have shown that the lesions behind the acetabulum are larger than the dimension expected from preoperative radiographs [56, 57] . Not all osteolytic lesions (softened section of a patient's bone) lead to failure although they are considered as an unfavourable medical issue and are a predictive factor for later adverse consequences. ere are some osteolytic lesions that are stable and do not expand, whereas there are others which have a balloon-like features and can expand and propagate [58, 59] . Osteolytic lesions can diffuse or be localized. e diffuse lesion can progress along the interface or be stable. Depending on the location, the prognosis for a lesion can vary. Although the lesions behind the acetabular component may not progress, they are naturally expansive at this location [60] . Computational Intelligence and Neuroscience Furthermore, it seems also that cemented and cementless implants give rise to different biological processes [61] . In cemented acetabular components, there is a tendency to show a pattern of osteolysis leading to losing the implant, whereas cementless components tend more to exhibit expansile, localized lesions with the cup of the implant remaining stable (Figure 3 ) [62] . Additionally, although loosening of the component of the implant may precede the progress of an expansile lesion, it is less often (Figure 4 ). e most popular method of detection of the extent of osteolytic lesions is computed tomography whereas the most sensitive method is magnetic resonance imaging (MRI). e studies have demonstrated that the sensitivity of CT and plain radiography in the detection of osteolysis is 74.7% and 51.7%, respectively, while the corresponding figure for MRI is 95.4% [63] . Furthermore, thanks to MRI images, it is not only possible to detect intracapsular synovial disease well before discernible loss of bone, but also to detect the granulomatous tissues caused by the wear debris, which are actually the merits of this tomographic modality [64] . Conversely, it is a difficult task to identify osteolytic lesions on plain radiographic images because of two reasons. First, the visualization of the cancellous bone adjacent to the prosthesis can be blocked by the tibial and femoral components. Second, before detecting any changes on the quantity of the skeletal calcium on the radiographic images, 50% of them are normally lost [65] . Currently, various methods are employed for the detection and analysis of the osteolytic lesions. However, the application of these methods is highly dependent on the knowledge of the medical profession, available facilities, and severity of the condition. Diagnosis of the osteolysis can be implemented manually using radiographic, CT, and MRI images or can be carried out with semiautomatic or automatic computer-based techniques, algorithms, toolkits, and software. Recently, thanks to the advances in the artificial neural networks, deep learning algorithms are widely used by orthopedists and medical profession with the assistance of biomedical research for the detection of a variety of bone tumors and fractures, analysis of prosthetic movements, etc. Although fewer computerbased studies have been carried out specifically on the topic of osteolysis as there are very limited public datasets for this topic, because of strict resemblance between anatomical regions of interest, tissue properties, biological behaviors, detection, and analytic modalities, the majority of these Computational Intelligence and Neuroscience methods are applicable for the analysis and detection of osteolysis. Following the most recent and state-of-the-art potential computer-based methods from semiautomatic to automatic methods, software and algorithms that can be employed to assist medical profession in the analysis of osteolytic lesions are briefly discussed. (1) Method 1: AnoGAN. It is important to detect osteolytic lesions in advance during the follow-up sessions before they reach to a level leading to surgical treatments. Since osteolytic lesions are very small and have abnormal shape from unpredictable adjacent areas in the bone, the image collection with osteolytic lesions is challenging compared to major cancers. In this case, it can be determined that abnormality recognition is a proper idea. In the anomaly detection, the network is trained with a large number of images without lesions and then judges whether images show lesions or not based on a determined abnormality score [66] . As the number of images with lesions is limited in comparison to the number of images without lesions, the anomaly technique could be an efficient solution. Although labeled images are used for training the network, it is a heavy burden for clinicians to annotate the images. In order to reduce the labor for medical professionals, unsupervised learning methods are normally considered. is is due to the fact that all CAD systems can be more helpful in detecting a target more accurately in the image. Watanabe et al. [67] carried out a study using a similar modality and called it AnoGAN, which can be useful in detecting osteolytic lesions. ey designed a classification method that functions based on adversarial learning but used bone tumor datasets to detect the metastatic bone tumors. Tumors lesions are distinguished based on computed abnormality scores. ese scores are defined by comparing a generated image with a test image in the image level and the future level. e suggested strategy takes only nonmetastatic bone tumor images and learns the normal image distributions based on a generative adversarial network (GAN). Although using unsupervised learning method is quite challenging in medical image processing, it is a significant task for clinical applications. Figure 5 demonstrates the anomaly detection framework proposed by Watanabe et al. [67] . As can be seen from Figure 5 , the proposed method is composed of two steps: training step and test step. During the training phase, the generative G and discriminator D networks compete to gain ability. After adversarial training, the generator network is able to produce nonmetastatic bone image from a latent noise vector z. is trained generator and the noise vector z are then used for the evaluation of the test samples in the next phase. e test phase employs the latent space to evaluate test samples. When a test image is imported, the method finds the finest z corresponding to the image G(z) generated by the generator network. e z value should be visually similar to X in iterative backpropagation stages. In the latent space, the best noise vector z can be found. e anomaly score is obtained with the following formula: where D(X) represents the discrimination loss that calculates the distinction of extracted features using the trained discriminator D in the features level. R(X) denotes the residual loss that computes the visual dissimilarity between G(z) and X in the image level. λ represents a weighted coefficient. e larger the value for the anomaly score, the greater the probability of anomaly detection. Figure 6 shows the experimental results using this method. (2) Method 2: SG-CNN. CNNs are very popular for classification tasks. ere are various classic models of CNN such as VGGNet [27] and AlexNet [68] that can be selected for categorization tasks. Although these networks demonstrate good performance for classification of natural images, they are not very efficient for classification of medical images especially when the goal is to detect tumor lesions like osteolysis. e reason is that, in natural images, most objects are normally at the center of image and the variation between objects is apparent, whereas medical image categorization demands fine-grained visual classification that cannot be done with classic CNNs with high classification accuracy. Although several methods are proposed to overcome the fine-grained labeling problem such as a mask-CNN structure based on annotations of part of images by Wei et al. [69] and part-based R-CNN structure for finegrained labeling by Zhang et al. [70] , these methods are time consuming on making datasets. To solve this problem, a novel CNN is proposed by Li et al. [71] ; it can produce ROI areas automatically by network independently without employing annotated images (Figure 7) . is model is a superlabel-guided convolutional neural network (SG-CNN) that classifies CT images of bone tumor. SG-CNN is composed of two subnetworks that are Calculation of anomaly score A(x) Searching optimal latent vector G D D Computational Intelligence and Neuroscience responsible for learning the whole image and focus on bone tumor lesions to learn more detailed data. e inputs of these networks are CT images with two labels in hierarchical relationship, and the outputs are fine-grained labels and superlabels that both are used for training the SG-CNN to gain classification accuracy. AlexNet CNN model was used as the network architecture. During training phase, images are fed into the superlabel subnetwork. When all the feature maps of the guide convolution layer of the subnetwork are added together, a heatmap is generated (Figure 8 ). e heatmap generated during this phase, superimposed on the points corresponding to the image parts, is used as the input of the fine-grained label subnetwork. As the image is cropped, most of the background parts of the image are removed. us, the model emphasizes the potential tumor areas. Finally, the output of the network predicts finegrained label whose classification accuracy is considered by two structure branches simultaneously. Employing this artificial neural network model could effectively be used for the classification of osteolytic lesions, as the same obstacles are involved in the detection and classification of bone tumors, and the appearance of osteolytic lesions in CT images is very similar to that of bone tumors. Considering the successful results obtained using a bone tumor dataset on the SG-CNN by Li et al. [71] , this method could be highly reliable. (3) Method 3: U-Net. Convolutional CNNs used for the classification are composed of convolutional layers followed by several fully connected layers that map the feature image produced by kernels into a fixed-size feature vector [72] . Nevertheless, one of the defects of CNN is that each time a convolutional operation is accomplished for the classification, feature map resolution is diminished by half. is can reduce the accuracy of the classification if the feature map achieved by the final classification is utilized. erefore, if convolutional neural networks are used for the classification of osteolysis, CNN may cause transition during the convolution operation. Although the performance of the model is rarely affected, the position of osteolytic lesions may simply be affected in the recognition step. Considering the fact that osteolytic lesions have abnormal shape and low contrast, this defect can visually affect the images by metallic artifacts and image noise [12, 25, 73, 74] . In comparison to the CNN, the fully convolutional layers in the fully convolutional networks (FCN), which accept input data (images) at any dimension, replace all the convolutional layers. In this network, a deconvolution layer is applied in order to perform upsampling for the feature map of the last convolutional layer to be at the same dimension of the input 2D data (image). us, while preserving the spatial information in the original input image, a prediction can be made for each pixel. Finally, the pixel-by-pixel classification can be performed on the upsampled feature map (Figure 9 ). Based on the residual network strategy, the FCN can solve the contradiction between the translation invariance in the target detection step and the translation invariance in the classification network. us, a semantic segmentation structure U-Net based on fully convolutional neural network can be an efficient method for the detection and classification of osteolytic legions. In a study carried out by Jian et al. [75] , a U-Net neural network consisting of a contraction path and an expanding path ( Figure 10 ) was used for the diagnosis of osteoporosis, which is the most common bone disease [76, 77] . For the detection of osteoporosis of the femoral neck, if a conventional CNN is employed, it could cause translation during the convolution operation. is could easily affect the position of the osteoporosis boundary box in the detection step that directly leads to a reduction in the segmentation accuracy of the femoral neck. Osteoporosis is normally specified by a reduction in the bone density, thinning of cortical bone, and thinning of trabecular bone. e U-Net network proposed by Jian et al. [75] was implemented on an image dataset of patients having undergone pelvic X-ray imaging. Among the patients, 30 had normal bone mass, 28 had lower bone mass, and 31 had osteoporosis. Normal bone mass X-ray images demonstrate thick cortical bone and high bone density ( Figure 11 ). e recognition results and classification of the U-Net network proposed by Liu et al. are as follows: the total recognition rate of lower bone mass images from normal bone mass is 83.88%, the corresponding figure for the osteoporosis from normal bone mass is 86.74%, and the one for osteoporosis from lower bone mass is 79.55%. e experimental results of the network demonstrate that it can successfully solve the influence of image interference for the bone density analysis. e proposed U-Net network with the recognition rate of above 81% for the detection of osteoporosis could be a highly functional solution to solve a similar problem for the detection of osteolysis. (4) Method 4: Seg-UNet. Do et al. [78] proposed a novel method for the detection of knee bone tumors from X-ray images using a multilevel Seg-UNet model with global-and patch-based techniques. is network is used as a computerbased assistive tool for the segmentation and classification of tumor regions into three labels: normal, benign, and malignant. Although this network is designed only for bone tumor detection around knee regions, as the anatomical region is at the same place where osteolysis will occur after TKA and there are similar visual characteristics in X-ray images, developing this network can effectively solve the problem of segmentation and classification of osteolytic lesion. e proposed Seg-UNet architecture is illustrated in Figure 12 . is multilevel network uses a combined globaland patch-based approach in order to not only detect small tumor regions but also achieve a high improvement in malignant tumor detection. is model has an encoder-decoder architecture to exploit the mutual advantage of segmentation and classification branches to learn the local texture features and global geometric context at every pixel. e encoding block E (X enc ) with the global encoding features X enc from the input image X is located at the left side of the model. e input of the model can be either a down scale image X G or an image patch X p from the original image X o with high-resolution. e classification branch in the middle of the model employs the global average pooling for the extraction of the encoding features, followed by dense and soft max layers for the classification of the input image. e network is composed of three outputs:Ŷ clas denotes the classification result,Ŷs eg presents the tumor segmentation result, and finallyŶ dist is the multilevel high-risk tumor result. eŶ clas determines Figure 9 : End-to-end network structure [75] . 8 Computational Intelligence and Neuroscience whether the input belongs to the normal, benign, or malignant labels. Because of the complexity and challenging conditions of the X-ray images of the knee bone, this separate classification branch at the global-context level was designed. e decoding block D (X enc ) on the right side of the model is the expanding path that maps the encoding feature into decoding feature X map at the pixel level. For outputting tumor segmentation maskŶs eg and multilevel distance featuresŶ dist , the 2D extracted features are improved by multitask learning at the pixel level between the high-risk tumor segmentation H dis and the pixel-tumor segmentation H seg.Ŷseg with the vector size of W×H×2 is used for the classification of pixels of the input image into normal or tumor groups.Ŷ dist with the vector size of W×H×5 decides the level of attention, i.e., normal, tumor, or high-risk, based on the distance to tumor in three levels of 1 to 3. As, in many cases, tumors are very small in comparison to the background regions, this model made an attempt to detect small tumors by learning mutual information from adjacent feature maps around tumors. e size of the high-resolution image compared to the very small size of the tumors is one of the challenges in knee bone tumor detection. Due to the memory limits, the input image is normally resized to be suitable for the global-based patch, leading to a loss of some image texture important for tumor recognition, especially for small tumors. e patch-based model that learns image texture detailed from image patches can solve this problem. e global-and patch-based models' data used to train the network are demonstrated in Figures 13 and 14 . e effectiveness and the ability of each model (global-based model and patch-based model) in the detection of bone tumors are illustrated in Figures 15 and 16 . Noise in small tumors, nontumor detection in variant pose, and noise in larger tumors are the failures of the patch-based model (Figure 15 ), whereas it assists global-based model for the detection of small, long, and large tumors (Figure 16 ). e Seg-UNet network proposed by Do et al. [78] demonstrated that the fusion model of the patch-and globalbased models could provide mean classification accuracy of 99.05% and segmentation mean IoU of 84.84% for bone tumor segmentation and classification when both globaland patch-based models are used. tRTA was developed as an alternative to the conventional method of diagnosis of osteolysis, in which the condition of patients is postsurgically monitored by several X-ray images Figure 10 : e U-Net network architecture. e contracting path is composed of two 3 × 3 convolution layers for repetitive processing, each accompanied with a modified linear unit (ReLU). e downsampling is composed of a single 2 × 2 maximum pooling operation [75] . Figure 12 : e multilevel Seg-UNet model with patch-and global-based approaches (top) and the block demonstration of the model (bottom). In the block diagram, E represents the encoding block; D depicts the decoding block; the three branch blocks H seg , H dist , and H clas denote multitask learning [78] . [78] . [79] . As osteolysis is a biological process that evolves and appears slowly, early follow-up and diagnosis of osteolysis in the radiographic images are cumbersome and in some cases impossible tasks. Furthermore, the symptoms of trabecular texture changes caused by osteolysis are difficult to observe normally until the lesions progress. Although CT images could ameliorate the detection process of osteolysis, the cost and exposure issues make them impractical for regular follow-up. tRTA is a computerized radiographic texture analysis method used as an alternative to RTA previously used for the measurement of the patterns of trabecular bone to assist the detection of osteolytic lesions [79] [80] [81] . In the tRTA method, ROIs including the potential osteolytic lesions are selected from the image database, taken during follow-up, and visually compared with the images of the previous sessions. en, texture features are computed from the selected ROIs to perform trend analysis with simple linear regression technique, BANN temporal analysis technique, and a LDA merging features technique. is method has the advantage of incorporating the absolute texture measures, as well as how these measures alter over time. In this method, the area of osteolytic lesions is measured based on the idea of cross intersect counting [82] . A morphometric grid is superimposed over the region of interest on the radiographic images, and then the number of test points overlapping the area of interest is counted (Figure 17 ). Based on the study carried out by Smith et al. [82] , the application results of morphometric method are compared with the estimations of professional orthopedists and proved to be reliable for the measurement of the area of osteolytic lesions if applied by trained orthopedic observers (nonmedical or medical health professionals). Various software packages and toolkits have been developed for the purpose of medical image processing. Although they are not specifically developed for a single medical purpose, a variety of them are used by the medical profession and researchers for tumor detection and analysis using a variety of imaging modalities. e most popular and commonly used software and toolkits are mentioned as follows. (1) ImageJ. is is an open-source image processing program developed in Java language inspired by NIH Image [83] . is program is currently available on all the current computer platforms and can be used with variety of plugins and macros. Although ImageJ is not specialized software, it is sometimes used by professionals and researchers for medical purposes. It performs its tasks in eight steps: dataset resizing, component hollowing, volume rendering, model slicing, image slicing, image cropping, standardization of the image size, and analysis. Nevertheless, ImageJ suffers from a variety of limitations such as a lake of 3D data analysis and requiring heavy interuser variability. (2) Osteolytica. It is image processing software specifically designed for the measurement of lytic bone lesions [84] . It employs novel graphic card acceleration, and it is capable of 3D rendering to make a rapid analysis and reconstruction of osteolysis. It is designed to be faster, more user-friendly, and less biased compared to manual osteolytic lesion measuring methods or ImageJ 2D analysis. e principal goal of Osteolytica is to measure the dimension and areas affected by osteolysis in a 3D bone analysis. It is composed of four operating processes: dataset resizing, dataset loading, selecting the maximum lesion size, and finally lesion analysis. Osteolytica employs the process of reconstructing the surface of a volume sample and then subtracting the reconstructed volume from the original surface through volumetric diffusion method ( Figure 18 ) [85] . (3) 3D Slicer. 3D Slicer is an open-source, free, and multiplatform software package that can be widely used for various medical applications such as virtual reality, real-time 3D ultrasound reconstruction, adaptive radiation therapy, tracked ultrasound for needle guidance, robot assistance intervention, surgical navigation, and image segmentation ( Figure 19 ) [85, 87] . Furthermore, this software provides multiorgan analysis from head to toe and supports various imaging modalities including ultrasound, computed tomography, magnetic resonance imaging, microscopy, and nuclear medicine imaging. It also provides real-time analysis, which is highly useful during surgical navigation. Nevertheless, 3D Slicer is not approved for clinical use, and the distribution is intended for research use although there is no restriction on its employment. Figure 20 ) [89, 90] . ITK with its extreme programming methodologies and spatially oriented architecture allows processing medical images in two, three, or more dimensions. is free, open-source software for the development of interactive medical image processing software [91, 92] . MITK is a class library based on ITK that provides leading edge segmentation and registration techniques and forms the basis of algorithms. It executes the visualization commands with the Visualization Toolkit (VTK) [93] . MITK workbench has a highly customizable and extensible end-user application providing all steps of a clinical workflow such as data retrieval, image analysis, image-guided therapy, diffusion imaging treatment planning, tool tracking, diagnosis, intervention support, and treatment control ( Figure 21 ). Various techniques and methods that can be automatic, semiautomatic, or manual can be employed for the detection and segmentation of bone tumors, to be more specific osteolytic lesions. Depending on the objective of the research, they can provide different accuracies. In the previous section, the state-of-the-art automatic and semiautomatic computer-based methods, as well as manual methods with the use of software and toolkits, were mentioned for this purpose. However, the methods and techniques are not limited to this. An overview of the mentioned methods with their key points, in order to provide a well comparison for selection and application, is presented in Table 1 . Datasets play a prominent role in the performance of neural networks. Comprehensive is unsupervised learning method is suitable when the dataset is limited is method performs anomaly detection by generating a large number of nonlesion images by GAN to detect images with lesions. Convolutional neural network (CNN) is method can automatically produce ROI areas independently through a superlabelguided CNN is method can improve classification accuracy by generating fine-grained labels and superlabels of the region of interest in medical images whose lesions of interest are not well apparent. U-Net U-Net structure deep neural network is method is suitable for the segmentation of lesions when they have abnormal shape and low contrast and are susceptible to transition during classification e U-Net structure performs semantic segmentation of the osteolytic lesions on the input image by concatenating the convolutional layers in the encoder path with the deconvolutional layers in the decoder path. Multilevel Seg-UNet is method is suitable for segmentation of lesions of interest on the input image when the lesion has abnormal shape and low contrast and the size of the lesion is very small compared to the input image size e Seg-UNet exploits U-Net structure as well as the global-and patch-based approach in order to improve the classification accuracy. tRTA Mathematical computation is method is a manual image processing method for the segmentation of the lesions of interest from dataset that requires trained medical practitioners e tRTA is a computerized radiographic texture analysis method for the evaluation of ROI through linear regression, BANN temporal analysis technique, and a LDA merging features technique. Manual computation e method employs the cross-intersect counting approach In this method, with the use of a morphometric grid that is superimposed onto the region of interest on the radiographic images, computation is performed. Manual image processing General-purpose image processing software ImageJ can take the advantage of different plugins and macros for various image processing goals. Manual image processing Specifically designed for the measurement of lytic bone lesions is image processing software is designed for 3D analysis of lesions and requires trained staff. datasets can well train the networks during the training phase. Subsequently, they can have a great impact on the effectiveness and accuracy of the proposed computer-based methods for the detection and segmentation of bone tumors. e larger and the more comprehensive the datasets are, the more valuable the provided data will be. It is important to note that not all the datasets are publicly available for use. Some datasets are private and require formal requests and legal permission for use. Furthermore, the images could be acquired through different imaging modalities such as MRI, CT, PET, SPECT, or conventional X-rays and with different anatomical positions that should be implemented for the evaluation of the networks based on the objective's similarities. Table 2 presents a brief overview of the private and public datasets that can provide highly valuable data regarding bone tumors that can be used for training and evaluation of the proposed deep learning networks for the detection of osteolytic lesions. Osteolytic lesions can be treated though two ways: surgical and nonsurgical treatment. Operative treatments are recommended to address prior or ongoing osteolytic lesions and to correct failing articulation. Surgical treatments of osteolytic regions of THA, TJA, and shoulder arthroplasties Open-source medical library Manual image processing is medical library is suitable for developers for medical image processing purposes. Open-source medical library Manual image processing is class medical library is based on the ITK library and provides segmentation and registration techniques. It also has a highly customizable workbench. 16 Computational Intelligence and Neuroscience follow a similar philosophy [100] . Nevertheless, the methods are varying depending on the structure, anatomical location, function, design, and materials employed. If the prosthetic components are aligned, well-fixed, functional, and proper modular replacement parts can be obtained, the operative surgical treatment is then focused on the revision of the bearing surfaces and potential grafting of bone. e osteolytic lesions caused by metal corrosion and byproducts from metal-on-metal implants are debrided, and modular metal-on-metal bearings are replaced with other ones, normally with ceramic on polyethylene [101] . Nonoperative treatment of osteolysis is a possible option for patients who are either not able to tolerate reoperation immediately or under low loads of prosthetic byproducts and susceptible to osteolysis. e main goal of nonoperative treatment is merely to postpone the need for reoperation and also to keep the size of bone defects limited. e common nonsurgical treatment methods are mentioned as follows. (1) Bisphosphonates. Bisphosphonates are the drugs that should be consumed orally or parenterally to treat metabolic disease, osteolysis associated with metastatic and osteoporosis. is medication is a synthetic analogue of pyrophosphate and is advised for osteolysis treatment in a physician-directed or off-label manner [102] . e action mechanism of bisphosphonates is mainly on the osteoclast that undergoes apoptosis leading to the inhibition of bone resorption [103, 104] . (2) Cell erapy. Delivery of cells as a local therapeutic platform can indirectly or directly affect osteolysis. is can heal the bone and provide paracrine and autocrine factors [105] . Autologous bone grafting is one type of local cell therapy. Crosstalk between MSCs and macrophages is also an ongoing process in all the inflammatory bone disorders and bone healing process [106, 107] . Osteolysis is a progressive and biological reaction to particulate wear debris. It is the most common indication for revision surgeries after total joint arthroplasty in long-term reviews. e biological mechanisms leading to osteolytic lesions are only now beginning to be understood. Although many research studies have been carried out to characterize the complex cellular interactions that result in the bone loss, it is still not apparent why some patients undergo early osteolysis and in others this phenomenon is postponed for many years or even never occurs. It is more probable that the observed differences between the conditions of patients indicate the differential sensitivity or differences in ability to mount a wear response and generate wear particles. To expand our knowledge about the process of osteolysis, the basic science studies must be translated into clinical studies and eventually clinical practice. Although in general there is no specific classification for osteolysis, it is categorized based on the measurement of volume of the affected lesions. Studies on the genetic profile may help to explain the variability in the rate of development and the extent of osteolysis in different patients. To detect this asymptomatic process, regular radiographic follow-up is necessary. Imaging modalities like MRI and CT scans are utilized where the extent of bone loss is uncertain. Nevertheless, it is important to note that each imaging modality has its pros and cons. In addition, various computer-based image processing methods have been developed as clinically assisted tools to facilitate this process. Among them, artificial neural networks have proved to be highly efficient for tumor detection and segmentation. e research results of the popular deep learning networks used for tumor detection such as CNN, U-Net, and Seg-UNet proved that these deep learning algorithms could assist early detection and analysis of osteolytic lesions. Further developments and studies in this field are highly crucial for the development of a solid and specialized network for the detection and segmentation of osteolysis. ere are numerous nonsurgical therapeutic interventions for the treatment of the osteolytic lesion, but they still require clinical approval to be verified. Some of them have already passed limited clinical evaluations. Nevertheless, it must be determined that progressive osteolysis resulting from wear debris is not only a biological process, but also related to modular interfaces, material issues, or failure of the bearing surface. In this respect, currently, there is no evidence to prove that nonsurgical treatment methods can clinically treat osteolysis except for delaying the process. Although it is hoped that changes in the design of implants can positively affect the reduction of wear particles and osteolysis development, only time will prove this. Fortunately, there are some strong research results supporting this issue. Nevertheless, long-term clinical follow-up is desperately required to make any new benchmarks. Data Availability e authors did not use any specific dataset. e authors declare that they have no conflicts of interest. A disease of access to fixation interfaces: clinical orthopaedics and related research ® e synovial-like membrane at the bone-cement interface in loose total hip replacements and its proposed role in bone lysis Reactions of the articular capsule to wear products of artificial joint prostheses Long-Term Survivorship of Total Hip Arthroplasty with Highly Cross-Linked Polyethylene for Osteonecrosis How long does a hip replacement last? A systematic review and meta-analysis of case series and national registry reports with more than 15 years of follow-up Characterization of long-term femoralhead-penetration rates Failure of a metal-on-metal total hip arthroplasty from progressive osteolysis Osteolysis in association with a total hip arthroplasty with Osteolysis in alloarthroplasty of the hip Complications of total hip arthroplasty associated with the Number OF polyethylene particles and osteolysis IN total joint replacements Brain tumor segmentation based on deep learning and an attention mechanism using MRI multi-modalities brain images Review of current machine learning approaches for anomaly detection in network traffic TPCNN: two-path convolutional neural network for tumor and liver segmentation in CT images using a novel encoding approach An extended approach to predict retinopathy in diabetic patients using the genetic algorithm and fuzzy C-means Liver CT sequence segmentation based with improved U-Net and graph cut Fully convolutional neural network with augmented atrous spatial pyramid pool and fully connected fusion path for high resolution remote sensing image segmentation Artificial intelligence in nuclear medicine Artificial intelligence, machine (deep) learning and radio(geno)mics: definitions and nuclear medicine imaging applications Deep learning for coral classification Lung and pancreatic tumor characterization in the deep learning era: novel supervised and unsupervised learning approaches Medical breast ultrasound image segmentation by machine learning U-net: convolutional networks for biomedical image segmentation SegNet: a deep convolutional encoder-decoder architecture for image segmentation V-net: fully convolutional neural networks for volumetric medical image segmentation Visualizing and understanding convolutional networks Very deep convolutional networks for large-scale image recognition Gradientbased learning applied to document recognition Going deeper with convolutions Densely connected convolutional networks Deep residual learning for image recognition Automated diagnosis of breast ultrasonography images using deep neural networks Automated liver and tumor segmentation based on concave and convex points using fuzzy c-means and mean shift clustering Deep learning with convolutional neural network for differentiation of liver masses at dynamic contrast-enhanced CT: a preliminary study Lung infection segmentation for COVID-19 Computational Intelligence and Neuroscience pneumonia based on a cascade convolutional network from CT images Investigation of effective climatology parameters on COVID-19 outbreak in Iran Lung pattern classification for interstitial lung diseases using a deep convolutional neural network Influence of patient activity on femoral osteolysis at five and ten years following hybrid total hip replacement Bone morphogenetic protein and bone metastasis, implication and therapeutic potential e basic science of periprosthetic osteolysis Studies of the mechanism by which the mechanical failure of polymethylmethacrylate leads to bone resorption A literature review of the association between wear rate and osteolysis in total hip arthroplasty e lysis threshold: an erroneous and perhaps misleading concept e lexicon of polyethylene wear in artificial joints Ballooning osteolysis in 71 failed total ankle arthroplasties Ceramic prosthesis surfaces induce an inflammatory cell response and fibrotic tissue changes Metal hypersensitivity reactions to orthopedic implants e osteolytic response of macrophages to challenge with particles of Simplex P, Endurance, Palacos R, and Vertebroplastic bone cement particlesin vitro Aseptic loosening of a total knee prosthesis caused by delayed hypersensitivity to bone cement Wear particles and ions from cemented and uncemented titanium-based hip prostheses-A histological and chemical analysis of retrieval material e effect of particle size and electrical charge on macrophage-osteoclast differentiation and bone resorption Effect of Size and Dose on Bone Resorption Activity of Macrophages by in Vitro Clinically Relevant Ultra High Molecular Weight Polyethylene Particles Fluid pressure causes bone resorption in a rabbit model of prosthetic loosening e differences in toxicity and release of bone-resorbing mediators induced by titanium and cobaltchromium-alloy wear particles Polyethylene wear debris and tissue reactions in knee as compared to hip replacement prostheses Clinical and radiographic evaluation of the harris-galante cup e role of acetabular component screw holes and/or screws in the development of pelvic osteolysis A new classification system for the management of acetabular osteolysis after total hip arthroplasty Femoral bone loss in total hip arthroplasty: classification and preoperative planning Acetabular bone loss during revision total hip replacement: preoperative investigation and planning -pubmed Cellular profile and cytokine production at prosthetic interfaces Patterns of osteolysis around total hip components inserted with and without cement Efficient bone metastasis diagnosis in bone scintigraphy using a fast convolutional neural network architecture Magnetic resonance imaging after total hip arthroplasty: eva Bone and joint imaging A primitive study on unsupervised anomaly detection with an autoencoder in emergency head CT volumes Bone metastatic tumor detection based on AnoGAN using CT images On the momentum term in gradient descent learning algorithms Mask-CNN: localizing parts and selecting descriptors for fine-grained image recognition Partbased R-CNNs for fine-grained category detection Classification of bone tumor on CT images using deep convolutional neural network Nondestructive diagnosis and analysis of computed microtomography images via texture descriptors Detection of brain lesion location in MRI images using convolutional neural network and robust PCA LNPSS: SAR image despeckling based on local and non-local features using patch shape selection and edges linking Diagnostic and gradation model of osteoporosis based on improved deep U-net network Cluster prototypes and fuzzy memberships jointly leveraged cross-domain maximum entropy clustering Affinity and penalty jointly constrained spectral clustering with all-compatibility, flexibility, and robustness Multi-level seg-unet model with global and patch-based X-ray images for knee bone tumor detection Investigation of temporal radiographic texture analysis for the detection of periprosthetic osteolysis Computerized radiographic analysis of osteoporosis: Preliminary evaluation Computerized radiographic texture measures for characterizing bone strength: a simulated clinical setup using femoral neck specimens Use of morphometry to quantify osteolysis after total hip arthroplasty NIH Image to ImageJ: 25 years of image analysis Osteolytica: an automated image analysis software package that rapidly measures cancer-induced osteolytic lesions in in vivo models with greater reproducibility compared to other commonly used methods Filling holes in complex surfaces using volumetric diffusion Volumetric CT-based segmentation of NSCLC using 3D-Slicer 3D slicer as an image computing platform for the quantitative imaging network Automating the extraction of 3D models from medical images for virtual reality and haptic simulations Itk: enabling reproducible research and open science International news -diversity of technical papers offered at SAMPE e medical imaging interaction toolkit e medical imaging interaction toolkit: challenges and advances Visualizing with VTK: a tutorial Deep neural network based artificial intelligence assisted diagnosis of bone scintigraphy for cancer bone metastasis Joint segmentation of bones and muscles using an intensity and histogram-based energy 20 Computational Intelligence and Neuroscience minimization approach Bone segmentation on whole-body CT using convolutional neural network with novel data augmentation techniques Deep learning-based classification of primary bone tumors on radiographs: a preliminary study Regenerative semi-supervised bidirectional w-networkbased knee bone tumor classification on radiographs guided by three-region bone segmentation An approach for the detection and classification of tumor cells from bone MRI using wavelet transform and KNN classifier Evaluating the progression of osteolysis after total knee arthroplasty Metal-on-metal total hip arthroplasty: risk factors for pseudotumours and clinical systematic evaluation Are biologic treatments a potential approach to wear-and corrosion-related problems? Effect of alendronate on pseudomembrane cytokine expression in patients with aseptic osteolysis Bisphosphonates in orthopedic applications Mesenchymal stem cellmacrophage crosstalk and bone healing Establishment of NF-κB sensing and interleukin-4 secreting mesenchymal stromal cells as an on-demand drug delivery system to modulate inflammation Preconditioned or IL4-secreting mesenchymal stem cells enhanced osteogenesis at different stages