key: cord-0044171-4spwwymm
authors: Ribeiro, Rui; Pilastri, André; Moura, Carla; Rodrigues, Filipe; Rocha, Rita; Morgado, José; Cortez, Paulo
title: Predicting Physical Properties of Woven Fabrics via Automated Machine Learning and Textile Design and Finishing Features
date: 2020-05-06
journal: Artificial Intelligence Applications and Innovations
DOI: 10.1007/978-3-030-49186-4_21
sha: c30153b02bc450d5d3b1d8b8d8b93624065269e4
doc_id: 44171
cord_uid: 4spwwymm

This paper presents a novel Machine Learning (ML) approach to support the creation of woven fabrics. Using data from a textile company, two CRoss-Industry Standard Process for Data Mining (CRISP-DM) iterations were executed, aiming to compare three input feature representation strategies related with fabric design and finishing processes. During the modeling stage of CRISP-DM, an Automated ML (AutoML) procedure was used to select the best regression model among six distinct state-of-the-art ML algorithms. A total of nine textile physical properties were modeled (e.g., abrasion, elasticity, pilling). Overall, the simpler yarn representation strategy obtained better predictive results. Moreover, for eight fabric properties (e.g., elasticity, pilling) the addition of finishing features improved the quality of the predictions. The best ML models obtained low predictive errors (from 2% to 7%) and are potentially valuable for the textile company, since they can be used to reduce the number of production attempts (saving time and costs).

The introduction of the Industry 4.0 concept is transforming diverse industry sectors due to the adoption of Information Technology (IT), such as Internet of Things (IoT), Big Data, Cloud Computing and Artificial Intelligence (AI) [10, 23] . In particular, the Industry 4.0 transformation can enhance the textile industry by improving the production efficiency (e.g., reducing costs) and assisting in the design of woven fabrics.

In this work, we address a textile company that is being transformed by the Industry 4.0. The company produces custom made woven fabrics for diverse clients. Currently, the fabric design is mostly based on the designer experience and intuition, which results in the execution of several trial-and-error production experiments that require resources (production materials, machines, human labour) and time. Each new fabric production attempt also requires laboratory quality tests, to verify if the produced fabric complies with quality standards and the client requirements. If a fabric is not approved, a new design attempt is set, resulting in an additional production time and costs. All these production steps generate data that can be explored by AI tools, namely Data Mining (DM) and Machine Learning (ML), to support the design of new woven fabrics.

In this paper, we report the implementation of a CRoss-Industry Standard Process for DM (CRISP-DM) [24] project for the prediction of the final fabric physical properties, as measured by nine laboratory quality tests (e.g., abrasion, pilling). The goal is to use a ML model as an "oracle", providing estimates of the fabric real physical properties for several input design options, thus aiding the textile design experts and reducing the number of fabric production attempts. To better focus on input feature selection and transformation, we adopt an Automated Machine Learning (AutoML) procedure during the modeling stage of CRISP-DM, allowing to automatically select and tune the hyperparameters of the predictive ML models [19] . In particular, we focus on input variables that can be set during the textile design phase, namely based on fabric design (e.g., composition, amount of finished threads) and finishing (e.g., washing, drying, singeing) features. In total, we executed two major CRISP-DM iterations, in which we explored different input feature engineering strategies. This paper is organized as follows: Sect. 2 introduces the related work; Sect. 3 presents the two CRISP-DM iterations; Sect. 4 details the obtained results; and finally Sect. 5 presents the main conclusions.

The creation of a new woven fabric is composed of several phases (e.g., design, production, testing). In particular, fabric testing has a crucial role in assessment product quality and performance, ensuring regulatory compliance and it provides information about the properties of the fabrics [18] . The overall process of new fabric creation generates large amounts of data, which under the Industry 4.0 concept can be used by AI tools (DM and ML) to extract valuable knowledge [17] . Following the increasing interest in DM, the CRISP-DM was proposed as a standard methodology to support the execution of real DM projects [24] . The methodology involves interactions between business domain and DM/ML experts and several iterations that can include up to six main phases: business understanding, data understanding, data preparation, modeling, evaluation and deployment. Regarding the textile domain, use of DM techniques is more recent, involving mainly classification tasks, such as defect detection and estimating the quality of yarns [26] .

In what concerns a data-driven modeling of textile quality tests, the research is more scarce. Fan and Hunter [12] used a backpropagation Neural Network (NN) with one single hidden layer with 30 inputs based on fibre, yarn, and fabric constructional parameters to predict nine fabric properties (e.g., abrasion, seam slippage). A similar NN model was adopted in [1] to predict the pilling propensity of fabrics. In other study, Support Vector Machines (SVM) with 17 input features related to fiber and yarn were used to predict 8 different rates of pilling [25] . In the same study, backpropagation NN were also proposed to predict other textile properties, such as seam strength and elongation. In a more recent study [11] , a simple multiple regression model was used to estimate the relationship between fabric tear strength and other input variables, such as yarn tensile strength, yarn count and fabric linear density. In previous work [22] , we performed an initial exploration of ML algorithms, such as Random Forests (RF) and Gradient Boosting Machines (GBM), to predict two fabric properties (tear strength in warp and weft directions). While interesting results were achieved, the study explored a very limited set of inputs (e.g., no finishing data was used).

The performance of a ML algorithm is dependent on a correct feature engineering and ML model selection [9] . Most of the mentioned state-of-the-art works adopt a simple and fixed set of input variables (defined a priori) when predicting the textile physical properties. Moreover, the prediction models were obtained by using an empirical trial-and-error process that often requires a substantial effort from the ML expert [15] . In contrast with the related works (e.g., [1, 12, 25] ), we employ in this paper an AutoML that automatically selects the best among several state-of-the-art ML algorithms. The adoption of the AutoML procedure allowed us to better focus on feature engineering, which is a non-trivial task in this domain. For instance, finishing features were considering a future challenge and thus were excluded from the predictive study performed in [12] . Woven fabric feature engineering is a complex task due to two main reasons. First, the design and finishing processes of the fabric creation includes a variable number of input features (e.g., yarns, finishing operations) that can influence the targeted textile physical properties. Second, most of these features are nominal and often present a high cardinality. Since most regression ML algorithms work only work with numeric values, a nominal to numeric transform is needed.

Business Understanding: The creation of a new woven fabric starts with the definition desired characteristics. The fabric developer uses its experience and intuition, taking into account the textile requirements and starts to analyse the most similar fabrics already produced. Then, a several design elements are initially set, such as the type and number of fibers and the pick count. Some of these design elements involve a single value per fabric (e.g., number of picks), while others involve a variable number of choices (e.g., which and how many yarns to use). Next, a physical sample is produced using several materials (e.g., yarns) and machines (e.g., loom). The final production stage includes a variable sequence of finishing operations (e.g., washing, drying). Then, the produced fabric is tested via laboratory instruments, allowing to infer the physical properties and check if it meets the desired characteristics. If the fabric does not comply with the quality standards or client requirements, then the whole fabric creation process is repeated. In practice, several iterations are executed until a quality fabric is achieved, which results in additional production time and costs. The analyzed textile company expressed the need to get a fast and cheap estimate of the true fabric physical properties by adopting a ML approach. The goal is to use the predictive ML models as "oracles", quickly checking some fabric design and finishing alternative choices, thus reducing the number of attempts necessary to produce a woven fabric.

In total, the company identified nine target properties: abrasion, seam slippage (warp and weft directions), elasticity (warp and weft directions), dimensional stability to steam (warp and weft directions), bias distortion and pilling. All these nine properties are measured using numeric values. In this work, each property is measured as a separate regression task.

The data was collected from two main data sources: the company Enterprise Resource Planning (ERP), with fabric production records, and the laboratory testing database, with fabric quality tests performed between February 2012 to March 2019. The ERP data included 90,034 examples with 2,391 features per row. Using a manual analysis and domain expert knowledge, the ERP features were filtered into a total of 805 potentially relevant attributes. The laboratory dataset had 149,388 examples with the results for the nine selected physical tests. To aggregate all data, a Data Warehouse system was implemented, using an Extraction, Transform, Load (ETL) process to merge and preprocess the two data sources. During the ETL, some records were discarded since they had missing features (e.g., no yarns or no composition values).

When analysing the obtained historical data, we identified a small fraction of laboratory database entries (around 1%) that included slight different physical test values for the same fabric. After consulting the laboratory analysts, it become clear that the differences were due to the execution of laboratory tests at different fabric finishing procedures (e.g., before or after drying). Since the laboratory database did not include when such tests were executed, we opted to compute average values, in order to get a single number per fabric and test.

The initial set of input features explored in this CRISP-DM iteration is presented in Table 1 . Figure 1 exemplifies how some of these features are related with the textile fabric. The first 11 rows of the table are related with a fixed set of design attributes that are defined for all fabrics. Each fabric is composed by two main elements warp and weft, each including a variable mixture of yarns, from 1 (minimum) to a maximum of 21 (in our database). Moreover, each yarn has four main characterizing features plus the number of its repetitions in the warp or weft (these features are shown in the last five rows of Table 1 ). Thus, the proper preprocessing of yarn data, to feed the regression models, is a non-trivial issue. In this work, we propose the following yarn representation. For each fabric, we use a sequence that has a maximum of max y yarns for warp and then another sequence of max y yarns for weft. Each sequence is thus composed by the elements <y 1 , ..., y maxy >, where y i denotes the i-th yarn representation data. In this work, we adopt the threshold of max y = 6 yarns per warp and weft. This value allows the representation of 99.7% of the fabrics without any information loss, while using a larger threshold would increase the sparseness of the input space, increasing the complexity of the predictive models. When a fabric does not have 6 yarns, we use a zero padding to fill the "empty" yarn values, which is a popular text preprocessing technique that adds null values (e.g., 0) to nonexistent features. Finally, we explore two yarn alternative representations: A, use of the code (unique value) and its number of repetitions, where y i is set as the tuple (code i , repetitions i ); and B, use of all yarn characterizing elements except the code, where y i is set as (composition i , folds i , count i , repetitions i ). In total, the A representation assumes 35 input variables (11 + 2 × 6 × 2), while the B encoding results in 59 input features (11 + 2 × 6 × 5). Before feeding the ML algorithms, all the numeric inputs were standardized to a zero mean and one standard deviation. As for the nominal variables, several of them contain a high cardinality. For instance, the analyzed database includes 6,265 distinct types of yarns. A popular nominal to numeric transform is the one-hot encoding, which assigns one boolean value per nominal level. However, this transformation would highly increase the input space, resulting in a very sparse representation that would prejudice the learning of the regression models, also enlarging the computational memory and effort. To handle this issue, in this work we transform all nominal attributes with the Inverse Document Frequency (IDF) function:

where n is the total number of examples in the training set and n l is the number of examples that contain the level l in the analyzed attribute [5] . The advantage of this transform is that is encodes a nominal attribute into a single numeric value, with the most frequent levels being set near the zero (but with a larger "space" between them), and the less frequent ones being more close to each other and near a IDF (l) maximum value. Table 2 presents the nine output targets. The last column (Range) shows the admissible range values for each target, as defined by the textile company. All examples outside such range were considered outliers (e.g., uncommon military fabrics) and thus removed from the dataset. Since different quality tests can be assigned to different fabrics (depending on the client requirements), a variable number of examples is presented for each output (column Examples).

The experiments were conducted in a personal computer using two different computational environments: the R statistical tool and its rminer package for data manipulation and evaluation of ML algorithms [8] , and H20 software which implements a AutoML procedure [7] . As previously discussed, during the first iteration of CRISP-DM we explored the issue of yarn representation, thus two main strategies as compared: A and B. During this modeling stage, to find the best ML algorithm we adopt an AutoML procedure. The AutoML was configured to automatically select the regression model and its hyperparameters based on the best Mean Absolute Error (MAE), using a internal 5-fold cross-validation applied over the training data. We adopted the same AutoML configuration executed in [13] . The computational experiments were executed on a desktop computer and each ML algorithm was trained using a maximum running time of 3,600 s. After selecting the best ML algorithm, its best set of hyperparameters are fixed and the ML is retrained with all training data. A total of 6 different regression families are searched by the AutoML tool: RF, Extremely Randomized Trees (XRT), Generalized Linear Models (GLM), GBM, XGBoost (XG) and Stacked Ensemble (SE). RF is an ensemble method that typically combines a large set of tree predictors, such that each tree depends on a random sample of features and training examples [4] . XRT is another tree ensemble that consists of randomizing both attribute and cut-point choices when splitting a tree node [14] . GLM estimates regression models for outcomes following exponential distributions (e.g., Gaussian, Poisson, gamma) [21] . GBM performs an ensemble of weak successive decision trees, sequentially building regression trees for all data features [20] . XG is another popular boosting decision tree algorithm [6] . Finally, the SE combines the predictions of the previous individual ML algorithms by using a second-level ML algorithm [3] . The H20 tool sets RF and XRT with their default hyperparameters, performs a grid search to set the hyperparameters for GLM (1 hyperparameter), GBM (9 hyperparameters) and XG (10 hyperparameters), and uses GLM as the second-level learner for SE.

Evaluation: An external 3-fold cross-validation was executed to evaluate the regression models. Several metrics were selected to measure the quality of the predictions: MAE, Normalized MAE (NMAE), Adjusted R2 (Adj.R 2 ) and classification accuracy for a given tolerance T (Acc@T ). Regarding MAE and NMAE, the lower the values, the better are the predictions. The NMAE measure normalizes the MAE by the range of the output target on the test set, thus it provides a percentage that is easy to interpret and is scale independent. In the case of Adj.R 2 and Acc@T (from 0 to 1), higher values indicate better predictions. The Acc@T value is based on the Regression Error Characteristic (REC) curves and it measures the percentage of correctly classified examples when assuming a fixed absolute error tolerance (T ) [2] . In this work we use T ∈ {5%, 10%, 20%}. We note that the percentage of error tolerance is computed by considering the range of the target values. The first CRISP-DM iteration results are discussed in Sect. 4.

After showing the Sect. 4 results to the textile experts, it was decided to perform a second CRISP-DM iteration to check the utility of finishing features. During a new business understanding phase, it become clear that the finishing process should influence the final fabric properties. The finishing consists of a predefined sequence of operations that are applied to a fabric with the goal to increase the attractiveness or serviceability of the textile product [16] . At the new data understanding and preparation stages, we identified that the company had a total of 61 different types of finishing operations. Moreover, the sequence of finishing operations can be different for each fabric and it can include repetitions of the operations (e.g., several wash and dry cycles). In the analyzed database, the number of executed finishing operations ranged from 1 (minimum) to 39 (maximum), with an average of 6.82. Table 3 presents the top ten most used types of finishing and the respective number of usages (column Examples). In the table, we added the special value "Others" to represent a merge of distinct finishing operations for which there was no description data. Similarly to the yarn encoding strategy, in this work we will assume a sequence with a maximum of max f finishing operations to represent the finishing process: <f 1 , ..., f max f >, where f i denotes the i-th finishing operation. In this work, we set max f = 10 as a reasonable value that represents around 85% of all fabrics without information loss, helping to reduce the number of inputs that are fed into the ML models. To encode each finishing operation (nominal attribute) we adopt the same IDF transform (Eq. 1). In the modeling phase, the best previous input encoding (A) is compared with the new encoding C that merges all A inputs with the finishing features, resulting in 45 (35 + 10) input variables. The evaluation phase was executed similarly to the first CRISP-DM iteration. 

In all experiments performed the AutoML always selected the GBM or SE algorithms. GBM provided the best overall results (lowest NMAE averaged over the external 3 cross-validation iterations), while for some targets and specific folds (e.g., Bias Distortion and third fold experiment), the selected model was SE.

To compare the feature strategy results, we always assume the best algorithm (GBM or SE) per external fold validation, denoting this as the AutoML model. Table 4 summarizes the predictive performance results, in terms of the 3-fold average NMAE values for the best AutoML model that were obtained during the first and second CRISP-DM iterations. For the first iteration, it becomes clear that A is the best yarn representation strategy. It provides the lowest NMAE results for eight of the nine fabric targets and it also obtains the lowest average value, over all output tasks (difference of 0.66% points when compared with B). Moreover, A has the additional advantage of producing less inputs (35 and not 59), leading to predictive models that require less computational memory and fitting effort. Following these results, we adopted the A encoding to represent the yarns. In the second iteration, the usage of fabric finishing features (C) improves the prediction results for eight of the nine targets. Overall, C provides the lowest average NMAE, with a 0.25% point improvement when compared with A. Table 5 complements the results by showing the other predictive measures for A and C (represented in column Str.). In general, when C obtains the lowest MAE error, it also outperforms the A strategy for the other measures (Acc@T and Adj.R 2 ). For demonstrative purposes, Fig. 2 shows the AutoML elasticity (warp) predictions (xaxis) versus the target values for a particular external 3-fold iteration. The plot includes the tolerance ranges of the T = 5% and T = 10%, showing that an interesting percentage of the values are correctly predicted within those ranges (e.g., 43% of accuracy for T = 10%). 

This paper addresses a textile company that is being transformed under the Industry 4.0 concept and that identified the need to reduce the number of production attempts when designing new woven fabrics by using a Machine Learning (ML) approach. To handle this goal, we implemented two iterations of the CRoss-Industry Standard Process for Data Mining (CRISP-DM) methodology. Each iteration focused on a feature engineering task, aiming to check the value of input fabric yarn and finishing feature representations. During the modeling stage of CRISP-DM, an Automated ML (AutoML) was used to select the best among six state-of-the-art ML algorithms. The best results were achieved by an input set of features that includes a fixed sequence with a simple yarn code representation and another fixed sequence with fabric finishing operations (strategy C). Interesting predictive results were achieved for nine targeted fabric properties, with an average NMAE error that ranges from 2% to 7%. The results were shown to the textile company, which considered them valuable to reduce the number of fabric creation attempts, thus having a potential to save the production time and costs. In future work, we intend to apply a similar approach in the prediction of other fabric quality tests, such as residual extension and traction.

Predicting the pilling propensity of fabrics through artificial neural network modeling

Regression error characteristic curves

Stacked regressions

Random forests

On the evaluation of unsupervised outlier detection: measures, datasets, and an empirical study

XGBoost: a scalable tree boosting system

Practical Machine Learning with H2O: Powerful, Scalable Techniques for Deep Learning and AI

Modern Optimization with R

A few useful things to know about machine learning

Industrie 4.0: Hit or hype?

The prediction of tear strength of plain weave fabric using linear regression models

A worsted fabric expert system: Part II: an artificial neural network model for predicting the properties of worsted fabrics

An automated and distributed machine learning framework for telecommunications risk management

Extremely randomized trees

Which method to use? An assessment of data mining methods in environmental data science

Finishing of technical textiles

Data Mining: Concepts and Techniques

Fabric Testing

Scaling tree-based automated machine learning to biomedical big data with a feature set selector

Gradient boosting machines, a tutorial

Generalized linear models

Predicting the tear strength of woven fabrics via automated machine learning: an application of the CRISP-DM methodology

Implementing smart factory of Industrie 4.0: an outlook

CRISP-DM: towards a standard process model for data mining

Prediction of wool knitwear pilling propensity using support vector machines

Data mining and machine learning in textile industry

Acknowledgments. This work was carried out within the project "TexBoost: less Commodities more Specialities" reference POCI-01-0247-FEDER-024523, co-funded by Fundo Europeu de Desenvolvimento Regional (FEDER), through Portugal 2020 (P2020).