S t u d i e S i n c o n S e rvat i o n 5 6 ( 2 0 1 1 ) pa g e S 1 0 4 – 1 1 4 104 Received August 2010 On the Utility of Spectral-Maximum- Based Automated Thread Counting from X-Radiographs of Paintings on Canvas c. richard Johnson, Jr., don H. Johnson, naoto Hamashima, Heui Sung Yang and ella Hendriks INTRODUCTION Manual thread counting is possible from the weave pattern visible at sufficient enlargement of the intensity pattern in an X-radiograph of a painting on canvas. a recently introduced algorithm [1] for automated thread counting relies upon the topography of the two- dimensional Fourier transform of the X-radiograph. (For a description of the two-dimensional Fourier transform see the appendix.) a procedure based on a two-dimensional Fourier transform is at the heart of ongoing automated thread counting campaigns among paintings by van gogh, vermeer and the delft School, Matisse, renoir, Monet and gauguin. in combination with knowledge of the artist’s studio practice, such as van gogh’s rapid usage of rolls of pre-primed canvas during his late French period, thread count data can be used to assist in art historical issues such as dating and authentication. this paper presents the results of a number of numer ical exper iments intended to elaborate on This paper establishes that the two-dimensional Fourier transform, spectral-maximum-based extraction of thread density appears suited to automatic thread counting from scanned X-radiographs of paintings for a range of European painters from the seventeenth century to the early twentieth century. With regularly woven canvas, striping occurring in color-coded maps of local thread count can be used to identify rollmate candidates originally separated by as much as a few meters, maybe more. These results suggest that recently developed spectral-maximum-based thread counting algorithms are sufficiently sophisticated to support major efforts in archival thread counting as key forensic data in a variety of art historical investigations. Still, the canvas and priming used by some artists require a more refined approach to automated thread counting than a simple spectral-maximum-based scheme. some of the presumptions in a spectral-maximum- based approach to automated thread counting from X-radiographs of paintings on ‘plain’ weave canvas. the degree of satisfaction of these presumptions is critical to the utility of such an algorithm. For example, reliance on the dominant spike in the two-dimensional Fourier transform is validated by experiments during algorithm development (repeated here) that indicated the spectral- maximum-based scheme produced a result close to the corresponding spot count in a high percentage of cases for paintings by vincent van gogh. is this dominant spike assumption of a spectral-maximum-based scheme valid for other artists and other periods as well? this paper presents an experiment on X-radiographs of paintings on plain weave canvas by van gogh, vermeer, Matisse and rembrandt. it is discovered that the dominant spike presumption, while quite valid for van gogh and satisfactory for other artists such as Matisse and vermeer, is much less often satisfied by the X-radiographs of paintings of other artists, such as rembrandt. When the dominant spike presumption is suitable, the thread counts in overlapping evaluation squares (e.g. 1 cm squares on 0.5 cm centers) can be color-coded spectral-maximum-based automated thread counting: x-radiographs oF paintings on canVas 105 S t u d i e S i n c o n S e rvat i o n 5 6 ( 2 0 1 1 ) pa g e S 1 0 4 – 1 1 4 and provide a map of the weave density [2]. one feature that emerges from these weave maps is striping. a bundle of threads, either hor izontal or vertical, can maintain roughly the same density (in threads cm–1) across an entire painting. therefore, why not on into the originally neighboring canvas? is the weave density pattern retained further (e.g. meters) along the same bundle of threads? a suitably normalized cross-correlation of density profiles is used to provide a measure of similarity [3]. cross-correlations internal to large paintings by Matisse are examined, and it is observed that the level of cross-correlation between X-radiographs with shared threads only gradually decays within 2–3 m of separation. this means that pieces of canvas sharing threads need not be adjacent to show a strong cross-correlation. thus, rollmates can be identified even when canvas between two rollmate paintings is not available. one loss is that a weak connection between separation distance and cross-correlation means that the cross-correlation values cannot be used to quantify, or even rank, precisely the distance of separation. Finally, an experiment is conducted to test the utility of X-radiographs for thread counting relative to direct photographs of the canvas. the presumption is that the grooves between the canvas threads are filled evenly and uniformly with radio-opaque paint so that the X-radiograph is essentially identical, in terms of thread counting, to a photograph of the raw canvas. there are numerous practicalities that challenge this presumption. the reason verso photographs are not used is that the vast majority of old master paintings in museums are lined on their backs with another canvas such that the original canvas is hidden. an additional problem with a photograph of the back of an unlined painting is that the stretcher hides the edges thereby blocking observation of cusping at the canvas edge. For examples from the few unlined paintings by van gogh in the collection of the van gogh Museum, the weave maps generated from the X-radiographs are compared with those from photos of the backs. the weave maps for each painting are strikingly similar. altogether the experiments reported here support the claim of broad, if not complete, applicability for spectral-maximum-based thread counting from scanned X-radiographs. the automated collection and analysis of thread count data expands the information available for supporting advances in a variety of art historical inquiries. SPOT COUNT COMPARISON TO SURROUNDING SWATCH COUNT in the original development of a two-dimensional Fourier-transform-maximum-based scheme relying on the dominant peak in the frequency range of interest corresponding to the thread density, a set of test line segments and the number of crossing threads were recorded for over 900 spots in 21 van gogh paintings. roughly the same number of horizontal and vertical spot counts were taken from the following 21 paintings in the collection of the van gogh Museum: Still Life: French Novels, The White Orchard, Peach Tree in Blossom, Blossoming Peach Tree, Vincent’s Bedroom in Arles, Orchard in Blossom, Wheat Fields with Reaper at Sunrise, Cypresses and Two Women, The Pietà (after Delacroix), The Plough and the Harrow (after Millet), The Sheep-Shearers (after Millet), Les Peiroulets Ravine, Evening: The Watch (after Millet), The Walk: Falling Leaves, The Garden of Saint-Paul Hospital, Reaper with Sickle (after Millet), The Thresher (after Millet), At the Foot of the Mountains, Entrance to a Quarry, Tree Trunks with Ivy and Tree Roots and Trunks. Manual counts were performed independently by two individuals for each spot and archived upon agreement. a 751 × 751 pixel square swatch (which for 600 dpi is approximately 1.25 × 1.25 inches or 3.18 × 3.18 cm) centered on the midpoint of the test line segment for each spot was extracted from the X-radiograph scans. this provided a test bed for algorithm assessment. the spectral-maximum-based algorithm [1] begins by performing a two-dimensional Fourier transform on each evaluation square after high-pass pre-processing. With the X-radiograph displaying waves of vertically and horizontally oriented threads, the tall peaks quanti- fying thread density are to be found near the horizontal and vertical axes in the two-dimensional Four ier transform. the algorithmically generated thread count is then compared to the manual spot count associated with the test line segment. this match is not expected to be precise. after all, the thread density across the test line segment need not be fully representative of the ‘average’ behavior extracted by the two-dimensional Fourier transform across the entire surrounding evaluation square. the original experiments during algorithm develop- ment suggested that this strategy could achieve an accuracy of within 1 thread cm–1 of the associated spot count over 90% of the swatches. However, a number of situations were observed where the tallest peak was just slightly higher than the next tallest. this challenges confidence that the tallest peak is the overwhelming 106 c. r. Johnson, d. h. Johnson, n. hamashima, h. s. Yang and e. hendriKs S t u d i e S i n c o n S e rvat i o n 5 6 ( 2 0 1 1 ) pa g e S 1 0 4 – 1 1 4 dominant component in the periodic thread pattern and can be used to quantify the thread count. therefore, more recently experiments were conducted with a modification of the algorithm to provide a count only when the height of the tallest peak exceeded that of the the next-tallest peak by an amount labeled the dominance factor (δ). the test set was also increased to 1000 swatches by adding swatches from Peasant Woman Cutting Straw (after Millet) from the collection of the van gogh Museum and the modified algorithm was run for all 1000. For each dominance factor, the absolute error was recorded between the manual spot count and the corresponding automatic swatch count for all swatches satisfying that dominance factor. excluding swatches with less dominant peaks reduces the absolute error bound for 90% compliance to less than 1 thread cm–1, as shown in Figure 1. For Figure 1, all swatches passing a particular domin- ance threshold are ordered from smallest to largest in terms of absolute error between the manual spot count and automatic swatch count. to produce a percentage, the index of each term in a dominance-threshold- specific set of errors is divided by the total number of swatches satisfying the specific dominance threshold. For each swatch in the dominance-threshold-specific set of ordered errors, a point is plotted with the x-coordinate as the ordered percentage and the y-coordinate as its absolute error. For each dominance factor, the resulting monotonically non-decreasing curve begins at (0, 0). For a dominance factor of one, the plotted maximum of 1 thread cm–1 occurs for over 90% of the swatches. a maximum absolute error of 0.8 thread cm–1 is met by just over 80% of the swatches. to assure an absolute error of less than 0.8 thread cm–1 for 90% of those satisfying the associated dominance-threshold δ requires a δ greater than 1.5. over 90% compliance with a lower absolute error requires an even higher δ. However, as δ increases, the percentage of satisfactory swatches decreases, as indicated in Figure 2. the percentage of swatches satisfying a dominance factor of 3 or more is less than 40%. DOMINANT SPIKE PRESUMPTION While the X-radiographs of the late nineteenth-century paintings of vincent van gogh appear to provide a high percentage of swatches that satisfy the dominant spike presumption that the location of the dominant spike designates the weave periodicity, this condition need not be met by X-radiographs of the paintings of other artists. testing of the satisfaction of such a presumption by the X-radiographs of paintings on canvas by other artists and from other periods relied on scanned X-radiographs of early twentieth-century paintings by Henri Matisse, and seventeenth-century paintings by Johannes vermeer and rembrandt van rijn. For 400 count comparisons, swatches were taken sur rounding 200 coincident pairs of vertical and horizontal spot counts on four Henri Matisse paintings in the collection of the Museum of Modern art: Bather, Goldfish and Palette, Gourds and The Moroccans. 0 10 20 30 40 50 60 70 80 90 100 0 0.2 0.4 0.6 0.8 1 % A b so lu te T h re a d C o u n t E rr o r (t h /c m ) Figure 1 Spot–swatch count comparison ordered by absolute error among 1000 Van Gogh test swatches passing particular dominance threshold δ: curves left to right for δ = 1, 1.5, 2, 2.5 and 3. 1 1.2 1.4 1.6 1.8 2 2.2 2.4 2.6 2.8 3 0 10 20 30 40 50 60 70 80 90 100 dominance threshold % Figure 2 Van Gogh dominance threshold percentages: passing dominance test (solid), absolute error less than 1 thread cm–1 (dashed). spectral-maximum-based automated thread counting: x-radiographs oF paintings on canVas 107 S t u d i e S i n c o n S e rvat i o n 5 6 ( 2 0 1 1 ) pa g e S 1 0 4 – 1 1 4 the resulting diagram of percentages passing with suffi ciently large peak dominance factor δ and of percentages – among those passing – performing within an absolute error less than 1 thread cm–1 is plotted in Figure 3. in comparison to the plot for 1000 swatches (each surrounding single, either vertical or horizontal, spot counts) from van gogh paintings in Figure 2, the Matisse paintings exhibit a lower percentage of swatches displaying satisfactory performance for the same dominance threshold. the dominance threshold needs to be set to approximately 1.4 or higher to get the per centage of dominance-threshold-satisfying swatches with satisfactory performance (i.e. automatic counts within 1 thread cm–1 of the corresponding spot counts) above 90%. Just over 60% of the swatches surpass a dominance threshold of 1.4. Fewer still satisfy higher thresholds. also, notice that in comparison to the van gogh swatches producing Figure 2, the number of Matisse swatches possessing a dominance factor greater than 3 is much lower. a similar exercise was performed for 200 swatches (each surrounding a pair of spot counts, one for vertical threads and one for horizontal threads) from four vermeer paintings: Diana and Her Companions and Girl with a Pearl Earring from the Mauritshuis and The Milkmaid and The Little Street from the rijksmuseum. the same was done with four rembrandt paintings: Self- Portrait from the Mauritshuis, Man in Oriental Costume from the Metropolitan Museum of art and The Jewish Bride and The Denial of Peter from the rijksmuseum. the results for the vermeer paintings and the rembrandt paintings are shown in Figures 4 and 5, respectively. despite the large differences in the ages of the works, the results for vermeer’s paintings are similar to those for Matisse (i.e. in the nearly 60% of swatches satisfying a dominance threshold of 1.4, just under 90% are within 1 thread cm–1 of the corresponding spot count). the results displayed in Figure 5 for rembrandt’s canvases are considerably worse. a dominance factor approaching 3 is required to get over 90% of the swatches satisfying this dominance factor to have absolute error less than 1 thread cm–1. unfortunately, less than 20% of the original swatches remain in consideration once the dominance factor is above 2.5. thus, the basic spectral-maximum- based procedure is not suited for application with the X-radiographs of these paintings by rembrandt. the X-radiographs of the rembrandt paintings reveal a weave pattern of substantial irregularity both in the number of threads cm–1 and in individual thread thickness. their image quality can be low in places, as well. the suitability of a spectral-maximum-based thread counting strategy to the vermeer paintings counters the feasible speculation that satisfactory canvas regularity might be a feature only of more modern canvases (such as used by van gogh and Matisse). typically, the level of satisfactory performance by those swatches passing the dominance threshold test is different in the two directions. For example, consider a pair of numbers – one for each thread direction – stating the percentage of spot counts matched to within 1 thread cm–1 by the tallest peak (i.e. with a unity dominance threshold) [4]. among the four vermeer 1 1.2 1.4 1.6 1.8 2 2.2 2.4 2.6 2.8 3 0 10 20 30 40 50 60 70 80 90 100 dominance threshold % Figure 3 Matisse dominance threshold percentages: passing dominance test (solid), absolute error less than 1 thread cm–1 (dashed). 1 1.2 1.4 1.6 1.8 2 2.2 2.4 2.6 2.8 3 0 10 20 30 40 50 60 70 80 90 100 dominance threshold % Figure 4 Vermeer dominance threshold percentages: passing dominance test (solid), absolute error less than 1 thread cm–1 (dashed). 108 c. r. Johnson, d. h. Johnson, n. hamashima, h. s. Yang and e. hendriKs S t u d i e S i n c o n S e rvat i o n 5 6 ( 2 0 1 1 ) pa g e S 1 0 4 – 1 1 4 paintings examined, these pairs of percentages listing the higher value first in each pair were approximately 94:50, 77:70, 78:58 and 88:80. these four combined to produce the y-axis intercept in Figure 4 of approximately 74% by the upper dashed curve. in Figures 3 through 5, the dips in the percentage with satisfactory absolute error between the spot and swatch counts as the dominance threshold increases to 2 or greater can be attributed to a small group of swatches with strongly dominant, substantially mislocated peaks. characterizing these errant swatches to permit count avoidance triage or to inform algorithmic resolution is a current topic of interest. a suitability index is proposed for the use of a spectral-maximum-based thread counter. consider the percentage of swatches exceeding a particular dominance threshold and the separate percentage among this dominance-threshold-surpassing group that have less than 1 thread cm–1 error with their corresponding spot counts. the proposed suitability index is the value at which these two percentages match for the same dominance threshold. this value is the y-coordinate of the intersection point of the ‘x’ formed by the solid and dashed curves in Figures 2 through 5. For these figures, the suitability indices (also known as the x-factor) for van gogh, Matisse, vermeer and rembrandt are approximately 92, 84, 82 and 73, respectively. Based on experience to date, an x-factor of 80% is proposed as the cutoff for utilization of the current spectral-maximum- based thread counting software. indeed as the 80% value is approached from above, it becomes likely that, in some density weave maps composed from the current software of paintings from the group in question, one of the two thread directions will appear speckled rather than striped. PROFILE CROSS-CORRELATION TESTING assuming spectral-maximum-based thread counting is suitable for the X-radiograph of interest, the thread counts within overlapping swatches covering the entire X-radiograph can be accumulated with very little human user time. user time is devoted primarily to image cropping to exclude elements in the X-radiograph image from outside the canvas and to taking a few manual spot counts to acquire sufficient information for tuning the user-selected range over which the search for the maximum spectral peak is to be conducted. Labeling the center of each swatch with a color related to its calculated thread density (with one figure for horizontal threads and another for vertical threads) produces weave maps that readily reveal patterns in the density variations. one immediate observation in the weave maps of paintings by van gogh is the appearance of stripes (Figures 8 and 9): horizontally oriented ones in the maps of the density of horizontal threads and vertically oriented in maps of the density of vertical threads. in each of these four-part figures, the left column provides the horizontal and vertical weave maps for a painting (with the black rectangle indicating the location of the inner edge of the stretcher). For each canvas, the observed str ipes display a particular pattern of variation that can be described by averaging thread counts along the stripe direction and assembling a density profile vector for each thread direction. presumably, this profile would be a close match to the profile of its original neighbor on the pre-primed canvas roll. assessing the similarity of two signals is a common engineering task accomplished with computation of their cross-correlation at various relative shifts. at the relative shift that aligns the peaks and valleys of one signal with those of a similarly shaped signal this cross-correlation will take on a large value. thus, the location of a strong peak in a cross-correlation curve drawn as a function of the relative shift between two similar curves indicates the relative shift providing the most agreeable alignment. a strong peak in the cross- correlation of two X-radiograph profiles suggests the two radiographs (and their associated paintings) shared threads on the same canvas bolt [2, 3, 5, 6]. a reasonable question to ask is whether or not the strength of a normalized cross-correlation is related to 1 1.2 1.4 1.6 1.8 2 2.2 2.4 2.6 2.8 3 0 10 20 30 40 50 60 70 80 90 100 dominance threshold % Figure 5 Rembrandt dominance threshold percentages: passing dominance test (solid), absolute error less than 1 thread cm–1 (dashed). spectral-maximum-based automated thread counting: x-radiographs oF paintings on canVas 109 S t u d i e S i n c o n S e rvat i o n 5 6 ( 2 0 1 1 ) pa g e S 1 0 4 – 1 1 4 the distance of separation of two paintings exhibiting a weave match. to test this question we rely on the X-radiographs of large paintings on canvas by Matisse. Such uncut expanses are obviously from a single roll. Fifteen Matisse paintings were examined: Bather, Blue Window, Goldfish and Palette, Gourds, The Moroccans, The Rose Marble Table, Still Life after Jan Davidsz. de Heem’s ‘La Desserte’ and Woman on a High Stool from Museum of Modern art and Apples, Bathers by a River, The Geranium, The Green Sash, Interior at Nice, Lorette with Cup of Coffee and Woman before an Aquarium from the art institute of chicago. these canvases were up to 18 X-radiographs ‘wide’ horizontally and 11 X-radiographs ‘high’ vertically. the standard X-radiograph size used is 42.5 × 30.5 cm. the cross-correlation of profiles of any pair of X-radiographs within a row (column) that shares hori- zontal (vertical) threads is expected to show a prominent peak. if the painting is five X-radiographs wide, with the X-radiographs numbered left to right as one to five, then X-radiographs one and two, as well as X-radiographs two and three, three and four, and four and five are all one X-radiograph apart. Similarly, X-radiographs one and three, two and four, and three and five are all two X-radiographs apart. X-radiographs one and four are three X-radiographs apart, as are X-radiographs two and five. the pair of X-radiographs one and five is the only pair four X-radiographs apart. For each of these pairs it is possible to compute the peak in the normalized cross-correlation and plot these peak values as the y-coordinate and the number of separating X-radiographs as the x-coordinate. Similar values can be obtained for vertically aligned X-radiographs. values from the 15 Matisse paintings for which data were available are plotted in Figure 6. the formula for normalized cross-correlation from [3] compensates for different amounts of overlap between different X-radiographs in the same row. this is needed because X-radiographs are typically not laid down according to a precise grid and have varying degrees of overlap with their immediate neighbors. alignment is even more haphazard when considering X-radiographs from different paintings relative to their potential original arrangement. For the vertical cluster at each separation value, the square marker indicates the average and the triangular markers the standard deviation on either side of this average. admittedly, the distance measure along the x-axis is woefully imprecise due to the typically casual assembly of the X-radiograph films laid out to cover a painting. However, it is apparent that the average values in Figure 6 exhibit a slight decay with increasing Figure 6 Profile correlations versus separation for 15 paintings by Matisse: Squares indicate mean at each separation value; ∇ one standard deviation above mean; Δ one standard deviation below mean. 110 c. r. Johnson, d. h. Johnson, n. hamashima, h. s. Yang and e. hendriKs S t u d i e S i n c o n S e rvat i o n 5 6 ( 2 0 1 1 ) pa g e S 1 0 4 – 1 1 4 distance. However, this decay rate (per separating X-radiograph) is quite small in Figure 6 relative to the variability of the cross-correlation peak values, as measured by their standard deviation. While the cross- correlation distribution exhibited in Figure 6 may not be so helpful in ordering placement of near neighbors, its strength retention over 15 separating X-radiographs allows rollmate candidates to be identified despite unfilled gaps in roll reconstruction of paintings such as those by van gogh that are typically three or less X-radiographs across in either dimension. the histogram of all cross-correlation values for all aligned X-radiograph pairs within all paintings in both directions (i.e. the data in Figure 6) and a similar histogram for unaligned X-radiograph pairs from the same painting over the set of 15 Matisse paintings appear in Figure 7. the top plot indicates that 0.2 would be a normalized cross-correlation threshold with few false negatives (i.e. rejecting pairs that are actually aligned). However, the bottom plot indicates that with a threshold of 0.2 there would be a number of false positives (i.e. passing pairs that are not actually aligned). as shown in [7], the histograms of X-radiographs sharing threads and of those not sharing threads have much less overlap among a group of van gogh paintings tested. the lower tail of the thread-sharing group’s histogram for the van gogh paintings is smaller for higher cross- correlation values than the Matisse paintings, while the cross-correlation value histogram for the group not sharing threads is similar in range and shape to that for the Matisse paintings in the bottom plot in Figure 7. threshold selection is clearly critical to the candidate weave matches that will be identified numerically. Setting the cross-correlation threshold midway between the peaks of the two histograms balances the incidence of false positives and false negatives. Setting the threshold higher, e.g. for this Matisse example to 0.5 or 0.6, reduces the incidence of false positives, under the presumption that false negatives will be made up for by other X-radiograph pairings from the candidate paintings. X-RADIOGRAPH AND UNLINED BACK PHOTO WEAVE MAPS COMPARISON thread counts of lined paintings are typically performed on X-radiographs of the painting because the original canvas weave is usually no longer visible. this approach always raises the question of how accurately the X-radiographs, whose intensity measures the thickness of radio-opaque materials stacked on the canvas, reflect the canvas weave. in addition to the interference of Figure 7 Histograms of profile correlations of X-radiographs in the same Matisse painting: pairs sharing threads – top, pairs not sharing threads – bottom. spectral-maximum-based automated thread counting: x-radiographs oF paintings on canVas 111 S t u d i e S i n c o n S e rvat i o n 5 6 ( 2 0 1 1 ) pa g e S 1 0 4 – 1 1 4 the artist-applied paint, inappropriate X-ray exposures and canvas deterioration also inhibit thread counting. Fortunately, the image does not have to be a pristine replica to permit thread counting. as long as the suitably magnified intensity pattern reflects the periodic weave pattern, counting is feasible. unlined paintings offer an opportunity to determine if X-radiog raphs provide an image sufficient for automatically generating a weave map that depicts the actual weave accurately. For two unlined paintings by van gogh (Portrait of a Woman with Hat and At the Foot of the Mountains) in the collection of the van gogh Museum weave maps were not only drawn from the X-radiographs but also from high resolution photographs of the unlined backs of the canvas not covered by the stretcher. in Figures 8 and 9 the left column presents the weave maps from the X-radiograph for the vertical and hor izontal threads. the black spots are locations where the X-radiograph quality is determined by the software to be of insufficient quality for a reliable count to be obtained automatically, and therefore are blacked out. the striping is indicative of the orientation of the threads (i.e. vertical above horizontal). the black rectangle outlines within the maps in the left columns of Figures 8 and 9 indicate the inside edge of the stretcher. the right column stacks weave maps composed from cropped photographs of the unlined backs of the paintings leaving just the portion inside the stretcher. the similarities of the weave maps from the X-radiographs and those from photos of the unlined backs are quite striking. Figures 8 and 9 provide evidence of the potential for trustworthiness of X-radiograph data in revealing the weave pattern, as well as the utility of the spectral- maximum-based thread counting software on verso photos. CONCLUSIONS each of the experiments described in this paper helps enrich the understanding of the bounds of utility of recently developed spectral-maximum-based thread counting algorithms producing full-painting weave maps useful in identifying rollmate candidates [1–3, 5, 6]. evidence is presented of the accuracy of such automated thread counts collected in collaboration with the van gogh Museum as part of its studio practice project [8] for paintings done in the 1880s by vincent van gogh. it was confirmed that X-radiographs displaying a regular weave – with little in-band interference – of paintings by other artists as well as van gogh satisfy the naive Figure 8 Weave maps of X-radiograph and back photo of Portrait of a Woman with Hat: top row – density of vertically oriented threads; bottom row – density of horizontally oriented threads; left column – X-radiograph; right column – photo of unlined back; black outline in images in left column denotes inside of stretcher which matches full size of images in right column. dominant spike presumption relied upon by the current automated scheme. it was observed that X-radiographs from paintings by yet other artists will require either more time-consuming, less comprehensive semi- automatic procedures or more sophisticated algorithms for fully automated counting of less regular weaves. analyses done in conjunction with the ‘Matisse: radical invention 1913–1917’ exhibition at the art institute of chicago and the Museum of Modern art [9] show that the cross-correlation of X-radiograph-wide weave density profiles can be a strong indicator of shared threads despite a few meters of separation. using both X-radiographs and photographs of unlined canvases, it was possible to mediate potential concerns with thread counting from X-radiographs, which show only an impression of the threads and not the threads themselves. 112 c. r. Johnson, d. h. Johnson, n. hamashima, h. s. Yang and e. hendriKs S t u d i e S i n c o n S e rvat i o n 5 6 ( 2 0 1 1 ) pa g e S 1 0 4 – 1 1 4 ACKNOWLEDGEMENTS the authors are grateful to the following museums for access to the X-radiographs of their paintings used in this study: van gogh Museum, Museum of Modern art, Metropolitan Museum of art, rijksmuseum, royal picture gallery Mauritshuis, and art institute of chicago. in addition, the authors thank the van gogh Museum for the digitized photos of the unlined backs of Portrait of a Woman with Hat and At the Foot of the Mountains with rulers included for scaling. APPENDIX Two-dimensional Fourier transform as a visualization of the calculation of a two-dimensional Fourier transform, consider standing in shallow water with regularly separated plane waves coming at you from a particular direction. Further assume you have an image from directly above the water’s surface that registers the height of the water with a grayscale intensity pattern of tilted, uniform-width stripes. the two-dimensional Fourier transform of this image of waves will produce an image with two narrow tall peaks in its grayscale intensity pattern. the two peaks are a symmetric pair about the origin, i.e. one peak is located at (x, y) and the other at (–x, –y). the line through the origin joining these two peaks is perpendicular to the wavefront. the wave frequency (i.e. thread count per unit distance) is proportional to the distance of either peak to the origin. thus, the location of these peaks can be used to compute the frequency and orientation of the waves in the or iginal image. the scanned X-radiograph of a painting on canvas is a snapshot of an ocean dominated by two sets of waves – threads – approaching you from directions (approximately) 90° apart. the two-dimensional Fourier transform of this ideal image consists of four peaks, a pair for each wave. From this viewpoint, thread counting for horizontal and vertical threads – and finding their angles as well – can be performed simultaneously by computing the two-dimensional Fourier transform and finding peak locations near the positive portions of the x and y axes. if the waves are not in the shape of sinusoids, then the two-dimensional Fourier transform is not simply two narrow peaks. non-sinusodially shaped, but regularly periodic, waves will add a variety of minor peaks to the two-dimensional Fourier transform. the previously solitary peaks will now be the dominant ones among a landscape of peaks. add the slice of a vessel across the waves, i.e. artist’s brushstrokes, and more peaks appear. due to the smoother, i.e. less oscillatory, pattern of the mark in the waves made by the vessel, the added peaks can be tall but will not be in the vicinity of the peaks representing the pattern of the waves. the presence of disturbances in the periodicity of the waves from a highly regular pattern further complicates the computational task. Here, the case of interest is when the wave pattern is sufficiently regular that the location of the tallest peak in its two-dimensional Fourier transform quantifies the wave frequency. REFERENCES 1 Johnson, d.H., Johnson, c.r. Jr., Klein, a.g., Sethares, W.a., Lee H., and Hendriks, e., ‘a thread counting algorithm for art forensics’, in Proceedings of the 13th IEEE DSP Workshop, Marco Island, FL, 4–7 January 2009 (2009) 679–684, doi:10.1109/ dSp.2009.4786009. `Figure 9 Weave maps of X-radiograph and back photo of At the Foot of the Mountains: top row – density of vertically oriented threads; bottom row – density of horizontally oriented threads; left column – X-radiograph; right column – photo of unlined back; black outline in images in left column denotes inside of stretcher which matches full size of images in right column. spectral-maximum-based automated thread counting: x-radiographs oF paintings on canVas 113 S t u d i e S i n c o n S e rvat i o n 5 6 ( 2 0 1 1 ) pa g e S 1 0 4 – 1 1 4 2 Johnson, c.r. Jr., Hendriks, e., noble, p., and Franken, M., ‘advances in computer-assisted canvas examination: thread counting algorithms’, American Institute for Conservation of Historic and Artistic Works Paintings Specialty Group Postprints 21 (2009) 25–33. 3 Johnson, d.H., Sun, L., Johnson, c.r. Jr. and Hendriks, e., ‘Matching canvas weave patterns from processing X-ray images of master paintings’, in Proceedings of the 35th International Conference on Acoustics, Speech and Signal Processing, Dallas, TX, 14–19 March 2010 (2010) 958–961, doi:10.1109/icaSSp. 2010.5495297. 4 Hamashima, n., Triage of X-ray images for automated spectral- maximum-based thread counting, Master of engineering report, cornell university (2010). 5 Hendriks, e., Johnson, d.H. and Johnson, c.r. Jr., ‘interpreting canvas weave matches’, http://people.ece.cornell.edu/johnson/ canvas.pdf (accessed 30 august, 2010). 6 Johnson, d.H., Hendriks, e., geldof, M. and Johnson, c.r. Jr., ‘do weave matches imply canvas roll matches?’, American Institute for Conservation of Historic and Artistic Works Paintings Specialty Group Postprints 22 (2011, in press). 7 Yang, H., Development of Evaluation Tools for Weave Matching Algorithm of Canvas X-Ray Images, Master of engineering report, cornell university (2010). 8 ‘research into van gogh’s studio practice in context’, http:// www3.vangoghmuseum.nl/vgm/index.jsp?page=13321 &lang=en (accessed 30 august, 2010). 9 d’alessandro, S., and elderfield, J., Matisse: Radical Invention, 1913–1917, art institute of chicago (2010). AUTHORS C. RiChaRd Johnson, JR. was born in Macon, georgia, uSa, in 1950. He received a phd in electr ical engineering from Stanford university, california, uSa along with the first phd minor in art History granted by Stanford, in 1977. He is currently the geoffrey S.M. Hedrick Senior professor of engineering and a Stephen H. Weiss presidential fellow at cornell university, ithaca, nY, uSa. at the start of 2007, after 30 years of research on adaptive feedback systems theory and blind equalization in communication receivers, professor Johnson accepted a five-year appointment as an adjunct research Fellow at the van gogh Museum, the netherlands charged with facilitating the interaction of art historians and conservation specialists with algorithm-building signal processors. Address: School of Electrical and Computer Engineering, 390 Rhodes Hall, Cornell University, Ithaca, NY 14853, USA. Email: johnson@ece.cornell.edu don h. Johnson received a phd in electrical engineer- ing from the Massachusetts institute of technology, uSa in 1974. in 1977, he joined the faculty of the electrical and computer engineer ing department at rice university, Houston, texas, uSa where he is currently the J.S. abercrombie professor emeritus. He is a former president of the Signal processing Society, a fellow of the institute of electrical and electronics engineers (ieee), uSa and a recipient of the ieee Millenium Medal and the Signal processing Society’s Meritorious Service award. He co-authored the graduate-level book Array Signal Processing, published in 1993, and holds six patents. His current research interests concern developing signal processing techniques for technical art history studies. Address: Department of Electrical and Computer Engineering, MS 380, Rice University, 6100 Main Street, Houston, TX 77005, USA. Email: dhj@rice.edu naoto hamashima currently works as a system engineer at nS Solutions corporation in Japan. He received a Master of engineering from cornell university, in electrical and computer engineering in 2010. He graduated from trinity college, Hartford, ct, uSa with a BS in electrical engineering in 2009. Address: 4-31-6- 302 Kichijoji-Honcho, Musashino-shi, Tokyo, Japan 180-000. Email: nhamashima@gmail.com or nh245@cornell.edu heui sung Yang was born in Seoul, republic of Korea, in 1987. He received a Bachelor of Science in 2009 and Master of engineering in 2010 both in electrical and computer engineering from cornell university. in october 2010, he began employment as an interface analyst at epic Systems in verona, Wi. Address: 2933 Fish Hatchery Road, #205, Madison, WI 53713, USA. Email: hy96@cornell.edu ella hendRiks has a Ba (university of Manchester,, uK, 1982) and a phd (university of amsterdam, the netherlands 2006) in art history (dissertation: ‘new views on van gogh’s development in antwerp and paris: an integrated art historical and technical study of his paintings in the van gogh Museum’). She was trained as a conservator of easel paintings at the Hamilton Kerr institute, university of cambridge, uK (1987). From 1988 to 1999 she was Head conservator at the Frans Hals Museum in Haarlem and since 1999 has been Head conservator at the van gogh Museum in amsterdam, the netherlands. Address: Van Gogh Museum, Paulus Potterstraat 7, 1071 CX Amsterdam, The Netherlands. Email: Hendriks@vangoghmuseum.nl 114 c. r. Johnson, d. h. Johnson, n. hamashima, h. s. Yang and e. hendriKs S t u d i e S i n c o n S e rvat i o n 5 6 ( 2 0 1 1 ) pa g e S 1 0 4 – 1 1 4 Résumé — Cet article établit que l’ extraction de la densité de fils, basée sur le maximum spectral par transformée de Fourier 2D semble fiable pour s’appliquer aux comptage automatique de fils des peintures utilisant des scans des radiographies des peintures et ca pour les oeuvres d’une serie de peintres européens du XVIIe siècle au début du XXe siècle. Sur une toile tissée régulièrement, l’entrelace s’étant reproduit sur les cartes en codes de couleur du comptage de fils locaux peut être utilisé pour identifier les candidats «rollmate» initialement séparés de quelques mètres, voire plus. Ces résultats suggèrent que les algorithmes récemment développés pour le comptage automatisé de fils basé sur le maximum spectral sont suffisamment sophistiqués pour soutenir des efforts importants de comptage de fils pour des banques de données qui serviront comme archive dés données clé de type médico- légal dans le contexte de l’histoire de l’art. Pourtant, les toiles et les preparations utilisées par certains artistes exigent une approche plus fine qu’un simple comptage. Zusammenfassung — Der Artikel weisst nach, daß die zweidimensionale, sich am spektralen Maximum orientierende Fouriertransform-Extraktion der Fadendichte für eine automatische Fadenzählung an gescannten Röntgenbildern von Gemälden einer Reihe europäischer Maler des 17. bis zum frühen 20. Jahrhundert geeignet zu sein scheint. Bei gleichmäßig gewobenem Maltuch kann das in farbkodierten Karten (maps) auftretende Streifenmuster der lokalen Fadenzahl dazu dienen, Tuchstücke von der gleichen Rolle zu identifizieren, auch wenn sie einige Meter auseinander liegen, vielleicht sogar mehr. Diese Ergebnisse lassen annehmen, daß kürzlich vorgestellte, auf spektralem Maximum gestützte fadenzählende Algorithmen hochentwickelt genug sind, um größere Projekte archivalischen Fadenzählens zu rechtfertigen, die so forensische Eckdaten für eine Vielzahl kunsthistorischer Untersuchungen zu liefern imstande sind. Allerdings verlangen die von einigen Künstlern benutzten Maltücher und Grundierungen einen, gegenüber einem einfachen Schema auf der Basis des spektralen Maximums verfeinerten Ansatz für automatisiertes Fadenzählen.. Resumen — Este artículo establece que la obtención de la densidad de hilos mediante el cálculo bidimensional por transformada de Fourier, basada en el máximo espectral, se presenta como método posible para un cuenteo automático de hilos, a partir de radiografías escaneadas de cuadros de pintores europeos entre el siglo XVII y comienzos del XX. En telas tejidas regularmente ciertas tiras que aparecen en mapas de códigos de color en cuenteos locales de hilos pueden ser usadas para identificar piezas de tela del mismo rollo que pudieran estar separadas por algunos metros e incluso más. Estos resultados sugieren que estos cálculos algorítmicos de cuenteo de hilos son lo suficientemente sofisticados para llevar a cabo esfuerzos mayores en el cuenteo de hilos como datos forenses claves en una amplia variedad de investigaciones en la historia del arte. Adicionalmente hay que considerar que los lienzos e imprimaciones utilizadas por algunos artistas requieren un criterio mucho más refinado de cuenteo automático de hilos que un simple esquema basado en el máximo espectral.