OP-LLCJ170045 1..14 ++ Computer stylometry of C. S. Lewis’s The Dark Tower and related texts ............................................................................................................................................................ 5 Michael P. Oakes RIILP, University of Wolverhampton, England ....................................................................................................................................... Abstract This article looks at the provenance of the unfinished novel The Dark Tower, 10 generally attributed to C. S. Lewis. The manuscript was purportedly rescued from a bonfire shortly after Lewis’s death by his literary executor Walter Hooper, but the quality of the text is hardly vintage Lewis. Using computer stylometric pro- grams made available by Eder et al.’s (2016: Stylometry with R: A package for computational text analysis. R Journal, 8(1): 107–21) ‘stylo’ package and a word 15 length analysis, samples of each chapter of The Dark Tower were compared with works known to be by Lewis, two books by Hooper and a hoax letter concerning the bonfire by Anthony Marchington. Initial experiments found that the first six chapters of The Dark Tower were stylometrically consistent with Lewis’s known works, but the incomplete Chapter 7 was not. This may have been due to an 20 abrupt change in genre, from narrative to pseudoscientific style. Using principal components analysis, it was found that the first and subsequent components were able to separate genre and individual style, and thus a plot of the second against the third principal components enabled the effects of genre to be filtered out. This showed that Chapter 7 was also consistent with the other samples of C. S. 25 Lewis’s writing. ................................................................................................................................................................................. 1 Introduction Clive Staples Lewis (1898–1963) was a prolific writer, and his best-loved fiction is probably his 30 Deep Space trilogy, The Screwtape Letters, and his ‘Narnia’ series of children’s books. Shortly after Lewis’ death, Walter Hooper, the literary executor for the Lewis Estate, claimed to have found an un- published fragment of fiction, which was published 35 much later (1977) as The Dark Tower. There is some overlap between The Dark Tower and the Deep Space trilogy, as they share a number of characters such as McPhee, Ransom, and even Lewis himself. For many years, C. S. Lewis had lived with his 40 brother Warren at a house called the Kilns, in Oxford. In the first paragraph of the preface to the version of The Dark Tower published by Fount, Hooper claimed that Warren wanted to dispose of his late brother’s old papers, and ordered the gar- 45dener to light a bonfire of them which ‘burned steadily for three days’. In Hooper’s own words: ‘Happily, however, the Lewis’s gardener, Fred Paxford, knew that I had the highest regard for anything in the master’s hand, and when he 50was given a great quantity of CS Lewis’s note- books and papers to lay on the flames, he urged the Major [Warren Lewis] to delay till I should have a chance to see them. One of the rescued notebooks contained the hand-written 55manuscript of The Dark Tower’. (Hooper, 1977, p. vii) Correspondence: Michael P. Oakes, University of Wolverhampton, RIILP, Stafford Street, MC Building, Wolverhampton WV1 1LY, United Kingdom E-mail: Michael.Oakes@wlv.ac.uk Digital Scholarship in the Humanities � The Author 2017. Published by Oxford University Press on behalf of EADH. All rights reserved. For Permissions, please email: journals.permissions@oup.com 1 of 14 doi:10.1093/llc/fqx043 Deleted Text: Deleted Text: Deleted Text: `` Deleted Text: '' Deleted Text: `` Deleted Text: ''. Deleted Text: , Deleted Text: `` Deleted Text: '' Deleted Text: '' Deleted Text: : Deleted Text: . The Dark Tower was eventually published with a number of C. S. Lewis short stories, all of which had been published before, except for the very brief The Man Born Blind which had been found in a note- 5 book given to Walter Hooper by Lewis’s brother. The story of the bonfire was later denied by Fred Paxford, and this denial was published in the journal Christianity and Literature (Lindskoog, 1978). Shortly afterwards, Christianity and Literature (1979 (28): 12– 10 13) also published a letter from Anthony Marchington, seemingly in support of Paxford’s denial, as it stated that a chemical analysis of the soil in Lewis’ garden had re- vealed that no major bonfire had been lit there. This letter is thought to be a hoax: its content is clearly 15 pseudoscientific, and Marchington was a close friend of Walter Hooper, at one time sharing lodgings with him. The Dark Tower itself is unfinished, possibly be- cause the plot hits something of a dead end. Opinions vary as to the quality of the writing, and the story 20 changes tack abruptly in the final chapter, where the protagonist Scudamour is left alone in a library in ‘Othertime’ to learn about the ‘Othertimers’ discoveries about time travel. Hooper (1977, p. viii) estimates that Lewis began writing The Dark Tower soon after com- 25 pleting Out of the Silent Planet in 1938. There are simi- larities with Madeleine L’Engle’s A Wrinkle in Time, although this was not written until 1962. All this has led a number of people, most notably Katherine Lindskoog, to conclude that The Dark Tower may not 30 be entirely written by C. S. Lewis. The most likely can- didates for writing at least parts of The Dark Tower, apart from Lewis himself, would be Walter Hooper and Anthony Marchington. Lindskoog (1988, pp. 53– 54) mainly suspects Marchington: 35 ‘No one thinks that Walter Hooper could have tackled all that ficto-science. The most obvious suspect is Anthony Marchington himself. He is a scientist, he is interested in the origin of The Dark Tower, and he has tricked Christianity 40 and Literature with a scientific spoof. Furthermore, he was about eight years old when Madeleine L’Engle published her chil- dren’s classic A Wrinkle in Time, and so he quite possibly read it as a child. That could 45 account for unconscious copying of Engle’s automaton scene in The Dark Tower’. The corresponding ‘automaton scene’ in The Dark Tower occurs in Chapter 2. 2 Previous Work 50In the past, a number of computer stylometric ana- lyses have been performed on The Dark Tower and related texts. The first of these was by Carla Faust Jones (1989), who used a computer program written by Jim Tankard which he had previously used 55to study the Federalist Papers (Tankard, 1986). First the program finds the frequencies of character n-grams (sequences of n consecutive characters, where n was 1 or 2) in the text, then normalizes these to frequencies per 1,000 characters, rounded 60to the nearest whole number. Spaces and punctu- ation were not considered, and upper and lower case characters were considered equivalent. For 1- gram (single characters), the index of difference be- tween two text samples was given by the expression: Xz a jfA � fBj: 65where fA is the frequency of a character in the first text sample, and fB is the frequency of that character in the second. The differences in these frequencies are found for every character in the alphabet, and 70then all added together. For the 2-grams, the expres- sion is analogous: we find the differences in the frequencies of every possible character pair in the two texts, and then add together all 26 � 26 differ- ences. Jones’ (1989) results are shown in Table 1. 75Both the 1-gram and 2-gram analyses show that the three complete science fiction novels by Lewis, Out of the Silent Planet, Perelandra, and That Hideous Strength are more similar to each other than they are to The Dark Tower. Although this is 80interesting, it does not prove that The Dark Tower was not written by Lewis. There is no comparison with Lewis’s other works nor any comparison with works by other candidate authors for The Dark Tower. 85Lindskoog (1994, pp. 247–48) describes a seem- ingly unpublished report by Andrew Queen Morton. He used a data visualization technique called a Cumulative Sum Control Chart (CUSUM) analysis, M. P. Oakes 2 of 14 Digital Scholarship in the Humanities, 2017 Deleted Text: - Deleted Text: `` Deleted Text: '' Deleted Text: `` Deleted Text: Othertimers' Deleted Text: '' Deleted Text: : Deleted Text: : Deleted Text: - Deleted Text: `` Deleted Text: '' Deleted Text: `` Deleted Text: '' Deleted Text: normalises Deleted Text: s Deleted Text: x Deleted Text: , Deleted Text: : Deleted Text: - Deleted Text: visualisation which has been used to detect changes in the quality of production line outputs in an industrial setting. Morton himself suggested that this technique could be used to detect discontinuities in writing style, 5 such as when one author breaks off and another begins in a multiple-authored text. A linguistic fea- ture such as word length or noun frequency is used to characterize the texts. The resulting graph shows an upward trend for those portions of the text which 10 show an above average (taken over the text as a whole) occurrence of the chosen feature, and a down- ward trend for those parts which show a below aver- age occurrence. Thus, if two authors who have contributed to a text show different rates of usage of 15 the chosen feature, the point where one writer hands over to another might show an abrupt change in the direction (upwards or downwards) of the graph. Morton took the first twenty-three sentences of Chapter 1 of The Dark Tower, the first twenty-four 20 sentences of Chapter 4, and the first twenty-five sen- tences of Chapter 7, alongside sections from Out of the Silent Planet and That Hideous Strength. Morton concluded that The Dark Tower was a composite work: Lewis did not write Chapters 1 and 4, but he 25 did write Chapter 7, the one with the library scene. The technique is highly controversial in studies of disputed authorship, but my feeling is that the choice of linguistic features may affect the success of the technique itself. For example, Merriam 30 (2000) achieved interesting results for the Shakespeare play Edward III with CUSUM charts using the frequencies of prosodic features, rare words, and function words, combined into a single chart using principal components analysis (PCA). 35 Unfortunately Lindskoog gives no details of which linguistic features Morton used to characterize the texts. Morton’s study also suffers from the brevity of the texts which were analysed. More recently, Thompson and Rasp (2009) used 40statistical techniques developed by Thisted and Efron (1987) for comparing smaller samples of un- known authorship (such as a newly discovered text) with a much larger canon with known authorship. If we define t as the size in words of the small sample 45divided by the size in words of the larger canon, n1 as the number of words occurring exactly once in the canon, n2 the number of words occurring twice, and so on, then in their ‘new words’ test, we can estimate bv0 , the number of words in the smaller text 50that do not appear in the larger canon, as follows: bv0 ¼ n1t � n2t 2 þ n3t 3 . . . : This formula depends on t being small, to ensure that the series converges. We want to see how close 55the estimated value of bv0 is to m0, which is the number of ‘new’ words actually found in the small sample but not in the canon. If these values differ greatly, it suggests that the small sample was not written by the author of the canon. 60They performed three other tests using related formulae—the ‘rare words’ test, where the estimated and true numbers of words occurring below an ar- bitrary threshold number of times are compared, and the ‘slope’ and ‘uniformity’ tests, which take 65into account the estimated and real numbers of words of every individual frequency up to a thresh- old. The tests were validated first by comparing sam- ples of George MacDonald’s writings with those known to be by Lewis. The ‘new words’ test was 70most successful, being able to discriminate between them 25% of the time with 95% confidence—we Table 1 Indexes of difference between The Dark Tower and C. S. Lewis’s three complete science fiction novels, found by Jones (1989) Comparison Texts compared I. D. (unigrams) I. D. (bigrams) A1 Silent Planet and Perelandra 76 1,778 A2 Silent Planet and Hideous Strength 60 1,890 A3 Perelandra and Hideous Strength 74 1,834 B1 Silent Planet and Dark Tower 113 2,427 B2 Perelandra and Dark Tower 83 2,137 B3 Hideous Strength and Dark Tower 91 2,327 I.D.¼ Index of Difference Computer stylometry of The Dark Tower Digital Scholarship in the Humanities, 2017 3 of 14 Deleted Text: characterise Deleted Text: 23 Deleted Text: one Deleted Text: 24 Deleted Text: four Deleted Text: 25 Deleted Text: seven Deleted Text: one Deleted Text: four Deleted Text: seven Deleted Text: characterise Deleted Text: `` Deleted Text: '' Deleted Text: `` Deleted Text: '' Deleted Text: -- Deleted Text: `` Deleted Text: '' Deleted Text: ; Deleted Text: `` Deleted Text: '' Deleted Text: `` Deleted Text: '' Deleted Text: `` Deleted Text: '' Deleted Text: -- would expect only 5% if we were looking at a single author. The ‘new words’ test also showed the best discriminatory power between short samples of The Dark Tower and two of Lewis’ science fiction novels 5 (those thought to have been written closest in time to The Dark Tower), namely, Out of the Silent Planet and Perelandra. The test found that 29% of The Dark Tower samples were significantly different to the ‘canon’ of two science fiction novels. Overall, 10 Thompson and Rasp felt that their results were in- conclusive. Even though the ‘new words’ test did discriminate between samples of The Dark Tower and the complete novels, this may not have been due to a difference in authorship, but because a 15 novel in draft form might differ from a complete, polished work. 3 Stylometry with R: The ‘Stylo’ Package Before describing the specific experiments carried 20 out for this article, I will describe some general fea- tures of the package that were used, ‘Stylometry with R’ (stylo), which was written in the R statistical programming language by Eder et al. (2016). Stylo enables a choice of measures of document dissimi- 25 larity, and I used the classic Burrows’ Delta, first described by Burrows (2002), throughout. Stylo also allows a variety of linguistic features to be used to characterize the texts, these being word and character overlapping n-grams, where n can 30 be any number, including 1 for single words or characters. An n-gram is a sequence of n tokens. For example, if n is 2, and we are interested in over- lapping character sequences, a word like ‘Lewis’ would be analysed into the four entities ‘Le’, ‘ew’, 35 ‘wi’, and ‘is’. Finally, stylo enables a number of kinds of graphical displays, each of which is a way of showing which documents are most similar to each other, by placing them close together on the page. For example, Fig. 1 is an example showing the 40 outputs for hierarchical agglomerative clustering. The relationships between the texts are shown on dendrograms, so called because they look like trees on their side. The branches on the extreme right each correspond to individual texts, and texts on 45nearby branches are similar to each other. The tech- nique for building a dendrogram is to first find the most similar pair of texts and join them together, so that thereafter they can be considered as a joint entity. In the subsequent series of steps, each time 50the most similar pair of single texts or joint entities is fused to form a larger group. This process con- tinues until all the texts are joined in a single struc- ture. When using Ward’s (1963) method, the default linkage method offered by stylo, the docu- 55ment similarities between a newly formed joint entity and all the other text groups formed so far are functions of the distances between each of the two constituents before fusion and the rest of the text groupings, and the number of texts in each 60entity. A series of dendrograms obtained for Fig. 1 Dendrogram of texts by Lewis. L’Engle and Tolkein, using the 100 most frequent single words M. P. Oakes 4 of 14 Digital Scholarship in the Humanities, 2017 Deleted Text: `` Deleted Text: '' Deleted Text: `` Deleted Text: '' Deleted Text: `` Deleted Text: '' Deleted Text: t Deleted Text: paper Deleted Text: `` Deleted Text: '' Deleted Text: characterise Deleted Text: `` Deleted Text: '' Deleted Text: `` Deleted Text: '', Deleted Text: `` Deleted Text: '', Deleted Text: `` Deleted Text: '' Deleted Text: `` Deleted Text: ''. Deleted Text: (HAC) Deleted Text: , Deleted Text: is a different numbers of linguistic features can be fused into a bootstrap consensus tree, such as that shown in Fig. 2. Branches between texts are shown when- ever such a branch was found in a selected propor- 5 tion of the dendrograms—I used the default value of 0.5 throughout. A third type of representation, called PCA, can be seen for example in Fig. 3. The technique aims to find groups of texts which are characterized by the common presence or ab- 10 sence of certain groups of linguistic features, which form a component. Texts with many of these fea- tures score highly on the component, while other texts with few of them have negative scores on this component. The component which accounts for the 15 greatest amount of variability between the texts is called the first principal component (PC1), but there are other components which successively ac- count for less variability between the texts. Normally the texts are plotted according to their positions on 20 the first two components (PC1 and PC2), but as we shall see in this article, if PC1 corresponds to genre rather than author, genre effects can sometimes be overcome by plotting the texts according to their scores on lower components (such as PC2 and 25PC3). PCA is often used to examine variation in language. For example, Holmes et al. (2001) used PCA to examine authorship of the ‘Pickett letters’ from the American Civil War, Binongo and Smith (1999) used PCA to study the authorship of the play 30Pericles, and Harris (2010) looked at possible genres in the corpus of Rongorongo from the Easter Islands. Biber (1988) used the closely related tech- nique of factor analysis to study functional linguistic variation arising from genre and register. Stylo 35allows a culling parameter to be set. For example, if this value is 20, then only features appearing in at least 20% of the texts will be considered in the ana- lysis. In all the experiments described in this article, the ‘culling’ parameter was set to 0; so for example if 40we are studying the frequencies of the top 100 words, the frequency of every one of these words will be considered. The 100 most frequent words (MFW) are the 100 MFW in the entire corpus, Fig. 2 Dendrogram of texts by Lewis. L’Engle and Tolkein, using the 100 most frequent single words Fig. 3 PCA of texts by Lewis. L’Engle and Tolkein, using the 100 most frequent single words Computer stylometry of The Dark Tower Digital Scholarship in the Humanities, 2017 5 of 14 Deleted Text: -- Deleted Text: principal components analysis Deleted Text: characterised Deleted Text: paper Deleted Text: `` Deleted Text: '' Deleted Text: paper Deleted Text: `` Deleted Text: '' Deleted Text: , Deleted Text: most frequent word Deleted Text: s rather than the 100 MFW in an individual sample. It is possible to use text samples of different sizes be- cause the word frequencies are normalized. Throughout the experiments the following text 5 pre-processing steps were adhered to. By selecting the ‘English’ button on the ‘Input and Language’ page of the stylo graphical user interface (GUI), contractions such as ‘don’t’ will be treated as the two single words ‘don’ and ‘t’. Hyphenated com- 10 pound words such as ‘topsy-turvy’ also become two single words, here ‘topsy’ and ‘turvy’ (Eder et al., 2015, p. 11). The ‘Preserve Case’ button was not selected, so all upper case characters were converted to lower case. I did not select the option to delete 15 pronouns, and no stop list was used, but did select the option to read in text as plain text files. By de- fault, all sequences of non-alphabetic characters were reduced to a single white space for n-grams longer than 1. Single words were treated as single 20 letters separated by spaces. It is possible to examine the full feature set with the R command stylo.re- sults¼stylo(), then running the GUI to select the desired feature set, and then examining the set with stylo.results$features (Eder et al., 2016, 25 p. 112). 4 Text Samples The set of text samples used in these experiments is summarized in Table 2. The four texts from Lord of the Rings are the Prologue, and the first chapter of 30each of three parts (called individually The Fellowship of the Ring, The Two Towers, and The Return of the King). The texts from the The Hobbit are Chapters 1–4, and the texts from the Narnia series are the first two chapters of The Lion, The 35Witch and the Wardrobe, the first chapter of The Voyage of the Dawn Treader, the first chapter of The Magician’s Nephew, and the first chapter of The Last Battle. The four samples of That Hideous Strength are the first four chapters, as is the case for 40Perelandra. However, the four samples of Out of the Silent Planet consist of the first two chapters; the third and fourth chapters; the fifth and sixth chap- ters; and the seventh and eighth chapter. The two shortest texts, The Man Born Blind and the 45Marchington letter, were used in their entirety, as was the Lefay Fragment. The seven samples of The Dark Tower consist of one chapter each, including the seventh and final (but unfinished) chapter. The four samples of Through Joy and Beyond consist of Table 2 Text samples used in the experiments described in this article Samples Author Year Title Sample length (words each) LOR0, LOR1, LOR2, LOR3 J. R. R. Tolkein 1954–55 Lord of the Rings 7,381, 9,820, 3,375, 13,039 HOB1, HOB2, HOB3, HOB4 J. R. R. Tolkein 1937 The Hobbit 8,652, 5,234, 2,874, 4,066 ENG1, ENG2, ENG3, ENG4 Madeleine L’Engle 1962 A Wrinkle in Time 4,652, 3,628, 3,819, 4,198 LWW, DAWN, MN, LB C. S. Lewis 1950–56 ‘Narnia’ series 3,869, 3,237, 3,035, 2,648 THS1, THS2, THS3, THS4 C. S. Lewis 1945 That Hideous Strength 9,069, 7,468, 8,941, 8,457 PER1, PER2, PER3, PER4 C. S. Lewis 1943 Perelandra 4,980, 4,208, 5,329, 5,508 OSP1, OSP2, OSP3, OSP4 C. S. Lewis 1938 Out of the Silent Planet 5,497, 3,635, 3,761, 3,794 MBB C. S. Lewis Unknown The Man Born Blind 1,769 LEFAY C. S. Lewis Unknown The ‘Lefay’ fragment 5,437 DT1, DT2, DT3, DT4, DT5, DT6, DT7 C. S. Lewis Unknown The Dark Tower 3,010, 4,190, 4,879, 5,645, 3,504, 3,580, 3,691 TJB1, TJB2, TJB3, TJB4 Walter Hooper 1982 Through Joy and Beyond 4,532, 6,676, 3,918, 5,400 PWD1_2, PWD3_4, PWD5, PWD6 Walter Hooper 1971 Past Watchful Dragons 4,574, 3,991, 2,299, 4,689 MLET Tony Marchington 1979 Letter to ‘Christianity and Literature’ 986 MC1, MC2 C. S. Lewis 1942–44 Mere Christianity 8,229, 9,192 PP1, PP2 C. S. Lewis 1940 The Problem of Pain 3,677, 3,001 M. P. Oakes 6 of 14 Digital Scholarship in the Humanities, 2017 Deleted Text: most frequent words Deleted Text: , Deleted Text: normalised Deleted Text: `` Deleted Text: '' Deleted Text: `` Deleted Text: '' Deleted Text: `` Deleted Text: '' Deleted Text: `` Deleted Text: '' Deleted Text: `` Deleted Text: ''. Deleted Text: `` Deleted Text: '' Deleted Text: `` Deleted Text: '' Deleted Text: `` Deleted Text: '' Deleted Text: : Deleted Text: `` Deleted Text: '' Deleted Text: : Deleted Text: are Deleted Text: summarised Deleted Text: to one part each of that book—and thus comprise the entirety of that book. The four samples of Past Watchful Dragons consist of: the first two chapters; the third and fourth chapters; the fifth chapter, 5 excluding the Lefay Fragment; and the sixth chapter. The two samples of Mere Christianity are Books 1 and 2 (Right and Wrong as a Clue to the Meaning of the Universe and What Christians Believe). Finally, the two samples of The Problem of Pain are Chapters 10 1 and 2 of that book. The Lewis texts are compared against the Tolkein texts because the two authors were close friends who regularly discussed their work at meetings of the lit- erary group called ‘The Inklings’, which met at the 15 ‘Eagle and Child’ pub in Oxford. They both wrote about other worlds, such as Middle Earth (Tolkein) and Narnia (Lewis). Like Lewis and Tolkein, Madeleine L’Engle also wrote children’s fantasy novel with a Christian theme, where children are 20 transported to faraway planets. As stated in the introduction, Lindskoog has noticed similarities be- tween The Dark Tower and A Wrinkle in Time. The Lefay Fragment is a long fragment of a draft of the sixth Narnia book, The Magician’s Nephew, also 25 found by Hooper in one of Lewis’s notebooks, and reproduced in Past Watchful Dragons (Hooper, 1971, pp. 48–65). Out of the Silent Planet, Perelandra, and That Hideous Strength are Lewis’s three science fic- tion works for adults. As described above, Walter 30 Hooper claimed to have discovered the short story A Man Born Blind and the unfinished novel The Dark Tower in notebooks, written in Lewis’s hand- writing, after Lewis’s death. However, the handwrit- ing in the Dark Tower notebook has never been 35 satisfactorily authenticated (Lindskoog, 1999). The two selected works by Walter Hooper himself are Past Watchful Dragons, a guide to the Narnia books, and Through Joy and Beyond, a biography of C. S. Lewis. A further sample used is the full 40 text of Tony Marchington’s hoax letter to Christianity and Literature. To the author’s best knowledge, Tony Marchington left no other pub- lished works, and thus it was not possible to use a larger sample of Marchington’s writing in these ex- 45 periments. Mere Christianity and The Problem of Pain are examples of Lewis’s non-fiction writing. 5 Experiments 1: Discrimination between Lewis and Two Other Authors of Fiction 50The first set of experiments, the baseline, was de- signed to show whether the multivariate statistical techniques available in the stylo package were able to distinguish between the three authors L’Engle, Tolkien, and Lewis. The results for the hierarchical 55clustering (Ward’s method) using the 100 MFW as linguistic features are shown in Fig. 1. The choice of 100 words is made following the recommendations of Burrows (2002) and Juola (2015, p. i108), as the 100 MFW are typically function words, giving in- 60formation about grammar and individual writing style rather than content. Here we see four main clusters, which from top to bottom correspond to (1) Tolkein, with the ex- ception of Lewis’s Voyage of the Dawn Treader; (2) 65Lewis’s first two books from the Deep Space trilogy; (3) children’s books written by L’Engle and Lewis, except for the second section of Perelandra; and (4) Lewis’s last book from the Deep Space trilogy. The same pattern is seen more clearly in the bootstrap 70consensus tree (also for the 100 most frequent single words), as shown in Fig. 2, where the two ‘Deep Space’ branches are placed closer together, effect- ively leaving three main clusters in the diagram. In Fig. 3, the data for 100 most frequent single 75words are displayed using PCA. Once again we see three main groupings, with samples by Lewis seen in the top right of the diagram, samples of children’s books in the middle left part, and samples by Tolkein in the bottom right section. 80Juola (2015) recommends running a series of in- dependent analyses in stylometric work. While a series of runs using the same feature set with different clustering algorithms (such as shown in Figs 1–3) are not independent of each other, experi- 85ments using distinct feature sets would be. In his experiment on the writing of J. K. Rowling, Juola states that ‘Tests were run on four separate feature sets: word lengths, character 4-grams, word pairs, and the 100 most frequent words’ (Juola, 2015, 90p. i108). Juola (personal communication) recom- mends using all character 4-grams and word pairs, Computer stylometry of The Dark Tower Digital Scholarship in the Humanities, 2017 7 of 14 Deleted Text: -- Deleted Text: one Deleted Text: two Deleted Text: `` Deleted Text: '', Deleted Text: `` Deleted Text: '' Deleted Text: : Deleted Text: - Deleted Text: were Deleted Text: most frequent words ( Deleted Text: ) Deleted Text: : Deleted Text: most frequent words Deleted Text: a Deleted Text: b Deleted Text: c Deleted Text: d Deleted Text: `` Deleted Text: '' Deleted Text: is Deleted Text: principal components analysis Deleted Text: are Deleted Text: `` Deleted Text: '' Deleted Text: : not just the top n. To achieve this as far as possible, I set n to the very high value of 5,000. Although these linguistic features are not completely inde- pendent of each other (for example, if a word has 5 high frequency, this will raise the frequencies of its constituent character n-grams), I endeavoured to follow his approach. The groupings produced by the hierarchical clustering when using either the 5,000 MFW 2-grams (see Fig. 4) or the 5,000 most 10 frequent character 4-grams (shown in Fig. 5) were the same as each other, producing somewhat clearer separation between the authors than was the case for the 100 most frequent single words. In Figs 4 and 5, we again see a cluster for chil- 15 dren’s authors, but this time we see more separation between those in Lewis’s Narnia series and those by Madeleine L’Engle, than we saw in Figs 1–3. The middle cluster consists entirely of Tolkein sam- ples, and the bottom cluster contains all the books in 20Lewis’s Deep Space trilogy. Thus it seems that it is possible to some extent to distinguish between the three authors of fiction, but the situation is partly confused because we are seeing both the effects of authorship and of genre. As a result we have two 25clusters for Lewis, one for his adult fiction, and an- other for his children’s fiction, which is only mar- ginally distinguished from another author (L’Engle) who also wrote in the children’s fiction genre. To separate authorship and genre, it is possible to use 30the technique of PCA. For example, Schöch (2013) used PCA to examine French plays by the brothers Pierre and Thomas Corneille. The PC1 separated the plays by author, but the second component sepa- rated them by genre: tragedy or comedy. An example 35of a feature which distinguished the plays by genre was the word ‘mort’ (death) which was much more prevalent in tragedies than comedies. One of the Fig. 4 Dendrogram of texts by Lewis. L’Engle and Tolkein, using the 5,000 MFW 2-grams Fig. 5 Dendrogram of texts by Lewis. L’Engle and Tolkein, using the 5,000 most frequent character 4-grams M. P. Oakes 8 of 14 Digital Scholarship in the Humanities, 2017 Deleted Text: most frequent word Deleted Text: principal component analysis ( Deleted Text: ) Deleted Text: first principal component features discriminating between the two authors was the function word ‘ces’ (these). Using the related technique of correspondence analysis, Linmans (1998) showed that samples taken from the 5 Synoptic Gospels were separated on the first compo- nent according to genre (discourse, aphorisms, nar- rative, or parable), and on the second component according to author (Mathew, Mark, or Luke). I ran a PCA on the most frequent 5,000 character 10 4-gram data, and achieved the plot shown in Fig. 6. As in all the previous experiments, we see three main groupings in the text samples. This time the Tolkein samples all appear in the top half of the plot, the children’s writing appears in bottom left 15 part, and the Deep Space samples by Lewis appear in the bottom right part. Thus Lewis’s texts still appear in two separate clusters—one for his children’s writing, and one for his adult science fiction. At the most coarse grained division of texts, we see writing 20 for children in the left half of the diagram, corres- ponding to negative scores on PC1, and writing for adults in the right division, corresponding to posi- tive scores on PC1. Although Lord of the Rings was not specifically written for children, it was written as a 25sequel to The Hobbit, which was. Thus we see the samples of The Hobbit appearing to the left of those from Lord of the Rings. In this experiment discrimin- ation by genre was seen to be more pronounced than discrimination by author, since the PC1 accounts for 30more variation in the data than any of the other prin- cipal components. We can remove the effect of genre by taking PC1 out of the diagram, and instead of plotting PC1 against PC2, plotting PC2 against PC3, as shown in Fig. 7. There is no option on the stylo 35GUI for plotting PCA components other than the first and second, but this may be done with the following series of R commands: >a¼stylo() >b¼a$pca.coordinates 40>PC2¼b[,2] >PC3¼b[,3] >labels¼names(PC2) >plot(PC2, PC3, pch¼““) >for (i in 1:length(labels)){ 45þtext(PC2[i], PC3[i], labels[i]) þ} Fig. 6 PCA of texts by Lewis. L’Engle and Tolkein, using the 5,000 most frequent character 4-grams Fig. 7 Plot of texts by Lewis, L’Engle, and Tolkein on the second and third principal components, using the 5,000 most frequent character 4-grams Computer stylometry of The Dark Tower Digital Scholarship in the Humanities, 2017 9 of 14 Deleted Text: s Deleted Text: -- Deleted Text: first principal component ( Deleted Text: ) This has the effect of grouping all the Lewis texts together, irrespective of genre, in the bottom left of the diagram. There is now also a distinct cluster for L’Engle in the top left corner, and the Tolkein sam- 5 ples all appear on the right-hand side. 6 Experiments 2: The Dark Tower in Relation to Texts by Lewis, Hooper, and Marchington The second set of experiments was designed to show 10 where the individual chapters of The Dark Tower lay in relation to known works by Lewis, Hooper, and the Marchington letter. The results are shown in Fig. 8 for hierarchical clustering by Ward’s method with the 100 most frequent single words. The coar- 15 sest (leftmost) subdivision separates most of the known works by Lewis from those by Hooper and Marchington. The posthumously discovered texts (MBB, LEFAY and DT1–DT6) all cluster very close together, and all are well within the main Lewis clus- 20 ter. This suggests that all these texts were indeed written by Lewis. The main surprise was that there was a small cluster of Lewis texts at the bottom, attached to the Hooper/Marchington cluster. The final chapter of The Dark Tower (DT7) appeared 25 in this small cluster, and thus seems to have stylistic similarities with works both by and not by Lewis. The experiment was repeated using the 5,000 most frequent character 4-grams, since this feature gave the most clear-cut results for the fiction texts. These 30 results are shown in Fig. 9. The results are more clear-cut when using the 5,000 most frequent character 4-grams (Fig. 9) than when using the top 100 single words (Fig. 8), and give two main clusters. All the samples of 35 Lewis’s known fiction appear in the bottom cluster, along with the posthumously published samples MBB, LEFAY, and Chapters 1–6 of The Dark Tower. The top cluster contains all the samples of Hooper’s works, clustered tightly together, the 40 Marchington Letter, and a tight grouping contain- ing Chapter 7 of The Dark Tower and four samples of Lewis’s non-fiction. The main division between the texts thus appears to be non-fictional (top clus- ter) versus fictional (bottom cluster). Once again we 45have a situation where genre and authorship con- found each other—does the final chapter of The Dark Tower appear in the top cluster because it is written by Hooper or Marchington, or because it is written in the style of non-fiction? The next step was 50to perform PCA experiments to first try and deter- mine whether the PC1 did indeed correspond to genre, and if so omit this component from a future analysis using PC2 and PC3. This would ide- ally extract the effects of genre, so that the results of 55authorship alone can be seen. The PCA analysis plotting the text samples according to their scores on the second and third principal components is shown in Fig. 10. The fictional text samples have all got positive (or only slightly negative) scores 60on PC1, and the non-fiction samples almost all negative (or only slightly positive) scores on PC1. Fig. 8 Dendrogram comparing The Dark Tower with text samples by Lewis, Hooper, and Marchington, using the 100 most frequent single words M. P. Oakes 10 of 14 Digital Scholarship in the Humanities, 2017 Deleted Text: right Deleted Text: to Deleted Text: clear Deleted Text: clear Deleted Text: to Deleted Text: 4 Deleted Text: -- Deleted Text: first principal component ( Deleted Text: ) Deleted Text: are Thus PC1 is polarized by genre, and was eliminated at the next step. DT7 is very close to works by Hooper, being almost superimposed on the cluster of text samples by Hooper in the bottom left quad- 5 rant. Is this because they were actually written by Hooper, or are they simply written in a stylistically similar non-fictional style? The small Marchington sample appears as a complete outlier at highly nega- tives scores on both PC1 and PC2. 10 In the next experiment, I removed the effect of genre which gave the polarity seen on PC1, where all the non-fiction texts are placed on the left-hand side, and all the fiction texts are placed on the right-hand side. This was done by omitting PC1, 15 and plotting PC2 against PC3. This plot is shown in Fig. 11. This plot was inconclusive, since the Hooper and Lewis samples appeared very close to- gether (albeit with a tendency for the Hooper sam- ples to appear near the top), and DT7 is almost 20 equidistant between samples by the two authors Hooper and Lewis. Further experimentation showed that Fig. 11 was probably distorted due to the outlying Marchington letter (M_LET) sample, which was much smaller than the others and thus 25probably contained much statistical noise. In add- ition, while it was pseudoscientific in style, it was also a letter, which would also put it in contrast with the other texts. After removing this sample, the character 4-gram frequencies in the corpus were 30recalculated to include only the remaining texts. When this sample was removed, I obtained the much clearer plot shown in Fig. 12. Here the Hooper texts are plotted at positive values of both PC2 and PC3, and thus form a cluster in the top 35right part of the graph. DT7 now plots much closer to the Lewis texts. Fig. 9 Dendrogram comparing The Dark Tower with text samples by Lewis, Hooper, and Marchington, using the 5,000 most frequent character 4-grams Fig. 10 Plot of The Dark Tower chapters and texts by Lewis, Hooper, and Marchington on the first and second principal components, using the 5,000 most fre- quent character 4-grams Computer stylometry of The Dark Tower Digital Scholarship in the Humanities, 2017 11 of 14 Deleted Text: polarised Deleted Text: left Deleted Text: right Deleted Text: - 7 Experiments 3: Word Length Experiments The one linguistic feature suggested by Juola (2015, p. i108) not yet examined in this article is mean 5 word length. The mean word lengths (in characters) for each of the text samples used in this article were found, using a program written in Perl by the author, and are shown in Table 3. Although average word length is often con- 10 sidered a blunt tool for assigning authorship, the results in Table 3 generally accord with the experi- ments performed on stylo before the effects of genre were filtered out by PCA. The thirteen texts with greatest average word length are all non-fictional, 15 which is the style in which DT7 is written. The texts by Hooper and Marchington and the final chapter of The Dark Tower are grouped together at the top of the table, as they have greater average word length than the other texts. The texts with 20 lowest average word length are the Narnia series, including the Lefay Fragment. The other children’s authors also tended to use shorter words: the aver- age word lengths for the Madeleine L’Engle samples were in the range 4.134–4.294; for Tolkein’s The 25Hobbit, the range was 4.092–4.252; and for Tolkein’s Lord of the Rings, it was from 4.045 to 4.294, except for the Prologue which was 4.389. Since word length is a single figure which depends on both genre and authorship, it is not possible to 30separate these out using this technique alone, and thus the final chapter of The Dark Tower appears close to the Marchington and Hooper samples, pos- sibly because they are all written in the style of non- fiction. To filter out the effect of genre, it might be 35possible to find the mean word lengths for the genres (children’s fiction; adult fiction; adult non- fiction) over a large range of authors, and to find the word lengths of our samples relative to these means. Word length as a feature has been found 40in several multi-dimensional studies, such as Biber (1988), revealing that word length has functional properties. Fig. 11 Plot of The Dark Tower chapters and texts by Lewis, Hooper, and Marchington on the second and third principal components, using the 5,000 most fre- quent character 4-grams Fig. 12 Plot of The Dark Tower chapters and texts by Lewis and Hooper on the second and third principal components, using the 5,000 most frequent character 4-grams M. P. Oakes 12 of 14 Digital Scholarship in the Humanities, 2017 Deleted Text: : Deleted Text: paper Deleted Text: paper Deleted Text: , Deleted Text: 13 Deleted Text: , Deleted Text: to Deleted Text: to Deleted Text: , Deleted Text: , 8 Conclusion From these analyses, I feel that it is clear that Lewis wrote the first six chapters of The Dark Tower, as well as The Man Born Blind and the Lefay Fragment, 5 all of which were found by Walter Hooper in note- books after Lewis’s death. Initial results did show that the final chapter of The Dark Tower was more stylistically consistent with the samples of Hooper and Marchington’s writing. However, this may be 10 more a question of genre than authorship, since the plot of The Dark Tower changes abruptly from a narrative account in the first six chapters, to a pseudoscientific description of how the people of ‘Othertime’ discovered time travel in the seventh 15 chapter. Marchington’s letter is also in pseudoscien- tific style, as it describes the results of a (ficticious) soil analysis. Although Hooper’s texts are not pseudoscientific, they are not narrative fiction either, which may explain why they initially clus- 20 tered with the Marchington letter and the final chapter of The Dark Tower. The use of PCA where factors corresponding to genre were not plotted proved to be an effective means of filtering out genre. Once the effects of genre were removed, 25 text sample DT7 did appear to be more typical of the Lewis texts than the Hooper texts. Discovering the contents of a library in another world is in fact a Lewisian motif, seen in The Voyage of the Dawn Treader, the third of the Narnia series, when Lucy 30 reads the contents of a book of magical spells in the library of the fallen star Coriakin. On the other hand, if an unfinished work was to be added to, it would be easier to add a new chapter at the end than at any other place in the text. 35 References Biber, D. (1988). Variation Across Speech and Writing. Cambridge, UK: Cambridge University Press. Binongo, J. and Smith, M. W. A. (1999). The application of principal component analysis to stylometry. Literary 40 and Linguistic Computing, 14(4): 445–65. Burrows, J. (2002). Delta: A measure of stylistic differ- ence and a guide to likely authorship. Literary and Linguistic Computing, 17(3): 267–87. Table 3 Average word lengths (in characters) for each of the text samples Text sample Words Characters Average word length M_LET 986 4,775 4.843 DT7 3,691 16,696 4.523 PWD6 4,689 21,110 4.502 PWD1_2 4,574 20,518 4.489 PWD3_4 3,991 17,844 4.471 PP1 3,677 16,422 4.466 PP2 3,001 13,317 4.378 OSP3 3,761 16,680 4.345 TJB1 4,532 20,075 4.430 TJB4 5,400 23,894 4.425 TJB3 3,918 17,336 4.425 TJB2 6,676 29,457 4.412 PWD5 2,299 10,098 3.392 LOR0 7,381 32,396 4.389 OSP4 3,794 16,612 4.378 THS1 9,069 39,528 4.359 THS2 7,468 32,433 4.343 PER4 5,508 23,906 4.340 ENG4 4,198 18,028 4.294 PER3 5,329 22,866 4.291 OSP2 3,635 15,559 4.280 DT1 3,010 12,837 4.265 HOB4 4,066 17,287 4.252 LOR1 9,820 41,593 4.236 THS4 8,457 35,807 4.234 DT3 4,879 20,614 4.225 THS3 8,941 37,728 4.220 DT6 3,580 15,105 4.219 DT2 4,190 17,670 4.217 ENG1 4,652 19,518 4.196 PER1 4,980 20,862 4.189 ENG2 3,628 15,171 4.182 OSP1 5,497 22,940 4.173 DAWN 3,237 13,443 4.153 DT4 5,645 23,388 4.143 ENG3 3,819 15,799 4.137 HOB1 8,652 35,654 4.121 HOB2 4,234 21,436 4.096 HOB3 2,874 11,761 4.092 DT5 3,504 14,327 4.089 MC2 9,192 37,505 4.080 LOR2 3,375 13,685 4.055 LOR3 13,039 52,740 4.045 MBB 1,769 7,125 4.028 MC1 8,229 33,106 4.023 PER2 4,208 16,905 4.017 MN 3,035 12,112 3.991 LEFAY 5,437 21,676 3.987 LWW 3,869 15,258 3.944 LB 2,648 10,395 3.926 Computer stylometry of The Dark Tower Digital Scholarship in the Humanities, 2017 13 of 14 Deleted Text: 6 Deleted Text: `` Deleted Text: '' Deleted Text: were Eder, M., Rybicki, J., and Kestemont, M. (2015). ‘Stylo’: A package for stylometric analyses. file://prs-store2.unv. wlv.ac.uk/home2$/in4326/home/Profile/Downloads/ stylo_howto%20(1).pdf (accessed 17 May 2017). 5 Eder, M., Rybicki, J., and Kestemont, M. (2016). Stylometry with R: A package for computational text analysis. R Journal, 8(1): 107–21. Harris, M. (2010). Corpus linguistics for the decipherment of Rongorongo. M. Res. Dissertation, Birkbeck University. 10 Holmes, D. I., Gordon, L. J., and Wilson, C. (2001). A widow and her soldier: Stylometry and the American civil war. Literary and Linguistic Computing, 16(4): 403–20. Hooper, W. (1971). Past Watchful Dragons. New York, NY: Collier Books. 15 Hooper, W. (1977). Preface. The Dark Tower and Other Stories. Lewis C.S. London: Fount. Jones, C. F. (1989). The literary detective computer analysis of stylistic differences between ‘‘The Dark Tower’’ and C. S. Lewis’ Deep Space Trilogy. Mythlore, 57: 11–15. 20 Juola, P. (2015). The Rowling case: A proposed standard analytic protocol for authorship questions. Digital Scholarship in the Humanities, 30(Suppl 1): i100–13. Lindskoog, K. (1978). Some problems in C.S. Lewis scholarship. Christianity and Literature, 27(4), 25 Summer 1978. Lindskoog, K. (1988). The C.S. Lewis Hoax. Portland, OR: Multnomah Books, pp. 43–61. Lindskoog, K. (1994). Light in the Shadowlands. Sisters, OR: Multnomah Books. 30Lindskoog, K. (1999). Katherine Lindskoog’s informal answer to the ninth non-proof. The Lewis Legacy issue 81, summer 1999. The C.S. Lewis Foundation for Truth in Publishing, June 1, 1999. http://www.discov- ery.org/a/944 (accessed 16 May 2017). 35Linmans, A. J. M. (1998). Correspondence analysis of the Synoptic Gospels. Literary and Linguistic Computing, 13(1): 1–13. Merriam, T. (2000). Edward III. Literary and Linguistic Computing, 15(2): 157–86. 40Schöch, C. (2013). Principal component analysis for liter- ary genre stylistics. The Dragonfly’s Gaze, September 29, 2013. http://dragonfly.hypotheses.org/472 (accessed 29 June 2017). Tankard, J. (1986). The Literary Detective, Byte, February 451986, pp. 231–8. Thisted, R. and Efron, B. (1987). Did Shakespeare write a newly discovered poem? Biometrika, 74(3): 445–55. Thompson, J. and Rasp, J. (2009). Did C. S, Lewis write The Dark Tower?: An examination of the small-sample 50properties of the Thisted-Efron tests of authorship. Austrian Journal of Statistics, 38(2): 71–82. Ward, J. H., Jr. (1963). Hierarchical grouping to optimize an objective function. Journal of the American Statistical Association, 58: 236–44. 55 M. P. Oakes 14 of 14 Digital Scholarship in the Humanities, 2017 file://prs-store2.unv.wlv.ac.uk/home2$/in4326/home/Profile/Downloads/stylo_howto%20(1).pdf file://prs-store2.unv.wlv.ac.uk/home2$/in4326/home/Profile/Downloads/stylo_howto%20(1).pdf file://prs-store2.unv.wlv.ac.uk/home2$/in4326/home/Profile/Downloads/stylo_howto%20(1).pdf http://www.discovery.org/a/944 http://www.discovery.org/a/944 http://dragonfly.hypotheses.org/472