mv: ‘./input-file.zip’ and ‘./input-file.zip’ are the same file Creating study carrel named subject-pacificArea-gutenberg Initializing database Unzipping Archive: input-file.zip creating: ./tmp/input/input-file/ inflating: ./tmp/input/input-file/21805.txt inflating: ./tmp/input/input-file/24998.txt inflating: ./tmp/input/input-file/23821.txt inflating: ./tmp/input/input-file/3818.txt inflating: ./tmp/input/input-file/48714.txt inflating: ./tmp/input/input-file/metadata.csv caution: excluded filename not matched: *MACOSX* === DIRECTORIES: ./tmp/input === DIRECTORY: ./tmp/input/input-file === metadata file: ./tmp/input/input-file/metadata.csv === found metadata file === updating bibliographic database Building study carrel named subject-pacificArea-gutenberg FILE: cache/48714.txt OUTPUT: txt/48714.txt FILE: cache/24998.txt OUTPUT: txt/24998.txt FILE: cache/21805.txt OUTPUT: txt/21805.txt FILE: cache/3818.txt OUTPUT: txt/3818.txt FILE: cache/23821.txt OUTPUT: txt/23821.txt 24998 txt/../ent/24998.ent === file2bib.sh === id: 24998 author: Becke, Louis title: Concerning "Bully" Hayes From "The Strange Adventure Of James Shervinton and Other Stories" - 1902 date: pages: extension: .txt txt: ./txt/24998.txt cache: ./cache/24998.txt Content-Encoding ISO-8859-1 Content-Type text/plain; charset=ISO-8859-1 X-Parsed-By ['org.apache.tika.parser.DefaultParser', 'org.apache.tika.parser.csv.TextAndCSVParser'] X-TIKA:embedded_depth 0 X-TIKA:parse_time_millis 1 resourceName b'24998.txt' Traceback (most recent call last): File "/data-disk/reader-compute/reader-classic/bin/file2bib.py", line 107, in text = textacy.preprocessing.normalize.normalize_quotation_marks( text ) File "/data-disk/python/lib/python3.8/site-packages/textacy/preprocessing/normalize.py", line 32, in normalize_quotation_marks return text.translate(QUOTE_TRANSLATION_TABLE) AttributeError: 'NoneType' object has no attribute 'translate' 21805 txt/../pos/21805.pos 24998 txt/../wrd/24998.wrd Traceback (most recent call last): File "/data-disk/reader-compute/reader-classic/bin/txt2keywords.py", line 54, in for keyword, score in ( yake( doc, ngrams=NGRAMS, topn=TOPN ) ) : File "/data-disk/python/lib/python3.8/site-packages/textacy/ke/yake.py", line 96, in yake word_scores = _compute_word_scores(doc, word_occ_vals, word_freqs, stop_words) File "/data-disk/python/lib/python3.8/site-packages/textacy/ke/yake.py", line 205, in _compute_word_scores freq_baseline = statistics.mean(freqs_nsw) + statistics.stdev(freqs_nsw) File "/data-disk/python/lib/python3.8/statistics.py", line 315, in mean raise StatisticsError('mean requires at least one data point') statistics.StatisticsError: mean requires at least one data point 21805 txt/../wrd/21805.wrd 24998 txt/../pos/24998.pos 21805 txt/../ent/21805.ent === file2bib.sh === id: 21805 author: Roosevelt, Franklin D. (Franklin Delano) title: Day of Infamy Speech: Given before the US Congress December 8 1941 date: pages: extension: .txt txt: ./txt/21805.txt cache: ./cache/21805.txt Content-Encoding ISO-8859-1 Content-Type text/plain; charset=ISO-8859-1 X-Parsed-By ['org.apache.tika.parser.DefaultParser', 'org.apache.tika.parser.csv.TextAndCSVParser'] X-TIKA:content_handler ToTextContentHandler X-TIKA:embedded_depth 0 X-TIKA:parse_time_millis 2 resourceName b'21805.txt' 48714 txt/../pos/48714.pos 48714 txt/../wrd/48714.wrd 23821 txt/../pos/23821.pos 23821 txt/../wrd/23821.wrd 48714 txt/../ent/48714.ent 23821 txt/../ent/23821.ent === file2bib.sh === id: 23821 author: Becke, Louis title: Yorke The Adventurer date: pages: extension: .txt txt: ./txt/23821.txt cache: ./cache/23821.txt Content-Encoding ISO-8859-1 Content-Type text/plain; charset=ISO-8859-1 X-Parsed-By ['org.apache.tika.parser.DefaultParser', 'org.apache.tika.parser.csv.TextAndCSVParser'] X-TIKA:content_handler ToTextContentHandler X-TIKA:embedded_depth 0 X-TIKA:parse_time_millis 3 resourceName b'23821.txt' === file2bib.sh === id: 48714 author: Nalty, Bernard C. title: Cape Gloucester: The Green Inferno date: pages: extension: .txt txt: ./txt/48714.txt cache: ./cache/48714.txt Content-Encoding ISO-8859-1 Content-Type text/plain; charset=ISO-8859-1 X-Parsed-By ['org.apache.tika.parser.DefaultParser', 'org.apache.tika.parser.csv.TextAndCSVParser'] X-TIKA:content_handler ToTextContentHandler X-TIKA:embedded_depth 0 X-TIKA:parse_time_millis 4 resourceName b'48714.txt' 3818 txt/../wrd/3818.wrd 3818 txt/../pos/3818.pos 3818 txt/../ent/3818.ent === file2bib.sh === id: 3818 author: Becke, Louis title: By Reef and Palm date: pages: extension: .txt txt: ./txt/3818.txt cache: ./cache/3818.txt Content-Encoding ISO-8859-1 Content-Type text/plain; charset=ISO-8859-1 X-Parsed-By ['org.apache.tika.parser.DefaultParser', 'org.apache.tika.parser.csv.TextAndCSVParser'] X-TIKA:content_handler ToTextContentHandler X-TIKA:embedded_depth 0 X-TIKA:parse_time_millis 3 resourceName b'3818.txt' Done mapping. Reducing subject-pacificArea-gutenberg === reduce.pl bib === id = 21805 author = Roosevelt, Franklin D. (Franklin Delano) title = Day of Infamy Speech: Given before the US Congress December 8 1941 date = pages = extension = .txt mime = text/plain words = 518 sentences = 27 flesch = 62 summary = United States of America was suddenly and deliberately attacked by naval and air forces of the Empire of Japan. The United States was at peace with that nation and, at the solicitation the American island of Oahu, the Japanese ambassador to the United deliberately sought to deceive the United States by false statements and The attack yesterday on the Hawaiian islands has caused severe damage to American naval and military forces. Yesterday, the Japanese government also launched an attack against Last night, Japanese forces attacked Hong Kong. Last night, Japanese forces attacked Guam. Last night, Japanese forces attacked the Philippine Islands. Last night, the Japanese attacked Wake Island. And this morning, the Japanese attacked Midway Island. The people of the United States have already formed their attack by Japan on Sunday, December 7th, 1941, a state of war has existed between the United States and the Japanese empire. cache = ./cache/21805.txt txt = ./txt/21805.txt === reduce.pl bib === id = 3818 author = Becke, Louis title = By Reef and Palm date = pages = extension = .txt mime = text/plain words = 38203 sentences = 2369 flesch = 90 summary = wife, is away at the other White Man's house no woman cometh inside the "Ay," said Robertson, the trading skipper, "and a good man with his "Aye," said the girl with the White Man's blood, "my right hand is for But the white man smiled, and said his boat was too head of the good and clever white man, as she rubbed his and the girl's "Let's have a look at her," said the skipper, a man of kind impulses, "Nay, mother," said the White Man, "not brother and sister, but only "Tell me," said the White Man. "'Then shalt thou live alone in thy shame,' said the old man; and he our white man, till he returned and said the woman was dead. "The white girl put her hands to her face and wept, and said: 'Bring me son to the white man that came here to look upon the Silent Faces; and cache = ./cache/3818.txt txt = ./txt/3818.txt === reduce.pl bib === id = 23821 author = Becke, Louis title = Yorke The Adventurer date = pages = extension = .txt mime = text/plain words = 20514 sentences = 905 flesch = 82 summary = "Guess he's a lunatic," said Captain Guest, turning to me with a laugh. like a drink of water, Yorke became the soul of hospitality, and told "Captain Yorke," I said, "I can cut it out, I am sure. "Guess he thinks a lot about those poor men of his," said Guest to me "Captain Yorke," said Guest, with a laugh, "you should not have told us "Don't believe that, Captain Yorke," I said, "but at the same time I "I got away on the same morning, and made a fine run right across St. George's Channel, and along the New Britain coast till I made Cape "It's a mighty smart dodge, Captain Yorke," said Guest, as we rose and "Well then, Captain Yorke," I said, "as Guest here leaves me to do all was strained, I picked myself up, handed the rifles to Yorke, and said I cache = ./cache/23821.txt txt = ./txt/23821.txt === reduce.pl bib === === reduce.pl bib === id = 48714 author = Nalty, Bernard C. title = Cape Gloucester: The Green Inferno date = pages = extension = .txt mime = text/plain words = 18450 sentences = 881 flesch = 63 summary = Rupertus, Commanding General, 1st Marine Division, battalions of the 1st Marines were to follow up the assault on Cape infantry unit, the 1st Battalion, 1st Marines, reached Yellow Beach 1 at On New Britain, the 1st Marine Division fought weather and terrain, 3d Battalion, 1st Marines, moved ahead, Company I and the supporting 1st Marine Division landed on New Britain._] Company B, 1st Marine Tank Battalion, with At Cape Gloucester, the 1st Marine Division had an air force of its own The division's air force landed at Cape Gloucester from LSTs on D-Day, Gayle, the new commander of the 2d Battalion, 5th Marines, who already Company K of Deakin's 3d Battalion, 5th Marines, attacked on 11 March, ambush sprung by the 2d Battalion, 5th Marines, killed 20 Japanese and component of the Marine division--Company A, 1st Tank Battalion. the 1st Marine Division received two new weapons--the M4 medium tank, cache = ./cache/48714.txt txt = ./txt/48714.txt Building ./etc/reader.txt 3818 48714 23821 48714 23821 3818 number of items: 5 sum of words: 77,685 average size in words: 19,421 average readability score: 74 nouns: man; hand; men; boat; wife; time; day; island; people; girl; night; captain; face; water; ship; woman; fire; eyes; beach; days; years; hands; enemy; way; house; head; natives; sea; morning; cutter; women; side; village; division; board; troops; trader; one; coast; attack; place; lagoon; crew; deck; air; wind; land; illustration; months; vessel verbs: was; had; is; said; were; be; have; do; came; are; been; see; did; made; come; ''s; went; go; get; took; saw; let; know; called; told; set; got; take; tell; looked; has; put; left; killed; turned; knew; am; make; fell; asked; stood; lay; brought; sat; gave; began; think; give; found; taking adjectives: little; old; white; good; japanese; other; native; many; young; more; first; few; great; heavy; big; 1st; small; own; 5th; long; black; american; dead; last; new; poor; open; much; dark; next; same; such; right; high; wild; blue; 7th; soft; low; better; only; light; bad; strong; short; deep; half; german; certain; whole adverbs: then; not; up; out; so; away; down; now; here; n''t; again; back; there; only; very; off; as; just; even; over; well; too; in; ashore; never; still; ever; once; about; yet; far; on; enough; soon; all; more; also; ago; first; perhaps; instead; always; almost; long; later; together; quite; much; quickly; nearly pronouns: i; he; his; her; it; we; you; she; my; me; him; they; them; us; our; their; your; its; himself; myself; thy; thee; herself; themselves; ourselves; yourself; itself; mine; ours; one; hers; theirs; ''s; ''em; yours; thyself; she----; hez; he''d proper nouns: _; marines; new; marine; cape; battalion; yorke; britain; japanese; guest; captain; island; gloucester; division; brantley; matsuda; colonel; pacific; thou; army; general; rabaul; lieutenant; company; defense; doris; january; islands; department; usmc; 3d; 1st; god; enderby; december; taplin; taku; o''shea; hill; 2d; south; march; pallou; mr; langton; beach; group; challis; rupertus; force keywords: new; japanese; britain; yorke; taplin; taku; pallou; pacific; o''shea; nalia; marines; man; langton; island; illustration; hand; guest; gloucester; fray; enderby; drake; doris; division; challis; captain; cape; brantley; bentos; battalion one topic; one dimension: said file(s): ./cache/21805.txt titles(s): Day of Infamy Speech: Given before the US Congress December 8 1941 three topics; one dimension: said; marines; yorke file(s): ./cache/3818.txt, ./cache/48714.txt, ./cache/23821.txt titles(s): By Reef and Palm | Cape Gloucester: The Green Inferno | Yorke The Adventurer five topics; three dimensions: said man away; marines japanese battalion; oahu secretary regret; oahu secretary regret; oahu secretary regret file(s): ./cache/3818.txt, ./cache/48714.txt, , , titles(s): By Reef and Palm | Cape Gloucester: The Green Inferno | Concerning "Bully" Hayes From "The Strange Adventure Of James Shervinton and Other Stories" - 1902 | Concerning "Bully" Hayes From "The Strange Adventure Of James Shervinton and Other Stories" - 1902 | Concerning "Bully" Hayes From "The Strange Adventure Of James Shervinton and Other Stories" - 1902 Type: gutenberg title: subject-pacificArea-gutenberg date: 2021-06-07 time: 13:06 username: emorgan patron: Eric Morgan email: emorgan@nd.edu input: facet_subject:"Pacific Area" ==== make-pages.sh htm files ==== make-pages.sh complex files ==== make-pages.sh named enities ==== making bibliographics id: 24998 author: Becke, Louis title: Concerning "Bully" Hayes From "The Strange Adventure Of James Shervinton and Other Stories" - 1902 date: words: nan sentences: nan pages: flesch: nan cache: txt: summary: id: 23821 author: Becke, Louis title: Yorke The Adventurer date: words: 20514.0 sentences: 905.0 pages: flesch: 82.0 cache: ./cache/23821.txt txt: ./txt/23821.txt summary: "Guess he''s a lunatic," said Captain Guest, turning to me with a laugh. like a drink of water, Yorke became the soul of hospitality, and told "Captain Yorke," I said, "I can cut it out, I am sure. "Guess he thinks a lot about those poor men of his," said Guest to me "Captain Yorke," said Guest, with a laugh, "you should not have told us "Don''t believe that, Captain Yorke," I said, "but at the same time I "I got away on the same morning, and made a fine run right across St. George''s Channel, and along the New Britain coast till I made Cape "It''s a mighty smart dodge, Captain Yorke," said Guest, as we rose and "Well then, Captain Yorke," I said, "as Guest here leaves me to do all was strained, I picked myself up, handed the rifles to Yorke, and said I id: 3818 author: Becke, Louis title: By Reef and Palm date: words: 38203.0 sentences: 2369.0 pages: flesch: 90.0 cache: ./cache/3818.txt txt: ./txt/3818.txt summary: wife, is away at the other White Man''s house no woman cometh inside the "Ay," said Robertson, the trading skipper, "and a good man with his "Aye," said the girl with the White Man''s blood, "my right hand is for But the white man smiled, and said his boat was too head of the good and clever white man, as she rubbed his and the girl''s "Let''s have a look at her," said the skipper, a man of kind impulses, "Nay, mother," said the White Man, "not brother and sister, but only "Tell me," said the White Man. "''Then shalt thou live alone in thy shame,'' said the old man; and he our white man, till he returned and said the woman was dead. "The white girl put her hands to her face and wept, and said: ''Bring me son to the white man that came here to look upon the Silent Faces; and id: 48714 author: Nalty, Bernard C. title: Cape Gloucester: The Green Inferno date: words: 18450.0 sentences: 881.0 pages: flesch: 63.0 cache: ./cache/48714.txt txt: ./txt/48714.txt summary: Rupertus, Commanding General, 1st Marine Division, battalions of the 1st Marines were to follow up the assault on Cape infantry unit, the 1st Battalion, 1st Marines, reached Yellow Beach 1 at On New Britain, the 1st Marine Division fought weather and terrain, 3d Battalion, 1st Marines, moved ahead, Company I and the supporting 1st Marine Division landed on New Britain._] Company B, 1st Marine Tank Battalion, with At Cape Gloucester, the 1st Marine Division had an air force of its own The division''s air force landed at Cape Gloucester from LSTs on D-Day, Gayle, the new commander of the 2d Battalion, 5th Marines, who already Company K of Deakin''s 3d Battalion, 5th Marines, attacked on 11 March, ambush sprung by the 2d Battalion, 5th Marines, killed 20 Japanese and component of the Marine division--Company A, 1st Tank Battalion. the 1st Marine Division received two new weapons--the M4 medium tank, id: 21805 author: Roosevelt, Franklin D. (Franklin Delano) title: Day of Infamy Speech: Given before the US Congress December 8 1941 date: words: 518.0 sentences: 27.0 pages: flesch: 62.0 cache: ./cache/21805.txt txt: ./txt/21805.txt summary: United States of America was suddenly and deliberately attacked by naval and air forces of the Empire of Japan. The United States was at peace with that nation and, at the solicitation the American island of Oahu, the Japanese ambassador to the United deliberately sought to deceive the United States by false statements and The attack yesterday on the Hawaiian islands has caused severe damage to American naval and military forces. Yesterday, the Japanese government also launched an attack against Last night, Japanese forces attacked Hong Kong. Last night, Japanese forces attacked Guam. Last night, Japanese forces attacked the Philippine Islands. Last night, the Japanese attacked Wake Island. And this morning, the Japanese attacked Midway Island. The people of the United States have already formed their attack by Japan on Sunday, December 7th, 1941, a state of war has existed between the United States and the Japanese empire. ==== make-pages.sh questions ==== make-pages.sh search ==== make-pages.sh topic modeling corpus Zipping study carrel