the digital librarian

{ "@context": [ "http://iiif.io/api/presentation/ /context.json", { "author": "http://purl.org/dc/terms/creator", "bibo": "http://purl.org/ontology/bibo/" } ], "@id": "http://example.org/iiif/book /manifest", "@type": ["sc:manifest", "bibo:book"], "metadata": [ { "label": "author", "property": "http://purl.org/dc/terms/creator", "value": "allen smithee" }, { "label": "license", "property": "http://purl.org/dc/terms/license", "value": "cc-by . " } ], "license": "http://creativecommons.org/licenses/by/ . /", "author": { "@id": "http://www.wikidata.org/entity/q ", "label": "allen smithee" } }

use test::more; use app::paia::tester; new_paia_test; paia qw(config); is stdout, "{}\n"; is error, undef; paia qw(config -c x.json --verbose); is error, "failed to open config file x.json\n"; ok exit_code; paia qw(config --config x.json --verbose foo bar); is output, "# saved config file x.json\n"; paia qw(config foo bar); paia qw(config base http://example.org/); is exit_code, ; is output, ''; paia qw(config); is_deeply stdout_json, { base => 'http://example.org/', foo => 'bar', }, "get full config" done_paia_test;

best of digital news design งานที่เราว่า ว้าว! เลยอยากแบ่งกันดู thanisara ruangdej (gg) recap – open data for democracy เปิดข้อมูลรัฐสู่สาธารณะ เพื่อประชาธิปไตย punch up team แสนล้าน กู้มาแล้วไปไหน? ชวนดูเครื่องมือติดตามและตรวจสอบเงินกู้โควิด- punch up team

นี่ว่าดี! งาน visual & data-driven stories จาก the pudding cup punch up team see all blog event recap – open data for democracy เปิดข้อมูลรัฐสู่สาธารณะ เพื่อประชาธิปไตย mar more info punch up x skooldio เวิร์คชอปออนไลน์ที่นำบรรยากาศห้องเรียนมาอยู่บนหน้าจอที่บ้าน jun more info พลิกโฉมสื่อไทยด้วยข้อมูลใน data journalism workshop jun more info punch up x wisesight : workshop แปลงโฉม data report ให้ย่อยง่าย น่าดู อ่านสนุก ตอบโจทย์ผู้อ่าน dec more info

no. building, th floor, soi patpong, surawong rd, bang rak, bangkok copyright , punch up say hi! hi@punchup.world or leave us a message notice: javascript is required for this content. principal - labiks ir para o conteúdo linkedininstagramtwitteryoutubefacebooke-mail buscar resultados para: quem somos biblioteca projetos relatório anual mapa painel de dados blog contato buscar resultados para: loading... sistema de bicicletas compartilhadas na amÉrica latina relatÓrio painel de dados os sistemaslatino-americanos navegue mapa latino-americano de sistemas de bicicletas compartilhadas visite principaladmin - - t : : + : latin american bike knowledge sharing a labiks nasce com a missão de reunir, compartilhar e potencializar conhecimento sobre os sistemas de bicicletas públicas da américa latina. nós realmente acreditamos no valor e contribuição das pesquisas para alcançarmos cidades e comunidades mais sustentáveis. sendo assim, nós trabalhamos por mais transparência e responsabilidade governamental na américa latina. conheÇa a labiks latin american bike knowledge sharing a labiks nasce com a missão de reunir, compartilhar e potencializar conhecimento sobre os sistemas de bicicletas públicas da américa latina. nós realmente acreditamos no valor e contribuição das pesquisas para alcançarmos cidades e comunidades mais sustentáveis. sendo assim, nós trabalhamos por mais transparência e responsabilidade governamental na américa latina. conheÇa a labiks nossos desafios transformar conhecimento em ação junte-se a labiks! para que mais cidades possam usufruir de sistemas de bicicletas compartilhadas de qualidade é importante estimular a capacitação de todos os atores sobre tendências e boas práticas aplicadas ao planejamento, financiamento, gestão e monitoramento destes sistemas. assim, a labiks convida pesquisadores, governos, indústria, financiadores e todos os interessados a serem parceiros desta iniciativa. junte-se a nós! latin american bike knowledge sharing site feito pela liquefeito ir ao topo managing remote conference presenters with zoom | disruptive library technology jester skip links skip to primary navigation skip to content skip to footer disruptive library technology jester about resume toggle search toggle menu peter murray library technologist, open source advocate, striving to think globally while acting locally follow columbus, ohio email twitter keybase github linkedin stackoverflow orcid email managing remote conference presenters with zoom posted on march , and updated on april , minute read bringing remote presenters into a face-to-face conference is challenging and fraught with peril. in this post, i describe a scheme using zoom that had in-person attendees forgetting that the presenter was remote! the code lib conference was this week, and with the covid- pandemic breaking through many individuals and institutions made decisions to not travel to pittsburgh for the meeting. we had an unprecedented nine presentations that were brought into the conference via zoom. i was chairing the livestream committee for the conference (as i have done for several years—skipping last year), so it made the most sense for me to arrange a scheme for remote presenters. with the help of the on-site a/v contractor, we were able to pull this off with minimal requirements for the remote presenter. list of requirements zoom pro accounts pc/mac with video output, as if you were connecting an external monitor (the “receiving zoom” computer) pc/mac (the “coordinator zoom” computer) usb audio interface hardwired network connection for the receiving zoom computer (recommended) the pro-level zoom accounts were required because we needed to run a group call for longer than minutes (to include setup time). and two were needed: one for the coordinator zoom machine and one for the dedicated receiving zoom machine. it would have been possible to consolidate the two zoom pro accounts and the two pc/mac machines into one, but we had back-to-back presenters at code lib, and i wanted to be able to help one remote presenter get ready while another was presenting. in addition to this equipment, the a/v contractor was indispensable in making the connection work. we fed the remote presenter’s video and audio from the receiving zoom computer to the contractor’s a/v switch through hdmi, and the contractor put the video on the ballroom projectors and audio through the ballroom speakers. the contractor gave us a selective audio feed of the program audio minus the remote presenter’s audio (so they wouldn’t hear themselves come back through the zoom meeting). this becomes a little clearer in the diagram below. physical connections and setup this diagram shows the physical connections between machines. the audio mixer and video switch were provided and run by the a/v contractor. the receiving zoom machine was the one that is connected to the a/v contractor’s video switch via an hdmi cable coming off the computer’s external monitor connection. in the receiving zoom computer’s control panel, we set the external monitor to mirror what was on the main monitor. the audio and video from the computer (i.e., the zoom call) went out the hdmi cable to the a/v contractor’s video switch. the a/v contractor took the audio from the receiving zoom computer through the video switch and added it to the audio mixer as an input channel. from there, the audio was sent out to the ballroom speakers the same way audio from the podium microphone was amplified to the audience. we asked the a/v contractor to create an audio mix that includes all of the audio sources except the receiving zoom computer (e.g., in-room microphones) and plugged that into the usb audio interface. that way, the remote presenter could hear the sounds from the ballroom—ambient laughter, questions from the audience, etc.—in their zoom call. (note that it was important to remove the remote presenter’s own speaking voice from this audio mix; there was a significant, distracting delay between the time the presenter spoke and the audio was returned to them through the zoom call.) we used a hardwired network connection to the internet, and i would recommend that—particularly with tech-heavy conferences that might overflow the venue wi-fi. (you don’t want your remote presenter’s zoom to have to compete with what attendees are doing.) be aware that the hardwired network connection will cost more from the venue, and may take some time to get functioning since this doesn’t seem to be something that hotels often do. in the zoom meeting, we unmuted the microphone and selected the usb audio interface as the microphone input. as the zoom meeting was connected, we made the meeting window full-screen so the remote presenter’s face and/or presentation were at the maximum size on the ballroom projectors. setting up the zoom meetings the two zoom accounts came from the open library foundation. (thank you!) as mentioned in the requirements section above, these were pro-level accounts. the two accounts were olf_host @openlibraryfoundation.org and olf_host @openlibraryfoundation.org. the olf_host account was used for the receiving zoom computer, and the olf_host account was used for the coordinator zoom computer. the zoom meeting edit page looked like this: this is for the “code lib remote presenter a” meeting with the primary host as olf_host @openlibraryfoundation.org. note these settings: a recurring meeting that ran from : am to : pm each day of the conference. enable join before host is checked in case the remote presenter got on the meeting before i did. record the meeting automatically in the cloud to use as a backup in case something goes wrong. alternative hosts is olf_host @openlibraryfoundation.org the “code lib remote presenter b” meeting was exactly the same except the primary host was olf_host , and olf_host was added as an alternative host. the meetings were set up with each other as the alternative host so that the coordinator zoom computer could start the meeting, seamlessly hand it off to the receiving zoom computer, then disconnect. preparing the remote presenter remote presenters were given this information: code lib will be using zoom for remote presenters. in addition to the software, having the proper audio setup is vital for a successful presentation. microphone: the best option is a headset or earbuds so a microphone is close to your mouth. built-in laptop microphones are okay, but using them will make it harder for the audience to hear you. speaker: a headset or earbuds are required. do not use your computer’s built-in speakers. the echo cancellation software is designed for small rooms and cannot handle the delay caused by large ballrooms. you can test your setup with a test zoom call. be sure your microphone and speakers are set correctly in zoom. also, try sharing your screen on the test call so you understand how to start and stop screen sharing. the audience will see everything on your screen, so quit/disable/turn-off notifications that come from chat programs, email clients, and similar tools. plan to connect to the zoom meeting minutes before your talk to work out any connection or setup issues. at the -minute mark before the remote presentation, i went to the ballroom lobby and connected to the designated zoom meeting for the remote presenter using the coordinator zoom computer. i used this checklist with each presenter: check presenter’s microphone level and sound quality (make sure headset/earbud microphone is being used!) check presenter’s speakers and ensure there is no echo test screen-sharing (start and stop) with presenter remind presenter to turn off notifications from chat programs, email clients, etc. remind the presenter that they need to keep track of their own time; there is no way for us to give them cues about timing other than interrupting them when their time is up the critical item was making sure the audio worked (that their computer was set to use the headset/earbud microphone and audio output). the result was excellent sound quality for the audience. when the remote presenter was set on the zoom meeting, i returned to the a/v table and asked a livestream helper to connect the receiving zoom to the remote presenter’s zoom meeting. at this point, the remote presenter can hear the audio in the ballroom of the speaker before them coming through the receiving zoom computer. now i would lock the zoom meeting to prevent others from joining and interrupting the presenter (from the zoom participants panel, select more then lock meeting). i hung out on the remote presenter’s meeting on the coordinator zoom computer in case they had any last-minute questions. as the speaker in the ballroom was finishing up, i wished the remote presenter well and disconnected the coordinator zoom computer from the meeting. (i always selected leave meeting rather than end meeting for all so that the zoom meeting continued with the remote presenter and the receiving zoom computer.) as the remote presenter was being introduced—and the speaker would know because they could hear it in their zoom meeting—the a/v contractor switched the video source for the ballroom projectors to the receiving zoom computer and unmuted the receiving zoom computer’s channel on the audio mixer. at this point, the remote speaker is off-and-running! last thoughts this worked really well. surprisingly well. so well that i had a few people comment that they were taken aback when they realized that there was no one standing at the podium during the presentation. i’m glad i had set up the two zoom meetings. we had two cases where remote presenters were back-to-back. i was able to get the first remote presenter set up and ready on one zoom meeting while preparing the second remote presenter on the other zoom meeting. the most stressful part was at the point when we disconnected the first presenter’s zoom meeting and quickly connected to the second presenter’s zoom meeting. this was slightly awkward for the second remote presenter because they didn’t hear their full introduction as it happened and had to jump right into their presentation. this could be solved by setting up a second receiving zoom computer, but this added complexity seemed to be too much for the benefit gained. i would definitely recommend making this setup a part of the typical a/v preparations for future code lib conferences. we don’t know when an individual’s circumstances (much less a worldwide pandemic) might cause a last-minute request for a remote presentation capability, and the overhead of the setup is pretty minimal. tags: code lib, howto, zoom categories: raw technology twitter facebook linkedin previous next you may also enjoy more thoughts on pre-recording conference talks minute read over the weekend, i posted an article here about pre-recording conference talks and sent a tweet about the idea on monday. i hoped to generate discussion abo... should all conference talks be pre-recorded? minute read the code lib conference was last week. that meeting used all pre-recorded talks, and we saw the benefits of pre-recording for attendees, presenters, and con... user behavior access controls at a library proxy server are okay minute read earlier this month, my twitter timeline lit up with mentions of a half-day webinar called cybersecurity landscape - protecting the scholarly infrastructure. ... as a cog in the election system: reflections on my role as a precinct election official minute read i may nod off several times in composing this post the day after election day. hopefully, in reading it, you won’t. it is a story about one corner of democ... enter your search term... twitter github feed © peter murray. powered by jekyll & minimal mistakes. half-life - wikipedia half-life from wikipedia, the free encyclopedia jump to navigation jump to search scientific and mathematical term this article is about the scientific and mathematical concept. for the video game, see half-life (video game). for other uses, see half-life (disambiguation). this article is missing information about the history of the term half-life. please expand the article to include this information. further details may exist on the talk page. (july ) number of half-lives elapsed fraction remaining percentage remaining ⁄ ⁄ ⁄ ⁄ . ⁄ . ⁄ . ⁄ . ⁄ . ... ... ... n / n / n half-life (symbol t ⁄ ) is the time required for a quantity to reduce to half of its initial value. the term is commonly used in nuclear physics to describe how quickly unstable atoms undergo radioactive decay or how long stable atoms survive. the term is also used more generally to characterize any type of exponential or non-exponential decay. for example, the medical sciences refer to the biological half-life of drugs and other chemicals in the human body. the converse of half-life is doubling time. the original term, half-life period, dating to ernest rutherford's discovery of the principle in , was shortened to half-life in the early s.[ ] rutherford applied the principle of a radioactive element's half-life to studies of age determination of rocks by measuring the decay period of radium to lead- . half-life is constant over the lifetime of an exponentially decaying quantity, and it is a characteristic unit for the exponential decay equation. the accompanying table shows the reduction of a quantity as a function of the number of half-lives elapsed. contents probabilistic nature formulas for half-life in exponential decay . half-life and reaction orders . decay by two or more processes . examples in non-exponential decay in biology and pharmacology see also references external links probabilistic nature[edit] simulation of many identical atoms undergoing radioactive decay, starting with either atoms per box (left) or (right). the number at the top is how many half-lives have elapsed. note the consequence of the law of large numbers: with more atoms, the overall decay is more regular and more predictable. a half-life usually describes the decay of discrete entities, such as radioactive atoms. in that case, it does not work to use the definition that states "half-life is the time required for exactly half of the entities to decay". for example, if there is just one radioactive atom, and its half-life is one second, there will not be "half of an atom" left after one second. instead, the half-life is defined in terms of probability: "half-life is the time required for exactly half of the entities to decay on average". in other words, the probability of a radioactive atom decaying within its half-life is %.[ ] for example, the image on the right is a simulation of many identical atoms undergoing radioactive decay. note that after one half-life there are not exactly one-half of the atoms remaining, only approximately, because of the random variation in the process. nevertheless, when there are many identical atoms decaying (right boxes), the law of large numbers suggests that it is a very good approximation to say that half of the atoms remain after one half-life. various simple exercises can demonstrate probabilistic decay, for example involving flipping coins or running a statistical computer program.[ ][ ][ ] formulas for half-life in exponential decay[edit] main article: exponential decay an exponential decay can be described by any of the following three equivalent formulas:[ ]: – n ( t ) = n ( ) t t / n ( t ) = n e − t τ n ( t ) = n e − λ t {\displaystyle {\begin{aligned}n(t)&=n_{ }\left({\frac { }{ }}\right)^{\frac {t}{t_{ / }}}\\n(t)&=n_{ }e^{-{\frac {t}{\tau }}}\\n(t)&=n_{ }e^{-\lambda t}\end{aligned}}} where n is the initial quantity of the substance that will decay (this quantity may be measured in grams, moles, number of atoms, etc.), n(t) is the quantity that still remains and has not yet decayed after a time t, t ⁄ is the half-life of the decaying quantity, τ is a positive number called the mean lifetime of the decaying quantity, λ is a positive number called the decay constant of the decaying quantity. the three parameters t ⁄ , τ, and λ are all directly related in the following way: t / = ln ⁡ ( ) λ = τ ln ⁡ ( ) {\displaystyle t_{ / }={\frac {\ln( )}{\lambda }}=\tau \ln( )} where ln( ) is the natural logarithm of (approximately . ).[ ]: half-life and reaction orders[edit] the value of the half-life depends on the reaction order: zero order kinetics: the rate of this kind of reaction does not depend on the substrate concentration. the rate law of zero order kinetics is as follows: [ a ] = [ a ] − k t {\displaystyle [a]=[a]_{ }-kt} in order to find the half life we have to replace the concentration value for the initial concentration divided by and isolate the time. if we do it, we find the equation of the half life of the zero order reaction: t / = [ a ] k {\displaystyle t_{ / }={\frac {[a]_{ }}{k }}} the t / formula for a zero order reaction suggests the half-life depends on the amount of initial concentration and rate constant. first order kinetics: in first order reactions, the concentration of the reaction will continue to decrease as time progresses until it reaches zero, and the length of half-life will be constant, independent of concentration. the time for [a] to decrease from [a] to ½ [a] in a first-order reaction is given by the following equation: k t / = − ln ⁡ ( / [ a ] [ a ] ) = − ln ⁡ = ln ⁡ {\displaystyle kt_{ / }=-\ln {\biggl (}{\frac { / [a]_{ }}{[a]_{ }}}{\biggr )}=-\ln {\frac { }{ }}=\ln } for a first-order reaction, the half-life of a reactant is independent of its initial concentration. therefore, if the concentration of a at some arbitrary stage of the reaction is [a], then it will have fallen to ½ [a] after a further interval of (\ln )/k. hence, the half-life of a first order reaction is given as the following: t / = ln ⁡ k {\displaystyle t_{ / }={\frac {\ln }{k}}} the half-life of a first order reaction is independent of its initial concentration and depends solely on the reaction rate constant, k. second order kinetics: in the second order reactions, the concentration of the reactant decrease following this formula: [ a ] = k t + [ a ] {\displaystyle {\frac { }{[a]}}=kt+{\frac { }{[a]_{ }}}} then, we replace [a] for [a] divided by in order to calculate the half-life of the reactant a and isolate the time of the half-life (t / ): t / = [ a ] k {\displaystyle t_{ / }={\frac { }{[a]_{ }k}}} as you can see, the half-life of the second order reactions depends on the initial concentration and rate constant. decay by two or more processes[edit] some quantities decay by two exponential-decay processes simultaneously. in this case, the actual half-life t ⁄ can be related to the half-lives t and t that the quantity would have if each of the decay processes acted in isolation: t / = t + t {\displaystyle {\frac { }{t_{ / }}}={\frac { }{t_{ }}}+{\frac { }{t_{ }}}} for three or more processes, the analogous formula is: t / = t + t + t + ⋯ {\displaystyle {\frac { }{t_{ / }}}={\frac { }{t_{ }}}+{\frac { }{t_{ }}}+{\frac { }{t_{ }}}+\cdots } for a proof of these formulas, see exponential decay § decay by two or more processes. examples[edit] half-life demonstrated using dice in a classroom experiment further information: exponential decay § applications and examples there is a half-life describing any exponential-decay process. for example: as noted above, in radioactive decay the half-life is the length of time after which there is a % chance that an atom will have undergone nuclear decay. it varies depending on the atom type and isotope, and is usually determined experimentally. see list of nuclides. the current flowing through an rc circuit or rl circuit decays with a half-life of ln( )rc or ln( )l/r, respectively. for this example the term half time tends to be used, rather than "half-life", but they mean the same thing. in a chemical reaction, the half-life of a species is the time it takes for the concentration of that substance to fall to half of its initial value. in a first-order reaction the half-life of the reactant is ln( )/λ, where λ is the reaction rate constant. in non-exponential decay[edit] the term "half-life" is almost exclusively used for decay processes that are exponential (such as radioactive decay or the other examples above), or approximately exponential (such as biological half-life discussed below). in a decay process that is not even close to exponential, the half-life will change dramatically while the decay is happening. in this situation it is generally uncommon to talk about half-life in the first place, but sometimes people will describe the decay in terms of its "first half-life", "second half-life", etc., where the first half-life is defined as the time required for decay from the initial value to %, the second half-life is from % to %, and so on.[ ] in biology and pharmacology[edit] see also: biological half-life a biological half-life or elimination half-life is the time it takes for a substance (drug, radioactive nuclide, or other) to lose one-half of its pharmacologic, physiologic, or radiological activity. in a medical context, the half-life may also describe the time that it takes for the concentration of a substance in blood plasma to reach one-half of its steady-state value (the "plasma half-life"). the relationship between the biological and plasma half-lives of a substance can be complex, due to factors including accumulation in tissues, active metabolites, and receptor interactions.[ ] while a radioactive isotope decays almost perfectly according to so-called "first order kinetics" where the rate constant is a fixed number, the elimination of a substance from a living organism usually follows more complex chemical kinetics. for example, the biological half-life of water in a human being is about to days,[ ] though this can be altered by behavior and other conditions. the biological half-life of caesium in human beings is between one and four months. the concept of a half-life has also been utilized for pesticides in plants,[ ] and certain authors maintain that pesticide risk and impact assessment models rely on and are sensitive to information describing dissipation from plants.[ ] in epidemiology, the concept of half-life can refer to the length of time for the number of incident cases in a disease outbreak to drop by half, particularly if the dynamics of the outbreak can be modeled exponentially.[ ][ ] see also[edit] half time (physics) list of radioactive nuclides by half-life mean lifetime median lethal dose references[edit] ^ john ayto, th century words ( ), cambridge university press. ^ muller, richard a. (april , ). physics and technology for future presidents. princeton university press. pp. – . isbn . ^ chivers, sidney (march , ). "re: what happens during half-lifes [sic] when there is only one atom left?". madsci.org. ^ "radioactive-decay model". exploratorium.edu. retrieved - - . ^ wallin, john (september ). "assignment # : data, simulations, and analytic science in decay". astro.glu.edu. archived from the original on - - .cs maint: unfit url (link) ^ a b rösch, frank (september , ). nuclear- and radiochemistry: introduction. . walter de gruyter. isbn - - - - . ^ jonathan crowe; tony bradshaw ( ). chemistry for the biosciences: the essential concepts. p. . isbn . ^ lin vw; cardenas dd ( ). spinal cord medicine. demos medical publishing, llc. p. . isbn - - - - . ^ pang, xiao-feng ( ). water: molecular structure and properties. new jersey: world scientific. p. . isbn . ^ australian pesticides and veterinary medicines authority ( march ). "tebufenozide in the product mimic wp insecticide, mimic sc insecticide". australian government. retrieved april . ^ fantke, peter; gillespie, brenda w.; juraske, ronnie; jolliet, olivier ( july ). "estimating half-lives for pesticide dissipation from plants". environmental science & technology. ( ): – . bibcode: enst... . f. doi: . /es p. pmid . ^ balkew, teshome mogessie (december ). the sir model when s(t) is a multi-exponential function (thesis). east tennessee state university. ^ ireland, mw, ed. ( ). the medical department of the united states army in the world war, vol. ix: communicable and other diseases. washington: u.s.: u.s. government printing office. pp. – . external links[edit] look up half-life in wiktionary, the free dictionary. wikimedia commons has media related to half times. welcome to nucleonica, nucleonica.net (archived ) wiki: decay engine, nucleonica.net (archived ) system dynamics – time constants, bucknell.edu researchers nikhef and uva measure slowest radioactive decay ever: xe- with billion trillion years v t e radiation (physics and health) main articles non-ionizing radiation acoustic radiation force infrared light starlight sunlight microwave radio waves ultraviolet ionizing radiation radioactive decay cluster decay background radiation alpha particle beta particle gamma ray cosmic ray neutron radiation nuclear fission nuclear fusion nuclear reactors nuclear weapons particle accelerators radioactive materials x-ray earth's energy budget electromagnetic radiation synchrotron radiation thermal radiation black-body radiation particle radiation gravitational radiation cosmic background radiation cherenkov radiation askaryan radiation bremsstrahlung unruh radiation dark radiation radiation and health radiation syndrome acute chronic health physics dosimetry electromagnetic radiation and health laser safety lasers and aviation safety medical radiography mobile phone radiation and health radiation protection radiation therapy radioactivity in the life sciences radioactive contamination radiobiology biological dose units and quantities wireless electronic devices and health radiation heat-transfer related articles half-life nuclear physics radioactive source radiation hardening list of civilian radiation accidents costa rica accident goiânia accident moroccan accident zaragoza accident see also: the categories radiation effects, radioactivity, radiobiology, and radiation protection authority control gnd: - ma: , retrieved from "https://en.wikipedia.org/w/index.php?title=half-life&oldid= " categories: chemical kinetics exponentials radioactivity hidden categories: cs maint: unfit url articles with short description short description is different from wikidata articles to be expanded from july commons category link is on wikidata wikipedia articles with gnd identifiers wikipedia articles with ma identifiers wikipedia articles with multiple identifiers navigation menu personal tools not logged in talk contributions create account log in namespaces article talk variants views read edit view history more search navigation main page contents current events random article about wikipedia contact us donate contribute help learn to edit community portal recent changes upload file tools what links here related changes upload file special pages permanent link page information cite this page wikidata item print/export download as pdf printable version in other projects wikimedia commons languages afrikaans العربية aragonés asturianu বাংলা bân-lâm-gú Беларуская भोजपुरी Български bosanski català Чӑвашла Čeština cymraeg dansk الدارجة deutsch eesti Ελληνικά español esperanto euskara فارسی français gaeilge galego 贛語 한국어 हिन्दी hrvatski bahasa indonesia Íslenska italiano עברית ಕನ್ನಡ ქართული Қазақша kiswahili kreyòl ayisyen latviešu lietuvių limburgs magyar Македонски മലയാളം bahasa melayu nederlands 日本語 nordfriisk norsk bokmål norsk nynorsk occitan oʻzbekcha/ўзбекча پنجابی plattdüütsch polski português română runa simi Русский simple english slovenčina slovenščina کوردی Српски / srpski srpskohrvatski / српскохрватски suomi svenska தமிழ் Татарча/tatarça తెలుగు ไทย türkçe Українська اردو tiếng việt 吴语粵語中文 edit links this page was last edited on march , at : (utc). text is available under the creative commons attribution-sharealike license; additional terms may apply. by using this site, you agree to the terms of use and privacy policy. wikipedia® is a registered trademark of the wikimedia foundation, inc., a non-profit organization. privacy policy about wikipedia disclaimers contact wikipedia mobile view developers statistics cookie statement the age of surveillance capitalism - wikipedia the age of surveillance capitalism from wikipedia, the free encyclopedia jump to navigation jump to search book published in the age of surveillance capitalism front cover author shoshana zuboff subject politics, cybersecurity publisher profile books publication date january , isbn the age of surveillance capitalism is a non-fiction book by professor shoshana zuboff which looks at the development of digital companies like google and amazon, and suggests that their business models represent a new form of capitalist accumulation that she calls "surveillance capitalism".[ ][ ] while industrial capitalism exploited and controlled nature with devastating consequences, surveillance capitalism exploits and controls human nature with a totalitarian order as the endpoint of the development.[ ] premise[edit] zuboff states that surveillance capitalism "unilaterally claims human experience as free raw material for translation into behavioural data [which] are declared as a proprietary behavioural surplus, fed into advanced manufacturing processes known as ‘machine intelligence’, and fabricated into prediction products that anticipate what you will do now, soon, and later." she states that these new capitalist products "are traded in a new kind of marketplace that i call behavioural futures markets."[ ] in a capitalist society, information, such as a users likes and dislikes, observed from accessing a platform like facebook is information that can be freely used by that platform to better the experience of a user by feeding them information that data obtained from their previous activity would have shown them to be interested in. this in many ways can be done through the use of an algorithm that automatically filters out information. the danger of surveillance capitalism is that platforms and tech companies are entitled to this information because it is free for them to access. there is very little supervision by governments and users themselves. because of this, there has been backlash on how these companies have used the information gathered. for example, google, which is said to be “the pioneer of surveillance capitalism”, zuboff ( )[ ] introduced a feature that used “commercial models…discovered by people in a time and place”, zuboff ( ).[ ] this means that not only are commercials being specifically targeted to you through your phone, but now work hand in hand with your environment and habits such as being shown an advertisement of a local bar when walking around downtown in the evening. advertising attempts this technical and specific can easily have an impact on the one's decision-making process in the activities they choose and in political decisions. thus the idea that these companies seemingly go unchecked whilst having the power to observe and control thinking is one of the many reasons tech companies such as google themselves are under so much scrutiny. furthermore, the freedom allotted to tech companies comes from the idea that “surveillance capitalism does not abandon established capitalist ‘laws’ such as competitive production, profit maximization, productivity and growth”, zuboff ( ),[ ] as they are principles any business in a capitalistic society should aim to excel in, in order to be competitive. zuboff ( )[ ] claims in an article that “new logic accumulation…introduces its own laws of motion”. in other words, this is a new phenomenon in capitalistic operations that should be treated as such and be instilled with its own specific restrictions and limitations. lastly, as invasive as platforms have been in terms of accumulating information, they have also led to what is now called a “sharing economy”, van dijck ( )[ ] in which digital information can be obtained by individuals carrying out their own surveillance capitalism through the aid of platforms themselves. thus “individuals can greatly benefit from this transformation because it empowers them to set up business”, van dijck ( ).[ ] small businesses will also benefit in potentially growing faster than they would have without knowing consumer demands and wants. this leaves surveillance capitalism as an exceptionally useful tool for businesses, but also an invasion of privacy to users. reception[edit] the new yorker listed the age of surveillance capitalism as one of its top non-fiction books of .[ ] former president of the united states barack obama also listed it as one of his favourite books of , which journalism researcher avi asher-schapiro noted as an interesting choice, given that the book heavily criticises the "revolving door of personnel who migrated between google & the obama admin”.[ ] sam dibella, writing for the lse blog, criticised the book's approach which could "inspire paralysis rather than praxis when it comes to forging collective action to counter systematic corporate surveillance."[ ] the financial times called the book a "masterwork of original thinking and research".[ ] references[edit] ^ bridle, james ( february ). "the age of surveillance capitalism by shoshana zuboff review – we are the pawns". the guardian. issn - . retrieved - - – via www.theguardian.com. cs maint: discouraged parameter (link) ^ naughton, john ( january ). "'the goal is to automate us': welcome to the age of surveillance capitalism". the observer. issn - . retrieved - - – via www.theguardian.com. cs maint: discouraged parameter (link) ^ "the new tech totalitarianism". www.newstatesman.com. retrieved - - . ^ naughton, john ( - - ). "'the goal is to automate us': welcome to the age of surveillance capitalism". the observer. issn - . retrieved - - . ^ a b c d zuboff, shoshana; möllers, norma; murakami wood, david; lyon, david ( - - ). "surveillance capitalism: an interview with shoshana zuboff". surveillance & society. ( / ): – . doi: . /ss.v i / . . issn - . ^ a b van dijck, josé; poell, thomas; de waal, martijn ( - - ). "the platform society". oxford scholarship online. doi: . /oso/ . . . isbn . ^ yorker, the new ( - - ). "our favorite nonfiction books of ". the new yorker (serial). issn - x. retrieved - - . ^ binder, matt. "obama praises book that slams his white house for its google relationship". mashable. retrieved - - . ^ november th; reviews, |book; democracy; comments, culture| ( - - ). "book review: the age of surveillance capitalism: the fight for a human future at the new frontier of power by shoshana zuboff". usapp. retrieved - - . ^ graphics, ft interactive. "the age of surveillance capitalism by shoshana zuboff". ft business book of the year award. retrieved - - . retrieved from "https://en.wikipedia.org/w/index.php?title=the_age_of_surveillance_capitalism&oldid= " categories: american non-fiction books non-fiction books books critical of capitalism hidden categories: cs maint: discouraged parameter articles with short description short description matches wikidata navigation menu personal tools not logged in talk contributions create account log in namespaces article talk variants views read edit view history more search navigation main page contents current events random article about wikipedia contact us donate contribute help learn to edit community portal recent changes upload file tools what links here related changes upload file special pages permanent link page information cite this page wikidata item print/export download as pdf printable version languages italiano edit links this page was last edited on april , at : (utc). text is available under the creative commons attribution-sharealike license; additional terms may apply. by using this site, you agree to the terms of use and privacy policy. wikipedia® is a registered trademark of the wikimedia foundation, inc., a non-profit organization. privacy policy about wikipedia disclaimers contact wikipedia mobile view developers statistics cookie statement unix philosophy - wikipedia unix philosophy from wikipedia, the free encyclopedia jump to navigation jump to search philosophy on developing software ken thompson and dennis ritchie, key proponents of the unix philosophy the unix philosophy, originated by ken thompson, is a set of cultural norms and philosophical approaches to minimalist, modular software development. it is based on the experience of leading developers of the unix operating system. early unix developers were important in bringing the concepts of modularity and reusability into software engineering practice, spawning a "software tools" movement. over time, the leading developers of unix (and programs that ran on it) established a set of cultural norms for developing software; these norms became as important and influential as the technology of unix itself; this has been termed the "unix philosophy." the unix philosophy emphasizes building simple, short, clear, modular, and extensible code that can be easily maintained and repurposed by developers other than its creators. the unix philosophy favors composability as opposed to monolithic design. contents origin the unix programming environment program design in the unix environment doug mcilroy on unix programming do one thing and do it well eric raymond's unix rules mike gancarz: the unix philosophy "worse is better" criticism see also notes references external links origin[edit] the unix philosophy is documented by doug mcilroy[ ] in the bell system technical journal from :[ ] make each program do one thing well. to do a new job, build afresh rather than complicate old programs by adding new "features". expect the output of every program to become the input to another, as yet unknown, program. don't clutter output with extraneous information. avoid stringently columnar or binary input formats. don't insist on interactive input. design and build software, even operating systems, to be tried early, ideally within weeks. don't hesitate to throw away the clumsy parts and rebuild them. use tools in preference to unskilled help to lighten a programming task, even if you have to detour to build the tools and expect to throw some of them out after you've finished using them. it was later summarized by peter h. salus in a quarter-century of unix ( ):[ ] write programs that do one thing and do it well. write programs to work together. write programs to handle text streams, because that is a universal interface. in their award-winning unix paper of [citation needed], ritchie and thompson quote the following design considerations:[ ] make it easy to write, test, and run programs. interactive use instead of batch processing. economy and elegance of design due to size constraints ("salvation through suffering"). self-supporting system: all unix software is maintained under unix. the whole philosophy of unix seems to stay out of assembler. — michael sean mahoney[ ] the unix programming environment[edit] in their preface to the book, the unix programming environment, brian kernighan and rob pike, both from bell labs, give a brief description of the unix design and the unix philosophy:[ ] rob pike, co-author of the unix programming environment even though the unix system introduces a number of innovative programs and techniques, no single program or idea makes it work well. instead, what makes it effective is the approach to programming, a philosophy of using the computer. although that philosophy can't be written down in a single sentence, at its heart is the idea that the power of a system comes more from the relationships among programs than from the programs themselves. many unix programs do quite trivial things in isolation, but, combined with other programs, become general and useful tools. the authors further write that their goal for this book is "to communicate the unix programming philosophy."[ ] program design in the unix environment[edit] brian kernighan has written at length about the unix philosophy in october , brian kernighan and rob pike published a paper called program design in the unix environment. in this paper, they criticize the accretion of program options and features found in some newer unix systems such as . bsd and system v, and explain the unix philosophy of software tools, each performing one general function:[ ] much of the power of the unix operating system comes from a style of program design that makes programs easy to use and, more important, easy to combine with other programs. this style has been called the use of software tools, and depends more on how the programs fit into the programming environment and how they can be used with other programs than on how they are designed internally. [...] this style was based on the use of tools: using programs separately or in combination to get a job done, rather than doing it by hand, by monolithic self-sufficient subsystems, or by special-purpose, one-time programs. the authors contrast unix tools such as cat, with larger program suites used by other systems.[ ] the design of cat is typical of most unix programs: it implements one simple but general function that can be used in many different applications (including many not envisioned by the original author). other commands are used for other functions. for example, there are separate commands for file system tasks like renaming files, deleting them, or telling how big they are. other systems instead lump these into a single "file system" command with an internal structure and command language of its own. (the pip file copy program found on operating systems like cp/m or rsx- is an example.) that approach is not necessarily worse or better, but it is certainly against the unix philosophy. doug mcilroy on unix programming[edit] doug mcilroy (left) with dennis ritchie mcilroy, then head of the bell labs computing sciences research center, and inventor of the unix pipe,[ ] summarized the unix philosophy as follows:[ ] this is the unix philosophy: write programs that do one thing and do it well. write programs to work together. write programs to handle text streams, because that is a universal interface. beyond these statements, he has also emphasized simplicity and minimalism in unix programming:[ ] the notion of "intricate and beautiful complexities" is almost an oxymoron. unix programmers vie with each other for "simple and beautiful" honors — a point that's implicit in these rules, but is well worth making overt. conversely, mcilroy has criticized modern linux as having software bloat, remarking that, "adoring admirers have fed linux goodies to a disheartening state of obesity."[ ] he contrasts this with the earlier approach taken at bell labs when developing and revising research unix:[ ] everything was small... and my heart sinks for linux when i see the size of it. [...] the manual page, which really used to be a manual page, is now a small volume, with a thousand options... we used to sit around in the unix room saying, 'what can we throw out? why is there this option?' it's often because there is some deficiency in the basic design — you didn't really hit the right design point. instead of adding an option, think about what was forcing you to add that option. do one thing and do it well[edit] as stated by mcilroy, and generally accepted throughout the unix community, unix programs have always been expected to follow the concept of dotadiw, or "do one thing and do it well." there are limited sources for the acronym dotadiw on the internet, but it is discussed at length during the development and packaging of new operating systems, especially in the linux community. patrick volkerding, the project lead of slackware linux, invoked this design principle in a criticism of the systemd architecture, stating that, "attempting to control services, sockets, devices, mounts, etc., all within one daemon flies in the face of the unix concept of doing one thing and doing it well."[ ] eric raymond's unix rules[edit] in his book the art of unix programming that was first published in ,[ ] eric s. raymond, an american programmer and open source advocate, summarizes the unix philosophy as kiss principle of "keep it simple, stupid."[ ] he provides a series of design rules:[ ] build modular programs write readable programs use composition separate mechanisms from policy write simple programs write small programs write transparent programs write robust programs make data complicated when required, not the program build on potential users' expected knowledge avoid unnecessary output write programs which fail in a way that is easy to diagnose value developer time over machine time write abstract programs that generate code instead of writing code by hand prototype software before polishing it write flexible and open programs make the program and protocols extensible. mike gancarz: the unix philosophy[edit] in , mike gancarz (a member of the team that designed the x window system), drew on his own experience with unix, as well as discussions with fellow programmers and people in other fields who depended on unix, to produce the unix philosophy which sums it up in nine paramount precepts: small is beautiful. make each program do one thing well. build a prototype as soon as possible. choose portability over efficiency. store data in flat text files. use software leverage to your advantage. use shell scripts to increase leverage and portability. avoid captive user interfaces. make every program a filter. "worse is better"[edit] main article: worse is better richard p. gabriel suggests that a key advantage of unix was that it embodied a design philosophy he termed "worse is better", in which simplicity of both the interface and the implementation are more important than any other attributes of the system—including correctness, consistency, and completeness. gabriel argues that this design style has key evolutionary advantages, though he questions the quality of some results. for example, in the early days unix used a monolithic kernel (which means that user processes carried out kernel system calls all on the user stack). if a signal was delivered to a process while it was blocked on a long-term i/o in the kernel, then what should be done? should the signal be delayed, possibly for a long time (maybe indefinitely) while the i/o completed? the signal handler could not be executed when the process was in kernel mode, with sensitive kernel data on the stack. should the kernel back-out the system call, and store it, for replay and restart later, assuming that the signal handler completes successfully? in these cases ken thompson and dennis ritchie favored simplicity over perfection. the unix system would occasionally return early from a system call with an error stating that it had done nothing—the "interrupted system call", or an error number (eintr) in today's systems. of course the call had been aborted in order to call the signal handler. this could only happen for a handful of long-running system calls such as read(), write(), open(), and select(). on the plus side, this made the i/o system many times simpler to design and understand. the vast majority of user programs were never affected because they did not handle or experience signals other than sigint and would die right away if one was raised. for the few other programs—things like shells or text editors that respond to job control key presses—small wrappers could be added to system calls so as to retry the call right away if this eintr error was raised. thus, the problem was solved in a simple manner. criticism[edit] in a article entitled "the truth about unix: the user interface is horrid"[ ] published in datamation, don norman criticized the design philosophy of unix for its lack of concern for the user interface. writing from his background in cognitive science and from the perspective of the then-current philosophy of cognitive engineering,[ ] he focused on how end-users comprehend and form a personal cognitive model of systems—or, in the case of unix, fail to understand, with the result that disastrous mistakes (such as losing an hour's worth of work) are all too easy. see also[edit] cognitive engineering unix architecture minimalism (computing) software engineering kiss principle hacker ethic list of software development philosophies everything is a file worse is better notes[edit] ^ a b c d e raymond, eric s. ( ). "basics of the unix philosophy". the art of unix programming. addison-wesley professional (published - - ). isbn - - - . retrieved - - . ^ doug mcilroy, e. n. pinson, b. a. tague ( july ). "unix time-sharing system: foreword". the bell system technical journal. bell laboratories: – .cs maint: multiple names: authors list (link) ^ dennis ritchie; ken thompson ( ), "the unix time-sharing system" (pdf), communications of the acm, ( ): – , doi: . / . , s cid ^ a b "an oral history of unix". princeton university history of science. ^ a b kernighan, brian w. pike, rob. the unix programming environment. . viii ^ a b rob pike; brian w. kernighan (october ). "program design in the unix environment" (pdf). ^ dennis ritchie ( ), "the evolution of the unix time-sharing system" (pdf), at&t bell laboratories technical journal, ( ): – , doi: . /j. - . .tb .x ^ douglas mcilroy. "remarks for japan prize award ceremony for dennis ritchie, may , , murray hill, nj" (pdf). retrieved - - . ^ bill mcgonigle. "ancestry of linux — how the fun began ( )". retrieved - - . ^ "interview with patrick volkerding of slackware". linuxquestions.org. - - . retrieved - - . ^ raymond, eric ( - - ). the art of unix programming. addison-wesley. isbn - - - . retrieved - - . ^ raymond, eric ( - - ). "the unix philosophy in one lesson". the art of unix programming. addison-wesley. isbn - - - . retrieved - - . ^ norman, don ( ). "the truth about unix: the user interface is horrid" (pdf). datamation. ( ). references[edit] the unix programming environment by brian kernighan and rob pike, program design in the unix environment – the paper by pike and kernighan that preceded the book. notes on programming in c, rob pike, september , a quarter century of unix, peter h. salus, addison-wesley, may , ( isbn - - - ) philosophy — from the art of unix programming, eric s. raymond, addison-wesley, september , ( isbn - - - ) final report of the multics kernel design project by m. d. schroeder, d. d. clark, j. h. saltzer, and d. h. wells, . the unix philosophy, mike gancarz, isbn - - - external links[edit] basics of the unix philosophy – by catb.org the unix philosophy: a brief introduction – by the linux information project (linfo) why the unix philosophy still matters retrieved from "https://en.wikipedia.org/w/index.php?title=unix_philosophy&oldid= " categories: software development philosophies unix hidden categories: cs maint: multiple names: authors list articles with short description short description matches wikidata all articles with unsourced statements articles with unsourced statements from march navigation menu personal tools not logged in talk contributions create account log in namespaces article talk variants views read edit view history more search navigation main page contents current events random article about wikipedia contact us donate contribute help learn to edit community portal recent changes upload file tools what links here related changes upload file special pages permanent link page information cite this page wikidata item print/export download as pdf printable version languages العربية Čeština deutsch español فارسی français 한국어 italiano 日本語 norsk bokmål português Русский 中文 edit links this page was last edited on march , at : (utc). text is available under the creative commons attribution-sharealike license; additional terms may apply. by using this site, you agree to the terms of use and privacy policy. wikipedia® is a registered trademark of the wikimedia foundation, inc., a non-profit organization. privacy policy about wikipedia disclaimers contact wikipedia mobile view developers statistics cookie statement futurearch, or the future of archives... futurearch, or the future of archives... a place for thoughts on hybrid archives and manuscripts at the bodleian library. this blog is no longer being updated born digital: guidance for donors, dealers, and archival repositories digital preservation: what i wish i knew before i started transcribe at the archive atlas of digital damages dayofdigitalarchives sprucing up the tikafileidentifier spruce mashup: th- th april media recognition: dv part media recognition: dv part media recognition: dv part digital preservation: what i wish i knew before i started what is ‘the future of the past of the web’? day of digital archives, another source for old software comparing software tools mobile forensics preserving born-digital video - what are good practices? hidden pages media recognition - floppy disks part preserving digital sound and vision: a briefing th april sharp font writer files got any older? world backup day advisory board meeting, march randori - wikipedia randori from wikipedia, the free encyclopedia jump to navigation jump to search free-style practice in japanese martial arts this article needs additional citations for verification. please help improve this article by adding citations to reliable sources. unsourced material may be challenged and removed. find sources: "randori" – news · newspapers · books · scholar · jstor (december ) (learn how and when to remove this template message) randori japanese name kanji 乱取り hiragana らんどり transcriptions revised hepburn randori randori (乱取り) is a term used in japanese martial arts to describe free-style practice (sparring). the term denotes an exercise in 取り tori, applying technique to a random ( 乱 ran) succession of uke attacks. the actual connotation of randori depends on the martial art it is used in. in judo, jujutsu, and shodokan aikido, among others, it most often refers to one-on-one sparring where partners attempt to resist and counter each other's techniques. in other styles of aikido, in particular aikikai, it refers to a form of practice in which a designated aikidoka defends against multiple attackers in quick succession without knowing how they will attack or in what order. contents in japan in judo in tenshin aikido in kendo in karate in ninjutsu see also references external links in japan[edit] the term is used in aikido, judo, and brazilian jiu-jitsu dojos outside japan. in japan, this form of practice is called taninzu-gake (多人数掛け), which literally means multiple attackers. in judo[edit] the term was described by jigoro kano, the founder of judo, in a speech at the los angeles olympic games: "randori, meaning "free exercise", is practiced under conditions of actual contest. it includes throwing, choking, holding the opponent down, and bending or twisting of the arms. the two combatants may use whatever methods they like provided they do not hurt each other and obey the rules of judo concerning etiquette, which are essential to its proper working." [ ] there are types of randori.[ ] [ ] in tenshin aikido[edit] in steven seagal's tenshin aikido federation (affiliated with the aikikai), randori is different from that of aikikai, in that the attackers can do anything to the defender (e.g. punch, grab, kick, etc.), and the randori continues on the ground until a pin. in kendo[edit] in kendo, jigeiko means "friendly" free combat, as in competition, but without counting points. in karate[edit] although in karate the word kumite is usually reserved for sparring, some schools also employ the term randori with regard to "mock-combat" in which both karateka move with speed, parrying and attacking with all four limbs (including knees, elbows, etc.). in these schools, the distinction between randori and kumite is that in randori, the action is uninterrupted when a successful technique is applied. (also known as ju kumite or soft sparring.) in ninjutsu[edit] randori is also practiced in bujinkan ninjutsu and usually represented to the practitioner when he reaches the "shodan" level. in ninjutsu, randori puts the practitioner in a position where he is armed or unarmed and is attacked by multiple attackers. see also[edit] kata sparring randori-no-kata references[edit] ^ original text of this speech available at the judo information site at http://judoinfo.com/kano .htm ^ ohlenkamp, neil ( may ). black belt judo. new holland. isbn – via google books. ^ tello, rodolfo ( august ). judo: seven steps to black belt (an introductory guide for beginners). amakella publishing. isbn – via google books. external links[edit] judo information site youtube: randori in tenshin aikido v t e japanese martial arts lists list of japanese martial arts list of koryū schools of martial arts ko-budō battōjutsu bōjutsu hojōjutsu iaijutsu jōjutsu jujutsu jittejutsu kenjutsu kyūjutsu naginatajutsu ninjutsu shurikenjutsu sōjutsu gendai budō aikido daitō-ryū aiki-jūjutsu iaido judo karate kendo kyūdō nippon kempo shorinji kempo sumo terms aiki budō dōjō kuzushi maai mushin randori uchi-deshi zanshin japanese martial arts • japan martial arts v t e martial arts list of styles history timeline hard and soft regional origin china europe india indonesia japan korea philippines unarmed techniques chokehold clinch footwork elbow strike headbutt hold kick knee strike joint lock punch sweep takedown throw weapons duel melee weapons knife fighting stick-fighting swordsmanship ranged weapons archery shooting training kata boxing gloves practice weapon punching bag pushing hands randori sparring grappling brazilian jiu-jitsu judo jujutsu sambo shuai jiao sumo wrestling striking bando boxing capoeira karate kickboxing lethwei muay thai pradal serey sanshou savate taekwondo vovinam internal aikido aikijutsu baguazhang tai chi xing yi quan full contact / combat sports professional boxing professional kickboxing knockdown karate mixed martial arts pankration submission wrestling vale tudo self-defense / combatives arnis bartitsu hapkido kajukenbo jieitaikakutōjutsu krav maga mcmap pencak silat systema wing chun legal aspects silat melayu eclectic / hybrids american kenpo chun kuk do jeet kune do shooto shorinji kempo unifight entertainment beat 'em up fighting game martial arts film (chanbara) professional wrestling stage combat wuxia portal outline retrieved from "https://en.wikipedia.org/w/index.php?title=randori&oldid= " categories: aikido japanese martial arts japanese martial arts terminology judo mock combat training hidden categories: articles with short description short description is different from wikidata articles needing additional references from december all articles needing additional references articles containing japanese-language text navigation menu personal tools not logged in talk contributions create account log in namespaces article talk variants views read edit view history more search navigation main page contents current events random article about wikipedia contact us donate contribute help learn to edit community portal recent changes upload file tools what links here related changes upload file special pages permanent link page information cite this page wikidata item print/export download as pdf printable version languages বাংলা deutsch español français italiano עברית nederlands 日本語 polski português Русский Српски / srpski suomi svenska Українська edit links this page was last edited on february , at : (utc). text is available under the creative commons attribution-sharealike license; additional terms may apply. by using this site, you agree to the terms of use and privacy policy. wikipedia® is a registered trademark of the wikimedia foundation, inc., a non-profit organization. privacy policy about wikipedia disclaimers contact wikipedia mobile view developers statistics cookie statement nfts: crypto grifters try to scam artists, again – attack of the foot blockchain skip to content attack of the foot blockchain blockchain and cryptocurrency news and analysis by david gerard about the author attack of the foot blockchain: the book book extras business bafflegab, but on the blockchain buterin’s quantum quest dogecoin ethereum smart contracts in practice icos: magic beans and bubble machines imogen heap: “tiny human”. total sales: $ . index libra shrugged: how facebook tried to take over the money my cryptocurrency and blockchain commentary and writing for others press coverage: attack of the foot blockchain press coverage: libra shrugged table of contents the conspiracy theory economics of bitcoin the dao: the steadfast iron will of unstoppable code search for: main menu nfts: crypto grifters try to scam artists, again th march th march - by david gerard - comments. non-fungible tokens, or nfts, are the crypto hype for — since defi ran out of steam in , and bitcoin’s pumped bubble seems to be deflating. the scam is to sell nfts to artists as a get-rich-quick scheme, to make life-changing money. there’s a gusher of money out there! you just create a token! and any number of crypto grifters would be delighted to assist you. for a small consideration. it’s con men with a new variety of magic beans to feed the bubble machine — and artists are their excuse this time. the nft grift works like this: tell artists there’s a gusher of free money! they need to buy into crypto to get the gusher of free money. they become crypto advocates, and make excuses for proof-of-work and so on. a few artists really are making life-changing money from this! you probably won’t be one of them. in a nicer, happier world, nfts would be fun little things you could make and collect and trade, and it’d be great. it’s a pity this is crypto. what is an nft? an nft is a crypto-token on a blockchain. the token is virtual — the thing you own is a cryptographic key to a particular address on the blockchain — but legally, it’s property that you can buy, own or sell like any other property. most crypto-tokens, such as bitcoins, are “fungible” — e.g., you mostly don’t care which particular bitcoins you have, only how much bitcoin you have. non-fungible tokens are a bit different. each one is unique — and can be used as an identifier for an individual object. the nft can contain a web address, or maybe just a number, that points somewhere else. an nft is just a pointer. if the place the nft points to is a site that claims to sell nfts that represent artworks — then you have what’s being called crypto-art! note that it’s only the token that’s non-fungible — the art it points to is on a website, under centralised control, and easily changeable. when i buy an nft, what do i get? the art itself is not in the blockchain — the nft is just a pointer to a piece of art on a website. you’re buying the key to a crypto-token. you’re not buying anything else. an nft doesn’t convey copyright, usage rights, moral rights, or any other rights, unless there’s an explicit licence saying so. it’s like a “certificate of authenticity” that’s in comic sans, and misspelt. at absolute best, you’re buying a piece of official merchandise — one that’s just a number pointing to a website. why is an nft? nfts exist so that the crypto grifters can have a new kind of magic bean to sell for actual money, and pretend they’re not selling magic beans. the purpose of nfts is to get you to give your money to crypto grifters. when the grifter has your money, the nft has done its job, and none of the fabulous claims about nfts need to work or be true past that point. nfts are entirely for the benefit of the crypto grifters. the only purpose the artists serve is as aspiring suckers to pump the concept of crypto — and, of course, to buy cryptocurrency to pay for “minting” nfts. sometimes the artist gets some crumbs to keep them pumping the concept of crypto. cryptokitties, in late , was the first popular nft. cryptokitties was largely fueled by bored holders of ether — the cryptocurrency for ethereum — spending their ether, that they had too much of to cash out easily, on some silly toys that they traded amongst themselves. since then, various marketers have tried to push the idea along. people pay real money for hats in video games, don’t they? then surely they’ll buy crypto tokens that allegedly represent their favourite commercial ip! these mostly haven’t taken off. the first real success is nba top shots, where you buy an official nba-marketed token that gives you a website trading card of a video snippet. this has taken off hugely. nba top shots has its own issues, which i’ll probably deal with in a later post. defi pumpers tried pushing nfts in october last year, but they couldn’t get the idea to stick. the recent bitcoin bubble feels like it’s running out of steam — so they’re pushing the nft idea again, and pumping it hard. with nba top shots and some heavily promoted big-money alleged sales, crypto art nfts are hitting the headlines. how do i make an nft? if you aren’t a technically-minded blockchain enthusiast, there are websites where you can “mint” an nft. first, you need to buy some ether. this covers the transaction fee to make your nft. you’ll need ethereum wallet software, probably metamask, which is a browser extension. how much do you need? well, guess and hope you’re lucky. ethereum transaction fees peaked at $ per transaction in february. lots of poor artists have tried making nfts and lost over $ they really couldn’t spare — so guess high! you might notice that this looks a lot like a vanity gallery scam, or pay-to-play. you’d be correct — the purpose is to suck your precious actual-money into the crypto economy. connect your ethereum wallet to one of the nft marketplaces. upload your file and its description. you have created a token! now you need to hope a bored crypto holder will buy it. what is “digital ownership”? without a specific contract saying otherwise, an nft does not grant ownership of the artwork it points to in any meaningful sense. all implications otherwise are lies to get your money. this is the “registration scam” — like selling your name on a star, or a square foot of land on the moon. musicians will know the “band name registry” scam, where the scammer sells something that they imply will work like a trademark on your name — but, of course, it doesn’t. (there have been multiple “register your band name on a blockchain” scams.) crypto grifters will talk about “digital ownership.” this is meaningless. the more detail you ask for what actual usable rights this “ownership” conveys, the vaguer the claims will get. the whole idea of bitcoin was property unconfiscatable by the government, that they could use as money. instead of a framework of laws and rights, they’d use … a blockchain! this notion is incoherent and stupid on multiple levels — money is a construct agreed upon in a society, property rights are a construct of law and social expectations — but it’s also what the bitcoiners believe and what they wanted. nfts try to justify themselves with variations on this claim as the marketing pitch. christie’s auction of an nft is a fabulous worked example. there’s a -page terms and conditions document, and if you wade through the circuitous verbiage, it finally admits that … you’re just buying the crypto-token itself: [christie’s, pdf, archive] you acknowledge that ownership of an nft carries no rights, express or implied, other than property rights for the lot (specifically, digital artwork tokenized by the nft). … you acknowledge and represent that there is substantial uncertainty as to the characterization of nfts and other digital assets under applicable law. the magic bean in question is bidding at $ million as i write this, which means christie’s stands to make about $ million commission. pretty good payday for a cryptographic hash. [christie’s] i don’t understand any of this. please explain it like i’m five. “would you like to watch your favourite cbeebies show — or would you like me to write on a piece of paper that you own the show? all you get is the piece of paper.” the trouble with explaining nfts to a five-year-old is that you’ll have a hard time convincing a five-year-old that this nonsense isn’t the nonsense it obviously is. it sounds unfathomably stupid because it’s unfathomably stupid. the k foundation burn a million nfts: crypto art’s ghastly co production proof-of-work is the reprehensible, planet-destroying mechanism that the ethereum and bitcoin blockchains use to decide who gets fresh ether or bitcoins. proof-of-work is inexcusable nonsense, and every single person making money in anything linked to ethereum or bitcoin should feel personal shame. (crypto grifters don’t possess a shame organ.) like bitcoin, ethereum uses an whole country’s worth of electricity just to keep running — and generates a country’s worth of co . the ethereum developers claim they’re totally moving off proof-of-work any day now — but they’ve been saying that since . crypto grifters making bad excuses for proof-of-work will often object to calculating their favourite magic bean’s per-transaction energy use, at all. the excuse is that adding more transactions doesn’t directly increase bitcoin or ethereum’s energy consumption. the actual reason is that the numbers for bitcoin and ethereum are bloody awful. [digiconomist; digiconomist] the grifters will routinely pretend it’s somehow impossible to do arithmetic, and divide the energy use by the work achieved with it — in the precise same manner we do for literally every other enterprise or industry that uses energy. but if you’re calculating energy efficiency — of bitcoin, ethereum, visa, twitter or banks — then taking the total energy used and dividing it by the total work done is the standard way to work that out. sites have sprung up to calculate the share of energy that crypto art spends. the site cryptoart.wtf picks a random piece of crypto art and calculates that transaction’s energy use. “these figures do not include the production or storage of the works, or even web hosting, but is simply for the act of using the pow ethereum blockchain to keep track of sales and activity.” the creator also has a blog post to explain the site, and address common bad excuses for proof-of-work. [cryptoart.wtf; medium] you may tell yourself “but my personal marginal effect is minimal” — but in that case, don’t pretend you’re not just another aspiring crypto grifter. there are other blockchains that don’t use proof-of-work. hardly anybody does nfts on these chains — almost nobody uses them, and the local cryptocurrency for your fees is a lot more work to get hold of. and even if you did use one of these other blockchains, all the other ways that nfts are a scam would still hold. but what about artists? they need money too artist pay is terrible. even quite successful artists whose names you know wonder if they could tap into the rich people status-and-vanity art market, and get life-changing money. (i’ve already seen one artist bedazzled by the prospect of nft money say that anyone who objects to crypto art must be a shill for big tech.) artists don’t know technology any more than anyone else does, so a lot of artists who tentatively essayed an nft were completely unaware of the ghastly co production involved in anything that touches cryptocurrency. several were shocked at the backlash over an issue they’d had no idea existed. famous artists are getting into nfts. grimes did an nft, and it’d be fair to say that elon musk’s partner isn’t going to be doing an nft for the money. even if it’s a bit at odds with her album about ecological collapse. but famous musicians have long had a habit of adopting some awful headline-friendly technology that’s utterly unready for prime time consumer use, in order to show that they are hep and up to speed with the astounding future. then they never speak of it again. remember björk’s cryptocurrency album in ? kings of leon are doing an nft of their new album — sort of. their page on nft site opensea suggests that you buy a digital download (not an nft), limited edition vinyl (not an nft), or a collectible artwork (a wallpaper). so what you’re actually buying is a vinyl record with a download, and in return, you not only give the band money, but hasten ecological collapse. some small artists have done very well indeed from nfts — and that’s excellent news! if you’ve made life-changing money from an nft, then that’s good for the world as well as for you — ‘cos now the money’s out of the hands of the crypto grifters. (for goodness’ sake, cash out now.) an important rule of crypto is: every number that can be faked is faked. nfts are the sort of con where a shill appears to make a ton of money, so you’ll think you can too. put a large price tag on your nft by buying it from yourself — then write a press release talking about your $ , sale, and you’re only out the transaction fee. journalists who can’t be bothered checking things will write this up without verifying that the buyer is a separate person who exists. just like the high-end art world! another thing that the high-end art world shares with crypto is money laundering. press coverage tends to focus on cultural value, and assume this stuff must be of artistic weight because someone spent a fortune on it. the part that functions as a money-laundering scam is only starting to get comment recently. [national law review, ; art & object, ] nfts will almost certainly be used for money laundering as well, because crypto has always been a favourite for that use case. banksying the unbanksied: fraudulent nfts there is no mechanism to ensure that an nft for an artwork is created by the artist. a lot of nfts are just straight-up fraud. if nfts weren’t a scam, there would be legal and technical safeguards to help ensure the nft was being created by someone who owned the work in question, to fend off scammers. but there aren’t any — the sites all work on the basis “we’ll clean it up later, maybe.” this is because nfts only exist to further the crypto grift. there are multiple nft sites — you could create an unlimited number of nfts that all claimed to be of a single particular work. there are a number of twitter bots that will make an nft of any tweet you point them at. the point is for the bot owner to make a commission from the sale of the nfts, before the suckers catch on. don’t expect twitter to do anything about these people — twitter ceo jack dorsey has a $ . million offer for an nft of his first tweet. the offer is from dorsey’s fellow crypto grifter justin sun. now, you might think these two massive crypto holders were just trying to get headlines for the nft market. [rolling stone] someone nfted all of dinosaur artist corbin rainbolt’s tweeted illustrations — and he took down the lot and put up watermarked versions. “i am not pleased that i have to take this sort of scorched earth policy with my artwork, frankly i am livid.” [twitter] you could go through and block and report all the twitter bots, though more will just spring up. [twitter] but think of all the good things you could do with nfts, you luddite when you point out that cryptocurrencies are terrible and nfts are a scam, crypto grifters will start talking about all the things that you could potentially do if nfts worked like they claim they do. this is a standard crypto grifter move — any clear miserable failure in the present will be answered with talking about the fabulous future! e.g., claiming bitcoin or blockchain promises will surely come true, because it’s just like the early internet. which, of course, it isn’t. what can artists and buyers do about fraudulent nfts? if the nft site has a copy of your artwork up, you can send a dmca notice to them, and to their upstream network provider. if the nft site is just claiming or implying that you created this nft when you did not, this is clearly fraudulent (misrepresentation, passing off) — but may be harder to get immediate action on. if you bought an nft thinking it was put up by the artist, and it wasn’t, then you’ve been defrauded, and should ask for a refund. if the nft site won’t refund you, then bring to bear absolutely everything you can on them. if the site is unresponsive to notices of fraud — which is quite common, because crypto grifters think “digital ownership” is a thing, and don’t care that other rights might exist in law or society — it is absolutely in order to shout from the rooftops that they are frauds, and blacken their name as best you can. contact their financial backers too. then talk about that as well. ask around to see if you have a lawyer friend, or a friend of a friend, who might be in a position to assist pro bono just because these grifters are that terrible. the most important thing for artists to do about nft fraud is to work to make nfts widely considered to be worthless, fraudulent magic beans, with massive co generation per transaction. this shouldn’t be terribly difficult, given that nfts are in fact worthless, fraudulent magic beans, with massive co generation per transaction. but is it art? you can tell that crypto art is definitely art, because so many proponents of it are insufferable manifesto bros. just the manifestos could cause runaway global warming from sheer volume of hot air. (“banksying the unbanksied” courtesy etienne beureux.) pleased to offer a nft version of neoreaction a basilisk, which you can obtain at https://t.co/spznjzigoi — el sandifer, rationality expert to the stars (@elsandifer) march , you claim to place such moral stock in "artists getting paid" yet do not subscribe to my patreon, curious — dr samantha keeper md (@samfatekeeper) march , we have a unique opportunity to help the planet and make culture better for future generations, and everyone can contribute simply by not giving a toss about nfts. — dan davies (@dsquareddigest) march , your subscriptions keep this site going. sign up today! share this: click to share on twitter (opens in new window) click to share on facebook (opens in new window) click to share on linkedin (opens in new window) click to share on reddit (opens in new window) click to share on telegram (opens in new window) click to share on hacker news (opens in new window) click to email this to a friend (opens in new window) taggedchristie'scorbin rainboltcryptokittiesethereumgrimesjack dorseyjustin sunkings of leonnba top shotsnftopenseaproof of work post navigation previous article news: india crypto ban, north korea, bitmex execs to appear, ibm blockchain dead, more mcafee charges next article foreign policy: it’s a $ million jpeg, but is it art? comments on “nfts: crypto grifters try to scam artists, again” adam achen says: th march at : am wait, so, nft don’t even typically include a license for use of the underlying?! reply david gerard says: th march at : am nope! note how even the christie’s contract basically says “we dunno wtf this thing is, have fun” reply k. paul says: th march at : am isn’t it curious that on the same day that billion tethers get minted on the tron blockchain, beeple’s nft gets sold for usd million worth of eth? apparently justin sun (founder of the tron blockchain) was the leading bidder until losing it to another crypto bro at the last bid. super curious, no? money laundering? reply david gerard says: th march at : am i’m sure it’s just coincidence, and that sun definitely didn’t snipe his own bid under another name for press release purposes. reply k. paul says: th march at : am lol reply david gerard says: th march at : pm it turns out it was bought by … a guy beeple was already in the crypto business with! https://amycastor.com/ / / /metakovan-the-mystery-beeple-art-buyer-and-his-nft-defi-scheme/ so the $ m (in eth) to christie’s is correctly viewed as a marketing expense reply k. paul says: th march at : am i think sun, beeple, christie’s, vignesh, musk, etc. are all working together to push nfts. it all just seems so… planned and organized in advance. look at what musk is doing now. meanwhile, tether printer goes brrrrrrrrr!!! wk says: th march at : pm thanks; this was an interesting read. i’ve been reading about “crypto” on and off for a while now, trying to understand what it’s all about because it seems like nonsense. my initial skepticism has so far been reinforced and i completely fail to see how bitcoin or any other digital currency is independent of actual existing hard currencies. this nft business ($ . m for a tweet?) is headscratchingly ridiculous. reply john s says: st march at : pm crypto is a perfect way to take money from “midwits.” lower iq people instinctively know it’s dumb and the barrier of entry keeps them out. genuinely smart people (i’m not a genius but i would place myself in this category) read all the claims and conclude that there is no intrinsic value, regardless of limits on supply etc. people in the middle read the claims and convince themselves they understand this stuff and the marketing (better than fiat, banks, libertarian utopia etc) are true and get burned. the people who make money in crypto are either insiders or they know it’s crap and sell it during bubble periods instead of holding with the expectation that the value will perpetually increase due to magical properties. reply ingvar says: th march at : pm jwz on nfts. worth a read, including the comments (which, frankly, is not something i am used to saying). reply blaise says: th march at : pm great work: i now have a much better understanding of nft and your “contrarian” view makes perfect sense. reply adam burns says: th march at : pm > it’s [nfts are] like a “certificate of authenticity” that’s in comic sans, and misspelt. written in crypto crayons. for the love of humanity! oh … but wait. check out these ‘humanitarians’ https://www.proofofhumanity.id/ reply jetblack says: rd march at : am just more proof of the unmitigated stupidity of the world we live in. wow. and grimes just sold a bunch of ntfs for a tidy sum. hmm… i wonder who bought those? is she connected to anyone with a lot of disposable income with an vested interest in bitcoin and crypto? reply alex says: th march at : am hello! i have a question, help me out. let’s say an artist put up his/her work in a conditional nft market, an auction started and he/she successfully sold it. after this event – what rights does the artist have towards the auctioned work? or the work is still the intellectual property of the artist? reply david gerard says: th march at : am all the rights, unless explicitly stated otherwise in the sale of the nft. the purchaser might try to claim implied rights – e.g. a limited right to reproduce the work for the purpose of saying “this is what i bought an nft of” – but not major rights like copyright or reproduction without an explicit license. though i am not your lawyer, so ask one if it’s important. reply leave a reply cancel reply your email address will not be published. required fields are marked * comment name * email * website notify me of follow-up comments by email. notify me of new posts by email. this site uses akismet to reduce spam. learn how your comment data is processed. search for: click here to get signed copies of the books! get blog posts by email! email address subscribe support this site on patreon! hack through the blockchain bafflegab: $ /month for early access to works in progress! $ /month for early access and even greater support! $ /month corporate rate, for your analyst newsletter budget! buy the books! libra shrugged us paperback uk/europe paperback isbn- : kindle: uk, us, australia, canada (and all other kindle stores) — no drm google play books (pdf) apple books kobo smashwords other e-book stores attack of the foot blockchain us paperback uk/europe paperback isbn- : kindle: uk, us, australia, canada (and all other kindle stores) — no drm google play books (pdf) apple books kobo smashwords other e-book stores available worldwide rss - posts rss - comments recent blog posts news: coinbase goes public, bitcoin hashrate goes down, nfts go down, proof-of-space trashes hard disk market stilgherrian: the pm dumb anarcho-capitalist blockchain scams with david gerard podcast: i don’t speak german # : crypto fascists, with david gerard desperate investors, neoliberalism and keynes: how to increase returns new york’s excelsior pass for covid- , on ibm blockchain: doing the wrong thing, badly excerpts from the book table of contents the conspiracy theory economics of bitcoin dogecoin buterin’s quantum quest icos: magic beans and bubble machines ethereum smart contracts in practice the dao: the steadfast iron will of unstoppable code business bafflegab, but on the blockchain imogen heap: “tiny human”. total sales: $ . index about press coverage for attack of the foot blockchain press coverage for libra shrugged my cryptocurrency and blockchain press commentary and writing facebook author page about the author contact the content of this site is journalism and personal opinion. nothing contained on this site is, or should be construed as providing or offering, investment, legal, accounting, tax or other advice. do not act on any opinion expressed here without consulting a qualified professional. i do not hold a position in any crypto asset or cryptocurrency or blockchain company. amazon product links on this site are affiliate links — as an amazon associate i earn from qualifying purchases. (this doesn’t cost you any extra.) copyright © – david gerard powered by wordpress and hitmag. send to email address your name your email address cancel post was not sent - check your email addresses! email check failed, please try again sorry, your blog cannot share posts by email. everybody's libraries everybody's libraries libraries for everyone, by everyone, shared with everyone, about everything public domain day : honoring a lost generation it&# ;s public domain day again. in much of europe, and other countries with &# ;life+ years&# ; copyright terms, works by authors who died in , such as george orwell, karin michaelis, george bernard shaw, and edna st. vincent millay, have joined &# ; continue reading &# ; counting down to in the public domain we&# ;re rapidly approaching another public domain day, the day at the start of the year when a year&# ;s worth of creative work joins the public domain. this will be the third year in a row that the us will have &# ; continue reading &# ; from our subjects to yours (and vice versa) (tl;dr: i&# ;m starting to implement services and publish data to support searching across library collections that use customized subject headings, such as the increasingly-adopted substitutes for lcsh terms like &# ;illegal aliens&# ;. read on for what i&# ;m doing, why, and where &# ; continue reading &# ; everybody&# ;s library questions: finding films in the public domain welcome to another installment of everybody&# ;s library questions, where i give answers to questions people ask me (in comments or email) that seem to be useful for general consumption. before i start, though, i want to put in a plug &# ; continue reading &# ; build a better registry: my intended comments to the library of congress on the next register of copyrights the library of congress is seeking public input on abilities and priorities desired for the next register of copyrights, who heads the copyright office, a department within the library of congress. the deadline for comments as i write this is &# ; continue reading &# ; welcome to everybody&# ;s online libraries as coronavirus infections spread throughout the world, lots of people are staying home to slow down the spread and save lives. in the us, many universities, schools, and libraries have closed their doors. (here&# ;s what happening at the library where &# ; continue reading &# ; public domain day : coming around again i&# ;m very happy for to be arriving. as the start of the s, it represents a new decade in which we can have a fresh start, and hope to make better decisions and have better outcomes than some of &# ; continue reading &# ; vision # : rhapsody in blue by george gershwin it&# ;s only a few hours from the new year where i write this, but before i ring in the new year, and a new year&# ;s worth of public domain material, i&# ;d like to put in a request for what music &# ; continue reading &# ; vision # : ding dong merrily on high by george ratcliffe woodward and others it&# ;s beginning to sound a lot like christmas everywhere i go. the library where i work had its holiday party earlier this week, where i joined librarian colleagues singing christmas, hanukkah, and winter-themed songs in a pick-up chorus. radio stations &# ; continue reading &# ; vision # : the most dangerous game by richard connell &# ;be a realist. the world is made up of two classes&# ;the hunters and the huntees. luckily, you and i are hunters.&# ; sanger rainsford speaks these words at the start of &# ;the most dangerous game&# ;, one of the most famous short &# ; continue reading &# ; fail!lab fail!lab technology, libraries and the future! luddites, trumpism and change: a crossroads for libraries &# ;globalization is a proxy for technology-powered capitalism, which tends to reward fewer and fewer members of society.&# ; &# ; om malik corner someone and they will react. we may be seeing this across the world as change, globalization, technology and economic dislocation force more and more people into the corner of benefit-nots. they are reacting out [&# ;] is d printing dying? inc.&# ;s john brandon recently wrote about the slow, sad, and ultimately predictable decline of d printing. uh, not so fast. d printing is just getting started. for libraries whose adopted mission is to introduce people to emerging technologies, this is a fantastic opportunity to do so. but it has to be done right. another dead [&# ;] the state of the library website t&# ;was a time when the library website was an abomination. those dark days have lightened significantly. but new clouds have appeared on the horizon. darkest before the dawn in the dark ages of library websites, users suffered under ux regimes that were rigid, unhelpful and confusing. this was before responsive design became a standard in [&# ;] virtual realty is getting real in the library my library just received three samsung s devices with gear vr goggles. we put them to work right away. the first thought i had was: wow, this will change everything. my second thought was: wow, i can&# ;t wait for apple to make a vr device! the samsung gear vr experience is grainy and fraught with [&# ;] w c’s css framework review i&# ;m a longtime bootstrap fan, but recently i cheated on my old framework. now i&# ;m all excited by the w c&# ;s new framework. like bootstrap, the w c&# ;s framework comes with lots of nifty utilities and plug and play classes and ui features. even if you have a good cms, you&# ;ll find many of their code libraries [&# ;] ai first looking to the future, the next big step will be for the very concept of the “device” to fade away. over time, the computer itself—whatever its form factor—will be an intelligent assistant helping you through your day. we will move from mobile first to an ai first world. google founder&# ;s letter, april my library [&# ;] google analytics and privacy collecting web usage data through services like google analytics is a top priority for any library. but what about user privacy? most libraries (and websites for that matter) lean on google analytics to measure website usage and learn about how people access their online content. it&# ;s a great tool. you can learn about where people [&# ;] the l word i&# ;ve been working with my team on a vision document for what we want our future digital library platform to look like. this exercise keeps bringing us back to defining the library of the future. and that means addressing the very use of the term, &# ;library.&# ; when i first exited my library (and information science) [&# ;] locking down windows i&# ;ve recently moved back to windows for my desktop computing. but windows comes with enormous privacy and security issues that people need to take into account&# ;and get under a semblance of control. here&# ;s how i did it. there has been much written on this subject, so what i&# ;m including here is more of a [&# ;] killer apps & hacks for windows did the ux people at microsoft ever test windows ? here are some must have apps and hacks i&# ;ve found to make life on windows quick and easy. set hotkeys for apps sometimes you just want to launch an app from your keyboard. using a method on laptopmag.com, you can do this for most [&# ;] library hat library hat http://www.bohyunkim.net/blog/ blockchain: merits, issues, and suggestions for compelling use cases * this post was also published in acrl techconnect.*** blockchain holds a great potential for both innovation and disruption. the adoption of blockchain also poses certain risks, and those risks will need to be addressed and mitigated before blockchain becomes mainstream. a lot of people have heard of blockchain at this point. but many are [&# ;] taking diversity to the next level ** this post was also published in acrl techconnect on dec. , .*** getting minorities on board i recently moderated a panel discussion program titled “building bridges in a divisive climate: diversity in libraries, archives, and museums.” participating in organizing this program was interesting experience. during the whole time, i experienced my perspective constantly shifting [&# ;] from need to want: how to maximize social impact for libraries, archives, and museums at the ndp at three event organized by imls yesterday, sayeed choudhury on the “open scholarly communications” panel suggested that libraries think about return on impact in addition to return on investment (roi). he further elaborated on this point by proposing a possible description of such impact. his description was that when an object or [&# ;] how to price d printing service fees ** this post was originally published in acrl techconnect on may. , .*** many libraries today provide d printing service. but not all of them can afford to do so for free. while free d printing may be ideal, it can jeopardize the sustainability of the service over time. nevertheless, many libraries tend to worry [&# ;] post-election statements and messages that reaffirm diversity these are statements and messages sent out publicly or internally to re-affirm diversity, equity, and inclusion by libraries or higher ed institutions. i have collected these &# ; some myself and many others through my fellow librarians. some of them were listed on my blog post, &# ;finding the right words in post-election libraries and higher ed.&# ; [&# ;] finding the right words in post-election libraries and higher ed ** this post was originally published in acrl techconnect on nov. , .*** this year’s election result has presented a huge challenge to all of us who work in higher education and libraries. usually, libraries, universities, and colleges do not comment on presidential election result and we refrain from talking about politics at work. but [&# ;] say it out loud – diversity, equity, and inclusion i usually and mostly talk about technology. but technology is so far away from my thought right now. i don’t feel that i can afford to worry about internet surveillance or how to protect privacy at this moment. not that they are unimportant. such a worry is real and deserves our attention and investigation. but [&# ;] cybersecurity, usability, online privacy, and digital surveillance ** this post was originally published in acrl techconnect on may. , .*** cybersecurity is an interesting and important topic, one closely connected to those of online privacy and digital surveillance. many of us know that it is difficult to keep things private on the internet. the internet was invented to share things with others [&# ;] three recent talks of mine on ux, data visualization, and it management i have been swamped at work and pretty quiet here in my blog. but i gave a few talks recently. so i wanted to share those at least. i presented about how to turn the traditional library it department and its operation that is usually behind the scene into a more patron-facing unit at the recent american library association midwinter [&# ;] near us and libraries, robots have arrived ** this post was originally published in acrl techconnect on oct. , .*** the movie, robot and frank, describes the future in which the elderly have a robot as their companion and also as a helper. the robot monitors various activities that relate to both mental and physical health and helps frank with various house chores. [&# ;] zotero zotero collect, organize, cite, and share your research move zotero citations between google docs, word, and libreoffice last year, we added google docs integration to zotero, bringing to google docs the same powerful citation functionality — with support for over , citation styles — that zotero offers in word and libreoffice. today we&# ;re adding a feature that lets you move documents between google docs and word or libreoffice while preserving active zotero citations. [&# ;] retracted item notifications with retraction watch integration zotero can now help you avoid relying on retracted publications in your research by automatically checking your database and documents for works that have been retracted. we&# ;re providing this service in partnership with retraction watch, which maintains the largest database of retractions available, and we&# ;re proud to help sustain their important work. how it works [&# ;] scan books into zotero from your iphone or ipad zotero makes it easy to collect research materials with a single click as you browse the web, but what do you do when you want to add a real, physical book to your zotero library? if you have an iphone or ipad running ios , you can now save a book to zotero just by [&# ;] zotero comes to google docs we&# ;re excited to announce the availability of zotero integration with google docs, joining zotero&# ;s existing support for microsoft word and libreoffice. the same powerful functionality that zotero has long offered for traditional word processors is now available for google docs. you can quickly search for items in your zotero library, add page numbers and other [&# ;] improved pdf retrieval with unpaywall integration as an organization dedicated to developing free and open-source research tools, we care deeply about open access to scholarship. with the latest version of zotero, we&# ;re excited to make it easier than ever to find pdfs for the items in your zotero library. while zotero has always been able to download pdfs automatically as you [&# ;] introducing zoterobib: perfect bibliographies in minutes we think zotero is the best tool for almost anyone doing serious research, but we know that a lot of people — including many students — don’t need all of zotero’s power just to create the occasional bibliography. today, we’re introducing zoterobib, a free service to help people quickly create perfect bibliographies. powered by the same technology [&# ;] zotero . . : new pdf features, faster citing in large documents, and more the latest version of zotero introduces some major improvements for pdf-based workflows, a new citing mode that can greatly speed up the use of the word processor plugin in large documents, and various other improvements and bug fixes. new pdf features improved pdf metadata retrieval while the &# ;save to zotero&# ; button in the zotero connector [&# ;] zotero . and firefox: frequently asked questions in a unified zotero experience, we explained the changes introduced in zotero . that affect zotero for firefox users. see that post for a full explanation of the change, and read on for some additional answers. what&# ;s changing? zotero . is available only as a standalone program, and zotero . for firefox is being replaced [&# ;] new features for chrome and safari connectors we are excited to announce major improvements to the zotero connectors for chrome and safari. chrome the zotero connector for chrome now includes functionality that was previously available only in zotero for firefox. automatic institutional proxy detection many institutions provide a way to access electronic resources while you are off-campus by signing in to a [&# ;] a unified zotero experience since the introduction of zotero standalone in , zotero users have had two versions to choose from: the original firefox extension, zotero for firefox, which provides deep integration into the firefox user interface, and zotero standalone, which runs as a separate program and can be used with any browser. starting with the release of zotero [&# ;] none none none none collaborations workshop - keynotes live stream - invidious true invidious log in collaborations workshop - keynotes live stream video unavailable. watch on youtube show annotations download is disabled. genre: family friendly? no wilson score: . rating: . / engagement: . % softwaresaved subscribe | - shared march , hi! looks like you have javascript turned off. click here to view comments, keep in mind they may take a bit longer to load. play next by default: : collaborations workshop - panel live stream softwaresaved views : : python software carpentry workshop march - version control with git module softwaresaved views : : python software carpentry workshop - nov - building programs with python (part ) softwaresaved views : : april series: enhancing learning using google for education tools franco nicolo addun k views : : python software carpentry workshop - nov - automating tasks with the unix shell softwaresaved views : : april series | communicating using google for education tools franco nicolo addun k views : fellowship programme launch webinar softwaresaved views : : resurrecting retail virtual launch party retail prophet k views : chris hartgerink keynote talk on "the social model of inaccessibility" softwaresaved views : : april series: organizing life and work using google for education tools franco nicolo addun k views : : volt europa general assembly . . | #votevolt volt europa . k views : : python software carpentry workshop - march - building programs with python module part softwaresaved views released under the agplv by omar roth. btc: dpzymxu ryd yqzjs n kgkwcyry bch: qq ptclkzej eza a et ggc hxsq aylqut npk liberapay view javascript license information. / view privacy policy. current version: . . - ba @ master none how to tweet – what is a tweet, keyboard shortcuts, and sources open menu help center help topics using twitter managing your account safety and security rules and policies guides new user faq glossary a safer twitter our rules my privacy getting started guide contact us provide feedback search go to twitter sign out sign in search this site search goglobalwithtwitterbanner tweets search using twitter tweets adding content to your tweet search and trends following and unfollowing blocking and muting direct messages twitter on your device website and app integrations using periscope twitter voices fleets managing your account login and password username, email, and phone account settings notifications verified accounts suspended accounts deactivate and reactivate accounts safety and security security and hacked accounts privacy spam and fake accounts sensitive content abuse rules and policies twitter rules and policies general guidelines and policies law enforcement guidelines research and experiments help center tweets how to tweet how to tweet a tweet may contain photos, gifs, videos, links, and text. looking for information on how to tweet at someone? check out our article about how to post replies and mentions on twitter. view instructions for: how to tweet tap the tweet compose icon compose your message (up to characters) and tap tweet. how to tweet tap on the tweet compose icon enter your message (up to characters), and then tap tweet. a notification will appear in the status bar on your device and will go away once the tweet successfully sends. how to tweet type your tweet (up to characters) into the compose box at the top of your home timeline, or click the tweet button in the navigation bar. you can include up to photos, a gif, or a video in your tweet. click the tweet button to post the tweet to your profile. to save a draft of your tweet, click the x icon in the top left corner of the compose box, then click save. to schedule your tweet to be sent at a later date/time, click on the calendar icon at the bottom of the compose box and make your schedule selections, then click confirm. to access your drafts and scheduled tweets, click on unsent tweets from the tweet compose box. tweet source labels tweet source labels help you better understand how a tweet was posted. this additional information provides context about the tweet and its author. if you don’t recognize the source, you may want to learn more to determine how much you trust the content. click on a tweet to go to the tweet details page. at the bottom of the tweet, you’ll see the label for the source of the account’s tweet. for example, twitter for iphone, twitter for android, or twitter for web. tweets containing the twitter for advertisers label indicate they are created through the twitter ads composer and not whether they are paid content or not. paid content contains a promoted badge across all ad formats. in some cases you may see a third-party client name, which indicates the tweet came from a non-twitter application. authors sometimes use third-party client applications to manage their tweets, manage marketing campaigns, measure advertising performance, provide customer support, and to target certain groups of people to advertise to. third-party clients are software tools used by authors and therefore are not affiliated with, nor do they reflect the views of, the tweet content. tweets and campaigns can be directly created by humans or, in some circumstances, automated by an application. visit our partners page for a list of common third-party sources. deleting tweets read about how to delete a tweet. note that you can only delete your own tweets. you cannot delete tweets which were posted by other accounts. instead, you can unfollow, block or mute accounts whose tweets you do not want to receive. read about how to delete or undo a retweet. keyboard shortcuts the following are a list of keyboard shortcuts to use on twitter.com. actions n = new tweet l = like r = reply t = retweet m = direct message u = mute account b = block account enter = open tweet details o = expand photo / = search cmd-enter | ctrl-enter = send tweet navigation ? = full keyboard menu j = next tweet k = previous tweet space = page down . = load new tweets timelines g and h = home timeline g and o = moments g and n = notifications tab g and r = mentions g and p = profile g and l = likes tab g and i = lists tab g and m = direct messages g and s = settings and privacy g and u = go to someone’s profile bookmark or share this article scroll to top twitter platform twitter.com status card validator privacy center transparency center twitter, inc. about the company twitter for good company news brand toolkit jobs and internships investors help help center using twitter twitter media ads help center managing your account safety and security rules and policies contact us developer resources developer home documentation forums communities developer blog engineering blog developer terms business resources advertise twitter for business resources and guides twitter for marketers marketing insights brand inspiration twitter data twitter flight school © twitter, inc. cookies privacy terms and conditions english help center english español 日本語 한국어 português deutsch türkçe français italiano العربيّة nederlands bahasa indonesia Русский हिंदी সহায়তা কেন্দ্র मदत केंद्र સહાયતા કેન્દ્ર உதவி மையம் ಸಹಾಯ ಕೇಂದ್ರ by using twitter’s services you agree to our cookies use. we use cookies for purposes including analytics, personalisation, and ads. ok none everybody's libraries | libraries for everyone, by everyone, shared with everyone, about everything everybody's libraries libraries for everyone, by everyone, shared with everyone, about everything skip to content home about about the free decimal correspondence free decimal correspondence ils services for discovery applications john mark ockerbloom the metadata challenge ← older posts public domain day : honoring a lost generation posted on january , by john mark ockerbloom it’s public domain day again. in much of europe, and other countries with “life+ years” copyright terms, works by authors who died in , such as george orwell, karin michaelis, george bernard shaw, and edna st. vincent millay, have joined the public domain. canada, and other countries that still have the berne convention’s “life+ years” copyright terms, get works by authors like e. m. forster, nelly sachs, bertrand russell, elsa triolet, and other authors who died in in the public domain. and in the united states, copyrights from that are still in force have expired, introducing to the public domain a wide variety of works i’ve covered in my prior blog post. the new public domain work that i’ve seen most widely noted is f. scott fitzgerald’s jazz age novel the great gatsby. my library has a copy of the first edition, and its scan of the volume became available on hathitrust today. though he doesn’t use the term in gatsby, fitzgerald and many other authors writing around are often considered both members and chroniclers of the “lost generation”. the term was coined by gertrude stein, and made famous by ernest hemingway, who used it in the epigraph to his novel the sun also rises (one of many more works scheduled to join the us public domain a year from now). the lost generation describes an age cohort that was disrupted by the first world war, and all the deaths caused by that war and by the influenza pandemic that arose in its wake. society would never be the same afterwards. it’s ironic that some of the definitive creations of that generation are themselves part of a largely lost generation. at the time of their publication, they were supposed to enter the public domain after years at most, but that maximum term has been extended by more years, well over a generation’s worth of time. the creators of these works that got the full copyright term are almost all now dead, and many of the less famous works in this cohort have also become lost from most people’s memories. some, including many fragile films of that era, now have all copies lost as well. the generation that now sees these works joining the public domain also has many of the makings of a new “lost generation”. the number of deaths from covid- in the united states, which badly botched its response compared to many similar countries, far exceeds the number of american deaths in world war i, and is a sizable and rapidly growing fraction of all the american deaths from the - flu pandemic. many more people who have dealt with illness and quarantine have also experienced what feels like a lost year, one that hasn’t ended yet despite today’s change in the calendar. but it’s also important to recognize the key role of the public domain and of open access publications in preventing further loss. while philadelphia, where i live, has been hit hard by this pandemic, it hasn’t been hit as hard as some other places, in part because masking and other behavioral changes have been more widely used and accepted here. not long before the current pandemic started, the mutter museum’s spit spreads death exhibit reminded us of the horrifying death toll of the flu pandemic here, caused in large part by failing to stop mass gatherings that made the flu spread like wildfire here. the exhibit’s narrative, which many other local media outlets further elaborated on, was able to freely draw on a wide variety of source materials of the era that were all in the public domain due to their age. the freely available sources from helped spread public health awareness here in . open access to resources also spurred the rapid development and testing of effective treatments against covid. open sharing of the novel coronavirus genomes, and related scientific data, enabled research on the virus and effective responses to be carried out by many different labs across the globe, and many of the resulting research papers and research materials have also been made freely available in venues that are usually limited to paid subscribers. while much of this work is not public domain, strictly speaking, it is being shared and built on largely as if it were. that has enabled vaccines to be safely rolled out much more quickly than they have been for other diseases. while we celebrate today’s belated additions to the public domain, it’s also important to promote and protect it, because there are still efforts to freeze it or roll it back. the successor to the nafta trade deal requires canada to add years to its copyright terms, for instance (though canada has not yet implemented that provision). and while there is no current legislation to extend us copyright terms any further, such extensions have been proposed in the past, and we’ve just seen in congress’s recent funding bill how questionable changes to copyright law can be jammed into “must-pass” legislation with little or no warning or recourse. the public domain enriches our culture, reminds us and lets us learn from our past, and helps us make better futures. as gives us opportunities to turn the page, let’s celebrate the new opportunities we have to enjoy, share, reuse, and build on our newly public domain works. and let’s make sure we don’t lose any more generations. posted in online books, open access, publicdomain | comments counting down to in the public domain posted on december , by john mark ockerbloom we’re rapidly approaching another public domain day, the day at the start of the year when a year’s worth of creative work joins the public domain. this will be the third year in a row that the us will have a full crop of new public domain works (after a prior -year drought), and once again, i’m noting and celebrating works that will be entering the public domain shortly. approaching , i wrote a one-post-a-day advent calendar for works throughout the month of december, and approaching , i highlighted a few works, and related copyright issues, in a series of december posts called vision. this year i took to twitter, making one tweet per day featuring a different work and creator using the #publicdomaindaycountdown hashtag. tweets are shorter than blog posts, but i started days out, so by the time i finish the series at the end of december, i’ll have written short notices on more works than ever. since not everyone reads twitter, and there’s no guarantee that my tweets will always be accessible on that site, i’ll reproduce them here. (this post will be updated to include all the tweets up to .) the tweet links have been reformatted for the blog, a couple of -tweet threads have been recombined, and some typos may be corrected. if you’d like to comment yourself on any of the works mentioned here, or suggest others i can feature, feel free to reply here or on twitter. (my account there is @jmarkockerbloom. you’ll also find some other people tweeting on the #publicdomaindaycountdown hashtag, and you’re welcome to join in as well.) september : it’s f. scott fitzgerald’s birthday. his best-known book, the great gatsby, joins the us public domain days from now, along with other works with active copyrights. #publicdomaindaycountdown (links to free online books by fitzgerald here.) september : c. k. scott-moncrieff’s birthday’s today. he translated proust’s remembrance of things past (a controversial title, as the public domain review notes). the guermantes way, his translation of proust’s rd volume, joins the us public domain in days. #publicdomaindaycountdown september : today is t.s. eliot’s birthday. his poem “the hollow men” (which ends “…not with a bang but a whimper”) was first published in full in , & joins the us public domain in days. #publicdomaindaycountdown more by & about him here. september : lady cynthia asquith, born today in , edited a number of anthologies that have long been read by children and fans of fantasy and supernatural fiction. her first major collection, the flying carpet, joins the us public domain in days. #publicdomaindaycountdown september : as @marketplace reported tonight, agatha christie’s mysteries remain popular after years. in days, her novel the secret of chimneys will join the us public domain, as will the expanded us poirot investigates collection. #publicdomaindaycountdown september : homer hockett’s and arthur schlesinger, sr.’s political and social history of the united states first came out in , and was an influential college textbook for years thereafter. the first edition joins the public domain in days. #publicdomaindaycountdown september : inez haynes gillmore irwin died years ago this month, after a varied, prolific writing career. this blog post looks at of her books, including gertrude haviland’s divorce, which joins the public domain in days. #publicdomaindaycountdown october : for some, spooky stories and themes aren’t just for october, but for the whole year. we’ll be welcoming a new year’s worth of weird tales to the public domain in months. see what’s coming, and what’s already free online, here. #publicdomaindaycountdown october : misinformation and quackery has been a threat to public health for a long time. in weeks, the book the patent medicine and the public health, by american quack-fighter arthur j. cramp joins the public domain. #publicdomaindaycountdown october : sophie treadwell, born this day in , was a feminist, modernist playwright with several plays produced on broadway, but many of her works are now hard to find. her play “many mansions” joins the public domain in days. #publicdomaindaycountdown october : it’s edward stratemeyer’s birthday. books of his syndicate joining the public domain in days include the debuts of don sturdy & the blythe girls, & further adventures of tom swift, ruth fielding, baseball joe, betty gordon, the bobbsey twins, & more. #publicdomaindaycountdown october : russell wilder was a pioneering diabetes doctor, testing newly invented insulin treatments that saved many patients’ lives. his book diabetes: its cause and its treatment with insulin joins the public domain in days. #publicdomaindaycountdown october : queer british catholic author radclyffe hall is best known for the well of loneliness. hall’s earlier novel a saturday life is lighter, though it has some similar themes in subtext. it joins the us public domain in days. #publicdomaindaycountdown october : edgar allan poe’s stories have long been public domain, but some work unpublished when he died (on this day in ) stayed in © much longer. in days, the valentine museum’s book of his previously unpublished letters finally goes public domain. #publicdomaindaycountdown october : in , the nobel prize in literature went to george bernard shaw. in days, his table-talk, published that year, will join the public domain in the us, and all his solo works published in his lifetime will be public domain nearly everywhere else. #publicdomaindaycountdown october : author and editor edward bok was born this day in . in twice thirty ( ), he follows up his pulitzer-winning memoir the americanization of edward bok with a set of essays from the perspective of his s. it joins the public domain in days. #publicdomaindaycountdown october : in the silent comedy “the freshman”, harold lloyd goes to tate university, “a large football stadium with a college attached”, and goes from tackling dummy to unlikely football hero. it joins the public domain in days. #publicdomaindaycountdown october : it’s françois mauriac’s birthday. his le desert de l’amour, a novel that won the grand prix of the académie française, joins the us public domain in days. published translations may stay copyrighted, but americans will be free to make new ones. #publicdomaindaycountdown october : pulitzer-winning legal scholar charles warren’s congress, the constitution, and the supreme court ( ) analyzes controversies, some still argued, over relations between the us legislature and the us judiciary. it joins the public domain in days. #publicdomaindaycountdown october : science publishing in was largely a boys’ club, but some areas were more open to women authors, such as nursing & science education. i look forward to maude muse’s textbook of psychology for nurses going public domain in days. #publicdomaindaycountdown #adalovelaceday october : happy birthday to poet e. e. cummings, born this day in . (while some of his poetry is lowercase he usually still capitalized his name when writing it out) his collection xli poems joins the public domain in days. #publicdomaindaycountdown october : it’s pg wodehouse’s birthday. in days more of his humorous stories join the us public domain, including sam in the suburbs. it originally ran as a serial in the saturday evening post in . all that year’s issues also join the public domain then. #publicdomaindaycountdown october : playwright and nobel laureate eugene o’neill was born today in . his “desire under the elms” entered the us public domain this year; in days, his plays “marco’s millions” and “the great god brown” will join it. #publicdomaindaycountdown october : not everything makes it to the end of the long road to the us public domain. in days, the copyright for the film man and maid (based on a book by elinor glyn) expires, but no known copies survive. maybe someone will find one? #publicdomaindaycountdown october : corra harris became famous for her novel a circuit rider’s wife and her world war i reporting. the work she considered her best, though, was as a woman thinks. it joins the public domain in days. #publicdomaindaycountdown october : edna st. vincent millay died years ago today. all her published work joins the public domain in days in many places outside the us. here, magazine work like “sonnet to gath” (in sep vanity fair) will join, but renewed post-’ work stays in ©. #publicdomaindaycountdown october : all songs eventually reach the public domain. authors can put them there themselves, like tom lehrer just did for his lyrics. but other humorous songs arrive by the slow route, like tilzer, terker, & heagney’s “pardon me (while i laugh)” will in days. #publicdomaindaycountdown october : sherwood anderson’s winesburg, ohio wasn’t a best-seller when it came out, but his dark laughter was. since joycean works fell out of fashion, that book’s been largely forgotten, but may get new attention when it joins the public domain in days. #publicdomaindaycountdown october : artist nc wyeth was born this day in . the brandywine museum near philadelphia shows many of his works. his illustrated edition of francis parkman’s book the oregon trail joins the public domain in days. #publicdomaindaycountdown october : today (especially at : , on / ) many chemists celebrate #moleday. in days, they’ll also get to celebrate historically important chemistry publications joining the us public domain, including all issues of justus liebigs annalen der chemie. #publicdomaindaycountdown october : while some early alfred hitchcock films were in the us public domain for a while due to formality issues, the gatt accords restored their copyrights. his directorial debut, the pleasure garden, rejoins the public domain (this time for good) in days. #publicdomaindaycountdown (addendum: there may still be one more year of copyright to this film as of ; see the comments to this post for details.) october : albert barnes took a different approach to art than most of his contemporaries. the first edition of the art in painting, where he explains his theories and shows examples from his collection, joins the public domain in days. #publicdomaindaycountdown october : prolific writer carolyn wells had a long-running series of mystery novels featuring fleming stone. here’s a blog post by the passing tramp on one of them, the daughter of the house, which will join the public domain in days. #publicdomaindaycountdown october : theodore roosevelt was born today in , and died over years ago, but some of his works are still copyrighted. in days, volumes of his correspondence with henry cabot lodge, written from - and published in , join the public domain. #publicdomaindaycountdown october : american composer and conductor howard hanson was born on this day in . his choral piece “lament for beowulf” joins the public domain in days. #publicdomaindaycountdown october : “skitter cat” was a white persian cat who had adventures in several children’s books by eleanor youmans, illustrated by ruth bennett. the first of the books joins the public domain in days. #publicdomaindaycountdown #nationalcatday october : “secret service smith” was a detective created by canadian author r. t. m. maitland. his first magazine appearance was in ; his first original full-length novel, the black magician, joins the public domain in weeks. #publicdomaindaycountdown october : poet john keats was born this day in . amy lowell’s -volume biography links his romantic poetry with her imagist poetry. ( review.) she finished and published it just before she died. it joins the public domain in days. #publicdomaindaycountdown november : “not just for an hour, not for just a day, not for just a year, but always.” irving berlin gave the rights to this song to his bride in . both are gone now, and in months it will join the public domain for all of us, always. #publicdomaindaycountdown november : mikhail fokine’s the dying swan dance, set to music by camille saint-saëns, premiered in , but its choreography wasn’t published until , the same year a film of it was released. it joins the public domain in days. #publicdomaindaycountdown (choreography copyright is weird. not only does the term not start until publication, which can be long after st performance, but what’s copyrightable has also changed. before it had to qualify as dramatic; now it doesn’t, but it has to be more than a short step sequence.) november : herbert hoover was the only sitting president to be voted out of office between & . before taking office, he wrote the foreword to carolyn crane’s everyman’s house, part of a homeowners’ campaign he co-led. it goes out of copyright in days. #publicdomaindaycountdown november : “the golden cocoon” is a silent melodrama featuring an election, jilted lovers, and extortion. the ruth cross novel it’s based on went public domain this year. the film will join it there in days. #publicdomaindaycountdown november : investigative journalist ida tarbell was born today in . her history of standard oil helped break up that trust in , but her life of elbert h. gary wrote more admiringly of his chairmanship of us steel. it joins the public domain in days. #publicdomaindaycountdown november : harold ross was born on this day in . he was the first editor of the new yorker, which he established in coöperation with his wife, jane grant. after ninety-five years, the magazine’s first issues are set to join the public domain in fifty-six days. #publicdomaindaycountdown november : “sweet georgia brown” by ben bernie & maceo pinkard (lyrics by kenneth casey) is a jazz standard, the theme tune of the harlem globetrotters, and a song often played in celebration. one thing we can celebrate in days is it joining the public domain. #publicdomaindaycountdown november : today i hiked on the appalachian trail. it was completed in , but parts are much older. walter collins o’kane’s trails and summits of the white mountains, published in when the at was more idea than reality, goes public domain in days. #publicdomaindaycountdown november : in sinclair lewis’ arrowsmith, a brilliant medical researcher deals with personal and ethical issues as he tries to find a cure for a deadly epidemic. the novel has stayed relevant well past its publication, and joins the public domain in days. #publicdomaindaycountdown november : john marquand was born today in . he’s known for his spy stories and satires, but an early novel, the black cargo, features a sailor curious about a mysterious payload on a ship he’s been hired onto. it joins the us public domain in days. #publicdomaindaycountdown november : the first world war, whose armistice was years ago today, cast a long shadow. among the many literary works looking back to it is ford madox ford’s novel no more parades, part of his “parade’s end” tetralogy. it joins the public domain in days. #publicdomaindaycountdown november : anne parrish was born on this day in . in , the dream coach, co-written with her brother, got a newbery honor , and her novel the perennial bachelor was a best-seller. the latter book joins the public domain in days. #publicdomaindaycountdown november : in “the curse of the golden cross”, g. k. chesterton’s father brown once again finds a natural explanation to what seem to be preternatural symbols & events. as of today, friday the th, the story is exactly weeks away from the us public domain. #publicdomaindaycountdown november : the pop standard “yes sir, that’s my baby” was the baby of walter donaldson (music) and gus kahn (lyrics). it’s been performed by many artists since its composition, and in days, this baby steps out into the public domain. #publicdomaindaycountdown november : marianne moore, born on this day in , had a long literary career, including editing the influential modernist magazine the dial from on. in days, all issues of that magazine will be fully in the public domain. #publicdomaindaycountdown november : george s. kaufman, born today in , wrote or directed a play in every broadway season from till . in days, several of his plays join the public domain, including his still-performed comedy “the butter and egg man”. #publicdomaindaycountdown november : shen of the sea was a newbery-winning collection of stories presented as “chinese” folktales, but written by american author arthur bowie chrisman. praised when first published, seen more as appropriation later, it’ll be appropriable itself in days. #publicdomaindaycountdown november : i share a birthday today with jacques maritain, a french catholic philosopher who influenced the universal declaration of human rights. his book on reformers (luther, descartes, and rousseau) joins the public domain in days. #publicdomaindaycountdown november : prevailing views of history change a lot over years. the pulitzer history prize went to a book titled “the war for southern independence”. the last volume of edward channing’s history of the united states, it joins the public domain in days. #publicdomaindaycountdown november : alfred north whitehead’s science and the modern world includes a nuanced discussion of science and religion differing notably from many of his contemporaries’. (a recent review of it.) it joins the us public domain in weeks. november : algonquin round table member robert benchley tried reporting, practical writing, & reviews, but soon found that humorous essays & stories were his forte. one early collection, pluck and luck, joins the public domain in days. #publicdomaindaycountdown november : i’ve often heard people coming across a piano sit down & pick out hoagy carmichael’s “heart and soul”. he also had other hits, one being “washboard blues“. his original piano instrumental version becomes public domain in days. #publicdomaindaycountdown november : harpo marx, the marx brothers mime, was born today in . in his oldest surviving film, “too many kisses” he does “speak”, but silently (like everyone else in it), without his brothers. it joins the public domain in days. #publicdomaindaycountdown november : in the man nobody knows, bruce barton likened the world of jesus to the world of business. did he bring scriptural insight to management, or subordinate christianity to capitalism? it’ll be easier to say, & show, after it goes public domain in days. #publicdomaindaycountdown november : before virgil thomson (born today in ) was well-known as a composer, he wrote a music column for vanity fair. his first columns, and the rest of vanity fair for , join the public domain in days. #publicdomaindaycountdown november : “each moment that we’re apart / you’re never out of my heart / i’d rather be lonely and wait for you only / oh how i miss you tonight” those staying safe by staying apart this holiday might appreciate this song, which joins the public domain in days. #publicdomaindaycountdown (the song, “oh, how i miss you tonight” is by benny davis, joe burke, and mark fisher, was published in , and performed and recorded by many musicians since then, some of whom are mentioned in this wikipedia article.) november : feminist author katharine anthony, born today in , was best known for her biographies. her biography of catherine the great, which drew extensively on the empress’s private memoirs, joins the public domain in days. #publicdomaindaycountdown november : tonight in “barn dance” (soon renamed “grand ole opry”) debuted in nashville. most country music on it & similar shows then were old favorites, but there were new hits too, like “the death of floyd collins”, which joins the public domain in days. #publicdomaindaycountdown (the song, with words by andrew jenkins and music by john carson, was in the line of other disaster ballads that were popular in the s. this particular disaster had occurred earlier in the year, and became the subject of song, story, drama, and film.) november : as many folks get ready for christmas, many christmas-themed works are also almost ready to join the public domain in days. one is the holly hedge, and other christmas stories by temple bailey. more on the book & author. #publicdomaindaycountdown november : in john maynard keynes published the economic consequences of sterling parity objecting to winston churchill returning the uk to the gold standard. that policy ended in ; the book’s us copyright lasted longer, but will finally end in days. #publicdomaindaycountdown december : du bose heyward’s novel porgy has a distinguished legacy of adaptations, including a broadway play, and gershwin’s opera “porgy and bess”. when the book joins the public domain a month from now, further adaptation possibilities are limitless. #publicdomaindaycountdown december : in dorothy black’s romance — the loveliest thing a young englishwoman “inherits a small sum of money, buys a motor car and goes off in search of adventure and romance”. first serialized in ladies’ home journal, it joins the public domain in days. #publicdomaindaycountdown december : joseph conrad was born on this day in , and died in , leaving unfinished his napoleonic novel suspense. but it was still far enough along to get serialized in magazines and published as a book in , and it joins the public domain in days. #publicdomaindaycountdown december : ernest hemingway’s first us-published story collection in our time introduced his distinctive style to an american audience that came to view his books as classics of th century fiction: it joins the public domain in days. #publicdomaindaycountdown december : libertarian author rose wilder lane helped bring her mother’s “little house” fictionalized memoirs into print. before that, she published biographical fiction based on the life of jack london, called he was a man. it joins the public domain in days. #publicdomaindaycountdown december : indiana naturalist and author gene stratton-porter died on this day in . her final novel, the keeper of the bees, was published the following year, and joins the public domain in days. one review. #publicdomaindaycountdown december : willa cather was born today in . her novel the professor’s house depicts s cultural dislocation from a different angle than f. scott fitzgerald’s better-known great gatsby. it too joins the public domain in days. #publicdomaindaycountdown december : the last symphony published by finnish composer jean sibelius (born on this day in ) is described in the grove dictionary as his “most remarkable compositional achievement”. it joins the public domain in the us in days. #publicdomaindaycountdown december : when the habsburg empire falls, what comes next for the people & powers of vienna? the novel old wine, by phyllis bottome (wife of the local british intelligence head) depicts a society undergoing rapid change. it joins the us public domain in days. #publicdomaindaycountdown december : lewis browne was “a world traveler, author, rabbi, former rabbi, lecturer, socialist and friend of the literary elite”. his first book, stranger than fiction: a short history of the jews, joins the public domain in days. #publicdomaindaycountdown december : in , john scopes was convicted for teaching evolution in tennessee. books explaining the science to lay audiences were popular that year, including henshaw ward’s evolution for john doe. it becomes public domain in weeks. #publicdomaindaycountdown december : philadelphia artist jean leon gerome ferris was best known for his “pageant of a nation” paintings. three of them, “the birth of pennsylvania”, “gettysburg, ”, and “the mayflower compact”, join the public domain in days. #publicdomaindaycountdown december : the queen of cooks, and some kings was a memoir of london hotelier rosa lewis, as told to mary lawton. her life story was the basis for the bbc and pbs series “the duchess of duke street”. it joins the public domain in days. #publicdomaindaycountdown december : today we’re celebrating new films being added to the national film registry. in days, we can also celebrate more registry films joining the public domain. one is the clash of the wolves, starring rin tin tin. #publicdomaindaycountdown december : etsu inagaki sugimoto, daughter of a high-ranking japanese official, moved to the us in an arranged marriage after her family fell on hard times. her memoir, a daughter of the samurai, joins the public domain in days. #publicdomaindaycountdown december : on the trail of negro folk-songs compiled by dorothy scarborough assisted by ola lee gulledge, has over songs. scarborough’s next of kin (not gulledge, or any of their sources) renewed its copyright in . but in days, it’ll be free for all. #publicdomaindaycountdown december : virginia woolf’s writings have been slowly entering the public domain in the us. we’ve had the first part of her mrs. dalloway for a while. the complete novel, and her first common reader essay collection, join it in days. #publicdomaindaycountdown december : lovers in quarantine with harrison ford sounds like a movie made for , but it’s actually a silent comedy (with a different harrison ford). it’ll be ready to go out into the public domain after a -day quarantine. #publicdomaindaycountdown december : ma rainey wrote, sang, and recorded many blues songs in a multi-decade career. two of her songs becoming public domain in days are “shave ’em dry” (written with william jackson) & “army camp harmony blues” (with hooks tilford). #publicdomaindaycountdown december : for years we’ve celebrated the works of prize-winning novelist edith wharton as her stories join the public domain. in days, the writing of fiction, her book on how she writes her memorable tales, will join that company. #publicdomaindaycountdown december : albert payson terhune, born today in , raised and wrote about dogs he kept at what’s now a public park in new jersey. his book about wolf, who died heroically and is buried there, will also be in the public domain in days. #publicdomaindaycountdown december : in the s it seemed buster keaton could do anything involving movies. go west, a feature film that he co-wrote, directed, co-produced, and starred in, is still enjoyed today, and it joins the public domain in days. #publicdomaindaycountdown december : in days, not only will theodore dreiser’s massive novel an american tragedy be in the public domain, but so will a lot of the raw material that went into it. much of it is in @upennlib‘s special collections. #publicdomaindaycountdown december : johnny gruelle, born today in , created the raggedy ann doll, and a series of books sold with it that went under many christmas trees. two of them, raggedy ann’s alphabet book and raggedy ann’s wishing pebble, join the public domain in days. #publicdomaindaycountdown december : written in hebrew by joseph klausner, translated into english by anglican priest herbert danby, jesus of nazareth reviewed jesus’s life and teachings from a jewish perspective. it made a stir when published in , & joins the public domain in days. #publicdomaindaycountdown december : “it’s a travesty that this wonderful, hilarious, insightful book lives under the inconceivably large shadow cast by the great gatsby.” a review of anita loos’s gentlemen prefer blondes, also joining the public domain in days. #publicdomaindaycountdown december : “on revisiting manhattan transfer, i came away with an appreciation not just for the breadth of its ambition, but also for the genius of its representation.” a review of the john dos passos novel becoming public domain in days. #publicdomaindaycountdown december : all too often legal systems and bureaucracies can be described as “kafkaesque”. the kafka work most known for that sense of arbitrariness and doom is der prozess (the trial), reviewed here. it joins the public domain in days. #publicdomaindaycountdown december : chocolate kiddies, an african american music and dance revue that toured europe in , featured songs by duke ellington and jo trent including “jig walk”, “jim dandy”, and “with you”. they join the public domain in days. #publicdomaindaycountdown december : lon chaney starred in of the top-grossing movies of . the phantom of the opera has long been in the public domain due to copyright nonrenewal. the unholy three, which was renewed, joins it in the public domain in days. #publicdomaindaycountdown (if you’re wondering why some of the other big film hits of haven’t been in this countdown, in many cases it’s also because their copyrights weren’t renewed. or they weren’t actually copyrighted in .) december : “…you might as well live.” dorothy parker published “resumé” in , and ultimately outlived most of her algonquin round table-mates. this poem, and her other writing for periodicals, will be in the public domain tomorrow. #publicdomaindaycountdown posted in copyright, publicdomain | comments from our subjects to yours (and vice versa) posted on december , by john mark ockerbloom (tl;dr: i’m starting to implement services and publish data to support searching across library collections that use customized subject headings, such as the increasingly-adopted substitutes for lcsh terms like “illegal aliens”. read on for what i’m doing, why, and where i would value advice and discussion on how to proceed.) i’ve run the forward to libraries service for a few years now. as i’ve noted in earlier posts here, it’s currently used on the online books page and in some wikipedia articles to search for resources in your local library (or any other library you’re interested in) on a subject you’re exploring. one of the key pieces of infrastructure that makes it work is the library of congress subject headings (lcsh) system, which many research libraries use to describe their holdings. using the headings in the system, along with mappings between it and other systems for describing subjects (such as the english wikipedia article titles that forward to libraries knows how to relate to lcsh) allows researchers to find materials on the same subjects across multiple collections, using common terminology. there are limitations to relying on lcsh for cross-collection subject searches, though. first of all, many libraries, particularly those outside the us, do not use lcsh. some use other subject vocabularies. if a mapping has been defined between lcsh and another subject vocabulary (as has been done, for example, with mesh) one can use that mapping to determine search terms to use in libraries that use that subject vocabulary. we don’t yet have that capability in forward to libraries, but i’m hoping to add it eventually. changing the subjects i’m now also seeing more libraries that use lcsh, but that also use different terms for certain subjects that they find more appropriate for their users. while there is a process for updating lcsh terms (and its terms get updated on a monthly basis) the process can be slow, hard for non-specialists to participate in, and contentious, particularly for larger-scale subject heading changes. it can also be subject to pressure by non-librarians. the library of congress ultimately answers to congress (as its name suggests), and members of congress have used funding bills to block changes in subject headings that the librarian-run process had approved. they did that in for the subject heading “illegal aliens”, where librarians had recommended using other terms to cover subjects related to unauthorized immigration. the documentary film “change the subject” (linked with context in this article) has a detailed report on this controversy. four years after the immigration subject changes were blocked, some libraries have decided not to wait for lcsh to change, and are introducing their own subject terms. the university of colorado boulder, for example, announced in that they would use the term “undocumented immigrants” where the library of congress had “illegal aliens”. other libraries have recently announced similar changes. some library consortia have organized systematic programs to supersede outdated and offensive terms in lcsh in their catalogs. some groups now maintain specialized subject vocabularies that can both supplement and supersede lcsh terms, such as homosaurus for lgbt+-related subjects. and there’s also been increasing interest in using subject terms and classifications adapted to local communities. for instance, the brian deer classification system is intended to be both used and shaped by local indigenous communities, and therefore libraries in different locations that use it may well use different terms for some subjects, depending on local usage and interests. supporting cross-collection search in a community of localized catalogs we can still search across collections that use local terms, as long as we know what those terms are and how to translate between them. forward to libraries already uses a data file indicating wikipedia article titles that correspond closely to lcsh subjects, and vice versa. by extension, we can also create a data file indicating terms to use at a given library that correspond to terms in lcsh and other vocabularies, so we can see what resources are available at different places on a given topics. you can see how that works in practice at the online books page. as i write this, we’re still using the unaltered lcsh subjects (updated to october ), so we have a subject page showing free online books on “illegal aliens”. you can follow links from there to see what other libraries have. if you select the “elsewhere” link in the upper left column and choose the library of congress as the library to search, you’ll see what they hold under that subject heading. but if you instead choose the university of colorado boulder, you’ll see what they have under “undocumented immigrants”, the subject term they’ve adopted there. similar routing happens from wikipedia. the closest related wikipedia article at present is “illegal immigration”, and if you go down to the further reading section and select links in the library resources box, selecting “online books” or most libraries will currently take you to their “illegal aliens” subject search. but selecting university of colorado boulder (from “resources in other libraries” if you don’t already have it specified as your preferred library in wikipedia) will take you to their “undocumented immigrants” search. this routing applies two mappings, one from wikipedia terms to lcsh terms, and another from lcsh terms to local library terms. a common data resource these sorts of transformations are fundamentally data-driven. my forward to libraries github repository now includes a data file listing local subject terms that different libraries use, and how they relate to lcsh subject terms. (the library codes used in the file are the same ones that are used in my libraries data file, and are based on oclc and/or isil identifiers.) the local subject terms file is very short for now– as i write this, it only has enough data for the examples i’ve described above, but i’ll be adding more data shortly for other libraries that have announced and implemented subject headings changes. (and i’ll be glad to hear about more so i can add them.) as with other data in this repository, the data in this file is cc , so it can be used by anyone for any purpose. in particular, it could be be used by services other than my forward to libraries tool, such as by aggregated catalogs that incorporate data from multiple libraries, some of which might use localized subject terms that have lcsh analogues. where to go next what i’ve shown so far is not far removed from a proof-of-concept demo, but i hope it suggests ways that services can be developed to support searches among and across library collections with diverse subject headings. as i mentioned, i’ll be adding more data on localized subject headings as i hear about it, as well as adding more functionality to the forward to libraries service (such as the ability to link from a collection with localized subject headings, so i can support them in the online books page, or in other libraries that have such headings and want to use to the service). there are some extensions that could be done to the basic data model to support scaling up these sorts of localizations, such as customizations used by all the libraries in a given consortium, or ones that adopt wholesale an alternative set of subjects, whether that be mesh, homosaurus, or the subject thesaurus of a national library outside the us. even with data declarations supporting those sorts of “bulk” subject mappings, a universal subject mapping knowledge base could get large over time. i’ve created my own mapping file for my services, and for now i’m happy to grow it as needed and share the data freely. but if there is another suitable mapping hub already available or in the works, i’m happy to consider using that instead. it’s important to support exploration across a community of diverse libraries with a diverse array of subject terms and descriptions. i hope the tools and data i’ve described here will help advance us towards that goal, and that i can help grow them from their current nascent state to make them more broadly useful. posted in discovery, metadata, subjects, wikipedia | leave a comment everybody’s library questions: finding films in the public domain posted on march , by john mark ockerbloom welcome to another installment of everybody’s library questions, where i give answers to questions people ask me (in comments or email) that seem to be useful for general consumption. before i start, though, i want to put in a plug for your local librarians. even though many library buildings are closed now (as they should be) while we’re trying to get propagation and treatment for covid- under control, many of those libraries offer online services, including interactive online help from librarians. (many of our libraries are also expanding the scope and hours of these services during this health crisis.) your local librarians will have the best knowledge of what’s available to you, can find out more about your needs when they talk to you, and will usually be able to respond to questions faster than i or other specific folks on the internet can. check out your favorite library’s website, and look for links like “get help” or “online chat” and see what they offer. ok, now here’s the question, extracted from a comment made by nicholas escobar to a recent post: i am currently studying at the university of edinburgh getting masters degree in film composition. for my final project i am required to score a minute film. i was thinking of picking a short silent film (any genre) in the public domain that is minutes (or very close to that length) and was wondering if you had any suggestions? there are three questions implied by this one: first, how do you find out what films exist that meet your content criteria? second, how do you find out whether films in that set are in the public domain? finally, how can you get access to a film so you can do things with it (such as write a score for it)? there are a few ways you can come up with films to consider. one is to ask your local librarian (see above) or professor to recommend reference works or data sources that feature short films. (information about feature films, which run longer, are often easier to find, but there’s a fair bit out there as well on short films.) another is to search some of the reference works and online data sources i’ll mention in the other answers below. the answer to the copyright question depends on where you are. in the united states, there are basically three categories of public domain films: first, there are films copyrighted before . all such films’ copyrights have now expired in the us. this covers most, but not all, of the commercial silent-film era; once the jazz singer came out in , movie studies quickly switched to films with sound. second, there are us films that entered the public domain because they did not take the steps required to secure or maintain their copyrights. researching whether this has occurred with a particular film can be complicated, but because there’s been so much interest in cinema history, others have already researched the copyright history of many us films. the wikipedia article “list of films in the public domain in the united states” cites a number of reference sources you can check for the status of various films. (it also lists specific films believed to be in the public domain, but you should check sources cited in the article for those films, and not just take the word of what could be a random internet user before relying on that information.) third, there are films created in their entirety by the us government. there’s a surprisingly large number of these, in various genres and lengths, with tens of thousands or more digitized in the internet archive’s united states government film collection or listed in the national archives catalog. you can do lots of things with works of the united states government, which are generally not subject to copyright. that’s the situation in the united states, at least. however, if you’re not in the united states, different rules may apply. in edinburgh and elsewhere in the united kingdom (and in most of the rest of europe), works are generally copyrighted until the end of the th year after the death of the last author. in the uk, the authors of a film are considered to be the principal director, the screenwriter(s), and the composer(s). (for more specifics, see the relevant portion of uk law.) however, some countries will also let the copyrights of foreign works expire when they do in their country of origin, and in those a us film that’s in the public domain in the us would also be public domain in those countries. as you can see in the uk law section i link to, the uk does apply such a “rule of the shorter term” to films from outside the european economic area (eea), if none of the authors are eea nationals. so you might be good to go in the uk with many, but not all, us films that are public domain in the us. (i’m not a uk copyright expert, though; you might want to talk to one to be sure.) let’s suppose you’ve come up with some suitable possible films, either ones that are in the public domain, ones that have suitable creative commons licenses or you can otherwise get permission to score, or ones that are in-copyright but that you could score in the context of a study project, even if you couldn’t publish the resulting audiovisual work. (educational fair use is a thing, though its scope also varies from country to country. here a guide from the british library on how it works in the uk.) we then move on to the last question: how do you get hold of a copy so you can write a score for it? the answer to that question depends on your situation. right now, the situation for many of us is that we’re stuck at home, and can’t visit libraries or archives in person. (and our ability to get physical items like dvds or videotapes may be limited too.) so for now, you may be limited to films you can obtain online. there are various free sources of public domain films: i’ve already mentioned the internet archive, whose moving image archive includes many films that are in the public domain (and many that are not, so check rights before choosing one to score). the library of congress also offers more than , compilations and individual films free to all online. and your local library may well offer more, as digital video, or as physical recordings (if you can still obtain those). a number of streaming services that libraries or individuals can subscribe to offer films in the public domain that you can free free to set to music. check with your librarian or browse the collection of your favorite streaming service. i’m not an expert in films myself. folks reading this who know more, or have more suggestions, should feel free to add comments to this post while comments are open. in general, the first librarians you talk to won’t usually be experts about the questions you ask. but even when we can’t give definitive answers on our own, we’re good at sending researchers in productive directions, whether that’s to useful research and reference sources, or to more knowledgeable people. i hope you’ll take advantage of your librarians’ help, especially during this health crisis. and, for my questioner and other folks who are interested in scoring or otherwise building on public domain films, i’ll be very interested in hearing about the new works you produce from them. posted in copyright, publicdomain, questions | comments off on everybody’s library questions: finding films in the public domain build a better registry: my intended comments to the library of congress on the next register of copyrights posted on march , by john mark ockerbloom the library of congress is seeking public input on abilities and priorities desired for the next register of copyrights, who heads the copyright office, a department within the library of congress. the deadline for comments as i write this is march , though i’m currently having trouble getting the form to accept my input, and operations at the library, like many other places, are in flux due to the covid- pandemic. below i reproduce the main portion of the comments i’m hoping to get in before the deadline, in the hope that they will be useful for both them and others interested in copyright. i’ve added a few hyperlinks for context. at root, the register of copyrights needs to do the job the position title implies: build and maintain an effective copyright registry. a well designed, up-to-date digital registry should make it easy for rightsholders to register, and for the public to use registration information. using today’s copyright registry involves outdated, cumbersome, and costly technologies and practices. much copyright data is not online, and the usability of what is online is limited. the library of congress is now redesigning its catalogs for linked data and modern interfaces. its copyright office thus also has an opportunity to build a modern copyright registry linked to library databases and to the world, with compatible linked data technologies, robust apis, and free open bulk downloads. the copyright office’s registry and the library of congress’s bibliographic and authority knowledge bases could share data, using global identifiers to name and describe entities they both cover, including publications, works, creators, rightsholders, publishers, serials and other aggregations, registrations, relationships, and transactions. the copyright office need not convert wholesale to bibframe, or to other library-specific systems. it simply needs to create and support identifiers for semantic entities described in the registry (“things, not strings“), associate data with them, and exchange data in standard formats with the library of congress catalog and other knowledge bases. as a comprehensive us registry for creative works of all types, the copyright office is uniquely positioned to manage such data. the deep backfile project at the university of pennsylvania (which i maintain) provides one example of uses that can be made of linked copyright data. at is a page showing selected copyrights associated with collier’s magazine ( - ). it links to online copies of public domain issues, contents and descriptive information from external sources like fictionmags, wikidata, and wikipedia, and rights contact information for some of its authors. the information shown has no rights restrictions, and can be used by humans and machines. json files, and the entire deep backfile knowledge base, are available from this page and from github. it is not the copyright office’s job to produce applications like these. but it can provide data that powers them. much of our deep backfile data was copied manually from scanned catalog of copyright entries pages, and from online catalogs lacking easily exported or linked data. the copyright office and the library of congress could instead produce such data natively (first prospectively, eventually retrospectively). in the process, they could also cross-pollinate each other’s knowledge bases. to implement this vision, the register needs to understand library standards and linked open data technologies, gather and manage a skilled implementation team, and be sufficiently persuasive, trusted, and organized to bring stakeholders together inside and outside the copyright office and the library of congress to support and fund a new system’s development. if explained and implemented well, a registry of the sort described here could greatly benefit copyright holders and copyright users alike. the register of copyrights should also know copyright law thoroughly, implement sensible regulations required by copyright law and policy, and be a trusted and inclusive expert that rightsholders, users, and policymakers can consult. i expect other commenters to go into more detail about these skills, which are also useful in building a trustworthy registry of the sort i describe. but the copyright office is long overdue to be led by a register who can revitalize its defining purpose: register copyrights, in up-to-date, scalable, and flexible ways that encourage wide use of the creations they cover, and thus promote the progress of science and useful arts. update, march : as of the late afternoon on the day of the deadline, the form appears to be still rejecting my submission, without a clear error message. it did, however, accept a very short submission without any attachment, and with a url pointing here. so below i include the rest of my intended comment, listing top priorities. (the essay above was for the longer comment asked for about knowledge, skills, and abilities.) these priorities largely restate in summary form what i wrote above. if anyone else reading this was unable to post their full comment by the deadline due to technical difficulties, you can try emailing something to me (or leaving a comment to this post) and posting a simple comment to that effect on the lc site, and i’ll do my best to get your full comment posted on this blog. priority # : make copyright registration data easy to use: data should be easy to search, consult, and analyze, individually and in bulk, by people and machines, linked with the library of congress’s rich bibliographic data, facilitating verification of copyright ownership, licensing from rightsholders, and cataloging and analysis by libraries, publishers, vendors, and researchers. priority # : make effective copyright registration easy to do: ensure copyright registration is simple, inexpensive, supports a variety of electronic and physical deposits, and where possible supports persistent, addressible identifiers and accompanying data for semantic entities described in registrations, and their relationships. priority # : be a trusted, inclusive resource for understanding copyright and its uses: creators, publishers, consumers, and policymakers all are concerned with copyright, and with possible reforms. the register should help all understand their rights, and provide expert and impartial advice and mediation for diverse copyright stakeholders and policymaking priorities. other factors: the register of copyrights should also be capable of creating, implementing, and keeping up to date appropriate regulations and practices required or implied by congressional statutes. (for the “additional comments” attachment, i had a static pdf attachment showing the collier’s web page linked from my main essay, as it was on march .) posted in copyright, data, metadata, open access, serials | comments off on build a better registry: my intended comments to the library of congress on the next register of copyrights welcome to everybody’s online libraries posted on march , by john mark ockerbloom as coronavirus infections spread throughout the world, lots of people are staying home to slow down the spread and save lives. in the us, many universities, schools, and libraries have closed their doors. (here’s what happening at the library where i work, which as i write this has closed all its buildings.) but lots of people are still looking for information, to continue studies online, or just to find something good to read. libraries are stepping up to provide these things online. many libraries have provided online information for years, through our own websites, electronic resources that we license, create, or link to, and other online services. during this crisis, as our primary forms of interaction move online, many of us will be working hard to meet increased demand for digital materials and services (even as many library workers also have to cope with increased demands and stresses on their personal lives). services are likely to be in flux for a while. i have a few suggestions for the near term: check your libraries’ web sites regularly. they should tell you whether the libraries are now physically open or closed (many are closed now, for good reason), and what services the library is currently offering. those might change over time, sometimes quickly. our main library location at penn, for instance, was declared closed indefinitely last night, less than hours before it was next due to reopen. on the other hand, some digitally mediated library services and resources might not be available initially, but then become available after we have safe and workable procedures set up for them and sufficient staffing. many library web sites also prominently feature their most useful electronic resources and services, and have extensive collections of electronic resources in their catalogs or online directories. they may be acquiring more electronic resources to meet increased user demand for online content. some providers are also increasing what they offer to their library customers during the crisis, and sometimes making some of their material free for all to access. if you need particular things from your library during this crisis, reach out to them using the contact information given on their website. when libraries know what their users need, they can often make those needs a priority, and can let you know if and when they can provide them. check out other free online library services. i run one of them, the online books page, which now lists over million books and serials freely readable online due to their public domain status or the generosity of their rightsholders. we’ll be adding more material there over the next few weeks as we incorporate the listings of more collections, and respond to your requests. there are many other services online as well. wikipedia serves not only as a crowd-sourced collection of articles on millions of topics, but also as a directory of further online resources related to those topics. and the internet archive also offers access millions of books and other information resources no longer readily commercially available, many through controlled digital lending and other manifestations of fair use. (while the limits of fair use are often subject to debate, library copyright specialists make a good case that its bounds tend to increase during emergencies like this one. see also kyle courtney’s blog for more discussion of useful things libraries can do in a health crisis with their copyright powers.) support the people who provide the informative and creative resources you value. the current health crisis has also triggered an economic crisis that will make life more precarious for many creators. if you have funds you can spare, send some of them their way so they can keep making and publishing the content you value. humble bundles, for instance, offer affordable packages of ebooks, games, and other online content you can enjoy while you’re staying home, and pay for to support their authors, publishers, and associated charities. (i recently bought their tachyon sf bundle with that in mind; it’s on offer for two more weeks as i write this.) check the websites of your favorite authors and artists to see if they offer ways to sponsor their work, or specific projects they’re planning. buy books from your favorite independent booksellers (and if they’re closed now, check their website or call them to see if you can buy gift cards to keep them afloat now and redeem them for books later on). pay for journalism you value. support funding robust libraries in your community. consider ways you can help build up online libraries. many research papers on covid- and related topics have been opened to free access by their authors or publishers since the crisis began. increasing numbers of scholarly and other works are also being made open access, especially by those who have already been paid for creating them. if you’re interested in sharing your work more broadly, and want to learn more about how you can secure rights to do so, the authors’ alliance has some useful resources. as libraries shift focus from in-person to online service, some librarians may be busy with new tasks, while others may be left hanging until new plans and procedures get put into motion. if you’re in the latter category, and want something to do, there are various library-related projects you can work on or learn about. one that i’m running is the deep backfile project to identify serial issues that are in the public domain in less-than-obvious ways, and to find or create free digital copies of these serials (so that, among other things, people who are stuck at home can read them online). i’ve recently augmented my list of serial backfiles to research to include serials held by the library in which i work, in the hopes that we could eventually find or produce digital surrogates for some of them that our readers (and anyone else interested) could access from afar. i can also add sets for other libraries; if you’re interested in one for yours, let me know and i can go into more detail about the data i’m looking for. (i’m not too worried about creating too many serial sets to research, especially since once information about a serial is added into one of the serial sets, it also gets automatically added into any other sets that include that serial.) take care of yourself, and your loved ones. whether you work in libraries of just use them, this is a stressful time. give yourself and those around you room and resources to cope, as we disengage from much of our previous activities, and deal with new responsibilities and concerns. i’m gratified to see the response of the wikimedia foundation, for instance, which is committed both to keeping the world well-informed and up-to-date through wikipedia and related projects, and also to letting its staff and contractors work half-time for the same pay during the crisis, and waiving sick-day limits. among new online community support initiatives, i’m also pleased to see librarian-created resources like the ontario library association’s pandemic information brief, with useful information for library users and workers, and the covid glam discord community, a discussion space to support the professional and personal needs of people working in libraries, archives, galleries and museums. these will be difficult times ahead. our libraries can make a difference online, even as our doors are closed. i hope you’ll be able to put them to good use. posted in libraries, online books, open access | comments public domain day : coming around again posted on january , by john mark ockerbloom i’m very happy for to be arriving. as the start of the s, it represents a new decade in which we can have a fresh start, and hope to make better decisions and have better outcomes than some of what we’ve gone through in recent years. and i’m also excited to have a full year’s worth of copyrighted works entering the public domain in much of the world, including in the us for the second year in a row after a -year public domain freeze. outside the us, in countries that still use the berne convention‘s “life plus years” copyright terms, works by authors who died in are now in the public domain. (such countries include canada, new zealand, and a number of other countries mostly in asia and africa.) many other countries, including most european countries, have extended copyright terms to life of the author(s) plus years, often under pressure from the united states or the european union. in those countries, works by authors who died in are now in the public domain. the public domain review has a “class of ” post featuring some of these authors, along with links to lists of other people who died in the relevant years. in the us, nearly all remaining copyrights from have now expired, just as copyrights from expired at the start of last year. (the exceptions are sound recordings, which will still be under copyright for a little while longer. but thanks to recent changes in copyright law, those too will join the public domain soon instead of remaining indefinitely in state copyright.) i discussed some of the works joining the public domain in a series of blog posts last month, in the last one linking to some posts by others that mentioned new public domain arrivals from . but i’m happy not just because of these specific works, but also because new arrivals to the us public domain are now an annual event, and not just something that happens with published works at rare intervals. i could get used to this. it isn’t all good news this year. the most recent draft of the intellectual property chapter of the us-canada-mexico trade agreement requires canada to extend its copyrights another years, making it freeze its public domain not long after we’ve unfrozen our own in the us. but the agreement hasn’t yet been ratified, and could conceivably still be changed or rejected. and the continued force of copyrights from the second half of the previous ’ s while we’re entering a new set of ’ s is a reminder that us copyright terms remain overlong; so long, in fact, that many works from that era are lost or severely deteriorated before their copyrights expire. but there’s now an annual checklist of things to do for me and for many other library organizations. for me, some of the things to do for the online books page include: updating our documentation on what’s public domain (done) and on what versions of our site are public domain (also done; as in previous years, i’m dedicating to the public domain works that i wrote whose copyrights i control that are were published more than years ago. this year that includes the copyrights to the online books page.) removing the “no us access” notices from books i’d linked to at non-us sites, when i couldn’t previously establish that they were public domain here; and removing “us access only” notices for volumes at hathitrust, which over the next few days will be making -year-old volumes globally accessible without requiring author-death-date review. (this and other activities below will start tomorrow and continue until done.) updating our list of first active renewals for serials and our “determining copyright status of serial issues” decision guide to reflect the expiration of ’s copyrights. as part of this process, i’ll be deleting all the serial issue and contribution renewals currently recorded in our serials knowledge base, since they’re no longer in force. if anyone wants to know what they were for historical or other analytical purposes, i have a zipped collection of all our serial renewals records as of the end of , available on request. they can also be found in the january , commit of this github directory. adding newly opened or scanned books to our listings, through our automated oai harvests of selected digital collections, readers’ suggestions and requests, surveys of prize winners and other relevant collections, and our own bibliographer selections. all of this is work i’m glad to be doing this year, and hope to be doing more in the years to come. (and i’m already streamlining our processes to make it easier to do in years to come.) its the job of libraries to collect and preserve works of knowledge and creativity and make them easy for people to discover, access, and use. it’s also our job to empower our users to draw on those works to make new ones. as the public domain grows, we can freely collect and widely share more works, and our users can likewise build on and reuse more public domain works in their own creations. supporting the public domain, then, is supporting the work and mission of libraries. i therefore hope that all libraries and their users will support a robust public domain, and have more works to celebrate and work with every year. happy public domain day! posted in publicdomain | comments off on public domain day : coming around again vision # : rhapsody in blue by george gershwin posted on december , by john mark ockerbloom it’s only a few hours from the new year where i write this, but before i ring in the new year, and a new year’s worth of public domain material, i’d like to put in a request for what music to ring it in with: george gershwin’s rhapsody in blue, which joins the public domain in the us as the clock strikes twelve, over years after it was first performed. the unofficial song for public domain day turned out to be “yes! we have no bananas”, one of the members of the first big class of us public domain works in the last years. that’s a fun novelty song, and certainly memorable, but not something i necessarily want to hear a lot. in contrast, for me rhapsody in blue has a freshness that makes it a joy for me to hear repeatedly, right from the opening clarinet glissando (apparently the idea of clarinetist ross gorman, who took the scale that gershwin had composed for the piece and gave it the bendy, slidy wail that tells you right away that this is no ordinary concert piece). it’s brought together classical, popular, high-art and everyday music, as it’s been played and recorded countless times by jazz bands (the original scoring is for jazz band and piano), symphony orchestras, and pop musicans like billy joel. even its licensing as an theme tune for an airline hasn’t diminished it. there’s lots of other work joining the public domain along with gershwin’s tune. i’ve only had a chance to mention a few others in my short series, but others have mentioned more works you may find of interest. at the internet archive’s blog, elizabeth townsend gard writes about vera brittain’s not without honour and other works that will be in the public domain very soon. duke’s public domain day post mentions various books, films, and musical compositions joining the public domain as well (and has more to say on rhapsody in blue). wikipedia’s various articles also mention various works that will either be joining the public domain, or becoming more clearly established there. and hathitrust will begin opening access to tens of thousands of scanned volumes from over the next few days. i’ll have more to say on the new arrivals tomorrow, sometime after the midnight bells chime. by tradition, the first tune played in the new year is usually the public domain song “auld lang syne”. but after that, at your new years’ party or at a later public domain celebration, you might enjoy hearing or playing gershwin’s new arrival in the public domain. posted in publicdomain | comments off on vision # : rhapsody in blue by george gershwin vision # : ding dong merrily on high by george ratcliffe woodward and others posted on december , by john mark ockerbloom it’s beginning to sound a lot like christmas everywhere i go. the library where i work had its holiday party earlier this week, where i joined librarian colleagues singing christmas, hanukkah, and winter-themed songs in a pick-up chorus. radio stations and shopping centers play a familiar rotation of popular seasonal songs whose biggest hits are from a surprisingly narrow date range centered in the s. and more traditional familiar christmas carols, hymns, and songs are being sung and played in concert halls and churches well into january. the more “classic” christmas music often feels timeless to those of us singing and hearing it. but while their roots often go back far, the form in which we know them is often much newer that we might think. notice how the list in the previous link, for instance, includes “carol of the bells”, dated . that’s when it was first published as a christmas song, one that’s still under copyright. its roots are older, and darker, as is made clear in a recent slate article well worth reading. as noted there, the melody is based on a ukrainian folk tune (date unknown), its full musical setting composed by mykola leontovych (assassinated by a soviet agent in ), and christmas-themed lyrics written by the ukrainian-descended american musician peter wilhousky (who lived until ). while “carol of the bells” still has a number of years left to go on its copyright, another classic christmas carol will most likely be joining the public domain in the us in just under two weeks. like carol of the bells, “ding dong merrily on high” is based on a folk tune, in this case a secular dance tune first published in france in the th century under the title “branle de l’official”. in , george ratcliffe woodward, an english cleric already known for publishing collections of old songs, wrote lyrics for the tune recalling earlier ages, and included them in the cambridge carol-book, published that year by the society for promoting christian knowledge. charles wood, who’d collaborated with woodward on the earlier cowley carol book, wrote a harmonization to go with it. while you won’t hear it at every christmas service, it remains widely sung this time of year. that’s in large part because it’s so much fun to sing, with its dance-like rhythms, its long bell-like vocal runs on “gloria” (something also heard in “angels we have heard on high“), and its praise of various forms of music (musicians liking to hear good things about themselves as much as anyone else). i don’t actually know for sure that “ding dong merrily on high” is still under copyright here. i have not found a or copyright renewal for the song or the book it was published in, but i’m assuming that, if nothing else, gatt restoration retroactively secured and automatically renewed a us copyright for the song as published in the cambridge carol-book. (folks with more knowledge or legal expertise are free to correct me on that.) later published arrangements of the song may continue to have active copyrights, but only for material original to those arrangements. ’s remaining copyrights, on the other hand, all end in the us on january . (and since woodward and wood both died over years ago, the song’s already public domain in most other countries.) the arrival of , then, should at least clear up any ambiguity about the public domain status of the basic carol. i appreciate that, in part because this song, like many other christmas carols, lives in a sort of liminal space between the private property regimes set up for copyright holders and the older, more informal understandings of folk culture. both kinds of spaces have good reason to exist. on the one hand, it’s good to have more than a few people who can earn a living through music, and one important way many musicians do so is by controlling rights to their compositions. on the other hand, the folk process, which originally gave rise to the tunes for both “ding dong merrily on high” and “carol of the bells”, is also a very good way of creating and passing on shared cultural works. conflict can rage when two different sets of cultural expectations around creative works try to occupy the same space. that’s one reason we’ve seen decades of conflict in academia over open access, where scholarly work is largely published by companies that depend on its control and sale to earn money, while it’s largely written by scholars who earn their money in other ways, and tend to prefer free, widespread availability of their work. sometimes informal arrangements work best to keep the peace. publishers, for instance, have grown more used to free preprint servers, and memes and fan fiction communities have become more widely accepted (and even winning awards) as long as they stay well away from unauthorized commercial exploitation (where both big and small creators tend to draw the line). sometimes, though, it’s best to have a more formal understanding that works are free for anyone to freely use as we like. that’s what we’ll have when ’s copyrights end, and the works they cover, such as “ding dong merrily on high” are clearly seen to be in the public domain. and then, those of us who are so inclined can freely sing “hosanna in excelsis!“ posted in publicdomain | comments off on vision # : ding dong merrily on high by george ratcliffe woodward and others vision # : the most dangerous game by richard connell posted on december , by john mark ockerbloom “be a realist. the world is made up of two classes–the hunters and the huntees. luckily, you and i are hunters.” sanger rainsford speaks these words at the start of “the most dangerous game”, one of the most famous short stories of all time. first published in collier’s magazine in , it’s been reprinted in numerous anthologies, been adapted for radio, tv, and multiple movies, and assigned in countless middle and high school english classes. the tropes established in the story, in which a hunter finds himself a “huntee”, are so well-established in present-day american culture that there are lengthy tv tropes pages not just for the story itself, but for the trope named by its title. up until now, the story’s been under copyright in the us, as well as in europe and other countries that have “life plus years” copyright terms. (the author, richard connell, died just over years ago in , so as of january , it will be public domain nearly everywhere in the world.) anyone reprinting the story, or explicitly adapting it for drama or art has had to get permission or pay a royalty. on the other hand, many creators have reused its basic idea– humans being hunted for sport or entertainment– without getting such permission. that’s because ideas themselves are not copyrightable, but rather the expression of those ideas. and the basic idea long predates this particular story: consider, for instance, gladiators in roman arenas, or tributes being hunted down in the labyrinth by the minotaur of greek mythology. but the particular formulation in connell’s short story, in which general zaroff, a former nobleman bored with hunting animals, lures humans to his private island to hunt and kill them for sport, is both distinctively memorable, and copyrightable. stray too close to it, or quote too much from the story, and you may find yourself the target of lawyers. (but perhaps not if you yourself are dangerous enough game. i don’t know if the makers of “the incredibles“, which also featured a rich recluse using his wits and inventions to hunt humans on a private island, paid royalties to connell’s estate, or relied on fair use or arguments about uncopyrightable ideas. but in any case, disney is better equipped to either negotiate or defend themselves against infringement lawsuits than others would be.) rereading the story recently, i’m struck by both how it reflects its time in some ways, and in how its action is surprisingly economical. in , we were still living in the shadow of the first world war, in which multiple empires and noble houses fell, while others continued but began to teeter. the deadly spectacles of public executions and lynchings were still not uncommon in the united states. and the dividing of people into two classes– those who are inherently privileged and those who are left in the cold or even considered fair game– was particularly salient that year, as the second incarnation of the ku klux klan neared its peak in popularity, and as immigration law was changed to explicitly keep out people of the “wrong” national origin or race. those sorts of division haunt our society to this day. rainsford objects to zaroff’s dehumanizing game in what we now tend to think of the story’s setup, which actually takes most of the story’s telling. (the description of the hunt itself is relatively brief, and no words at all are used to describe the final showdown, which implicitly takes place in the gap between the story’s last two sentences.) in the end, though, rainsford prevails by beating his opponent at his own game. he doesn’t want to kill another human being, but when pressed to the extreme, he adopts his opponent’s rules (at the end giving zaroff the sporting warning “i am still a beast at bay… get ready”) and proves to be the better killer. with the story entering the public domain in less than three weeks, we’ll have the chance to reuse, adapt, and critique the story in quotation more freely than ever before. i hope we use the opportunity not just to recapitulate the story, but to go beyond it in new ways. that’s what happens in the best reuses of tropes. consider for instance, how in the hunger games books, the main character katniss repeatedly finds ways to subvert the trope of killing others for entertainment. instead of prevailing by beating opponents at the deadly human-hunting game the enemy has created, she and her allies find ways to reject the game’s premise, cut it short, or prevent its recurrence. when, in days, we get another year’s worth of public domain works, i hope we too find ways not just to revisit what’s come before, but make new and better work out of them. that’s something that the public domain allows everyone, and not just members of some privileged class, to do. posted in publicdomain | comments off on vision # : the most dangerous game by richard connell ← older posts search for: rss feed pages about free decimal correspondence ils services for discovery applications john mark ockerbloom the metadata challenge recent posts public domain day : honoring a lost generation counting down to in the public domain from our subjects to yours (and vice versa) everybody’s library questions: finding films in the public domain build a better registry: my intended comments to the library of congress on the next register of copyrights recent comments jason on public domain day : honoring a lost generation john mark ockerbloom on public domain day : honoring a lost generation norma bruce on public domain day : honoring a lost generation brent reid on counting down to in the public domain john mark ockerbloom on counting down to in the public domain archives january december march january december november october september july june january december october june january december september january october september july may january january june january october august april march february january december july may january october september june may april january december november october september august july june may april march february january december october september august july june may april march january december november october september august july june may april march february january december november access for all open access news copyrights and wrongs copyfight copyright & fair use freedom to tinker lawrence lessig general library-related news and comment lisnews teleread interesting folks jessamyn west john scalzi jonathan rochkind k. g. schneider karen coyle lawrence lessig leslie johnston library loon lorcan dempsey paul courant peter brantley walt crawford metadata and friends planet cataloging shiny tech boing boing o’reilly radar planet code lib tales from the repository repositoryman writing and publishing if:book making light publishing frontier everybody's libraries blog at wordpress.com. everybody's libraries blog at wordpress.com. email (required) name (required) website loading comments... comment × hublog by alf eatonsearch continuous deployment of a web service on cloud run march , creating and deploying a web service using cloud run's continuous deployment, github integration and cloud build's buildpacks cloud runnode.jsgithubbuilding amd docker images with arm (m ) macos march , using docker buildx bake to build docker images for different system architectures docker"git scraping" data from the office for national statistics api march , fetching and publishing regularly-updated data as a web service with github actions and datasette github actiondatasettecsvsqlitedocker on a raspberry pi december , using armv docker images on a raspberry pi dockerraspberry piarman express app as a web service in cloud functions july , deploying a simple web service to cloud functions node.jscloud functionsan express app as a web service in cloud run july , deploying a simple web service to cloud run node.jscloud runa single-author web app hosted on cloud run june , developing, building and deploying a single-author web app blogjavascriptexpressnode.jscloud rungithubsending a raw https request may , storing, editing and sending a multipart/form-data request over https converting pdf to png or jpeg september , tools and services for converting a page of a pdf to an image how to build a user interface april , the steps of designing a software product may , designing a user interface for moving data from one state to another openid connect march , a summary of the openid connect protocol and its usage for authentication in an spa serving a web application over https february , using nginx and letsencrypt to serve a web application over https janice: a prototype re-implementation of jane, using the semantic scholar open research corpus january , formatting a lacie external drive for time machine january , indexing semantic scholar's open research corpus in elasticsearch january , building an elasticsearch index of semantic scholar's open research corpus dataset a single-user blog october , building a simple blog using react and firebase recovering from a failed macos high sierra upgrade october , oauth in a chrome extension october , es export/import august , exporting/importing/re-exporting es modules styling and theming react components august , using css in js to style and theme react components async is more than await april , symfony forms march , symfony is best at allowing users to apply mutations to resources via html forms polymer + firebase makefile october , a makefile for deploying polymer apps to firebase distributed consensus april , what aaron understood september , what colour is a tree? september , collections of items in time and space fetching web resources september , using resource and collection interfaces to retrieve data from the web quantifying journals september , metrics for scoring and ranking journals it's a shame about google plus september , urls for people distributed asynchronous composable resources september , filling out data tables using promises and computed properties access-control-allow-origin: * april , add the access-control-allow-origin: * header to the data you publish no more documents april , client-side xml validation in javascript april , using an emscripten port of xmllint to validate xml against a dtd in a web browser. organising, building and deploying static web sites/applications march , using jekyll (remote or local) or yeoman (local) to build, serve and deploy a github pages site or application visualising political donations february , using tableau public to visualise donations to uk political parties force-directed tag clouds february , using artists as the dark matter in a graph of tags, to visualise the thematic content of radio shows exploring a personal twitter network january , using gephi to create a network graph showing the most highly-connected twitter friends of those i follow. searching for mergeable tables january , finding tabular data sets that can be merged, using urls for data types uk prospective parliamentary candidates january , the people who will be standing as candidates in the general election creating a map of grade i listed buildings january , filtering an environment agency shapefile to create a custom map uk parliamentary constituencies january , boundaries, names and codes of the uk's parliamentary constituencies the trouble with scientific software december , scientific software is often opaque, and difficult to obtain and cite archiving and displaying tweets with dat september , don't just publish json-ld june , publish plain, simple json, with a linked context document for consumers that want it vege-table: the data table that grows, with leaves may , the easiest, most resourceful way to harvest, explore and publish a collection of data. line-oriented data formats february , iterating arrays february , javascript methods for iterating arrays publishing research on the web january , two examples of publishing code, data and a human-readable report jquery microdata january , a jquery plugin for working with html microdata creating printable cards with html and css december , use html and css to fill a printed card with content post-humanist technology december , if you can't tell why a technology would be useful to you, it's for the robots collecting article metrics with openrefine december , using openrefine to collect article metrics data json templates december , using json templates to describe objects and query by example json-ld december , using context documents to map local property names to shared urls csv on the web, with php december , fetching, parsing and publishing csv publishing, versioning and persistence december , some rules for publishing a resource online select * from web december , ok guha describing objects december , using names and classes as shorthand for object properties switching off hubmed's rss and atom feeds august , hubmed's rss and atom feeds are discontinued web components july , using web components to define custom html elements internet surveillance june , methods of gathering information from the internet. citing articles within articles march , html markup for inline citations in scholarly articles open, social, academic bookmarking: save to app.net february , using app.net's file api to create an open, personal reading library. html metadata for journal articles november , a summary of ontologies for describing journal articles ten years of hubmed november , an overview of the ten years since hubmed was created publishing a podcast using google drive (in theory) september , generate a podcast feed for audio files stored on google drive, using apps script and yahoo pipes publishing articles using gists september , introducing macrodocs.org, a client-side renderer for articles stored in gists music seeds and more like these august , sources for music recommendation; querying by example querying data sets using google bigquery august , using google fusion tables to provide an api to data files august , resourceful web interfaces august , classlessness june , a resourceful alternative to oai-pmh june , adding files to google drive using php may , working with the harvard library bibliographic dataset april , bbc radio -> xspf bookmarklet march , how to text mine open access documents february , open access author manuscripts in pubmed central february , issn(l)s and serial title abbreviations february , extracting text from a pdf using only javascript november , open graph wins the semantic web september , citing with uris in google docs september , client-side pubmed searching july , capturing a manipulated web page with phantomjs march , this weblog in (some) urls march , a modular system for automatic entity extraction and manual annotation of academic papers february , getting and sending binary files with xmlhttprequest december , aoty november , reco: a music recommender october , artists october , creating a single file, lossless rip of a dvd chapter in ubuntu august , london cycle hire data/apps august , writing firefox add-ons with the jetpack sdk july , uk fuel consumption for energy use july , current uk reservoir stocks july , ecryptfs in ubuntu (lucid) june , using stix fonts with @font-face june , inline annotations/formatting in html may , command line twitter authentication using the pecl oauth library may , automatically mounting a remote directory in ubuntu using autofs + sshfs may , a simple hit counter with node.js and redis may , voting correlation (uk general election ) may , uk general election may , installing php . etc on ubuntu karmic ( . ) may , maps at the british library, and on the bbc may , mapstvbillions april , archiving timestamped copies of bookmarked web content march , a wsdl . description of the eutils efetch web service march , phpschemaxmlrest web services, xml and data typing march , phpxmlgoogle bookmarks lists march , googlelistsmapsa solr index of wikipedia on ec /ebs march , ec lucenesolrmapping xml named character references to unicode characters march , a pipe for new episodes in a bbc series march , bbce xjavascriptpipesrdfxmlyahooyqlindependent uk record labels on spotify march , adding spotify links to bbc radio playlists, via rdfa, using greasemonkey and rdfquery march , indexing json data in mongodb using php february , showing delicious bookmarks of pages within a domain february , elasticsearch in php february , describing rest apis with html forms february , the top google search result for each unicode character january , listing unicode characters january , spotify playlist: the hype machine top albums of january , on a bus updated january , using the bing maps web services in php january , publishing files using a public folder in google docs january , web applications january , installing platform-specific applications january , operating systems and application launching january , an os x single site browser with html storage support? january , a basic web app with a settings page, using jqtouch and php january , openurl + opensearch january , map overlays january , mapsthird-party cookies december , spotify lookup and playdar in aoty december , aotyplaydarspotifyopensearch + yql december , importing geoplanet data into mysql december , semantic assistants november , text mining november , soylatte: java . for -bit os x november , javaosxtransforming xml files with xslt . and saxon-he on os x, using an xml catalog october , xmllatest npg articles in pubmed central october , exploring pubchem via sparql october , bacode october , yahoo! apis terms of use changed october , using pubmed's autocomplete data in jquery september , html template september , htmlsheevaplug as a torrent seed box september , graphing weather time series data with timetric september , dataweatherconverting pdf to png using imagemagick or ghostscript august , the music industry (version) august , qr code testing on the iphone august , embedding chemical structure information in image files august , applescriptchemistryusing the tesco api with php august , apiphptopic modelling with mallet august , travel with an iphone august , iphonetravelmarking up a bibliographic reference with rdfa july , entities in scientific news stories june , onabus.com june , annotation of scientific articles june , annotationnow playing in songbird june , musicnow-playingsongbirda private radio archive june , notuberadiodealing with election results data june , adding bing search results to google june , extracting keyphrases from documents using mesh terms and kea june , scraping with yql execute june , scrapingclustering documents with cluto may , exploring an oai-pmh repository may , oaiyahoo! placemaker may , apilocationyahoofetching article citation counts from web of science may , apiphp, dom, dtds and named entities may , phpxmlphp, dom and xml encodings may , phpxmlrecording video from a webcam in ubuntu may , ubuntuvideoquerying bbc programmes in a talis data store may , bbcrdfurioai, yql and json may , phpyqlwhat's the unicode character for "irony"? may , updating local copies of databases and ontologies may , server-side dom scraping with javascript: options april , domjavascriptsolr/lucene on ec /ebs april , ec lucenesolrinstalling couchdb from source on os x april , everything? april , playdar as an openurl resolver? april , audiocoinsopenurlplaydarresolutiongraph of new albums added to spotify april , analysing 'science' bookmarks in delicious march , deliciousposting shared items from google reader to delicious march , deliciousphpresolving urls with php march , phpfinding all occurrences of a utf- -encoded needle in a utf- -encoded haystack march , phpusing yql and pipes to make a screensaver of the big picture march , pipesyqlpages tagged as 'science' on delicious, by co-tags march , delicioussciencepopular pages tagged as 'science' on delicious march , deliciousscienceselecting wikipedia articles by inchi march , chemistryinchirdfcontent hashing march , similarityyql open data tables march , scrapingyqlfestive spotify playlists march , playlistsradioxspftfl feeds march , semantic/scientific authoring add-ins for microsoft word march , publishingsemanticdata, science and stories march , datamusic recipe march , musicdelicious network meme tracker march , deliciouscomparing similar articles and categorisation with wikipedia march , fetching articles from the ny times api march , apiguardian + lucene = similar articles + categorisation march , guardian open platform march , apicloudmade february , mapsan open question to authors of text mining tools february , text-mininghtml + wmv -> xspf + mp february , phpvideoanalysing the tictocs collection of journal toc feeds february , freebase: types, topics, timelines and mentions february , freebaseontologygoogle, jquery and plugin loading february , googlejavascriptjqueryyoumomus february , bigmaps with modest maps january , mapsquestion for a map january , mapsbigmaps with cutycapt and xvfb january , ecsstract: scraping in xulrunner with json/css selectors january , generating standard chemical identifiers (standard inchi) january , pubmed xml in exist on os x january , difficult album titles of january , musicprivacy online: prevent tracking using adblock plus' site-specific filters january , adblockprivacyextracting a certificate/key pair from a java keystore january , spotified sxsw catalog january , greasemonkeyspotifydefining scraper mappings using css selectors january , an annotated timeline of u.s. public debt, using google spreadsheet and google calendar january , datagenerative art in second life january , installing an independent php . to run from the command line january , phpnotes on using the ubuntu ec ami january , events! january , radio now january , iplayerradioubuntu on ec january , displaying new episodes from bbc iplayer january , bbciplayertvzemanta api january , making a lucene index of wikipedia for morelikethis queries january , lucenephpwikipediaalbums of the year collages january , musicend-of-year tv (uk only) january , tvskyrails december , graphnetworkvisualisationspotification december , greasemonkeyuniprot / rdf / sparql december , rdfuniprotgetting a visitor's location (city) december , browse my privates december , firefox . , maxversion for extensions december , firefoxalbums of the year december , nokeepalive december , potatoo december , greasemonkeysongbird links and bookmarks december , libxml , php and utf- december , the most interesting regions in second life november , secondlifeon a bus november , iphonemapstransportsecond life person pseudo-apis november , second life region apis november , secondlifesecond life bigmap november , mouse coordinates bookmarklet november , bookmarkletencoding aac/mp audio files on os x november , audioosxinline wikipedia history, updated november , greasemonkeywikipediaintrepid vs nvidia november , ubuntunational public transport data repository data november , transportroyal mail paf data november , datatransport direct api november , apiphptransportgetting started in second life november , secondlifejsonp, google spreadsheet security october , securityuima october , minimal php script for downloading pubmed xml october , phppubmedminimal php script for downloading pubmed xml (with error checking) october , phppubmedhuffduffer october , who cares about open access october , publishingsciencesecond life: "teleport to camera position" october , second lifevideo encoding recommendations october , videomaximise os x windows with a keyboard shortcut september , query parameters in uris september , logout/login csrf september , pure data september , audiopuredataplaylist builder using freebase suggest september , freebasemetadataweb playlist tool september , mediaplaylistspreprints and categorisation september , creating a freebase data view september , datafreebaserasmus lerdorf on php performance september , phpubiquity pubmed search september , audacity september , php, simplexml, xpath and namespaced attributes september , removing 'for each' from javascript examples september , pubmed json api september , apijavascriptjsonpubmedlinux music players: compilations and watching folders september , audiolinuxbbc aod filter pipe september , audiobbcpipesgopubmed export api september , ubiquity commands september , national rail bus service sparklines september , businfographictransportgmail menuextra ssb in fluid august , veodia august , second lifevideopulseaudio resampling august , audioprojectm-pulseaudio august , audioubuntu,visualisationlondon: cycling and walking route maps august , london: visitors bus map, mobile tfl august , pulseaudio voodoo august , audioubuntuukpa negotiates a licence for commercial music podcasting august , podcastgeonames nearbywikipedia api august , apijavascriptjquerylocationwikipediaaiderss postrank api august , apijqueryuk postcode -> bus stop prototype august , maptransport , bus stops august , locationphpfull-text feeds as a route around censorship august , feedsepub and stanza august , epubliteraturepdfmendeley august , bibliographypdfcreating maptube maps with neighbourhood statistics data august , mapsan amazon wishlist competition/contest august , amazoncompetitionwishlistlisten later updated august , audiobbcextensionfirefoxmobile bus departures august , locationtransportupload to google docs bookmarklet august , bookmarkletgooglegrowl alerts for gmail messages from address book august , applescriptgmailradio comedy feeds august , bbcradiosending a url from safari to firefox august , applescriptosxwriting an atom feed in php august , atomphphow to share a social network august , portabilityprivacylocatory august , britain from above august , bbctvmeta-tv august , tviphone reader for google reader starred items august , feedsgoogleiphonefree august , iphonecount/distinct queries august , mysqlrdfxquerymanipulating forms in google spreadsheets august , googlespreadsheetstiddlywiki zoomable interface august , tiddlywikiiphone interface for delicious network august , deliciousiphonemapping statistics mini-presentation at barcamb august , mapsds game classics july , dsgameslondon age distribution maps part july , mapslondon age distribution maps july , mapsdata uri for a google search box july , googleiphonesecurity email addresses that are black holes july , securitybeware of the app july , iphonesecurityxmpp comments july , xmppupcoming api (php ) july , apiphpsend pdfs from skim to gmail july , applescriptgmailpdflisten direct /programmes july , bbcgreasemonkeyradioinstalling java advanced imaging in ubuntu hardy july , javaubuntuconvert amr files to wav in ubuntu hardy july , audiolinuxubuntuneighbourhood statistics api july , apiphpsoapordnance survey-based bigmap of the uk july , mapartist -> bbc radio shows lookup june , bbcmusicphpradiosoul bubbles june , gamesessential add-ons for firefox june , extensionsfirefoxchris wetherell on google reader june , feedsgooglepod-u-like june , app-enginepodcastsusing google to fetch all of a feed's items june , apifeedsgooglephpfirefox, opensearch and autocomplete june , firefoxopensearchopencalais api june , apiskim: open all with papers june , applescriptosxpdftumblr auto-pager june , greasemonkeywebclipcountupdown june , csssafaricreate a calendar from del.icio.us bookmarks june , calendardel.icio.uspod news june , podcastsbringing a publisher's content to the life science researcher may , presentationpublishingslidestext-miningthe rules of web . may , publications may , app-enginemedlineopensocialpublicationspythonon the rain-slick precipice of darkness, from penny arcade may , gameopensocial terminology may , opensocialmy speediest gatherers updated may , del.icio.usphpupgrading to gmail may , emailwith or without uids may , metadatapresentationsearchslidesxtechi'm feeling unlucky may , googlegreasemonkeymeta latest may , searchdealing with corrupt preference files on os x may , osxrecipebook april , drupalrecipebookrecipesmixingit april , drupalminingphpradiotextgazelle april , musicp pnow playing on the radio april , bbcradioxmpprealplayer for linux april , linuxradiorealplayerpubmed search url april , pubmedhow to make someone fetch a url with a blank referer header april , securityreification april , rdfsecurity against sql injection in wordpress april , phpsecuritywordpresstwubble april , twittergoogle docs april , code quality in contributed drupal modules april , drupalnds april , dsgamessecure password hashing april , drupalopenpasswordssecuritysourcewordpresshow identifight works april , identifightprivacysemgine's mymap for exploring semantic networks of information april , datagraphinterfacerdfvst instruments in linux march , linuxvstgoogle site search bookmarklet march , bookmarkletgooglebookmooch, librarything march , bookssneetchalizer march , audiolinuxlast.fm fingerprinting client march , last.fmmetadatamusicdecentralised music subscription services march , musicdrupalcpp march , drupals e march , tvpenguin march , booksidentifight additions march , identifightprivacyidentifight march , identifightprivacyspokeo march , identityphp script for downloading mp files from iplayer march , bbcphptopcited march , citationpublishingdownload tv shows from the bbc iplayer as mp march , bbctvclimate change march , climate-changeconferenceparticipationlocating london buses march , busesopen-databuilding a "now playing" wall february , amarokxmppmusicbrainz artist info api february , apimusicbrainzphplast.fm artist info api february , lastfmphp"relation" metadata february , metadatapublishingxmpp, publish-subscribe, pep and user tune february , pubsubxmppfull openurl metadata from crossref february , openurlsetting the height of a cross-domain iframe using postmessage february , htmljavascriptno frills fullscreen february , extensionsfirefoxkitte february , designwordpresscrossref citation plugin february , citationcrossref"play in sidebar" firefox extension february , audioextensionfirefoxplaylistvlccoding niggles february , codezowbar february , firefoxmetadatazoterorefactormycode february , codewindows-less? february , linuxmusicrenoisewindowscanon printers in ubuntu february , printubuntulinking to papers february , citationconversationsdisambiguationunpredictability of influence january , xmpp january , firefoxxmppupdating "selected text" bookmarklets january , bookmarkletsfinding conversations around academic publications january , citationconversationscintillacanonical pubmed urls january , pubmedlisten later january , bbcextensionfirefoxradiohow a firefox extension works january , extensionsfirefoxfsdl january , searchsingle window mode january , firefoxmozilla, chrome and fuel january , firefoxfirefox , del.icio.us posting extension january , del.icio.usextensionfirefoxopenurled january , openurlfeed deltas: what's changed? january , feedsblog remix january , musicxspfannotations in xml january , annotationsubmitting author manuscripts to pubmed central january , publishingdepositing nature articles in pubmed central january , natureopen-accesspublishingbpr markup january , citationmicroformatscrowbar january , scrapingzoterothe long arm of copyright january , copyrightgamesall nature papers now available online january , natureaccessing the umlsks soap web service using php january , apiphpcoverflow-ish for newest amarok albums january , amarokphpcommunicating with amarok from a local web page january , amarokhttp post in php january , phparchiving del.icio.us bookmarks january , del.icio.usdrupalprojectm in amarok december , audiovisualisationcontextlinks amarok plugin december , amarokamarok: record labels from musicbrainz december , amarokmusicbrainzpythonamarok: album release dates from musicbrainz december , amarokmusicbrainzpythontv on the internet december , bbctvbbc cross-platform iplayer december , bbcradiotvbest albums of lists december , drupalmusicpresenting replicates in a table december , datahtmlpublishingcharting features december , datajavascriptvisualisationair december , e xeasylistener bookmarklet december , bookmarkletplayraida toolkit entity extraction api december , medlineminingreasons for loving cash music december , musicmyexperiment december , bioinformaticsxnat workflow december , sciencefirefox's sandbox december , firefoxsecurityjquery in zotero november , jqueryzoterothe world is pixels at zoom level november , googlemapconditionally hiding html elements with jquery/css november , jqueryamazon aws api november , amazonapipubchem (eutilities) api november , apieutilsphpcross-platform javascript omissions november , javascriptrenoise november , musicrenoisesideload/mp tunes vs emi november , copyrightmusicpredictive accuracy is substantially improved when blending multiple predictors november , algorithmsserver-side scraping with javascript november , javascriptmetadatacurehunter's graph viewer november , visualisationscraping web pages with php november , phpscrapingmaking a screencast november , screencastvideorev="review" november , citationmicroformatspreserving pdf metadata november , metadatapdfgutsy november , ubuntubpr november , citationmicroformatsstill oink-less november , musicp pted november , p ptvmyspace -> file hd greasemonkey script november , greasemonkeymetadata scrapers october , metadatadel.icio.us / earlier october , del.icio.ushype machine october , musicsongbirdgetting a local copy of medline october , medlinephppubmedbbc radio player as a separate application, with webrunner october , bbcfirefoxradioi forgot my password october , securityxul ftw october , xulfix ssh in mac os x by reinstalling kerberos.framework october , osxmail -> thunderbird october , emailmethods for private atom/rss feeds october , feedssecuritylinux and wireless devices september , linuxwifigmail vulnerability september , emailsecuritythings that taste great together september , notes from drupalcon barcelona september , drupalupdate: artist popularity in specific countries september , lastfmncbi resource locator september , pubmedhcalendar, microformats and google calendar september , microformatsdealing with hard drives in ubuntu september , ubuntulondon cinema today september , cinemadrupalhigh usage of pubmed's "related articles" august , pubmedsearchinline wikipedia history august , greasemonkeywikipediaon the wire has a podcast august , radiomusicsun august , audioscrobblervisualisationnon-destructive faceted browsing august , searchvisualisationquite |kw?t| august , adding random email addresses to facebook august , geocoding apis august , apigeogene network api august , apibioinformaticsphppostgenomic api august , apicitationwhatizit api august , apibioinformaticsphppubmed api august , apiphppubmedclearforest sws api august , apimetadataphpcss workarounds for internet explorer < august , csswikipedia api july , apifreebase api july , apiscopus api july , apicitationrss nightmare july , feedslazytube july , screencastvideopublishing data tables july , datapublishinguser styles july , cssfarewell azureus july , p pepub and adobe digital editions july , publishingif it's ready, release it july , musicopera mini beta july , faceted search in solr/drupal july , drupalsearchmusic.of.interest july , musicpeel sessions july , drupalradioscintilla june , naturescintillacreate a google custom search engine on the fly june , googlesearchxtech science bof slides june , presentationxtechunofficial london rss feeds june , feedslondonbenchmarking php . . string manipulation june , phpbenchmarking php . . string manipulation june , phpexpanding abbreviations in hubmed june , hubmedmahalo june , searchpodule # may , poduleslast.fm listening graph may , lastfmvisualisationpodcast awards may , podcastxss vulnerabilities by pagerank may , securityriaa-safe top may , musicp pcompiling and installing xalan on os x may , osxprogramming language reference widgets for dashboard may , osxwebjay → last.fm playlists may , lastfmplaylistsnotes from xtech may , xtechpredictions/observations for may , mobile feed reader may , feedsreal-time, -bit audio processing may , audioplay this gene may , greasemonkeyitems you rated in amazon may , amazonrecommendationrate items quickly in amazon with greasemonkey april , amazongreasemonkeyxtech april , xtechmultiple "related articles" in pubmed april , pubmedr ds april , ds album covers april , musicvisualisationfetching cover art for a list of albums april , amarokmusicmusicbrainzpythonadd publications from hubmed to publicationslist.org april , greasemonkeyhubmedpublicationsmirror last.fm listening statistics april , lastfmmusicphpmobile mapping/gps april , gpszotero ? hubmed tags april , extensionfirefoxhubmedzoteroownership of user-contributed data march , data-portabilitytouchgraph relaunched march , touchgraphamarok → last.fm links (my first ruby) march , amaroklastfmmusicrubyadding musicbrainz data to an amarok database march , amarokmusicmusicbrainzpythonsmall pieces please march , p pthe sorry state of online music march , musicvideo aggregators march , video -bit firefox on -bit ubuntu march , ubuntuwhat's on your google homepage? march , googlevisual scrapers march , automationscrapingxhtml vs html march , xhtmlfour tenets of web security march , securityone column layouts march , cssfirefox offline browsing march , firefoxpublishinglast.fm user listening data march , lastfmscientific article conversations and distributed libraries february , citationsearch-and-replace february , bashwarning: don't use hpmount february , linuxplos one february , publishingopensearchfox february , extensionfirefoxcopying bookmarks from del.icio.us to connotea february , bookmarksdel.icio.use xgreasemonkeygetting an audio file in another format from an m p on os x february , audioosxmemcached and drupal february , drupaldrupal module for solr february , drupalsearchubuntu edgy, bluetooth and sony ericsson k i february , mobileubuntumame reviews february , drupalgamesa web interface to search and download albums in an amarok library february , amarokphpq: what am i using itunes for? january , musicposting machine tags to del.icio.us january , del.icio.usdrupal january , drupalberyl . beta january , berylubuntubt have been busy january , btthings you need to play arcade games january , gamesdvd::rip january , dvdlinuxnew server january , server days of london widget january , drupallondonosxplos too december , drupalpublishingmetrack december , del.icio.usgreasemonkeyautomatically play youtube videos in a full window december , greasemonkeyyoutubeplaying web video in fullscreen december , playliststits & sharks & acid december , audiomashupnautilus actions december , ubuntuplaying youtube videos in ubuntu december , ubuntuvideosound from a microphone on hda intel in ubuntu december , ubuntumetalicious november , javascriptperlmusicbrainz picard tagger november , metadatamusicmusicbrainzuniform requirements for manuscripts november , citationvisual jquery user style november , cssjquerycheap-ish windows xp november , windowsgeocoding uk postcodes with postcodeanywhere november , geophpyahoo! bookmarks november , bookmarksgmap geocoding uk postcodes november , geolast.fm events calendar october , lastfmnovelty vs necessity october , ubuntufitting in ubuntu october , firefoxubuntuamarok, mysql, json and greasemonkey october , amarokgreasemonkeyphpthe case of the disappearing comments october , wikipedia export format for citing papers from hubmed october , greasemonkeyhubmedunapiwikipediatgn analysis in the lancet october , immunologytgn hubmed speed october , hubmednnw sneak peek release october , netnewswirezotero and compound documents october , metadatapublishingzoterometaphors that have had their day october , google webpage gadgets october , googleprivacydreamhost promotion today october , msn.co.uk doesn't rank firefox october , firefoxsearchi candy september , ubuntuall you need on a (consumer) pc september , appsosxubuntuwindowsubuntu and core duo pcs september , ubuntumigrate movable type to drupal ( . ) september , drupalmtsecurity as a non-admin user in os x september , securitynetnewswire: "mark all as read and proceed" september , netnewswirebuilding a site to handle images in drupal september , drupalbt home hub september , btpodcasts for people who say they don't know any good podcasts september , podcastssharing a list of podcasts september , podcastsfirefox beta september , firefoxwhy is myspace popular august , myspace steps to making myspace nicer august , cssgreasemonkeymyspacescan in itunes august , applescriptitunesgenerate a bookmarklet to automate offprint requests august , bookmarkletpublishingnature.com css august , cssnaturestylishcleanliness july , osxnotate july , annotationpublishingaggademia july , aggregationdrupalnatureunapi link enabler for greasemonkey july , greasemonkeyunapihubmed paper in nucleic acids research july , hubmedsphere it! july , bookmarkletrss feeds for bloglines citation searches july , bloglinesfeedsferret: lucene for ruby july , lucenerubysearch - - data webs conference june , conferencemore about/like this page june , bookmarkletsmapping and tagging greasemonkey scripts june , connoteagreasemonkeygraph your connotea library june , connoteatouchgraphvisualisationxsl files for publishing from nlm xml may , publishingxsllucene . may , lucenesearchmesh information in hubmed may , hubmedmeshpodule # may , podulesquery statistics in hubmed may , hubmedsentence ordering in otmi may , text-mininggoogle co-op may , googlesearchadd hubmed links to google search results may , greasemonkeyhubmedadd radio commands to bbc radio player may , bbcgreasemonkeyradiostructure of a scientific article may , publishingrelated articles algorithms may , hubmedhealth-related queries in google may , googlerecommendations from hubmed may , hubmedrecommendationa plan for publishing journal articles may , publishingtreemaps of medline may , medlinevisualisationplaying streaming radio [realaudio, bbc, os x] through an airport express may , osxa network of politicians and interviewers/journalists on the bbc may , touchgraophvisualisationtouchgraph of bbc tv/radio collaborators april , bbctouchgraphvisualisationrecent papers in hubmed search results april , hubmedpersonalisation and privacy april , hubmedpersonalisationmarkov-chained text from medline abstracts april , medlineitunes alarm clock (ical + applescript) april , applescriptitunesosxemail notifications in gnome april , emailubuntudocument clustering in hubmed april , hubmedrecording streaming radio (improved) april , bashradiovlc, xspf, dapper and tango april , playlistsubuntuopen text mining interface (otmi) april , text-mininginterdb links in hubmed april , hubmedex-html april , htmlhubmed extension for mediawiki march , hubmedctla- -ig march , immunologyadobe xmp sdk beta march , metadatapdfxmppeer review with marginalia march , annotationpublishingfulltext links from hubmed's feeds march , hubmeda week of tv in pictures (comedy, mostly) march , tvcriticker march , filmrecommendationfeedback down march , playr's xspf player march , playrpodule # march , podulestgn march , immunologytgn my most played artists this week, from last.fm march , lastfmxhtml, svg and mathml march , xhtmlgetting document elements out of the clipboard march , bookmarklethobbs on rewind march , lots of comment spam march , mtgetting document elements into the clipboard march , bookmarkletexclusive photo of the new google colander march , googlecopy and paste with unapi march , unapilinking and storing supplementary data march , datanotepress for wordpress march , wordpressconnecting to the nintendo wfc march , dssupplementary data march , databluetooth intellimouse explorer on os x march , osxsound in ubuntu february , ubuntutorrentbot missed some episodes february , p pmanaging metadata for academic pdfs february , bibdeskbibtexmetadatapdfscalable bar charts with tables and css february , cssopenurl for music february , openurlpimp my paper! february , publishingwhere to download firefox february , firefoxsearchuniprot creative commons licensed, available as rdf february , rdfthe state of online biomedical full text articles february , publishingallfulltext february , bookmarkletshubmeda_list of podcasts february , podcaststhe state of biomedical pdfs february , publishingmanaging academic papers (almost) like mp s february , publishingquery expansion in hubmed february , hubmedwebphones february , audioauthor contributions in scientific paper metadata february , publishingupcoming.org simple event posting form february , calendareventsinterview with david lipman of the ncbi february , pubmedbritish albums vs mercury nominees february , musicpodule # january , podulesthings that reek of greatness january , normalising uris january , citationad-hoc xml databases with mysql . january , mysqlxlmpubmed lookup for structure blogging january , hubmednotepressrelevance-ranked search results in hubmed january , hubmedpopulate itunes with webjay playlists update january , applescriptituneswebjaynlm mods january , xsltcreating an atom feed in perl january , atomperltv january , tvcreating an openoffice document in perl january , openofficeperla suggestion for opensearch january , opensearchlistenable retrospectives january , feedsmusicbest albums of january , musicrvwhubmed bibtex changes january , bibtexhubmedciteproxy january , citationidentifiersmodsxmlreading feeds january , feedsosxsoftwaremp blog toplist updated january , blogsmusicvideo chat between mac and pc december , imosxvideoa script for slogger december , bookmarksextensionfirefoxmachineprose december , biomedicalontologypublishingrdfacademic metadata workflow december , metadatanow that's what i call weblogs... vol december , weblogsgoogle music search december , googleupdated opensearch templates for movable type december , mtmore useful firefox extensions december , firefoxdragthing [os x] december , osxsoftwareeasynews search plugin december , firefoxsearchpluginsspacer december , bookmarkletextracting knowledge from biomedical text december , biomedicalhubmedrdftextbioinformatics workflows december , bioinformaticsalbums of snapshot december , musicthings of interest added to hubmed december , hubmedindestructible user profiles december , searchgoggle update december , greasemonkeyrdf interoperability for social bookmarking tools december , feedshubmedrdftagspopulate itunes with webjay playlists november , applescriptituneswebjaysequence manipulation en-suite november , javascriptscienceindex diagnosticus november , searchbest of: bookmarklets november , bookmarkletsstylish november , firefoxbest of: games on os x november , gamesosxbest of: applescripts for itunes november , itunesosxbest of: firefox extensions november , extensionsfirefoxmechanical turking november , amazonrdf export from hubmed tags november , hubmedrdftagscontent negotiation for hubmed tags november , hubmedpiggybankrdftagsyahoo! canada movies feed november , cinemafeedcached web pages and spurl november , bookmarkscacheutf- citation export from hubmed november , hubmeda modular dynamic web page for bioinformatics searches november , bioinformaticsperllast.fm search plugin november , firefoxlastfmdated web page snapshots with my web november , bookmarkletcachemywebsubmitting reviews to google base november , googlegreasemonkeyrvwcreating a citable archive of a web page november , archivebookmarkscitationeeeeeeeeeeeevil november , googlethe wire on resonance fm podcast november , musicplayrpodcastmime types and feed handlers november , feedspodcastspodule # november , musicpodulesfoaf + hcard november , yahoo's my web . november , taggingvisible changes in hubmed this week november , hubmeddisabling caps lock in ubuntu november , ubuntuvisible changes to hubmed this week november , hubmedblogbridge november , softwarejedit november , softwaretemporary feed subscriptions and individual item archives november , feedsopenoffice on os x november , openofficeosxnotepress october , notepressresearchwordpressflock october , firefoxflockhealthline october , healthsearcha definite lack of standards for academic metadata october , metadatalooking for mp s? october , firefoxsearchpluginstweaking firefox for user-side accessibility october , accessibilitycssfirefoxstart.com gadgets october , a couple of music videos on slow servers october , musicpublishing whole documents using an open xml standard format october , publishingxmlsomeone comes to town october , torontosomeone leaves town october , parisfixation october , hubmedrvwtiny greasemonkey script for flickr page titles october , flickrgreasemonkeynews.com graph visualisation for related stories october , anti-personal portal aggregators october , blogpulse alert feed for recently played artists september , audioscrobblerfeedslast.fmmusicpodule # september , musicpodulesopensearch description and atom-based response templates for movable type september , atommovabletypeopensearchsearchubuntu breezy september , breezyubuntureplace the guardian logo september , greasemonkeyopensearch . september , firefoxopensearchsearchsruspyware . september , security september , fruityloopsmusicfirefox search plugin for google blog search september , firefoxgooglesearchpluginsmetadata in feeds (again) september , atommetadatardfreviewsthou shalt not make me squint september , accessibilitycssfirefoxflickr vs yahoo sign-up september , flickrsecurityyahoobookmark folders september , bookmarksfirefoxchanging feed format september , atomfeedsrdfrssdvd ripping to matroska september , dvdmatroskaoggxvidmovable type + tags september , movabletypetagaudioscrobbler browser update september , last.fmmusictouchgraphcoins browser extensions updated september , bookmarkletcoinsgreasemonkeyopenurltv august , torrentbottvsort del.icio.us popular (again) august , deliciousgreasemonkeygetting firefox bookmarks into spotlight august , firefoxspotlightdyld_fallback_library_path (os x) august , osxexport citations from hubmed to refworks august , bookmarklethubmedself-contained firefox search plugins august , firefoxsearchpluginsxsltfirefox search plugin template for a movable type weblog august , firefoxmovabletypesearchpluginstalk to google august , googleprivacyextracting microcontent (xslt, grddl, rdf) august , firefoxgreasemonkeymicrocontentrdfsome handy links for del.icio.us august , deliciouscoins to crossref resolver script august , coinsgreasemonkeymicrocontentopenurla better cite bookmarklet august , bookmarkletcitemicrocontentadding to last.fm with greasemonkey august , flittergreasemonkeylast.fmmusictorrentbot additions august , torrentbotturning a java jar into an application bundle (os x) august , jackson & his computer band (a test of bleep.com's web tools) august , hide flickr comments from specific users august , flickrgreasemonkeyplayr atom feed august , atomplayrpandora august , musicaudioscrobbler & last.fm relaunch august , audioscrobblerlast.fmdel.icio.us firefox extension security update august , deliciousfirefoxpodule # august , musicpodulesserver speed august , debianntrstng (os x) august , flickrperlhide flickr comments (greasemonkey) august , flickrgreasemonkeypublication of cytokines august , converting rtf to plain text (os x) july , osxgatherers of the month # july , deliciousmaking a big old map july , perlplayr's mp blogs section july , playrpodcastsopenurl coins july , coinsopenurlpodule # july , musicpodulesbbc air time july , bbcvisualisationatom . in hubmed july , atomfeedshubmedlinux applications july , linuxsoftwareubuntufirefox form widgets in os x july , firefoxosxatom . july , atomfeedsinsecure rss encryption july , greasemonkeysecuritydel.icio.us inbox in firefox's sidebar july , deliciousfirefoxos x url handler to open links to local files july , applescriptosxwordpressflickr pro accounts july , almost everything about hubmed july , hubmedstatistics in nature immunology july , publishingstatisticsitunes podcasts june , itunesplayrpodcastsextracting microcontent june , greasemonkeymicrocontentit's about the catalogue june , musicp peasynewzbin june , greasemonkeyhubmed tag storage june , hubmedtaggive us a big back! june , cssfirefoxspotlight june , osxsemantic weblog posts with movable type june , movabletyperdftv june , tvmore feeds please, vicar june , feedsclient-side m u generation june , playrgoggle update may , googlegreasemonkeyoperation d-elite may , p pweb assistants may , firefoxrdf data in hubmed may , hubmedpiggybankrdfparis may , mapsparisg.w.a. may , googleprivacyfof may , feedonfeedsabout reviews and microformats may , microcontentreviewsconcatenating multiple mp s into one big playable mp may , musicsoftwarepodules may , musicpodcastspodulestag search may , searchtagbagram may , m.i.a. loop may , musicgoggle may , firefoxgooglegreasemonkeyscreencast of hubmed and bibdesk may , bibliographyhubmedscreencastpubmed rss may , automator plug-ins may , osxgot adblock? may , firefoxopen sourcing apis april , opensourcethe hype machine april , musicemail reply notifications (mail and growl) april , applescriptgrowlosxhow to get a firefox that works (os x) april , firefoxosxtargeted advertising april , cssnetnewswireplay lhb daily downloads april , playrbrowser anti-aliasing april , firefoxitems for consideration april , bibliographyidentifiersopenurlsidÑ”É³otÑ” april , osxsoftwareskinning del.icio.us with firefox and uriid april , deliciousfirefoxfirefox search plugin for audioscrobbler april , audioscrobblerfirefoxsearchpluginssend me a file april , gpgjavascript benchmarks april , firefoxosxsafariswitching from safari to firefox april , firefoxosxfirefox search plugin installer april , firefoxsearchpluginsbest albums of april , musicreviewsprefetch google ad links april , googlegreasemonkeynew rvw! april , deliciousreviewsdeliciousify audioscrobbler april , audioscrobblerdeliciousgreasemonkeylesinrocksparis march , concertsfeedsparisflash + xspf in playr march , xspfupcoming api march , tv march , tvupdates march , why upcoming.org isn't more popular march , upcomingopensearch march , feedssearchflitter v . march , flittersearchgive me all your cookies march , greasemonkeyopenurl resolver bookmarklet march , bookmarkletopenurladd search links to audioscrobbler artist pages march , audioscrobblerbookmarkletgreasemonkeyartist popularity in specific countries march , audioscrobblermusicvisualisationremove pdf delay for journal articles march , pithhelmethubmed utf- export march , bibtexhubmedrisutf- us vs uk band popularity february , audioscrobblermusicvisualisationblackwell's 'author pays' publishing february , biomedicalskip sourceforge delay page february , pithhelmethide selected content with css (in the future) february , cssnetnewswiressl certificates for apache , courier, exim and jabberd on debian february , debiansslxml-based book authoring february , softwaresubversionwordpress . february , softwarewordpresschilibot february , biomedicaldatavisualisationradio rss feeds february , bbcfeedsradiobd graphit floating (modified) style for netnewswire february , cssnetnewswiretorrentbot moved february , torrentbotsorted lists of reviews, using rvw! and del.icio.us february , deliciousreviewsflickrdesk update february , flickrsoftwaresorting del.icio.us/popular february , bookmarkletdeliciousmap interfaces february , googlemapsgre.gario.us february , deliciousdynamic openurl resolver links february , openurlpubmed tabs and amplify february , biomedicalosxpubmedsoftwaremy speediest gatherers february , deliciousgatherers of the month february , deliciousswervedriver oddities february , musicpithhelmet hubmed redirect january , hubmedpithhelmetordnance survey copyright annoyance (again) january , mapsmfeeds january , feedsmusicplayrbeagle january , linuxsoftwarewpa january , ubuntumore paris rss feeds january , feedsparisoclc software contest january , a 's street photos january , mapsgraph del.icio.us subscriptions network january , deliciousvisualisationgraph del.icio.us related tags january , deliciousvisualisationexplosion january , playrm ucast update january , m ucastosxplayrsoftwaregames toplist january , deliciousblogresearch toplist january , deliciousmp blog toplist january , deliciousmusicyesterday's kexp january , musicradioflitter with del.icio.us links january , deliciousflitterlugradio interviews mark shuttleworth january , radioubuntucoral too slow january , p pplayrflitter with band photos january , flitterflickrdesk update january , flickrsoftwarejavascript drag-and-drop ordered lists january , javascriptunshuffly ipod january , svg maps from pdf january , mapspdfsvgxspf + swf january , xspflistmania january , liststechnorati touchgraph january , touchgraphvisualisationa mini svg map january , mapsparissvgupcoming.org for paris concerts january , feedsparisupcomingbreezy listening january , flitter bookmarklet january , flitteritunes music store gift certificates january , musicp pbands of january , musicbe the coolest december , bookmarkletdeliciouscatching up december , feedsmusicp ptvlocalopenurl and localsfx for hubmed pages december , openurlxslt export from omnioutliner pro december , osxsoftwarexsltparis concerts rss feed update december , concertsfeedspariscompiled live swervedriver albums december , search a restricted set of feeds december , feedssearchgoogle reviews december , googlereviewswi-fi ipod december , musicp phubmed tutorials? december , hubmedamiga emulation december , emulationwindows applications december , softwarewindowsvennmaster december , biomedicalvisualisationworlds apart december , musicflitter december , musicmusic in del.icio.us december , playlistshorizontal amazon music thing december , amazonpricenoia december , amazonucomics atom feeds for netnewswire november , feedperlgoogle scholar bookmarklet november , bookmarklettouchgraph browser for amazon citations november , touchgraphciteseer oai compliance november , citeseertouchgraph browser for google scholar november , googletouchgraphvisualisationpaper cd case november , john peel's final show november , musicadium groupchats november , links to google scholar november , free shipping for books at amazon france november , more tv november , mark steel lectures november , os x applications november , osxclusty pubmed november , dowser november , momentum november , citeulike november , a decent setup for writing (os x, with latex) november , sending files from one computer to another november , motorcasting november , fluxpod november , visitors october , flickrdesk: daily updated desktop pictures from flickr (os x) october , define the lie october , netgear wg fs and airport express on ubuntu october , ubuntu october , m ucast october , desktop pictures from flickr, using magpie and os x october , making podcasts from mp blogs october , what's in your menubar? october , realplayer update october , fulltext links from hubmed october , must... resist... october , torrentbot missed episodes september , bookmarklets september , review sites reviewed september , reviewsfirefox extensions updated september , glit_ september , hercules/powervr drivers september , storing or aggregating microcontent september , netnewswire . beta september , feed your reader september , mp .com community features september , albums for people who are bored of music september , deliciouspresidential candidates on science september , a buttload of bootlegs september , bootlegsnew web software releases september , softwaremolecular systems biology - a new open access journal from npg september , open-accessmedical literature info sorting overload september , aggregate feeds as impromptu record labels september , musicdivision of laura lee september , musicgoogle vs medline september , pubmedthe return of stg september , p pdocco september , documentssearchvisualisationdiebold's tamper-conducive vote counter august , securitycoral august , cachedata retention august , googleartificial meme tracking august , memenobel laureates call for open access to public-funded research august , open-accessthe p challenge august , radiofactory resetting an airport express august , appleibook/powerbook + ipod rebate august , appledevendra banhart august , musicthings google knows about you august , googleblogger's next blog tour august , ingentaconnect august , rss munging with urchin august , aws . beta august , rilo kiley + others august , i wish i had a tripod august , the guardian digital music survey august , alac droplet august , some people say 'tax the rich' august , audio channelling august , osxplanet is fantastic august , flac → alac august , what to do if you're running out of bandwidth august , rss feeds of playlist recommendations from playr august , inducement to hymn august , justeport streams audio to an airport express august , create a wishlist of videos to rent august , allmusic desertion august , caching media files august , brain hacks august , joggle tellybot august , tagged reviews august , cover art in del.icio.us rss feeds august , sidebar stars august , meme destruction project august , rvw! tool repurposed august , deliciousreview of an airport express august , allconsuming book list july , connected bands july , cross-cluster navigation july , nodule organiser july , hallowed be thy game july , punk voter playlist july , pagerank sites july , pubmed search field tags july , pdf warning from css july , a . gb torrent july , open source + television july , drm denial july , cachem u july , mobile rfid reader july , feed autodetection in firefox july , ipapers july , biomail july , omg - nomusic july , html wget m u mpg july , bbc news video console july , wget m u playlist files july , mp blog playlists july , simpler todo rss feed july , music to go to sleep by july , audioscrobbler browser bugfix july , twitch july , crossref/google search july , choose your open access with springer july , a ghost is born july , realplayer beta for os x july , getting the web out of the browser july , data auto-detection in browsers july , identifying papers with content hashes june , ris format confusion june , document publishing diagram june , hubmed.org june , apple tiger june , compressing pdfs containing colour images june , server switch june , mail.app, imap and courier june , torrentbot updates june , live music from the sonar festival june , webjay alarm clock june , clevercactus share june , soap interface for e-utilities june , suprnova discographies (-ish) feed june , firefox . june , favourite albums at/of the moment june , os x root email account forwarding june , rvw . june , jem archives june , blogdigger media june , muziekhobbyist webradio june , your kids need drugs june , pocket radio june , goliath x june , scopus links from hubmed june , drupaled and drupalblog june , ome june , hubmed history tab june , music file data hashes june , all back to mine may , rvw! formatter may , détente may , boosh on tv may , elsevier author self-archiving may , server co-op may , pnas offers open access option may , real bbc radio may , that man will not hang may , bastet may , swervedriver may , mp .com may , safari vulnerability may , google groups atom feeds may , printing from windows to a shared cups printer on panther using samba may , endnote incompatibility may , infomediaries may , sente: like itunes, for biomedical literature may , penance soirÃ©e may , red cross reports may , dropload may , fillable, home-networked file servers may , hubmed search box may , rss feeds for new album releases may , rock ahoy may , a big red blip may , synchronized multimedia working group may , printer sharing may , the golden apples of the sun may , hey hey k may , franz ferdinand - - may , prefuse visualisation toolkit may , electric chill noise may , searching scientific papers online may , unbinding may , ipod pricing may , returned-to albums may , fnac listings in paris concerts rss feed may , be not afraid april , blosxom april , itunes . april , flickr photo badging april , geneinfoviz april , inline del.icio.us april , random mp .com playlist april , inline musilog april , academic pdf workflow april , collaborative playlists april , current advantages of im clients april , follow mouse focus in x windows april , dafont april , lab notebook database system april , genotyping a meme april , os x fonts in gtk /gimp april , google ads april , unicode vs latin- april , pixies reunion show april , pdf browser plugin april , culturepool april , imagemagick april , musicompass april , tv torrent/rss automation april , spillsbury april , fibonacci ratios and musical intervals april , a shared word processor-bibliographic manager interface april , musiclogging from winamp april , torrentbot pt april , torrentbot april , smell the satire april , musiclogging april , mltorrents bookmarklet april , wonderfalls april , ml_www april , outcesticide set april , the live music archive is huge april , repaying generosity april , bittorrent command line client on os x april , bitcollider april , entrez search updates april , mediaseek april , link extraction bookmarklet for webjay march , user-centric data services march , aibrainz march , amg new releases march , gimp.app march , album cover art tagging for windows march , tv shows march , openurl router march , text mining march , science commons march , remembering konspire march , music ownership in an open, online database march , perlprimer march , music publishers, sales and metadata march , atomly march , hubmed svg graphs march , free software directory march , liars march , playr update march , time shifting march , pyget***** march , shn and flac tools march , x on os x march , acknowledgements march , latex add-ons march , latex for dummies march , latex for the modern age march , how to find (more of) what you want march , ipod whamb skin with volume march , join groups, find better music march , propa' mash-up ragga-rave dubplate bloodclot jungle tekno tour-de-force march , semantic hifi march , document handling linkdump march , /cores march , sparklines march , closed-access data from open-acces publications march , hubmed atom feeds march , power laws and purchasing priorities february , one world in march february , audioscrobbler browser update february , darkplace february , reviewskimya dawson february , netlabel catalogue february , releasing mac word . february , groupthinking, but on which side? february , the advancement of science and culture february , ranchero's big cat scripts plugin february , movable type 'edit this entry' bookmarklet february , advancemame february , zoom player february , latex february , gtk/panther february , ram february , yahoo search bookmarklet february , mperia february , jukebox mp collection for £ . february , recommend playlists with flickr february , berlin bastard lesson february , raster noton february , reviewr february , reviewssubviral rna february , america's sweetheart february , dear microsoft word february , listening post february , deepvacuum [os x] february , throttled [os x] february , citespace february , open-source audio, tools for endless music [os x] february , omniamea february , endnote pubmed import filter february , compiling an mp -playing helix client on panther february , helix developer grants february , netnewswire has no atom february , ecto final january , betterpropaganda january , quicktime for m u january , kazaa/kapsule test needed january , emotilinks january , usb floppy drive with red hat linux january , install classic after panther january , primer design for cdna amplification january , vienna rna january , it's called i like january , webjay.org january , nothing in return january , playlist bookmarklet update january , sente january , automatically update itunes library with daapd january , earth map desktop january , macgde january , musicplasma january , itunes music store rss generator january , mtcommentauthorlink january , m u playlists page january , artemis sequence viewer january , itunes opener january , jpegs progress january , pick of the bastard pops january , realplayer installation january , copper, prions and tse january , suprnova rss feed january , hubmed print-friendly pages january , scientific stories january , mp to m u or smil playlist january , return of the mac january , kx project and avg january , firebird tabbrowser extensions january , morality of software patches january , transmission january , pitchfork singles of january , are you feeling throaty? january , & free hosting january , largehearted boy january , what a difference a year makes january , linking to musicbrainz january , fountain of youth january , collections of music files from distributed sources january , trying this again january , ibook display problems? january , festive fifty january , happy new year january , some contributions to saving the internet december , who represents my points of view december , scientists for dean december , social software interfaces december , movable type plugins december , using religion for aggression december , playlist distribution december , gpsweb december , rpxp web service december , mldonkey [os x] december , vocal removal december , pester december , make gtk apps pretty december , mad plugin eats track numbers december , the signaling gateway december , movable type on windows xp december , xstream radio december , sourceforge december , no fink december , fluorescence microscopy movies december , gofigure december , rumoured demise of biomednet december , semblogbibman december , not safe for work december , rdf for last played tracks, via audioscrobbler december , mp blogs are switched on december , rss feed for paris concerts december , google irregularity december , festival octopus december , azureus december , day audio december , kid video december , album continuum december , laptop dj december , firebird [os x] december , playlist tracklisting update december , weasel december , electromagnetism december , qotsa/kyuss circle of collaboration december , musictouchgraphbbc radio interface november , the shins on kcrw november , unfree the music november , movable type spam vulnerability november , french-speaking weblog rankings november , qtfairuse november , got foaf november , cgi_buffer november , blam for trial november , vorbis updates november , bibdesk november , the perfect email november , singingfish november , glc november , bleep.com november , free- or donation-ware updates for panther november , you will require... november , xml for individual entries november , eugene garfield commentaries november , styling rss with css to make it browser-friendly november , scidev open access section november , reviews-enabled movable type november , sublime electronica november , excel add-in to remove low numbers november , fixed the blaxm reviews exchange november , blogware with reviews metadata november , a tune called grin october , longhorn october , itunes playlist hint october , facil-o-smil update for m u and cc october , phrase searching in pubmed october , the knowledge society october , flowjo october , wow. a big clock. october , playlouder msp october , soulseek recommendations october , winamp october , pdc pokemon october , fink upgrade for gcc . october , google glossary october , plos biology october , chutes too narrow october , constant playlist october , facil-o-smil october , weed october , gnu privacy guard october , os x im: msn is on october , plos biology trackbacks in hubmed october , plos biology october , from the ashes october , stop the leaks october , tuna october , empty pages in search results october , steam [os x] october , calendar events from xhtml october , sound october , in defence of open access october , emusic pricing changes october , digital accretion october , daily mp s from pitchforkmedia october , albums of the year (so far) october , dynamo playlist october , pure data dsp software october , open source democracy october , bad spiders october , mercora october , boom selected october , headcloud october , flat-fee p p model october , neuro-info-transmitters october , what's on my docks? october , metadata in the metaweblog api october , rdf review vocabulary october , the wellcome trust supports open access october , shareable playlists october , bloglines recommendations october , mini-links rss feed september , research mapper september , downstream september , iwebcal september , dynamic event files september , freak up, look smart september , x goodies september , export an event from a web page to ical september , terminally ill september , ical events from web pages september , fingertips september , natureevents september , kde september , share the music september , jumbled words september , equinox september , syncato september , wsil for blogroll autodiscovery september , tv listings and audio streaming licensing september , worst jobs in science september , digital marketplace summary september , uk data surveillance measures september , intellectual is not physical september , collective payment september , icaris september , the importance of open access for semantic research september , devonagent [os x] september , polished turds september , fame *and* fortune (if you're good enough) september , open source bibliography format september , biologging september , science and religion forum september , scientific publishing september , konspire radio channel september , subscribe to comments september , trackbacks september , nicotine september , waypath september , my.pubmed rss feeds september , musical interlude september , biotech protocols september , smokescreen september , openam.com subdomains august , armagetron tron clone august , netnewswire with webkit august , fun with xmltv august , peel sessions august , the return of openam august , bbc creative archive august , radiolaw enforcement against prohibition august , earthstation august , msn network rejigged august , tofu august , album cover artwork august , freakmachine august , human knowledge navigator august , peer review under scrutiny august , tools for handling information august , mcode august , classification of associations august , morale-o-meter august , miranda im august , prism for rdf august , who's going to pay? august , perception august , blufilter august , trillian pro . beta august , protein interaction browser august , music browser repaired august , august , jobs as rss extensions august , quick release august , test august , openam linking july , rock-it launcher july , aol journals july , myths and legends of file sharing july , os x show desktop july , the same thing, again july , digital sales network july , yapc july , buy back continues july , blosxom.com july , vague memories july , amazon tracks search box july , perl culture july , we're all going straight to hell :-) july , biomed central links july , faculty of links july , open bibliography software july , musical artifacts july , wiretap july , the marigolds july , the holy grail july , audioscrobbler + last.fm july , safari fullscreen bookmarklet july , biomed central articles in one big zip july , negative feedback on ebay july , copyright for scientific papers in eprint archives july , performance at the cost of expansibility july , i (heart symbol) mp july , when fireworks attack july , online electronic hardware stores july , touchgraph livejournal browser july , zane lowe on radio july , clutter update july , endnote v july , rss legacy july , did you know? july , open access conference reports june , phone gps june , searching for the social benefits of technological progress june , eff seeks p p licensing scheme june , cites and insights july june , public access to science act june , blosxom rating plugin june , pithhelmet june , nitle blog census api june , blogs ! us june , concept clustering june , molecular graphics on os x june , how i got soulseek to work on os x june , handy hints june , public library of science june , id card consultation figures june , nature pdf content extraction june , openurl draft standard june , politics and the english language june , four tet favourites june , costs of illicit mp downloading june , spoogefest june , unicode characters in hubmed june , paid for software june , site redesign june , ontologies in scientific research june , concert listings june , pdf annotation june , spared from internet hell june , political positioning june , technicalities of a p p music market june , modelling social interactions june , andromeda on os x may , back on track may , rvw specification may , jack valenti says may , kast/konspire b may , come together may , emergence may , itunes script may , emusic signs beggars group may , principles of emergent democracy may , lo-fi may , jabber notification of new referrers may , lamebrain may , kwiki and voodoopad may , music recommendations may , technorati api in blaxm! may , if they want to do this the hard way... may , sfx/openurl interview may , advertoys may , nodalpoint - moderated bioinformatics papers from pubmed may , rvw success may , test review for rvw markup in rss . may , photopal may , geograffiti may , rvw format in rss . may , video sans frontieres may , the scientist in rss may , rvw format in rss may , arrowsmith may , winamp aac/mp input plugin may , isuck may , itunes, again may , global friendster visualisation may , peer-to-peer search spidering may , drm within aac files may , processing soda may , improving science through online commentary may , rvw standard metadata format for reviews may , dj martian's page may , itunes download may , fos news catchup may , scrobbleyou may , on the wire may , emusic upgrade may , itunes may , electric six - fire april , itunes music store top downloads april , cd industry seeks niche april , echocloud april , music licensing april , semantic blogging demonstrator april , modular, extensible rdf april , touchgraph audioscrobbler browser april , antisocial behaviour in online communities april , laszlo april , the world live web april , finding people april , the wipers - box set -- is this real april , librarians on the offensive april , environmental noise retards auditory cortical development april , the liberation will not be nationalised april , fire with intent april , thinkbot april , globe alive april , journal of mammalogy april , open access april , equator april , last.fm april , completion of the human genome project april , rdf braindump april , wavefinder, dab april , microsound april , sumeria april , george boosh april , terrestrial jukebox april , not content april , summarise this april , queens of the stone age - feel good hit of the summer april , many to many april , digital video april , digital music streaming april , an excellent lab web page april , w c drafting, drifting april , internet explorer april , winamp . april , phoenix april , clarity of writing april , a miscommunication with civilians april , a few blam! and blaxm! updates april , not in my name ++ april , some radio shows april , automata and visualisation april , complexity digest rss april , political fiction april , distributing music on plastic discs one album at a time april , discussion from cto forum april , blueprint for phased access journals april , mp ripping and encoding benchmark april , new clinic album april , a tune april , rock & roll library april , more sites on sticks april , the day music became priceless april , blaxm!, foaf, rss march , mp track ids march , death of an activist march , iraqi opposition march , web applications march , anacubis visual google march , beos file system with metadata for os x march , bioinformatrix march , acromed march , foaf browser march , thinkbot march , lock down march , empire march , a global discussion forum, by invitation only. march , techgnosis march , blam! + radio march , blam! + blogger march , xnap hint for os x march , standardised review metadata march , blam! + moveable type march , oai searches from hubmed march , newzcrawler update march , imdb moveabletype hack march , amazon cd track listings march , not unexpectedly pleasant march , blam!: amazon review creator march , research buy-back march , endnote march , mp sushi march , but why? march , making money march , sciencedirect backfiles march , a simplified valuation of commoditised art march , apple java hooray march , ibook usb fm radio tuner march , more mini-things march , digital collection and peer review march , spirographx march , biopedia march , the ends of the internet march , biologging part march , oral traditions in online communication march , value of music march , biologging march , spiders march , citation trackbacks march , keyboard shortcuts march , giftbox march , cnps march , allabstracts bookmarklet march , citation maps february , science citation index february , medscape headlines in rss february , iscrobbler february , open access literature part iii february , open access literature part ii february , visualisations of political polarisation february , better late than never february , andromeda/php on os x february , open access literature february , hublink february , allmusic-to-magnet-uri bookmarklet february , semantic blogging and bibliographies february , linking services february , nice titles february , endnote import filter updated february , endnotehubmedexceptions to copyright february , latent semantic indexing february , fair use february , proper p p february , taking the internet outside february , the infography february , safari cookies february , cookiessafaritoc alerts january , alertsnewsreaderpushrsszetoc evaluation january , paratools january , citationparaciteparsingintegrated comments and trackbacks january , commentstrackbackfixed touchgraph scripts january , applettouchgraphvisualisationcitation parser update january , citationparsereferencescollaboration network browser january , analogies with trackback variants january , analogiesbiomedicalliteraturenetworksself-organisingtrackbackmake a list january , collaborativelisttrackbackweblogedina join-up january , openurlcitation matcher updated for multiple references january , citationparseris citation export file suffix january , exportfileriscitation matching january , citationopcitparsehublog rss update january , hublogrsstrackback january , trackbackalternative software for community-driven literature management january , blogcommunitysitesoftwarepersonal/group publishing january , knowledgeliteraturepersonalpublishingwebimmunolog launched january , collaborativejournaljoining the dots - advances in online biomedical literature management. january , biomedicalknowledgeliteraturemanagementsafari, touchgraph update january , touchgraphsfx lookup bookmarklet january , bookmarkletsfx - - : perl scripts for organising pdfs january , acrobatperl - - : library lookup issn bookmarklet january , bookmarkletlibrarylookup - - : experimental links january , citationdoi - - : gnutella p p january , gnutellamagnetp p - - : bibtex output january , bibtexpubmed - - : endnote and ris import filters january , endnoteexportpubmed - - : related articles algorithm january , articlespubmedrelated - - : linkout urls january , fulltextlinkoutpubmed - - : pubmed javascript january , javascriptpubmed - - : hubmed online. january , perlpubmedutilitiesxml none a quantitative analysis of the impact of arbitrary blockchain content on bitcoin roman matzutt , jens hiller , martin henze , jan henrik ziegeldorf , dirk müllmann , oliver hohlfeld , and klaus wehrle communication and distributed systems, rwth aachen university, germany, {matzutt,hiller,henze,ziegeldorf,hohlfeld,wehrle}@comsys.rwth-aachen.de data protection research institute, goethe university, frankfurt/main, muellmann@jur.uni-frankfurt.de abstract. blockchains primarily enable credible accounting of digital events, e.g., money transfers in cryptocurrencies. however, beyond this original purpose, blockchains also irrevocably record arbitrary data, rang- ing from short messages to pictures. this does not come without risk for users as each participant has to locally replicate the complete blockchain, particularly including potentially harmful content. we provide the first systematic analysis of the benefits and threats of arbitrary blockchain content. our analysis shows that certain content, e.g., illegal pornogra- phy, can render the mere possession of a blockchain illegal. based on these insights, we conduct a thorough quantitative and qualitative anal- ysis of unintended content on bitcoin’s blockchain. although most data originates from benign extensions to bitcoin’s protocol, our analysis re- veals more than files on the blockchain, over % of which are texts or images. among these files there is clearly objectionable content such as links to child pornography, which is distributed to all bitcoin partic- ipants. with our analysis, we thus highlight the importance for future blockchain designs to address the possibility of unintended data insertion and protect blockchain users accordingly. introduction bitcoin [ ] was the first completely distributed digital currency and remains the most popular and widely accepted of its kind with a market price of ∼ usd per bitcoin as of august st, [ ]. the enabler and key innovation of bit- coin is the blockchain, a public append-only and tamper-proof log of all transac- tions ever issued. these properties establish trust in an otherwise trustless, com- pletely distributed environment, enabling a wide range of new applications, up to distributed general-purpose data management systems [ ] and purely digital data-sharing markets [ ]. in this work, we focus on the arbitrary, non-financial data on bitcoin’s famous blockchain, which primarily stores financial transac- tions. this non-financial data fuels, e.g., digital notary services [ ], secure re- leases of cryptographic commitments [ ], or non-equivocation schemes [ ]. however, since all bitcoin participants maintain a complete local copy of the blockchain (e.g., to ensure correctness of blockchain updates and to bootstrap new users), these desired and vital features put all users at risk when objection- able content is irrevocably stored on the blockchain. this risk potential is exem- plified by the (mis)use of bitcoin’s blockchain as an anonymous and irrevocable content store [ , , ]. in this paper, we systematically analyse non-financial content on bitcoin’s blockchain. while most of this content is harmless, there is also content to be considered objectionable in many jurisdictions, e.g., the depic- tion of nudity of a young woman or hundreds of links to child pornography. as a result, it could become illegal (or even already is today) to possess the block- chain, which is required to participate in bitcoin. hence, objectionable content can jeopardize the currently popular multi-billion dollar blockchain systems. these observations raise the question whether or not unintended content is ultimately beneficial or destructive for blockchain-based systems. to address this question, we provide the first comprehensive and systematic study of unin- tended content on bitcoin’s blockchain. we first survey and explain methods to store arbitrary, non-financial content on bitcoin’s blockchain and discuss poten- tial benefits as well as threats, most notably w.r.t. content considered illegal in different jurisdictions. subsequently and in contrast to related work [ , , ], we quantify and discuss unintended blockchain content w.r.t. the wide range of insertion methods. we believe that objectionable blockchain content is a pres- suring issue despite potential benefits and hope to stimulate research to mitigate the resulting risks for novel as well as existing systems such as bitcoin. this paper is organized as follows. we survey methods to insert arbitrary data into bitcoin’s blockchain in section and discuss their benefits and risks in section . in section , we systematically analyze non-financial content in bitcoin’s blockchain and assess resulting consequences. we discuss related work in section and conclude this paper in section . data insertion methods for bitcoin beyond intended recording of financial transactions, bitcoin’s blockchain also allows for injection of non-financial data, either short messages via special trans- action types or even complete files by encoding arbitrary data as standard trans- actions. we first briefly introduce bitcoin transactions and subsequently survey methods available to store arbitrary content on the blockchain via transactions. bitcoin transactions transfer funds between a payer (sender) and a payee (receiver), who are identified by public-private key pairs. payers announce their transactions to the bitcoin network. the miners then publish these transactions in new blocks using their computational power in exchange for a fee. these fees vary, but averaged at satoshi per byte during august [ ] ( satoshi = − bitcoin). each transaction consists of several input scripts, which unlock funds of previous transactions, and of several output scripts, which specify who receives these funds. to unlock funds, input scripts contain a signature for the previous transaction generated by the owner of the funds. to prevent malicious scripts from causing excessive transaction verification overheads, bitcoin uses transaction script templates and expects peers to discard non-compliant scripts. data insertion methods input scriptsoutput scripts p pk p pkh p shp ms p sh injectors satoshicryptograffiti apertus standardop_ret. non-st. coinbase p shnon-st. fig. : bitcoin data insertion methods (italics show content insertion services) method payload costs/b eff. op ret. b . – . ct poor coinbase b — poor non-st. out. b . – . ct poor non-st. in. med. p pk b . – . ct high p pkh b . – . ct high p ms b . – . ct high p sh out. b . – . ct high p sh in. b . – . ct high table : payload, costs, and efficiency of low-level data insertion methods figure shows the insertion methods for non-financial data we identified in bitcoin. we distinguish low-level data insertion methods inserting small data chunks and content insertion services, which systematically utilize the low-level methods to insert larger chunks of data. in the following, we refer to non-financial blockchain data as content if it has a self-contained structure, e.g., a file or read- able text, or as data otherwise, e.g., fragments inserted via a low-level method. . low-level data insertion methods we first survey the efficiency of the low-level data insertion methods w.r.t. to in- sertable payload and costs per transaction (table ). to this end, we first explain our comparison methodology, before we detail i) intended data insertion meth- ods (op return and coinbase), ii) utilization of non-standard transactions, and iii) manipulation of standard transactions to insert arbitrary data. comparison methodology. we measure the payload per transaction (ppt), i.e., the number of non-financial bytes that can be added to a single standard- sized transaction (≤ b). costs are given as the minimum and maximum costs per byte (cpb) for the longest data chunk a transaction can hold, and for inserting b. costs are inflicted by paying transaction fees and possibly burning currency (at least satoshi per output script), i.e., making it unspendable. for our cost analysis we assume bitcoin’s market price of . usd as of august st, [ ] and the average fees of satoshi per byte as of august [ ]. note that high variation of market price and fees results in frequent changes of presented absolute costs per byte. finally, we rate the overall efficiency of an approach w.r.t. insertion of arbitrary-length content. intuitively, a method is efficient if it allows for easy insertion of large payloads at low costs. op return. this special transaction template allows attaching one small data chunk to a transaction and thus provides a controlled channel to an- notate transactions without negative side effects. e.g., in typical implementa- tions peers increase performance by caching spendable transaction outputs and op return outputs can safely be excluded from this cache. however, data chunk sizes are limited to b per transaction. coinbase. in bitcoin, each block contains exactly one coinbase transaction, which introduces new currency into the system to incentivize miners to dedi- cate their computational power to maintain the blockchain. the input script of coinbase transactions is up to b long and consists of a variable-length field encoding the new block’s position in the blockchain [ ]. stating a larger size than the overall script length allows placing arbitrary data in the resulting gap. this method is inefficient as only active miners can insert only small data chunks. non-standard transactions. transactions can deviate from the approved transaction templates [ ] via their output scripts as well as input scripts. in the- ory, such transactions can carry arbitrarily encoded data chunks. transactions using non-standard output scripts can carry up to . kib at comparably low costs. however, they are inefficient as miners ignore them with high probability. yet, non-standard output scripts occasionally enter the blockchain if miners in- sufficiently check them (cf. section . ). contrarily, non-standard input scripts are only required to match their respective output script. hence, input scripts can be altered to carry arbitrary data if their semantics are not changed, e.g., by using dead conditional branches. this makes non-standard input scripts slightly better suited for large-scale content insertion than non-standard output scripts. standard financial transactions. even standard financial transactions can be (mis)used to insert data using mutable values of output scripts. there are four approved templates for standard financial transactions: pay to public-key (p pk) and pay to public-key hash (p pkh) transactions send currency to a dedicated receiver, identified by an address derived from her private key, which is required to spend any funds received [ ]. similarly, multi-signature (p ms) transactions require m out of n private keys to authorize payments. pay to script hash (p sh) transactions refer to a script instead of keys to enable complex spending conditions [ ], e.g., to replace p ms [ ]. the respective public keys (p pk, p ms) and script hash values (p pkh, p sh) can be replaced with ar- bitrary data as bitcoin peers can not verify their correctness before they are ref- erenced by a subsequent input script. while this method can store large amounts of content, it involves significant costs: in addition to transaction fees, the user must burn bitcoins as she replaces valid receiver identifiers with arbitrary data (i.e., invalid receiver identities), making the output unspendable. using multi- ple outputs enables ppts ranging from . kib (p pkh) to . kib (p sh inputs) at cpbs from . ct to . ct. as they behave similarly w.r.t. data in- sertion, we collectively refer to all standard financial transactions as p x in the following. p sh scripts also allow for efficient data insertion into input scripts as p sh input scripts are published with their redeem script. due to miners’ verification of p sh transactions, transaction are not discarded if the redeem script is not template-compliant (but the overall p sh transaction is). we now survey different services that systematically leverage the discussed data insertion methods to add larger amounts of content to the blockchain. . content insertion services content insertion services rely on the low-level data insertion methods to add content, i.e., files such as documents or images, to the blockchain. we identify four conceptually different content insertion services and present their protocols. cryptograffiti. this web-based service [ ] reads and writes messages and files from and to bitcoin’s blockchain. it adds content via multiple p pkh output scripts within a single transaction, storing up to kib of content. to retrieve previously added content, cryptograffiti scans for transactions that either con- sist of at least % printable characters or contain an image file. satoshi uploader. the satoshi uploader [ ] inserts content using a single transaction with multiple p x outputs. the inserted data is stored together with a length field and a crc checksum to ease decoding of the content. p sh injectors. several services [ ] insert content via slightly varying p sh input scripts. they store chunks of a file in p sh input scripts. to ensure file integrity, the p sh redeem scripts contain and verify hash values of each chunk. apertus. this service [ ] allows fragmenting content over multiple transac- tions using an arbitrary number of p pkh output scripts. subsequently, these fragments are referenced in an archive stored on the blockchain, which is used to retrieve and reassemble the fragments. the chosen encoding optionally allows augmenting content with a comment, file name, or digital signature. to conclude, bitcoin offers various options to insert arbitrary, non-financial data. these options range from small-scale data insertion methods exclusive to active miners to services that allow any user to store files of arbitrary length. this wide spectrum of options for data insertion raises the question which benefits and risks arise from storing content on bitcoin’s blockchain. benefits and risks of arbitrary blockchain content bitcoin’s design includes several methods to insert arbitrary, non-financial data into its blockchain in both intended and unintended ways. in this section, we discuss potential benefits of engraving arbitrary data into bitcoin’s blockchain as well as risks of (mis)using these channels for content insertion. . benefits of arbitrary blockchain content besides the manipulation of standard financial transactions, bitcoin offers coin- base and op return transactions as explicit channels to irrevocably insert small chunks of non-financial data into its blockchain (cf. section ). as we discuss in the following, each insertion method has distinguishing benefits: op return. augmenting transactions with short pieces of arbitrary data is beneficial for a wide area of applications [ , , ]. different services use op return to link non-financial assets, e.g., vouchers, to bitcoin’s block- chain [ , ], to attest the existence of digital documents at a certain point of time as a digital notary service [ , , ], to realize distributed digital rights management [ , ], or to create non-equivocation logs [ , ]. coinbase. coinbase transactions differ from op return as only miners, who dedicate significant computational resources to maintain the blockchain, can use them to add extra chunks of data to their newly mined blocks. beyond advertisements or short text messages [ ], coinbase transactions can aid the mining process. adding random bytes to the coinbase transactions allows miners to increase entropy when repeatedly testing random nonces to solve the proof- of-work puzzle [ ]. furthermore, adding identifiable voting flags to transactions enables miners to vote on proposed features, e.g., the adoption of p sh [ ]. large-scale data insertion. engraving large amounts of data into the block- chain creates a long-term non-manipulable file storage. this enables, e.g., the archiving of historical data or censorship-resistant publication, which helps pro- tecting whistleblowers or critical journalists [ ]. however, their content is repli- cated to all users, who do not have a choice to reject storing it. hence, non-financial data on the blockchain enables new applications that leverage bitcoin’s security guarantees. in the following, we discuss threats of forcing honest users to download copies of all blockchain content. . risks of arbitrary blockchain content despite potential benefits of data in the blockchain, insertion of objectionable content can put all participants of the bitcoin network at risk [ , , ], as such unwanted content is unchangeable and locally replicated by each peer of the bitcoin network as benign data. to underpin this threat, we first derive an extensive catalog of content that poses high risks if possessed by individuals and subsequently argue that objectionable blockchain content is able to harm honest users. in the following, we identify five categories of objectionable content: copyright violations. with the advent of file-sharing networks, pirated data has become a huge challenge for copyright holders. to tackle this problem, copy- right holders predominantly target users that actively distribute pirated data. e.g., german law firms sue users who distribute copyright-protected content via file-sharing networks for fines on behalf of the copyright holders [ ]. in re- cent years, prosecutors also convicted downloaders of pirated data. for instance, france temporarily suspended users’ internet access and subsequently switched to issuing high fines [ ]. as users distribute their blockchain copy to new peers, copyright-protected material on the blockchain can thus provoke legal disputes about copyright infringement. malware. another threat is to download malware [ , ], which could poten- tially be spread via blockchains [ ]. malware has serious consequences as it can destroy sensitive documents, make devices inoperable, or cause financial losses [ ]. furthermore, blockchain malware can irritate users as it causes an- tivirus software to deny access to important blockchain files. e.g., microsoft’s antivirus software detected a non-functional virus signature from on the blockchain, which had to be fixed manually [ ]. privacy violations. by disclosing sensitive personal data, individuals can harm their own privacy and that of others. this threat peaks when individuals deliberately violate the privacy of others, e.g., by blackmailing victims under the threat of disclosing sensitive data about them on the blockchain. real-world manifestations of these threats are well-known, e.g., non-consensually releasing private nude photos or videos [ ] or fully disclosing an individual’s identity to the public with malicious intents [ ]. jurisdictions such as the whole european union begin to actively prosecute the unauthorized disclosure and forwarding of private information in social networks to counter this novel threat [ ]. politically sensitive content. governments have concerns regarding the leakage of classified information such as state secrets or information that other- wise harms national security, e.g., propaganda. although whistleblowers reveal nuisances such as corruption, they force all blockchain users to keep a copy of leaked material. depending on the jurisdiction, the intentional disclosure or the mere possession of such content may be illegal. while, e.g., the us government usually tends to prosecute intentional theft or disclosure of state secrets [ ], in china the mere possession of state secrets can result in longtime prison sen- tences [ ]. furthermore, china’s definition of state secrets is vague [ ] and covers, e.g., “activities for safeguarding state security” [ ]. such vague allega- tions w.r.t. state secrets have been applied to critical news in the past [ , ]. illegal and condemned content. some categories of content are virtually universally condemned and prosecuted. most notably, possession of child pornog- raphy is illegal at least in the countries [ ] that ratified an optional protocol to the convention on the rights of the child [ ]. religious content such as cer- tain symbols, prayers, or sacred texts can be objectionable in extremely religious countries that forbid other religions and under oppressive regimes that forbid re- ligion in general. as an example, possession of items associated with an objected religion, e.g., bibles in islamist countries, or blasphemy have proven risky and were sometimes even punished by death [ , ]. in conclusion, a wide range of objectionable content can cause direct harm if possessed by users. in contrast to systems such as social media platforms, file-sharing networks, or online storage systems, such content can be stored on blockchains anonymously and irrevocably. since all blockchain data is down- loaded and persistently stored by users, they are liable for any objectionable content added to the blockchain by others. consequently, it would be illegal to participate in a blockchain-based systems as soon as it contains illegal content. while this risk has previously been acknowledged [ ], definitive answers re- quire court rulings yet to come. however, considering legal texts we anticipate a high potential for illegal blockchain content to jeopardize blockchain-based sys- tem such as bitcoin in the future. our belief stems from the fact that, w.r.t. child pornography as an extreme case of illegal content, legal texts from countries such as the usa [ ], england [ ], ireland [ ] deem all data illegal that can be con- verted into a visual representation of illegal content. as we stated in section , it is easily possible to locate and reassemble such content on the blockchain. hence, even though convertibility usually covers creating a visual representation by, e.g., decoding an image file, we expect that the term can be interpreted to include blockchain data in the future. for instance, this is already covered implicitly by german law, as a person is culpable for possession of illegal content if she knowingly possesses an accessible document holding said content [ ]. it is criti- cal here that german law perceives the hard disk holding the blockchain as an document [ ] and that users can easily reassemble any illegal content within the blockchain. furthermore, users can be assumed to knowingly maintain control over such illegal content w.r.t. german law if sufficient media coverage causes the content’s existence to become public knowledge among bitcoin users [ ], as has been attempted by interpol [ ]. we thus believe that legislators will speak law w.r.t. non-financial blockchain content and that this has the potential to jeopardize systems such as bitcoin if they hold illegal content. blockchain content landscape to understand the landscape of non-financial blockchain data and assess its potentials and risks, we thoroughly analyze bitcoin’s blockchain as it is the most widely used blockchain today. especially, we are interested in i) the degree of utilization of data and content insertion methods, ii) the temporal evolution of data insertion, and iii) the types of content on bitcoin’s blockchain, especially w.r.t. objectionable content. in the following, we first outline our measurement methodology before we present an overview and the evolution of non-financial data on bitcoin’s blockchain. finally, we analyze files stored on the blockchain to derive if any objectionable content is already present on the blockchain. . methodology we detect data-holding transactions recorded on bitcoin’s blockchain based on our study of data insertion methods and content insertion services (cf. section ). we distinguish detectors for data insertion methods and detectors for content insertion services. to reduce false positives, e.g., due to public-key hash values that resemble text, we exclude all standard transaction outputs that include already-spent funds from analysis. this is sensible as data-holding transactions replace public keys or hashes such that spending requires computing correspond- ing private keys or pre-images, which is assumed to be infeasible. contrarily, even though we thoroughly analyzed possible insertion methods, there is still a chance that we do not exhaustively detect all non-financial data. nevertheless, our con- tent type analysis establishes a solid lower bound as we only consider readable files retrieved from bitcoin’s blockchain. in the following, we explain the key characteristics of the two classes of our blockchain content detectors. low-level insertion method detectors. the first class of detectors is tai- lored to match individual transactions that are likely to contain non-financial data (cf. section . ). these detectors detect manipulated financial transactions as well as op return, non-standard, and coinbase transactions. our text detector scans for p x output scripts for mutable values containing ≥ % printable ascii characters (to avoid false positives). the detector returns the concatenation of all output scripts of the same transaction that contain text. finally, we consider all coinbase and op return transactions as well as non-standard output scripts. we detect coinbase transactions based on the length field mismatch described in section . . op return scripts are detectable as they always begin with an op return operation. non-standard output scripts comprise all output scripts which are not template-conform. t ra n sa c ti o n s [# ] op ret. coinb. non-st. p x p sh input fig. : cumulative numbers of detected transactions per data insertion method . . . . . . . p re se n c e in t x s [% ] op ret. p x p sh input fig. : ratio of transactions that utilize data insertion methods service detectors. we implemented detectors specific to the content insertion services we identified in section . . these service-specific detectors enable us to detect and extract files based on the services’ protocols. these detectors also track the data insertion method used in service-created transactions. the cryptograffiti detector matches transactions with an output that sends a tip to a public-key hash controlled by its provider. for such a transaction, we concatenate all mutable values of output scripts that spend fewer than satoshi and store them in a file. this threshold is used to ignore non- manipulated output scripts, e.g., the service provider spending their earnings. to detect a satoshi uploader transaction, we concatenate all of its mutable values that spend the same small amount of bitcoins. if we find the first eight bytes to contain a valid combination of length and crc checksum for the transaction’s payload, we store the payload as an individual file. we detect p sh injector content based on redeem scripts containing more than one hash operation (standard transactions use at most one). we then ex- tract the concatenation of the second inputs of all redeem scripts (the first one contains a signature) of a transaction as one file. finally, the apertus detector recursively scans the blockchain for apertus archives, i.e., apertus-encoded lists of previous transaction identifiers. once a referred apertus payload does not constitute another archive, we retrieve its payload file and optional comment by parsing the apertus protocol. suspicious transaction detector. to account for less wide-spread insertion services, we finally analyze standard transactions that likely carry non-financial data but are not detected otherwise. we only consider transactions with at least suspicious outputs, i.e., roughly kib of content. we consider a set of outputs suspicious if all outputs i) spend the same small amount (< satoshi) and ii) are unspent. this detector trades off detection rate against false-positive rate. due to overlaps with service detectors, we exclude matches of this detector from our quantitative analysis, but discuss individual findings in section . . . utilization of data insertion methods data and content insertion in bitcoin has evolved over time, transitioning from single miners exploiting coinbase transactions to sophisticated services that en- able the insertion of whole files into the blockchain. we study this evolution in in se rt io n s/ m o n th [# ] p sh injectors cryptograffiti satoshi uploader apertus fig. : number of files inserted via con- tent insertion services per month t x si z e s [m ib ] p sh injectors cryptograffiti satoshi uploader apertus fig. : cumulative sizes of transactions from content insertion services terms of used data insertion methods as well as content insertion services and quantify the amount of blockchain data using our developed detectors. our key insights are that op return constitutes a well-accepted success story while content insertion services are currently only infrequently utilized. however, the introduction of op return did not shut down other insertion methods, e.g., p x manipulation, which enable single users to insert objectionable content. our measurements are based on bitcoin’s complete blockchain as of august st, , containing blocks and transactions with a total disk size of . gib. we first analyze the popularity of different data inser- tion methods and subsequently turn towards the utilization of content insertion services to assess how non-financial data enters the blockchain. data insertion methods. as described in section . , op return and coinbase transactions constitute intended data insertion methods, whereas p x and non-standard p sh inputs manipulate legitimate transaction templates to contain arbitrary data. figure shows the cumulative number of transactions containing non-financial data on a logarithmic scale. in total, our detectors found transactions carrying a total payload of . mib, i.e., only . % of bitcoin transactions contain non-financial data. however, we strive to further un- derstand the characteristics of non-financial blockchain content as even a single instance of objectionable content can potentially jeopardize the overall system. the vast majority of extracted transactions are op return ( . % of all matches) and coinbase ( . %) transactions. combined, they constitute . mib ( . % of all extracted data). out of all blocks, . % have content- holding coinbase transactions. while only . % of these contain ≥ % print- able text, . % of them contain ≥ consecutive printable ascii characters (mostly surrounded by data without obvious structure). of these short messages, . % contain voting flags for new features (cf. section . ). apart from this, miners often advertise themselves or leave short messages, e.g., prayer verses. op return transactions were introduced in to offer a benign way to augment single transactions with non-financial data. this feature is widely used, as shown by figure . among all methods, op return is the only one to be present with a rising tendency, with currently . % of all transactions containing op return outputs. these transactions predominantly manage off-blockchain assets or originate from notary services [ ]. while p x transactions are contin- uously being manipulated, they make up only . % of all transactions; p sh inputs are virtually irrelevant. hence, short non-financial data chunks are well- accepted, viable extensions to the bitcoin system (cf. section . ). p x transactions are asymmetric w.r.t. the number and sizes of data-carrying transactions. although constituting only . % of all detector hits, they make up . % of non-financial data ( . mib). this again highlights the high content- insertion efficiency of p x transactions (cf. section . ). finally, we discuss non-standard transactions and non-standard p sh in- put scripts. in total, we found transactions containing non-standard out- puts. the three first non-standard transactions (july ) repeatedly used the op checksig operation. we dedicate this to an attempted dos attack that tar- gets to cause high verification times. furthermore, we found p pkh transac- tions from october that contained op instead of a hash value. the steady increase of non-standard transactions in is due to scripts that consist of seemingly random bytes. contrarily, p sh input scripts sporadically carry non- standard redeem scripts and are then often used to insert larger data chunks (as they are used by p sh injectors). this is due to p sh scripts not being checked for template conformity. we found such transactions holding . mib of data. although peers should reject such transactions [ ], they still often man- age to enter the blockchain. non-standard p sh scripts even carry a substantial amount of data ( . % of the total data originate from p sh injectors). content insertion services. we now investigate to which extent content insertion services are used to store content on bitcoin’s blockchain. figure shows utilization patterns for each service and figure shows the cumulative size of non-financial data inserted via the respective service. notably, only few users are likely responsible for the majority of service-inserted content. in total, content insertion services account for . mib of non-financial data. more than a half of this content ( . mib) originates from p sh in- jectors. the remainder was mostly inserted using apertus ( . % of service- inserted data) and satoshi uploader ( . %). finally, cryptograffiti accounts for . mib ( . %) of content related to content insertion services. in the following, we study how the individual services have been used over time. our key observation is that both cryptograffiti and p sh injectors are in- frequently but steadily used; since we recognize on average . data items being added per month using these services. contrarily, apertus has been used only times since , while the satoshi uploader has not been used at all. in fact, the satoshi uploader was effectively used only during a brief period: . % of all transactions emerged in april . during this time, the service was used to upload four archives, six backup text files, and a pdf file. although apertus and the satoshi uploader have been used only infrequently, together they constitute . % of all p x data we detected. this stems from the utilization of those services to engrave files into the blockchain, e.g., archives or documents (satoshi uploader), or images (apertus). similarly, p sh injectors are used to backup conversations regarding development of the bitcoin client, especially online chat logs, forum threads, and emails, with a significant peak file via service? overall file via service? overall type yes no portion type yes no portion text . % archive . % images . % audio . % html . % pdf . % source code . % total . % table : distribution of blockchain file types according to our content-insertion- service and suspicious-transactions detectors. utilization between may and june ( . % of p sh injector matches). es- pecially apertus is well-suited for this task as files are spread over multiple trans- actions. based on the median, the average apertus file has a size of . kib and is spread over transactions, including all overheads. the largest aper- tus file is . kib large (including overheads), i.e., three times the size of a standard transaction, and is spread over transactions. the most heavily frag- mented apertus file is even spread over transactions. contrarily, . % of cryptograffiti matches are short text messages with a median length of byte. in conclusion, content insertion services are only infrequently used with vary- ing intentions and large portions of content was uploaded in bursts, indicating that only few users are likely responsible for the majority of service-inserted blockchain content. while cryptograffiti is mostly used to insert short text messages that also fit into one op return transaction, other services are pre- dominantly used to store, e.g., images or documents. as such files can constitute objectionable content, we further investigate them in the following. . investigating blockchain files after quantifying basic content insertion in bitcoin, we now focus on readable files that are extractable from the blockchain. we refer to files as findings of our content-insertion-service or suspicious-transaction detectors that are viewable using appropriate standard software. we reassemble fragmented files only if this is unambiguously possible, e.g., via an apertus archive. out of the . mib of blockchain data not originating from coinbase or op return transactions, we can extract and analyze files with meaningful content. in addition to these, we could extract files using our suspicious-transaction detector ( . % text). table summarizes the different file types of the analyzed files. the vast majority are text-based files and images ( . %). in the following, we discuss our findings with respect to objectionable con- tent. we manually evaluated all readable files with respect to the problematic categories we identified in section . . this analysis reveals that content from all those categories already exists in bitcoin’s blockchain today. for each of these categories, we discuss the most severe examples. to protect the safety and pri- vacy of individuals, we omit personal identifiable information and refrain from providing exact information on the location of critical content in the blockchain. copyright violations. we found seven files that publish (intellectual) property and showcase bitcoin’s potential to aid copyright violations. engraved are the text of a book, a copy of the original bitcoin paper [ , ], and two short textual white papers. furthermore, we found two leaked cryptographic keys: one rsa private key and a firmware secret key. finally, the blockchain contains a so-called illegal prime, encoding software to break the copy protection of dvds [ ]. malware. we could not find actual malware in bitcoin’s blockchain. how- ever, an individual non-standard transaction contains a non-malicious cross-site scripting detector. a security researcher inserted this small piece of code which, if interpreted by an online blockchain parser, notifies the author about the vul- nerability. such malicious code could become a threat for users as most websites offering an online blockchain parser also offer online bitcoin accounts. privacy violations. users store memorable private moments on the block- chain. we extracted six wedding-related images and one image showing a group of people, labeled with their online pseudonyms. furthermore, transactions contain online public chat logs, emails, and forum posts discussing bitcoin, in- cluding topics such as money laundering. storing private chat logs on the block- chain can, e.g., leak single user’s private information irrevocably. moreover, third parties can release information without knowledge nor consent of affected users. most notably, we found at least two instances of doxing, i.e., the complete dis- closure of another individual’s personal information. this data includes phone numbers, addresses, bank accounts, passwords, and multiple online identities. recently, jurisdictions such as the european union began to punish such serious privacy violations, including the distribution of doxing data [ ]. again, carrying out such assaults via blockchains fortifies the problem due to their immutability. politically sensitive content. the blockchain has been used by whistleblow- ers as a censorship-resistant permanent storage for leaked information. we found backups of the wikileaks cablegate data [ ] as well as an online news arti- cle concerning pro-democracy demonstrations in hong kong in [ ]. as stated in section . , restrictive governments are known to prosecute the pos- session of such content. for example, state-critical media coverage has already put individuals in china [ ] or turkey [ ] at the risk of prosecution. illegal and condemned content. bitcoin’s blockchain contains at least eight files with sexual content. while five files only show, describe, or link to mildly pornographic content, we consider the remaining three instances objectionable for almost all jurisdictions: two of them are backups of link lists to child pornog- raphy, containing links to websites, of which refer to tor hidden services. the remaining instance is an image depicting mild nudity of a young woman. in an online forum this image is claimed to show child pornography, albeit this claim cannot be verified (due to ethical concerns we refrain from providing a ci- tation). notably, two of the explicit images were only detected by our suspicious- transaction detector, i.e., they were not inserted via known services. while largely harmless, potentially objectionable blockchain content is infre- quently inserted, e.g., links to alleged child pornography or privacy violations. we thus believe that future blockchain designs must proactively cope with objec- tionable content. peers can, e.g., filter incoming transactions or revert content- holding transactions [ , ], but this must be scalable and transparent. related work previous work related to ours comprises i) mitigating the distribution of objec- tionable content in file-sharing peer-to-peer networks, ii) studies on bitcoin’s blockchain, iii) reports on bitcoin’s susceptibility for content insertion, and iv) approaches to retrospectively remove blockchain content. the trade-off between enabling open systems for data distribution and risking that unwanted or even illegal content is being shared is already known from peer-to-peer networks. peer-to-peer-based file-sharing protocols typically limit the spreading of objectionable public content by tracking the reputation of users offering files [ , , , ] or assigning a reputation to files themselves [ , ]. this way, users can reject objectionable content or content from untrustworthy sources. contrarily, distributed content stores usually resort to encrypt private files before outsourcing them to other peers [ , ]. by storing only encrypted files, users can plausibly deny possessing any content of others and can thus obliviously store it on their hard disk. unfortunately, these protection mechanisms are not applicable to blockchains, as content cannot be deleted once it has been added to the blockchain and the utilization of encryption cannot be enforced reliably. bitcoin’s blockchain was analyzed w.r.t. different aspects by numerous stud- ies. in a first step, multiple research groups [ , , , , ] studied the currency flows in bitcoin, e.g., to perform wealth analyses. from a different line of re- search, several approaches focused on user privacy and investigated the identities used in bitcoin [ , , , , ]. these works analyzed to which extent users can be de-anonymized by clustering identities [ , , , , ] and augmenting these clusters with side-channel information [ , , , ]. finally, the blockchain was analyzed w.r.t. the use cases of op return transactions [ ]. while this work is very close to ours, we provide a first comprehensive study of the complete landscape of non-financial data on bitcoin’s blockchain. the seriousness of objectionable content stored on public blockchains has been motivated by multiple works [ , , , , , ]. these works, however, fo- cus on reporting individual incidents or consist of preliminary analyses of the distribution and general utilization of content insertion. to the best of our knowl- edge, this paper gives the first comprehensive analysis of this problem space, including a categorization of objectionable content and a survey of potential risks for users if such content enters the blockchain. in contrast to previously considered attacks on bitcoin’s ecosystem [ , ], illegal content can be inserted instantly at comparably low costs and can put all participants at risk. the utilization of chameleon hash functions [ ] to chain blocks recently opened up a potential approach to mitigate unwanted or illegal blockchain con- tent [ ]. here, a single blockchain maintainer or a small group of maintainers can retrospectively revert single transactions, e.g., due to illegal content. to overcome arising trust issues, µchain [ ] leverages the consensus approach of traditional blockchains to vote on alterations of the blockchain history. as these approaches tackle unwanted content for newly designed blockchains, we seek to motivate a discussion on countermeasures also for existing systems, e.g., bitcoin. conclusion the possibility to store non-financial data on cryptocurrency blockchains is both beneficial and threating for its users. although controlled channels to insert non- financial data at small rates opens up a field of new applications such as digital notary services, rights management, or non-equivocation systems, objectionable or even illegal content has the potential to jeopardize a whole cryptocurrency. although court rulings do not yet exist, legislative texts from countries such as germany, the uk, or the usa suggest that illegal content such as child pornography can make the blockchain illegal to possess for all users. as we have shown in this paper, a plethora of fundamentally different meth- ods to store non-financial–potentially objectionable–content on the blockchain exists in bitcoin. as of now, this can affect at least countries in which pos- sessing content such as child pornography is illegal. this especially endangers the multi-billion dollar markets powering cryptocurrencies such as bitcoin. to assess this problem’s severity, we comprehensively analyzed the quantity and quality of non-financial blockchain data in bitcoin today. our quantitative analysis shows that . % of the roughly million transactions in bitcoin’s blockchain carry arbitrary data. we could retrieve over files, with new con- tent infrequently being added. despite a majority of arguably harmless content, we also identify different categories of objectionable content. the harmful poten- tial of single instances of objectionable blockchain content is already showcased by findings such as links to illegal pornography or serious privacy violations. acknowledgements this work has been funded by the german federal ministry of education and research (bmbf) under funding reference number kis . the responsibil- ity for the content of this publication lies with the authors. references . german criminal code, section ( ) . german criminal code, sections b and c ( ) . protection of children act, chapter , section ( ) . bitcoin transaction fees. https://bitcoinfees.info ( ) accessed / / . . general data protection regulation, section ( ) . aberer, k., despotovic, z.: managing trust in a peer- -peer information system. in: acm cikm. ( ) pp. – . adya, a., bolosky, w.j., castro, m., cermak, g., chaiken, r., douceur, j.r., howell, j., lorch, j.r., theimer, m., wattenhofer, r.p.: farsite: federated, available, and reliable storage for an incompletely trusted environment. sigops oper. syst. rev. (si) ( ) pp. – . ali, m., shea, r., nelson, j., freedman, m.j.: blockstack: a new decentralized internet. ( ) accessed / / . https://bitcoinfees.info . andresen, g.: block v (height in coinbase). https://github.com/bitcoin/ bips/blob/master/bip- .mediawiki ( ) accessed / / . . andresen, g.: pay to script hash. https://github.com/bitcoin/bips/blob/ master/bip- .mediawiki ( ) accessed / / . . ateniese, g., magri, b., venturi, d., andrade, e.: redactable blockchain – or – rewriting history in bitcoin and friends. in: ieee euros&p. ( ) pp. – . bartoletti, m., pompianu, l.: an analysis of bitcoin op return metadata. in: fc bitcoin workshop. ( ) . bellinger, j., hussain, m.: freedom of speech: the great divide and the common ground between the united states and the rest of the world. islamic law and international human rights law: searching for common ground? ( ) pp. – . blockchain.info: bitcoin charts. https://blockchain.info/charts ( ) ac- cessed / / . . camenisch, j., derler, d., krenn, s., pöhls, h.c., samelin, k., slamanig, d.: chameleon-hashes with ephemeral trapdoors. in: pkc ’ . ( ) pp. – . clark, j., essex, a.: commitcoin: carbon dating commitments with bitcoin. in: fc. ( ) pp. – . clarke, i., sandberg, o., wiley, b., hong, t.w.: freenet: a distributed anony- mous information storage and retrieval system. in: designing privacy enhanc- ing technologies: workshop on design issues in anonymity and unobservability. ( ) pp. – . committee to protect journalists: chinese journalist accused of illegally acquiring state secrets. https://cpj.org/x/ d ( ) accessed / / . . damiani, e., di vimercati, d.c., paraboschi, s., samarati, p., violante, f.: a reputation-based approach for choosing reliable resources in peer-to-peer net- works. in: acm ccs. ( ) pp. – . dell security: annual threat report. ( ) accessed / / . . douglas, d.m.: doxing: a conceptual analysis. ethics and information technology ( ) ( ) pp. – . eyal, i., sirer, e.g.: majority is not enough: bitcoin mining is vulnerable. in: fc. ( ) pp. – . fleder, m., kester, m., sudeep, p.: bitcoin transaction graph analysis. ( ) . freedom house: turkey freedom of the press report. https://freedomhouse. org/report/freedom-press/ /turkey ( ) accessed / / . . gracie, c.: hong kong stages huge national day democracy protests. http: //www.bbc.com/news/world-asia-china- ( ) accessed / / . . gupta, m., judge, p., ammar, m.: a reputation system for peer-to-peer net- works. in: acm nossdav. ( ) pp. – . heilman, e., kendler, a., zohar, a., goldberg, s.: eclipse attacks on bitcoin’s peer-to-peer network. in: usenix security. ( ) pp. – . herald union: copyright infringement by illegal file sharing in ger- many. http://www.herald-union.com/copyright-infringement-by-illegal- file-sharing-in-germany ( ) accessed / / . . hugpuddle: apertus – archive data on your favorite blockchains. http:// apertus.io ( ) accessed / / . . “hyena”: cryptograffiti.info. http://cryptograffiti.info accessed / / . . interpol: interpol cyber research identifies malware threat to virtual curren- cies. https://www.interpol.int/news-and-media/news/ /n - ( ) accessed / / . https://github.com/bitcoin/bips/blob/master/bip- .mediawiki https://github.com/bitcoin/bips/blob/master/bip- .mediawiki https://github.com/bitcoin/bips/blob/master/bip- .mediawiki https://github.com/bitcoin/bips/blob/master/bip- .mediawiki https://blockchain.info/charts https://cpj.org/x/ d https://freedomhouse.org/report/freedom-press/ /turkey https://freedomhouse.org/report/freedom-press/ /turkey http://www.bbc.com/news/world-asia-china- http://www.bbc.com/news/world-asia-china- http://www.herald-union.com/copyright-infringement-by-illegal-file-sharing-in-germany http://www.herald-union.com/copyright-infringement-by-illegal-file-sharing-in-germany http://apertus.io http://apertus.io http://cryptograffiti.info https://www.interpol.int/news-and-media/news/ /n - . irish office of the attorney general: child trafficking and pornography act, section . irish statue book ( ) pp. – . kondor, d., pósfai, m., csabai, i., vattay, g.: do the rich get richer? an empirical analysis of the bitcoin transaction network. plos one ( ) ( ) pp. – . labs, f.s.: ransomware: how to predict, prevent, detect & resond. threat response ( ) accessed / / . . le calvez, a.: non-standard p sh scripts. https://medium.com/@alcio/non- standard-p sh-scripts- fa df ( ) accessed / / . . lee, d.: france ends three-strikes internet piracy ban policy. http://www.bbc. com/news/technology- ( ) accessed / / . . lynch, l.: the leak heard round the world? cablegate in the evolving global mediascape. in brevini, b., hintz, a., mccurdy, p., eds.: beyond wikileaks: implications for the future of communications, journalism and society. palgrave macmillan uk ( ) pp. – . lyons, k., blight, g.: where in the world is the worst place to be a christian? ( ) accessed / / . . maesa, d.d.f., marino, a., ricci, l.: uncovering the bitcoin blockchain: an analysis of the full users graph. in: ieee dsaa. ( ) pp. – . matzutt, r., hohlfeld, o., henze, m., rawiel, r., ziegeldorf, j.h., wehrle, k.: poster: i don’t want that content! on the risks of exploiting bitcoin’s block- chain as a content store. in: acm ccs. ( ) . matzutt, r., müllmann, d., zeissig, e.m., horst, c., kasugai, k., lidynia, s., wieninger, s., ziegeldorf, j.h., gudergan, g., spiecker gen. döhmann, i., wehrle, k., ziefle, m.: mynedata: towards a trusted and user-controlled ecosystem for sharing personal data. in eibl, m., gaedke, m., eds.: informatik, gesellschaft für informatik, bonn ( ) pp. – . mcafee labs: threats report (december ). ( ) accessed / / . . mcreynolds, e., lerner, a., scott, w., roesner, f., kohno, t.: cryptographic currencies from a tech-policy perspective: policy issues and technical directions. in: springer lncs. volume . ( ) pp. – . meiklejohn, s., pomarole, m., jordan, g., levchenko, k., mccoy, d., voelker, g.m., savage, s.: a fistful of bitcoins: characterizing payments among men with no names. in: imc. ( ) pp. – . nakamoto, s.: bitcoin: a peer-to-peer electronic cash system. ( ) https: //bitcoin.org/bitcoin.pdf. . ober, m., katzenbeisser, s., hamacher, k.: structure and anonymity of the bit- coin transaction graph. future internet ( ) ( ) pp. – . office of the law revision counsel of the united states house of representatives: u.s. code, title , chapter , § ( ) . okupski, k.: bitcoin developer reference. technical report ( ) . peerenboom, r.p.: assessing human rights in china: why the double standard. ( ) accessed / / . . poex co., ltd: proof of existence. https://proofofexistence.com ( ) ac- cessed / / . . puddu, i., dmitrienko, a., capkun, s.: µchain: how to forget without hard forks. iacr cryptology eprint archive / ( ) accessed / / . . reid, f., harrigan, m.: an analysis of anonymity in the bitcoin system. in: security and privacy in social networks. ( ) pp. – . ron, d., shamir, a.: quantitative analysis of the full bitcoin transaction graph. in: fc. ( ) pp. – https://medium.com/@alcio/non-standard-p sh-scripts- fa df https://medium.com/@alcio/non-standard-p sh-scripts- fa df http://www.bbc.com/news/technology- http://www.bbc.com/news/technology- https://bitcoin.org/bitcoin.pdf https://bitcoin.org/bitcoin.pdf https://proofofexistence.com . scheller, s.h.: a picture is worth a thousand words: the legal implications of revenge porn. north carolina law review ( ) ( ) pp. – . selcuk, a.a., uzun, e., pariente, m.r.: a reputation-based trust management system for p p networks. in: ieee ccgrid. ( ) pp. – . shirriff, k.: hidden surprises in the bitcoin blockchain and how they are stored: nelson mandela, wikileaks, photos, and python software. http://www. righto.com/ / /ascii-bernanke-wikileaks-photographs.html ( ) ac- cessed / / . . sleiman, m.d., lauf, a.p., yampolskiy, r.: bitcoin message: data insertion on a proof-of-work cryptocurrency system. in: acm cw. ( ) pp. – . snow, p., deery, b., lu, j., johnston, d., kirby, p.: factom: business processes secured by immutable audit trails on the blockchain. https://www.factom.com/ devs/docs/guide/factom-white-paper- - ( ) accessed / / . . spagnuolo, m., maggi, f., zanero, s.: bitiodine: extracting intelligence from the bitcoin network. in: fc. ( ) pp. – . standing committee of the national people’s congress: law of the people’s re- public of china on guarding state secrets. ( ) accessed / / . . taylor, g.: concepts of intention in german criminal law. oxford journal of legal studies ( ) ( ) pp. – . tomescu, a., devadas, s.: catena: efficient non-equivocation via bitcoin. in: ieee s&p. ( ) pp. – . tucker, e.: a look at federal cases on handling classified in- formation. http://www.military.com/daily-news/ / / /a-look-at- federal-cases-on-handling-classified-information.html ( ) accessed / / . . united nations: appendix to the optional protocols to the convention on the rights of the child on the involvement of children in armed conflict and on the sale of children, child prostitution and child pornography ( ) . united nations: optional protocols to the convention on the rights of the child on the involvement of children in armed conflict and on the sale of children, child prostitution and child pornography. ( ) pp. – . waldman, m., rubin, a.d., cranor, l.: publius: a robust, tamper-evident, censorship-resistant and source-anonymous web publishing system. in: usenix security. ( ) pp. – . walsh, k., sirer, e.g.: experience with an object reputation system for peer- to-peer filesharing. in: nsdi. ( ) . wei, w.: ancient ’stoned’ virus signatures found in bitcoin block- chain. https://thehackernews.com/ / /microsoft-security-essential- found.html ( ) accessed / / . . wood, g.: ethereum: a secure decentralised generalised transaction ledger. ethereum project yellow paper ( ) accessed / / . . zeilinger, m.: digital art as ‘monetised graphics’: enforcing intellectual property on the blockchain. philosophy & technology ( ) . ziegeldorf, j.h., grossmann, f., henze, m., inden, n., wehrle, k.: coinparty: secure multi-party mixing of bitcoins. in: acm codaspy. ( ) pp. – . ziegeldorf, j.h., matzutt, r., henze, m., grossmann, f., wehrle, k.: secure and anonymous decentralized bitcoin mixing. fgcs ( ) – . zimmermann, t., rüth, j., wirtz, h., wehrle, k.: maintaining integrity and reputation in content offloading. in: ieee/ifip wons. ( ) pp. – http://www.righto.com/ / /ascii-bernanke-wikileaks-photographs.html http://www.righto.com/ / /ascii-bernanke-wikileaks-photographs.html https://www.factom.com/devs/docs/guide/factom-white-paper- - https://www.factom.com/devs/docs/guide/factom-white-paper- - http://www.military.com/daily-news/ / / /a-look-at-federal-cases-on-handling-classified-information.html http://www.military.com/daily-news/ / / /a-look-at-federal-cases-on-handling-classified-information.html https://thehackernews.com/ / /microsoft-security-essential-found.html https://thehackernews.com/ / /microsoft-security-essential-found.html a quantitative analysis of the impact of arbitrary blockchain content on bitcoin none issues · lostrses/escape-room · github skip to content sign up sign up why github? features → mobile → actions → codespaces → packages → security → code review → project management → integrations → github sponsors → customer stories→ team enterprise explore explore github → learn and contribute topics → collections → trending → learning lab → open source guides → connect with others the readme project → events → community forum → github education → github stars program → marketplace pricing plans → compare plans → contact sales → education → in this repository all github ↵ jump to ↵ no suggested jump to results in this repository all github ↵ jump to ↵ in this organization all github ↵ jump to ↵ in this repository all github ↵ jump to ↵ sign in sign up sign up {{ message }} lostrses / escape-room notifications star fork code issues pull requests actions projects security insights more code issues pull requests actions projects security insights labels milestones labels milestones new issue have a question about this project? sign up for a free github account to open an issue and contact its maintainers and the community. pick a username email address password sign up for github by clicking “sign up for github”, you agree to our terms of service and privacy statement. we’ll occasionally send you account related emails. already on github? sign in to your account open closed open closed author filter by author author: filter by this user label filter by label use alt + click/return to exclude labels. projects filter by project milestones filter by milestone assignee filter by who’s assigned sort sort by newest oldest most commented least commented recently updated least recently updated most reactions 👍 👎 😄 🎉 😕 ❤️ 🚀 👀 add some books about python to the bookshelf in room # opened apr , by jezcope weave a name for our absent rse into the narrative for continuity # opened apr , by jezcope add small pictures for some of the interactable objects in the room # opened apr , by jezcope add some breadcrumb navigation to the story website # opened apr , by jezcope check description of rooms for consistency # opened apr , by jezcope add illustrations for items in the rse office # opened apr , by tlestang make sure all images have alt text # opened apr , by jezcope move image credits from laptop page to readme for assets folder # opened apr , by lauracarter create a presentation for the end of the hackday # opened apr , by jezcope how do we get the a&h to use this? how do we get the word out? # opened apr , by marionbweinzierl make it shinier! # opened apr , by marionbweinzierl updating the readme to include all the information required for judging criteria for cw hackday ( april ) # opened apr , by lauracarter of create puzzles for software sustainability # opened apr , by marionbweinzierl create puzzle for research software engineering # opened apr , by marionbweinzierl create puzzles for software testing and ci # opened apr , by marionbweinzierl create puzzle for version control # opened apr , by marionbweinzierl create puzzle for licenses # opened apr , by marionbweinzierl create puzzle for readmes # opened apr , by marionbweinzierl protip! find all open issues with in progress development work with linked:pr. © github, inc. terms privacy security status docs contact github pricing api training blog about you can’t perform that action at this time. you signed in with another tab or window. reload to refresh your session. you signed out in another tab or window. reload to refresh your session. ¡el futuro es ahora! | future lab ☰ ✎ edit navigation inicio nosotros eventos blog contacto somos future lab, la comunidad del futuro. desarrollamos tecnología y compartimos conocimiento. trabajamos por el futuro que queremos ver. síguenos en facebook desarrollamos proyectos de base científica y tecnológica ya sea que nosotros mismo metamos mano en el desarrollo, o que sea a través de mentorías, nos encanta ser capaces de innovar y poder aportar al desarrollo de nuevas tecnologías; llevarlo a diferentes partes de méxico y poder presentar nuestros proyectos en diferentes eventos con empresas que están en el medio. compartimos conocimiento y fomentamos la educación en tecnología a través de talleres, charlas, conferencias y participaciones en eventos compartimos conocimientos técnicos y de cultura para el desarrollo de tecnología. vinculamos y creamos comunidad nos encanta empoderar a nuestra comunidad, apoyar y poder vincular con quien pueda potencializar a las grandes mentes del futuro que se nos acercan. nuestra visión en future lab es poder desarrollar nuestro futuro, compartir conocimientos y poder crear las conexiones que ayuden a nuestra comunidad. rodolfo ferro, co-fundador de future lab. ¡conoce todo lo que estamos haciendo! future lab en facebook ✎ edit footer © future lab. aha! | an arts & humanities adventure view on github aha! an arts & humanities adventure you are a researcher in the classics department. as part of your current research project, you have become interested in the life of a woman called fabrica collaborare, who lived in roman britain. there’s not much written specifically about fabrica, but you have seen her name mentioned in several texts from that time. you are not looking forward to the task of having to look at lots more texts to find out where fabrica - and the collaborare family - are mentioned. on your way out of the library to get a cup of coffee, you meet your colleague priya, and tell her about your problem. she tells you about a group at the university who might be able to help. you haven’t heard of the rse team before: priya tells you that ‘rse’ stands for research software engineering, and that their office is in room . . go to room . aha! maintained by lostrses published with github pages intro to the fediverse erambler home about series tags talks rdm resources intro to the fediverse date: - - tags: [fediverse] [social media] [twitter] wow, it turns out to be years since i wrote this beginners guide to twitter. things have moved on a loooooong way since then. far from being the interesting, disruptive technology it was back then, twitter has become part of the mainstream, the establishment. almost everyone and everything is on twitter now, which has both pros and cons. so what’s the problem? it’s now possible to follow all sorts of useful information feeds, from live updates on transport delays to your favourite sports team’s play-by-play performance to an almost infinite number of cat pictures. in my professional life it’s almost guaranteed that anyone i meet will be on twitter, meaning that i can contact them to follow up at a later date without having to exchange contact details (and they have options to block me if they don’t like that). on the other hand, a medium where everyone’s opinion is equally valid regardless of knowledge or life experience has turned some parts of the internet into a toxic swamp of hatred and vitriol. it’s easier than ever to forget that we have more common ground with any random stranger than we have similarities, and that’s led to some truly awful acts and a poisonous political arena. part of the problem here is that each of the social media platforms is controlled by a single entity with almost no accountability to anyone other than shareholders. technological change has been so rapid that the regulatory regime has no idea how to handle them, leaving them largely free to operate how they want. this has led to a whole heap of nasty consequences that many other people have done a much better job of documenting than i could (shoshana zuboff’s book the age of surveillance capitalism is a good example). what i’m going to focus on instead are some possible alternatives. if you accept the above argument, one obvious solution is to break up the effective monopoly enjoyed by facebook, twitter et al. we need to be able to retain the wonderful affordances of social media but democratise control of it, so that it can never be dominated by a small number of overly powerful players. what’s the solution? there’s actually a thing that already exists, that almost everyone is familiar with and that already works like this. it’s email. there are a hundred thousand email servers, but my email can always find your inbox if i know your address because that address identifies both you and the email service you use, and they communicate using the same protocol, simple mail transfer protocol (smtp) . i can’t send a message to your twitter from my facebook though, because they’re completely incompatible, like oil and water. facebook has no idea how to talk to twitter and vice versa (and the companies that control them have zero interest in such interoperability anyway). just like email, a federated social media service like mastodon allows you to use any compatible server, or even run your own, and follow accounts on your home server or anywhere else, even servers running different software as long as they use the same activitypub protocol. there’s no lock-in because you can move to another server any time you like, and interact with all the same people from your new home, just like changing your email address. smaller servers mean that no one server ends up with enough power to take over and control everything, as the social media giants do with their own platforms. but at the same time, a small server with a small moderator team can enforce local policy much more easily and block accounts or whole servers that host trolls, nazis or other poisonous people. how do i try it? i have no problem with anyone for choosing to continue to use what we’re already calling “traditional” social media; frankly, facebook and twitter are still useful for me to keep in touch with a lot of my friends. however, i do think it’s useful to know some of the alternatives if only to make a more informed decision to stick with your current choices. most of these services only ask for an email address when you sign up and use of your real name vs a pseudonym is entirely optional so there’s not really any risk in signing up and giving one a try. that said, make sure you take sensible precautions like not reusing a password from another account. instead of… try… twitter, facebook mastodon, pleroma, misskey slack, discord, irc matrix whatsapp, fb messenger, telegram also matrix instagram, flickr pixelfed youtube peertube the web interplanetary file system (ipfs) which, if you can believe it, was formalised nearly years ago in and has only had fairly minor changes since then! ↩︎ comments you can comment on this post, "intro to the fediverse", by: replying to its tweet on twitter or its toot on mastodon sending a webmention from your own site to https://erambler.co.uk/blog/intro-to-the-fediverse/ using this button: comments & reactions haven't loaded yet. you might have javascript disabled but that's cool 😎. me elsewhere :: keyoxide | keybase | mastodon | matrix | twitter | github | gitlab | orcid | pypi | linkedin © jez cope | built by: hugo | theme: mnemosyne build status: except where noted, this work is licensed under a creative commons attribution . international license. geoladies ph geoladiesph home join about projects contact geoladiesph we advocate for community diversity, collaborative participation, and affirmative spaces especially for women and under-represented communities. cheers to the ladies! 😉 join our latest workshop! for our latest workshop, we will provide an introduction to how drones are used in geospatial aerial surveys. apply for a slot below. according to the civil aviation authority of the philippines, only % of licensed drone pilots are women. with this, please note that women applicants will be prioritized to generate interest from an under-represented sector, in which our group would like to focus on. powered by typeform learn more about geoladies ph about our core team jen one of jen's advocacies is mapping breastfeeding stations in the philippines to help fellow wives and mommies. 🤱🏽 andi andi advocates for mapping mental health resources and services and promoting mental health awhereness to fight the stigma on it. 🌻 leigh drone expert! 🛫 cham artist mapper 🎨 nalie nalie is an advocate for sustainable living and she maps for work and voluntarily. 🌿 feye disaster response mapper 🌊 👉🏽resources for knowledge sharing ✨ view/download here. projects recent projects, workshops, and activities drone't you wish your girl friend could fly like me? a workshop on how drones are used in geospatial aerial surveys. women applicants are prioritized to generate interest from an under-represented sector in this field. pista ng mapa we've participated at the pista ng mapa ! more workshops more workshops! more projects more projects! follow us on social media for more updates or email us. follow us on facebook! email us here! view larger map copyright © geoladies ph fail!lab | technology, libraries and the future! fail!lab technology, libraries and the future! menu skip to content home about luddites, trumpism and change: a crossroads for libraries posted on december , by mryanhess “globalization is a proxy for technology-powered capitalism, which tends to reward fewer and fewer members of society.” – om malik corner someone and they will react. we may be seeing this across the world as change, globalization, technology and economic dislocation force more and more people into the corner of benefit-nots. they are reacting out of desperation. it’s not rational. it’s not pretty. but it shouldn’t be surprising. years ago at a library conference, one of the keynote speakers forecast that there would be a return to the analog (sorry my twitter-based memory does not identify the person). the rapidity of digitization would be met by a reaction. people would scurry back to the familiar, he said. they always do. fast forward to , where the decades-long trends toward globalization, borderless labor markets, denationalization, exponential technological change and corresponding social revolutions has hit the wall of public reaction. brexit. global trumpism. call it what you will. we’re in a change moment. the reaction is here. reacting to the reaction people in the blue zones, the technorati, the beneficiaries of cheap foreign labor, free trade and technological innovation are scratching their heads. for all their algorithms and ai, they didn’t see this coming. everything looked good on their feeds. no danger could possibly burst their self-assured bubble of inevitability. all was quiet. it was like a clear blue, september , morning in new york city. it was like the boardroom in the federal reserve in . the serenity was over in an instant. since brexit, and then trump’s election, the glittery digitarians have initiated a period of introspection. they’re looking up from their stock tickers and gold-plated smart watches to find a grim reality: the world is crowded with people that have lost much ground at the expense of the global maelstrom that has elevated a very small, lucky few to greatness. they are now seeing, as for the first time, the shuttered towns. the empty retail stores. the displaced and homeless. suddenly their confident talk of personal ai assistants has turned from technolust to terror. their success suddenly looks short-sighted. om malik wrote in his recent new yorker op-ed, that silicon valley may soon find itself equated with the super villains on wall street. he posits that a new business model needs to account for the public good…or else. i recently read throwing rocks at the google bus: how growth became the enemy of prosperity by douglas rushkoff. if you haven’t read it, now would be a good time. like bernie sanders and others, rushkoff has been warning of this kind of reaction for awhile. the system is not designed for the public good, but only around a narrow set of shareholder requirements. all other considerations do not compute. my reaction let me put this in personal perspective. in my work, i engage the public in “the heart of silicon valley” on what they want from their community and what’s missing. what i hear is concern about the loss of quiet, of connection to others, of a pace of life that is not / always a click away. this is consistent. people feel overwhelmed. as one of the chief technologists for my library, this puts me in a strange place. and i’ve been grappling with it for the past few months. on the one hand, people are curious. they’re happy to try the next big thing. but you also hear the frustration. meanwhile, the burden of the tech industry is more than inflated rents and traffic. there’s a very obvious divide between long-time residents and newcomers. there’s a sense that something has been lost. there’s anger too, even here in the shadow of google and facebook. the library as a philosophy the other day, i was visited by a eurpean library director who wanted to talk about vr. he asked me where i thought we’d be in ten years. i hesitated. my thoughts immediately went back to the words of despair that i’d been hearing from the public lately. of course, the genie’s out of the bottle. we can’t stop the digital era. vr interface revolutions will likely emerge. the robots will come. but we can harness this change to our benefit. we can add rules to heal it to our collective needs. this is where the library comes in. we have a sharing culture. a model that values bridging divides, pooling resources and re-distributing knowledge. it’s a model that is practically unique to the library if you think about it. as i read rushkoff, i kept coming back to the librarian’s philosophy on sharing. in his book, he contends that we need to re-imagine (re-code) our economy to work for people. he recalls technologies like http and rss which were invented and then given away to the world to share and re-use. this sounded very ‘librarian’ to me. we share knowledge in the form of access to technology, after all. we host training on new maker gear, coding, robotics, virtual reality. perhaps we need to double-down on this philosophy. perhaps, we can be more than just a bridge. maybe we can be the engine driving our communities to the other side. we can not just advocate, but do. have a hackathon? build a public alternative to the airbnb app to be used by people in your town. know the future in the end, libraires, technologists and digitarians need to tell a better story. we need to get outside our bubbles and tell that story with words that resonate with the benefit-nots. and more, we need that story to be backed up with real-world benefits. it starts with asking the community what kind of world they want to live it? what obstacles keep them from living that way? and then how the library and technology can help make change. we have the philosophy, we have the spaces and we have public permission. let’s get to work. posted in innovation, librarianship, society, technology, uncategorized | leave a comment is d printing dying? posted on october , by mryanhess inc.’s john brandon recently wrote about the slow, sad, and ultimately predictable decline of d printing. uh, not so fast. d printing is just getting started. for libraries whose adopted mission is to introduce people to emerging technologies, this is a fantastic opportunity to do so. but it has to be done right. another dead end? brandon cites a few reasons for his pessimism: d printed objects are low quality and the printers are finicky d printing growth is falling behind initial estimates people in manufacturing are not impressed and the costs are too high i won’t get into all that’s wrong with this analysis, as i feel like most of it is incorrect, or at the very least, a temporary problem typical of a new technology. instead, i’d like to discuss this in the library maker context. and in fact, you can apply these ideas to any tech project. how to make failure a win—no matter what libraries are quick to jump on tech. remember those qr codes that would revolutionize mobile access? did your library consider a second life branch? how about those chromebooks! inevitably, these experiments are going to fail. but that’s okay. as this blog often suggests, failure is a win when doing so teaches you something. experimenting is the first step in the process of discovery. and that’s really what all these kinds of projects need to be. in the case of a d printing project at your library, it’s important to keep this notion front and center. a d printing pilot with the goal of introducing the public to the technology can be successful if people simply try it out. that seems easy enough. but to be really successful, even this kind of basic d printing project needs to have a fair amount of up-front planning attached to it. chicago public library created a successful maker lab. their program was pretty simple: hold regular classes showing people how to use the d printers and then allow those that completed the introductory course to use the printers in open studio lab times. when i tried this out at cpl, it was quite difficult to get a spot in the class due to popularity. the grant-funded project was so successful, based on the number of attendees, that it was extended and continues to this day. as a grant-funded endeavor, cpl likely wrote out the specifics before any money was handed over. but even an internally-funded project should do this. keep the goals simple and clear so expectations on the front line match those up the chain of command. figure out what your measurements of success are before you even purchase the first printer. be realistic. always document everything. and return to that documentation throughout the project’s timeline. taking it to the next level san diego public library is an example of a maker project that went to the next level. uyen tran saw an opportunity to merge startup seminars with their maker tools at her library. she brought aspiring entrepreneurs into her library for a startup weekend event where budding innovators learned how the library could be a resource for them as they launched their companies. d printers were part of this successful program. it’s important to note that uyen already had the maker lab in place before she launched this project. and it would be risky for a library to skip the establishment of a rudimentary d printer program before trying for this more ambitious program. but it could be done if that library was well organized with solid project managers and deep roots in the target community. but that’s a tall order to fill. what’s the worst thing that could go wrong? the worst thing that could go wrong is doubling down on failure: repeating one failed project after another without changing the flawed approach behind it. i’d also add that libraries are often out ahead of the public on these technologies, so dead ends are inevitable. to address this, i would also add one more tactic to your tech projects: listening. the public has lots of concerns about a variety of things. if you ask them, they’ll tell you all about them. many of their concerns are directly related to libraries, but we can often help. we have permission to do so. people trust us. it’s a great position to be in. but we have to ask them to tell us what’s on their mind. we have to listen. and then we need to think creatively. listening and thinking outside the box was how san diego took their d printers to the next level. the long future of d printing the wright brothers first flight managed only feet in the air. a year later, they flew miles. these initial attempts looked nothing like the jet age and yet the technology of flight was born from these humble experiments. already, d printing is being adopted in multiple industries. artists are using it to prototype their designs. astronauts are using it to print parts aboard the international space station. bio-engineers are now looking at printing stem-cell structures to replace organs and bones. we’re decades away from the jet age of d printing, but this tech is here to stay. john brandon’s read is incorrect simply because he’s looking at the current state and not seeing the long-term promise. when he asks a ford engineer for his take on d printing in the assembly process, he gets a smirk. not a hotbed of innovation. what kind of reaction would he have gotten from an engineer at tesla? at apple? fundamentally, he’s approaching d printers from the wrong perspective and this is why it looks doomed. libraries should not make this mistake. the world is changing ever more quickly and the public needs us to help them navigate the new frontier. we need to do this methodically, with careful planning and a good dose of optimism. posted in innovation, technology | tagged d printing, innovation, project planning | comments the state of the library website posted on september , by mryanhess t’was a time when the library website was an abomination. those dark days have lightened significantly. but new clouds have appeared on the horizon. darkest before the dawn in the dark ages of library websites, users suffered under ux regimes that were rigid, unhelpful and confusing. this was before responsive design became a standard in the library world. it was before search engine optimization started to creep into library meetings. it was before user experience became an actual librarian job title. we’ve come a long way since i wrote the ugly truth about library websites. most libraries have evolved beyond the old “website as pamphlet” paradigm to one that is dynamic and focused on user tasks. public libraries have deployed platforms like bibliocommons to serve responsive, task-oriented interfaces that integrate their catalogs, programming and website into a single social platform. books, digital resources, programs and even loanable equipment are all accessible via a single search. what’s more, the critical social networking aspects of library life are also embedded along the user’s path. celebrated examples of this integrated solution include the san francisco public library and chicago public library. queens is also hard at work to develop a custom solution. in the academic realm, libraries have turned to unified discovery layers like worldcat discovery and ebsco discovery service to simplify (googlize) the research process. these systems put a single-search box front and center that access resources on the shelf, but also all those electronic resources that make up the bulk of academic budgets. and while there are still many laggards, few libraries ignore these problems outright. the storm ahead while the general state of online library interfaces has improved, the unforgiving, hyperbolic curve of change continues to press forward. and libraries cannot stay put. indeed, we need to quicken our pace and prepare our organizations for ongoing recalibration as the tempo of change increases. the biggest problem for library websites, is that there is little future for the library website. that’s because people will get less and less information through web browsers. indeed, consider how often you use a web browser on your phone versus an app. developments in ai, augmented reality and virtual reality will compound that trend. if you’re like chris milk, videographer and vr evangelist, you see the writing on the wall. the modes of how we experience information are about to undergo a fundamental revolution. milk likens the current state of vr to the old black and white silent films at the dawn of motion pictures. i’d extend this line of thinking to the web page. within a decade or two, i expect people will look back on web pages as a brief, transitory medium bridging print information to linked data. and as our ai, vr and ar technologies take off, they will liberate information from the old print paradigms altogether. in short, people will interact with information in more direct ways. they will ask a computer to provide them the answer. they will virtually travel to a “space” where they can experience the information they seek. get ready to re-invent the library…again so where does the library fit into this virtualized and automated future? one possibility is that the good work to transform library data into linked data will enable us to survive this revolution. in fact, it may be our best hope. another hope is that we continue to emphasize the library as a social space for people to come together around ideas. whether its a virtual library space or a physical one, the library can be the place in both local and global communities where people meet their universal thirst for connecting with others. the modes of those ideas (books, ebooks, videos, games) will matter far less than the act of connecting. in a sense, you could define the future online library as something between an mmorpg, meetup.com and the ted conference. so, the library website is vastly improved, but we won’t have long to rest on our laurels. ready player one? put on your vr goggles. call up siri. start rethinking everything you know about the library website. posted in information architecture, librarianship | tagged internet, libraries, user experience, web design, websites | comment virtual realty is getting real in the library posted on june , by mryanhess my library just received three samsung s devices with gear vr goggles. we put them to work right away. the first thought i had was: wow, this will change everything. my second thought was: wow, i can’t wait for apple to make a vr device! the samsung gear vr experience is grainy and fraught with limitations, but you can see the potential right away. the virtual reality is, after all, working off a smartphone. there is no high-end graphics card working under the hood. really, the goggles are just a plastic case holding the phone up to your eyes. but still, despite all this, it’s amazing. within twenty-four hours, i’d surfed beside the world’s top surfers on giant waves off hawaii, hung out with the masai in africa and shared an intimate moment with a pianist and his dog in their (new york?) apartment. it was all beautiful. we’ve been here before remember when the internet came online? if you’re old enough, you’ll recall the crude attempts to chat on digital bulletin board systems (bbs) or, much later, the publication of the first colorful (often jarringly so) html pages. it’s the hello world! moment for vr now. people are just getting started. you can tell the content currently available is just scratching the surface of potentialities for this medium. but once you try vr and consider the ways it can be used, you start to realize nothing will be the same again. the internet will disappear so said google ceo erik schmidt in . he was talking about the rise of ai, wearable tech and many other emerging technologies that will transform how we access data. for schmidt, the internet will simply fade into these technologies to the point that it will be unrecognizable. i agree. but being primarily a web librarian, i’m mostly concerned with how new technologies will translate in the library context. what will vr mean for library websites, online catalogs, ebooks, databases and the social networking aspects of libraries. so after trying out vr, i was already thinking about all this. here are some brief thoughts: visiting the library stacks in vr could transform the online catalog experience library programming could break out of the physical world (virtual speakers, virtual locations) vr book discussions could incorporate virtual tours of topics/locations touched on in books collections of vr experiences could become a new source for local collections vr maker spaces and tools for creatives to create vr experiences/objects year zero? still, vr makes your eyes tired. it’s not perfect. it has a long way to go. but based on my experience sharing this technology with others, it’s addictive. people love trying it. they can’t stop talking about it afterward. so, while it may be some time before the vr revolution disrupts the internet (and virtual library services with it), it sure feels imminent. posted in innovation, librarianship, technology | tagged gear vr, internet, oculus, samsung, virtual reality, vr | leave a comment w c’s css framework review posted on may , by mryanhess i’m a longtime bootstrap fan, but recently i cheated on my old framework. now i’m all excited by the w c’s new framework. like bootstrap, the w c’s framework comes with lots of nifty utilities and plug and play classes and ui features. even if you have a good cms, you’ll find many of their code libraries quite handy. and if you’re cms-deficient, this framework will save you time and headaches! why a framework? frameworks are great for saving time. you don’t have to reinvent the wheel for standard ui chunks like navigation, image positioning, responsive design, etc. all you need to do is reference the framework in your code and you can start calling the classes to make your site pop. and this is really great since not all well-meaning web teams have an eye for good design. most quality frameworks look really nice, and they get updated periodically to keep up with design trends. and coming from this well-known standards body, you can also be assured that the w c’s framework complies with all the nitty-gritty standards all websites should aspire to. things to love some of the things i fell in love with include: css-driven navigation menus. there’s really no good reason to rely on javascript for a responsive, interactive navigation menu. the w c agrees. icon support. this framework allows you to choose from three popular icon sets to bring icons right into your interface. image support: lots of great image styling including circular cropping, shadowing, etc. cards. gotta love cards in your websites and this framework has some very nice looking card designs for you to use. built-in colors. nuff sed. animations. there are plenty of other nice touches like buttons that lift off the screen, elements that drop into place and much more. i give it a big thumbs up! check it out at the w c.org. posted in reviews | tagged css, frameworks, w c, web design | comment ai first posted on may , by mryanhess looking to the future, the next big step will be for the very concept of the “device” to fade away. over time, the computer itself—whatever its form factor—will be an intelligent assistant helping you through your day. we will move from mobile first to an ai first world. google founder’s letter, april my library recently finalized a vision document for our virtual library presence. happily, our vision was aligned with the long-term direction of technology as understood by movers and shakers like google. as i’ve written previously, the library website will disappear. but this is because the internet (as we currently understand it) will also disappear. in its place, a new mode of information retrieval and creation will move us away from the paper-based metaphor of web pages. information will be more ubiquitous. it will be more free-form, more adaptable, more contextualized, more interactive. part of this is already underway. for example, people are becoming a data set. and other apps are learning about you and changing how they work based on who you are. your personal data set contains location data, patterns in speech and movement around the world, consumer history, keywords particular to your interests, associations based on your social networks, etc. ai emerging all of this information makes it possible for emerging ai systems like siri and cortana to better serve you. soon, it will allow ai to control the flow of information based on your mood and other factors to help you be more productive. and like a good friend that knows you very, very well, ai will even be able to alert you to serendipitous events or inconveniences so that you can navigate life more happily. people’s expectations are already being set for this kind of experience. perhaps you’ve noticed yourself getting annoyed when your personal assistant just fetches a wikipedia article when you ask it something. you’re left wanting. what we want is that kernel of gold we asked about. but what we get right now, is something too general to be useful. but soon, that will all change. nascent ai will soon be able to provide exactly the piece of information that you really want rather than a generalized web page. this is what google means when they make statements like “ai first” or “the web will die.” they’re talking about a world where information is not only presented as article-like web pages, but broken down into actual kernels of information that are both discrete and yet interconnected. ai first in the library library discussions often focus on building better web pages or navigation menus or providing responsive websites. but the conversation we need to have is about pulling our data out of siloed systems and websites and making it available to all modes like ai, apps and basic data harvesters. you hear this conversation in bits and pieces. the ongoing linked data project is part of this long-term strategy. so too with next-gen opacs. but on the ground, in our local strategy meetings, we need to tie every big project we do to this emerging reality where web browsers are increasingly no longer relevant. we need to think ai first. posted in librarianship, society, tech industry | tagged artificial intelligence, google, internet, libraries, linked data | leave a comment google analytics and privacy posted on april , by mryanhess collecting web usage data through services like google analytics is a top priority for any library. but what about user privacy? most libraries (and websites for that matter) lean on google analytics to measure website usage and learn about how people access their online content. it’s a great tool. you can learn about where people are coming from (the geolocation of their ip addresses anyway), what devices, browsers and operating systems they are using. you can learn about how big their screen is. you can identify your top pages and much much more. google analytics is really indispensable for any organization with an online presence. but then there’s the privacy issue. is google analytics a privacy concern? the question is often asked, what personal information is google analytics actually collecting? and then, how does this data collection jive with our organization’s privacy policies. it turns out, as a user of google analytics, you’ve already agreed to publish a privacy document on your site outlining the why and what of your analytics program. so if you haven’t done so, you probably should if only for the sake of transparency. personally identifiable data fact is, if someone really wanted to learn about a particular person, it’s not entirely outside the realm of possibility that they could glean a limited set of personal attributes from the generally anonymized data google analytics collects. ip addresses can be loosely linked to people. if you wanted to, you could set up filters in google analytics that look at a single ip. of course, on the google side, any user that is logged into their gmail, youtube or other google account, is already being tracked and identified by google. this is a broadly underappreciated fact. and it’s a critical one when it comes to how approach the question of dealing with the privacy issue. in both the case of what your organization collects with google analytics and what all those web trackers, including google’s trackers, collect, the onus falls entirely on the user. the internet is public over the years, the internet has become a public space and users of the web should understand it as such. everything you do, is recorded and seen. companies like google, facebook, mircosoft, yahoo! and many, many others are all in the data mining business. carriers and internet service providers are also in this game. they deploy technologies in websites that identify you and then sell what your interests, shopping habits, web searches and other activities are to companies interested in selling to you. they’ve made billions on selling your data. ever done a search on google and then seen ads all over the web trying to sell you that thing you searched last week? that’s the tracking at work. only you can prevent data fires the good news is that with little effort, individuals can stop most (but not all) of the data collection. browsers like chrome and firefox have plugins like ghostery, avast and many others that will block trackers. google analytics can be stopped cold by these plugins. but it won’t solve all the problems. users also need to set up their browsers to delete cookies websites save to their browsers. and moving off of accounts provided from data mining companies “for free” like facebook accounts, gmail and google.com can also help. but you’ll never be completely anonymous. super cookies are a thing and are very difficult to stop without breaking websites. and some trackers are required in order to load content. so sometimes you need to pay with your data to play. policies for privacy conscious libraries all of this means that libraries wishing to be transparent and honest about their data collection, need to also contextualize the information in the broader data mining debate. first and foremost, we need to educate our users on what it means to go online. we need to let them know its their responsibility alone to control their own data. and we need to provide instructions on doing so. unfortunately, this isn’t an opt-in model. that’s too bad. it actually would be great if the world worked that way. but don’t expect the moneyed interests involved in data mining to allow the us congress to pass anything that cuts into their bottom line. this ain’t germany, after all. there are ways with a little javascript to create a temporary opt-in/opt-out feature to your site. this will toggle tags added by google tag manager on and off with a single click. but let’s be honest. most people will ignore it. and if they do opt-out, it will be very easy for them to overlook everytime without a much more robust opt-in/opt-out functionality baked in to your site. but for most sites and users, this is asking alot. meanwhile, it diverts attention from the real solution: users concerned about privacy need to protect themselves and not take a given websites word for it. we actually do our users a service by going with the opt-out model. this underlines the larger privacy problems on the wild wild web, which our sites are a part of. posted in online security & privacy, society | tagged data mining, google analytics, online security & privacy | comments the l word posted on march , by mryanhess i’ve been working with my team on a vision document for what we want our future digital library platform to look like. this exercise keeps bringing us back to defining the library of the future. and that means addressing the very use of the term, ‘library.’ when i first exited my library (and information science) program, i was hired by adobe systems to work in a team of other librarians. my manager warned us against using the word ‘librarian’ among our non-librarian colleagues. i think the gist was: too much baggage there. so, we used the word ‘information specialist.’ fast forward a few years to my time in an academic environment at depaul university library and this topic came up in the context of services the library provided. faculty and students associated the library in very traditional ways: a quiet, book-filled space. but the way they used the library was changing despite the lag in their semantic understanding. the space and the virtual tools we put in place online helped users not only find and evaluate information, but also create, organize and share information. a case in point was our adoption of digital publishing tools like bepress and omeka, but also the scholar’s lab. i’m seeing a similar contradiction in the public library space. say library and people think books. walk into a public library and people do games, meetings, trainings and any number of online tasks. this disconnect between what the word ‘library’ evokes in the mind’s eye and what it means in practice is telling. we’ve got a problem with our brand. in fact, we may need a new word. taken literally, a library has been a word for a physical collection of written materials. the library of alexandria held scrolls for example. even code developers rely on ‘libraries’ today, which are collections of materials. in every case, the emphasis is on the collection of things. now, i’m not suggesting that we move away from books. books are vessels for ideas and libraries will always be about ideas. in fact, this focus on ideas rather than any one mode for transmitting ideas is key. in today’s library’s people not only read about ideas, they meet to discuss ideas, they brainstorm ideas. i don’t pretend to have the magic word. in fact, maybe it’s taking so long for us to drop ‘library’ because there is not a good word in existence. maybe we need create a new one. one tactic that comes to mind as we navigate this terminological evolution is to retain the library, but subsume it inside of something new. i’ve seen this done to various degrees in other libraries. for example, loyola university in chicago built an entirely new building adjacent to the book-filled library. administratively, the building is run by the library, but it is called the klarchek information commons. in that rather marvelous space looking out over lake michigan, you’ll find the modern ‘library’ in all its glory. computers, collaboration booths, etc. i like this model for fixing our identity problem and i think it would work without throwing the baby out with the bathwater. however, its done, one thing is for sure. our users have moved on from ‘the library’ and are left with no accurate way to describe that place that they love to go to when they want to engage with ideas. let’s put our thinking caps on and puts a word on their lips that does justice to what the old library has become. let’s get past the l word. posted in librarianship | tagged branding, information commons | leave a comment locking down windows posted on march , by mryanhess i’ve recently moved back to windows for my desktop computing. but windows comes with enormous privacy and security issues that people need to take into account…and get under a semblance of control. here’s how i did it. there has been much written on this subject, so what i’m including here is more of a digest of what i’ve found elsewhere with perspective on how it worked out for me over time. windows tweaker this is a pretty good tool that does what windows should do out of the box: give you one-stop access to all windows’ settings. as it is, windows has spread out many settings, including those for privacy, to the settings screen as well as registry editor and group policy editor. there are dozens of look and feel tweaks, including an easy way to force windows to use the hidden dark theme. the privacy tab, however, is the single most important. there, you can easily turn of all the nasty privacy holes in windows , such as how the os sends things like keystrokes (that’s right!) back to microsoft. the list of holes it will close is long: telemetry, biometrics, advertising id, cortana, etc. cortana speaking of cortana, i was really excited that this kind of virtual assistant was embedded in windows . i looked forward to trying it out. but then i read the fine print. cortana is a privacy nightmare. she can’t be trusted. she’s a blabbermouth and repeats back everything you tell her to not just microsoft, but indirectly to all of their advertising partners. and who knows where all that data goes and how secure it is in the long run. yuck! turn her off. pull the plug. zero her out. the easiest way to disable her is to set up a local account. but there’s more info out there, including this at pc world. local account when you first install windows , unplug the ethernet and shut down wifi. then, when you’re certain that all of msft’s listeners can’t communicate with your machine, go through the installation set up process and when asked to create/log in to your microsoft account, don’t. instead, use the local account option. the down sides of going this route are that you can’t sync your experience, accounts and apps across devices. you also won’t be able to use cortana. the up sides are that using a local account means you will be far more secure and private in whatever you do with your computer (as long as you maintain the many other privacy settings). reduce risk and streamline your pc windows comes crammed with many programs you may not want. some of these may even be tracking and sharing, so if you don’t actually use it, why not lighten the load on your system and remove them. you can do this the slow way, one app at a time, or you can use the powershell nuclear option and kill them all at once. i did this and haven’t regretted it one bit. so fire away… privacy settings i won’t go into all of this. there is plenty of solid advise on reducing your exposure on other sites (like at pc world) and some lengthy youtube videos which you can easily find. but it is critical that you go into the settings panel and turn everything off at the very least. that’s my feeling. some tell you that you even need to set up ip blocks to keep your machine from reporting back to microsoft and its advertising partners. others say this is somewhat overblown, and not unique to windows, like over at lifehacker, so i’ll leave it to you to decide. conclusion it’s really too bad that operating systems have gone down this road. our pcs should be tools for us and not the other way around. imagine if everything that happened on your device stayed private. imagine if it was all encrypted and nobody could hack into your pc or microsoft’s servers or their advertisers’ databases and learn all kinds of things about you, your family, your work, your finances, your secrets. and yet, this is precisely what microsoft (and ios, android and others) did, intentionally. frankly, i think its bordering on criminal negligence, but good luck suing when your data gets exploited. better safe than sorry…that’s my take. do a little work and lock down your computer. good luck out there… posted in online security & privacy, technology | tagged microsoft, online security & privacy, security, windows | leave a comment killer apps & hacks for windows posted on march , by mryanhess did the ux people at microsoft ever test windows ? here are some must have apps and hacks i’ve found to make life on windows quick and easy. set hotkeys for apps sometimes you just want to launch an app from your keyboard. using a method on laptopmag.com, you can do this for most any program. i use this in combination with macros like those noted below. quick switch to vpn vpn macro if you’re a smart and secure internet user, you probably already use a vpn service to encrypt the data and web requests you send over the internet (especially while on public wif-fi networks). but windows makes connecting to your vpn service a bit of a chore (i use private internet access, by the way). it’s weird because windows actually placed the connect to vpn in the communications center, but you still need to click into that, then click the vpn you want and then click connect…that’s clicks if you’re counting. i’ve tried two methods to make this at least a little easier. one caveat on all of this: if you log in with an administrator account (which i don’t because i’m concerned about security after all!), you could have your vpn client launch at start, but you’d still need to click the connect button and anytime you put the machine to sleep, it would disconnect (why they do that is beyond me). with both methods, you need to manually add a vpn account to windows built-in vpn feature. anyway, here are my two methods: macro method you can record actions as a “macro” and then save it as an executable program. you can then save the program to your desktop, start or taskbar. it’s a bit of a chore and in the end, the best you get is two-click access to your vpn connection…not the one-click you would get on a mac. if my memory serves, this method only works if you log-in with an administrator account. otherwise, you’ll be prompted for an administrator password each time…an who wants that? create shortcut to settings page add a hotkey to shortcut: create macro using something like jitbit that uses the new hotkey. save as executable create a shortcut to the desktop and pin to start optionally, change the icon to look pretty pin the communicator vpn app to your start pane. this is actually how i ended up going in the end. to do this, you need to ‘hack’ a shortcut that points to your vpn settings panel (where the connect button resides). on your desktop, right-click and select new > shortcut a shortcut wizard will open paste ms-settings:network-vpn into the form now pin the shortcut to your start and you have quick access to the connect dialog for your vpn switch between audio devices sometimes i want to jump between my speakers and my headphones and because i hate clicking and loath jumping out of windows ’s metro design into the old-school looking audio device controller, i followed the advice from the windows club. their solution uses freeware called audio switcher to assign a hotkey to different audio devices. i added audio switcher to my startup to make this a little more automated. unfortunately, because i normally work in a non-administrator account on windows , i get asked for an admin password to launch this app at startup. egads! in my case, i can now click the f (headphones) and f (speakers) keys to switch playback devices for sound. overcoming the windows education or windows pro watermark windows embeds a horrible little windows education or windows pro watermark over the lower right corner of your desktop if you use one of those versions. there are two solutions to removing this remarkably distracting bit of text. use a white background to “disappear” the white text or, have an app sit over that space. i use musicbee (recommended by lifehacker) and set position the mini-version over that spot. supposedly there’s a regex trick where you delete the text but that’s a bit much work for me for such a slight annoyance. other tricks there are a couple other tricks that i’ve used to clean up windows. removing metro apps. this allows you to remove all the built-in apps that are there simply to confound your privacy and peddle your identity to microsoft’s advertising partners. remove them. removing default folders from explorer. if you’re like me and want better performance, you use a separate hard disk drive for your music, video and images and another drive (probably an ssd) for your os and programs. windows is confusing for people with this kind of set up by placing folders in the file explorer to your images, documents, etc. on your c drive. in my case, that’s not the right drive. so i used the method linked above to remove those from explorer. posted in technology | tagged life hacks, macros, vpn, windows | leave a comment post navigation ← older posts search search subscribe enter your email address to subscribe to this blog and receive notifications of new posts by email. join other followers email address: sign me up! recent luddites, trumpism and change: a crossroads for libraries is d printing dying? the state of the library website virtual realty is getting real in the library w c’s css framework review topics best practices ( ) case studies ( ) digital services ( ) green tech ( ) information architecture ( ) innovation ( ) international librarianship ( ) librarianship ( ) library management ( ) online security & privacy ( ) reviews ( ) society ( ) tech industry ( ) technology ( ) uncategorized ( ) tweets rt @techn joy: i often check other library websites for design inspiration. today, i found my very favorite stat on @uvalibrary 's page htt… years ago congress has sold your privacy. let's buy theirs - gofundme.com/buycongressdat… years ago luddites, trumpism and change: a crossroads for libraries faillab.wordpress.com/ / / /lud… years ago archives archives select month december october september june may april march january december november april february january december november september april march february november october july june may april march february january november october september august july march february january december october september august july june may april march february january december blog at wordpress.com. fail!lab blog at wordpress.com. email (required) name (required) website loading comments... comment × privacy & cookies: this site uses cookies. by continuing to use this website, you agree to their use. to find out more, including how to control cookies, see here: cookie policy none none none none talk: using light from the dumpster fire to illuminate a more just digital world – erin white erinrwhite published april , skip to content erinrwhite in libraries, richmond | april , talk: using light from the dumpster fire to illuminate a more just digital world this february i gave a lightning talk for the richmond design group. my question: what if we use the light from the dumpster fire of to see an equitable, just digital world? how can we change our thinking to build the future web we need? presentation is embedded here; text of talk is below. hi everybody, i’m erin. before i get started i want to say thank you to the rva design group organizers. this is hard work and some folks have been doing it for years. thank you to the organizers of this group for doing this work and for inviting me to speak. this talk isn’t about . this talk is about the future. but to understand the future, we gotta look back. the web in travel with me to . twenty-five years ago! i want to transport us back to the mindset of the early web. the fundamental idea of hyperlinks, which we now take for granted, really twisted everyone’s noodles. so much of the promise of the early web was that with broad access to publish in hypertext, the opportunities were limitless. technologists saw the web as an equalizing space where systems of oppression that exist in the real world wouldn’t matter, and that we’d all be equal and free from prejudice. nice idea, right? you don’t need to’ve been around since to know that’s just not the way things have gone down. pictured before you are some of the early web pioneers. notice a pattern here? these early visions of the web, including barlow’s declaration of independence of cyberspace, while inspiring and exciting, were crafted by the same types of folks who wrote the actual declaration of independence: the landed gentry, white men with privilege. their vision for the web echoed the declaration of independence’s authors’ attempts to describe the world they envisioned. and what followed was the inevitable conflict with reality. we all now hold these truths to be self-evident: the systems humans build reflect humans’ biases and prejudices. we continue to struggle to diversify the technology industry. knowledge is interest-driven. inequality exists, online and off. celebrating, rather than diminishing, folks’ intersecting identities is vital to human flourishing. the web we have known profit first: monetization, ads, the funnel, dark patterns can we?: innovation for innovation’s sake solutionism: code will save us visual design: aesthetics over usability lone genius: “hard” skills and rock star coders short term thinking: move fast, break stuff shipping: new features, forsaking infrastructure let’s move forward quickly through the past years or so of the web, of digital design. all of the web we know today has been shaped in some way by intersecting matrices of domination: colonialism, capitalism, white supremacy, patriarchy. (thank you, bell hooks.) the digital worlds where we spend our time – and that we build!! – exist in this way. this is not an indictment of anyone’s individual work, so please don’t take it personally. what i’m talking about here is the digital milieu where we live our lives. the funnel drives everything. folks who work in nonprofits and public entities often tie ourselves in knots to retrofit our use cases in order to use common web tools (google analytics, anyone?) in chasing innovation™ we often overlook important infrastructure work, and devalue work — like web accessibility, truly user-centered design, care work, documentation, customer support and even care for ourselves and our teams — that doesn’t drive the bottom line. we frequently write checks for our future selves to cash, knowing damn well that we’ll keep burying ourselves in technical debt. that’s some tough stuff for us to carry with us every day. the “move fast” mentality has resulted in explosive growth, but at what cost? and in creating urgency where it doesn’t need to exist, focusing on new things rather than repair, the end result is that we’re building a house of cards. and we’re exhausted. to zoom way out, this is another manifestation of late capitalism. emphasis on late. because… happened. what taught us hard times amplify existing inequalities cutting corners mortgages our future infrastructure is essential “colorblind”/color-evasive policy doesn’t cut it inclusive design is vital we have a duty to each other technology is only one piece together, we rise the past year has been awful for pretty much everybody. but what the light from this dumpster fire has illuminated is that things have actually been awful for a lot of people, for a long time. this year has shown us how perilous it is to avoid important infrastructure work and to pursue innovation over access. it’s also shown us that what is sometimes referred to as colorblindness — i use the term color-evasiveness because it is not ableist and it is more accurate — a color-evasive approach that assumes everyone’s needs are the same in fact leaves people out, especially folks who need the most support. we’ve learned that technology is a crucial tool and that it’s just one thing that keeps us connected to each other as humans. finally, we’ve learned that if we work together we can actually make shit happen, despite a world that tells us individual action is meaningless. like biscuits in a pan, when we connect, we rise together. marginalized folks have been saying this shit for years. more of us than ever see these things now. and now we can’t, and shouldn’t, unsee it. the web we can build together current state: – profit first – can we? – solutionism – aesthetics – “hard” skills – rockstar coders – short term thinking – shipping future state: – people first: security, privacy, inclusion – should we? – holistic design – accessibility – soft skills – teams – long term thinking – sustaining so let’s talk about the future. i told you this would be a talk about the future. like many of y’all i have had a very hard time this year thinking about the future at all. it’s hard to make plans. it’s hard to know what the next few weeks, months, years will look like. and who will be there to see it with us. but sometimes, when i can think clearly about something besides just making it through every day, i wonder. what does a people-first digital world look like? who’s been missing this whole time? just because we can do something, does it mean we should? will technology actually solve this problem? are we even defining the problem correctly? what does it mean to design knowing that even “able-bodied” folks are only temporarily so? and that our products need to be used, by humans, in various contexts and emotional states? (there are also false binaries here: aesthetics vs. accessibility; abled and disabled; binaries are dangerous!) how can we nourish our collaborations with each other, with our teams, with our users? and focus on the wisdom of the folks in the room rather than assigning individuals as heroes? how can we build for maintenance and repair? how do we stop writing checks our future selves to cash – with interest? some of this here, i am speaking of as a web user and a web creator. i’ve only ever worked in the public sector. when i talk with folks working in the private sector i always do some amount of translating. at the end of the day, we’re solving many of the same problems. but what can private-sector workers learn from folks who come from a public-sector organization? and, as we think about what we build online, how can we also apply that thinking to our real-life communities? what is our role in shaping the public conversation around the use of technologies? i offer a few ideas here, but don’t want them to limit your thinking. consider the public sector here’s a thread about public service. ⚖️🏛️ 💪🏼💻🇺🇸 — dana chisnell (she / her) (@danachis) february , i don’t have a ton of time left today. i wanted to talk about public service like the very excellent dana chisnell here. like i said, i’ve worked in the public sector, in higher ed, for a long time. it’s my bread and butter. it’s weird, it’s hard, it’s great. there’s a lot of work to be done, and it ain’t happening at civic hackathons or from external contractors. the call needs to come from inside the house. working in the public sector government should be – inclusive of all people – responsive to needs of the people – effective in its duties & purpose — dana chisnell (she / her) (@danachis) february , i want you to consider for a minute how many folks are working in the public sector right now, and how technical expertise — especially in-house expertise — is something that is desperately needed. pictured here are the old website and new website for the city of richmond. i have a whole ‘nother talk about that new richmond website. i foia’d the contracts for this website. there are accessibility errors on the homepage alone. it’s been in development for years and still isn’t in full production. bottom line, good government work matters, and it’s hard to find. important work is put out for the lowest bidder and often external agencies don’t get it right. what would it look like to have that expertise in-house? influencing technology policy we also desperately need lawmakers and citizens who understand technology and ask important questions about ethics and human impact of systems decisions. pictured here are some headlines as well as a contract from the city of richmond. y’all know we spent $ . million on a predictive policing system that will disproportionately harm citizens of color? and that earlier this month, city council voted to allow richmond and vcu pd’s to start sharing their data in that system? the surveillance state abides. technology facilitates. i dare say these technologies are designed to bank on the fact that lawmakers don’t know what they’re looking at. my theory is, in addition to holding deep prejudices, lawmakers are also deeply baffled by technology. the hard questions aren’t being asked, or they’re coming too late, and they’re coming from citizens who have to put themselves in harm’s way to do so. technophobia is another harmful element that’s emerged in the past decades. what would a world look like where technology is not a thing to shrug off as un-understandable, but is instead deftly co-designed to meet our needs, rather than licensed to our city for . million dollars? what if everyone knew that technology is not neutral? closing this is some of the future i can see. i hope that it’s sparked new thoughts for you. let’s envision a future together. what has the light illuminated for you? thank you! erinrwhite published april , write a comment cancel reply write a comment comment name email website categories categoriesselect category bikes conferences libraries life projects richmond archives archives select month april march may march march february september august january december september august may april march march february contact e-mail me follow @erinrwhite independent publisher empowered by wordpress none twarc/deletes.py at main · docnow/twarc · github skip to content sign up sign up why github? features → mobile → actions → codespaces → packages → security → code review → project management → integrations → github sponsors → customer stories→ team enterprise explore explore github → learn and contribute topics → collections → trending → learning lab → open source guides → connect with others the readme project → events → community forum → github education → github stars program → marketplace pricing plans → compare plans → contact sales → education → in this repository all github ↵ jump to ↵ no suggested jump to results in this repository all github ↵ jump to ↵ in this organization all github ↵ jump to ↵ in this repository all github ↵ jump to ↵ sign in sign up sign up {{ message }} docnow / twarc notifications star k fork code issues pull requests actions projects wiki security insights more code issues pull requests actions projects wiki security insights permalink main switch branches/tags branches tags nothing to show {{ refname }} default view all branches nothing to show {{ refname }} default view all tags twarc/utils/deletes.py / jump to code definitions no definitions found in this file. code navigation not available for this commit go to file go to file t go to line l go to definition r copy path copy permalink cannot retrieve contributors at this time executable file lines ( sloc) . kb raw blame open with desktop view raw view blame #!/usr/bin/env python """ this program assumes that you are feeding it tweet json data for tweets that have been deleted. it will use the metadata and the api to analyze why each tweet appears to have been deleted. note that lookups are based on user id, so may give different results than looking up a user by screen name. """ import json import fileinput import collections import requests import twarc import argparse import logging user_ok = "user_ok" user_deleted = "user_deleted" user_protected = "user_protected" user_suspended = "user_suspended" tweet_ok = "tweet_ok" tweet_deleted = "tweet_deleted" # you have been blocked by the user. tweet_blocked = "tweet_blocked" retweet_deleted = "retweet_deleted" original_tweet_deleted = "original_tweet_deleted" original_tweet_blocked = "original_tweet_blocked" original_user_deleted = "original_user_deleted" original_user_protected = "original_user_protected" original_user_suspended = "original_user_suspended" t = twarc.twarc() def main(files, enhance_tweet=false, print_results=true): counts = collections.counter() for count, line in enumerate(fileinput.input(files=files)): if count % == : logging.info("processed {:,} tweets".format(count)) tweet = json.loads(line) result = examine(tweet) if enhance_tweet: tweet['delete_reason'] = result print(json.dumps(tweet)) else: print(tweet_url(tweet), result) counts[result] += if print_results: for result, count in counts.most_common(): print(result, count) def examine(tweet): user_status = get_user_status(tweet) # go with user status first (suspended, protected, deleted) if user_status != user_ok: return user_status else: retweet = tweet.get('retweeted_status', none) tweet_status = get_tweet_status(tweet) # if not a retweet and tweet deleted, then tweet deleted. if tweet_status == tweet_ok: return tweet_ok elif retweet is none or tweet_status == tweet_blocked: return tweet_status else: rt_status = examine(retweet) if rt_status == user_deleted: return original_user_deleted elif rt_status == user_protected: return original_user_protected elif rt_status == user_suspended: return original_user_suspended elif rt_status == tweet_deleted: return original_tweet_deleted elif rt_status == tweet_blocked: return original_tweet_blocked elif rt_status == tweet_ok: return retweet_deleted else: raise "unexpected retweet status %s for %s" % (rt_status, tweet['id_str']) users = {} def get_user_status(tweet): user_id = tweet['user']['id_str'] if user_id in users: return users[user_id] url = "https://api.twitter.com/ . /users/show.json" params = {"user_id": user_id} # user_deleted: and {"errors": [{"code": , "message": "user not found."}]} # user_protected: and user object with "protected": true # user_suspended: and {"errors":[{"code": ,"message":"user has been suspended."}]} result = user_ok try: resp = t.get(url, params=params, allow_ =true) user = resp.json() if user['protected']: result = user_protected except requests.exceptions.httperror as e: try: resp_json = e.response.json() except json.decoder.jsondecodeerror: raise e if e.response.status_code == and has_error_code(resp_json, ): result = user_deleted elif e.response.status_code == and has_error_code(resp_json, ): result = user_suspended else: raise e users[user_id] = result return result tweets = {} def get_tweet_status(tweet): id = tweet['id_str'] if id in tweets: return tweets[id] # user_suspended: and {"errors":[{"code": ,"message":"user has been suspended."}]} # user_protected: and {"errors":[{"code": ,"message":"sorry, you are not authorized to see this status."}]} # tweet_deleted: and {"errors":[{"code": ,"message":"no status found with that id."}]} # or {"errors":[{"code": ,"message":"sorry, that page does not exist."}]} url = "https://api.twitter.com/ . /statuses/show.json" params = {"id": id} result = tweet_ok try: t.get(url, params=params, allow_ =true) except requests.exceptions.httperror as e: try: resp_json = e.response.json() except json.decoder.jsondecodeerror: raise e if e.response.status_code == and has_error_code(resp_json, ( , )): result = tweet_deleted elif e.response.status_code == and has_error_code(resp_json, ): result = user_suspended elif e.response.status_code == and has_error_code(resp_json, ): result = user_protected elif e.response.status_code == and has_error_code(resp_json, ): result = tweet_blocked else: raise e tweets[id] = result return result def tweet_url(tweet): return "https://twitter.com/%s/status/%s" % ( tweet['user']['screen_name'], tweet['id_str']) def has_error_code(resp, code): if isinstance(code, int): code = (code, ) for error in resp['errors']: if error['code'] in code: return true return false if __name__ == "__main__": parser = argparse.argumentparser() parser.add_argument('--enhance', action='store_true', help='enhance tweet with delete_reason and output enhanced tweet.') parser.add_argument('--skip-results', action='store_true', help='skip outputting delete reason summary') parser.add_argument('files', metavar='file', nargs='*', help='files to read, if empty, stdin is used') args = parser.parse_args() main(args.files if len(args.files) > else ('-',), enhance_tweet=args.enhance, print_results=not args.skip_results and not args.enhance) copy lines copy permalink view git blame reference in new issue go © github, inc. terms privacy security status docs contact github pricing api training blog about you can’t perform that action at this time. you signed in with another tab or window. reload to refresh your session. you signed out in another tab or window. reload to refresh your session. home - ilda about about ilda transparency report strategic areas community gender and inclusion developing technologies transparency and governance projects femicide data standard artificial intelligence regional open data barometer global data barometer resources papers reports tools blog contact español english português do brasil we work towards an open, equal and data-driven region featured projects proyectos status: active ilda: the next generation proyectos status: active empatía proyectos status: active femicide data standardization proyectos status: active global data barometer proyectos status: active regional open data barometer proyectos status: active data+art news posts / / open data standards design behind closed doors? recursos / / data for development – a road ahead recursos / / flow to identify femicides dirección legal rincon / montevideo - uruguay impact hub av. , entre calle y , san pedro san josé - costa rica home researches projects blog contacto suscribite a nuestro newsletter: leave this field empty if you're human: contactanos seguinos seguinos en: apoyan: go to hellman go to hellman if you wanna end war and stuff, you gotta sing loud! open access for backlist books, part ii: the all-stars open access for backlist books, part i: the slush pile creating value with open access books infra-infrastructure, inter-infrastructure and para-infrastructure we should regulate virality notes on work-from-home teams your identity, your library four-leaf clovers responding to critical reviews ra : technology is not the problem. ra doesn't address the yet-another-wayf problem. radical inclusiveness would. ra 's recommended technical approach is broken by emerging browser privacy features ra draft rp session timeout recommendation considered harmful ra rp does not require secure protocols. it should. fudge, and open access ebook download statistics on the surveillance techno-state towards impact-based oa funding a milestone for gitenberg ebook drm and blockchain play cryptokitty and mouse. and the winner is... my face is personally identifiable information the vast potential for blockchain in libraries the shocking truth about ra : it's made of people! choose privacy week: your library organization is watching you everything* you always wanted to know about voodoo (but were afraid to ask) holtzbrinck has attacked project gutenberg in a new front in the war of copyright maximization digitalno inovacijsko stičišče slovenije - digitalno inovacijsko stičišče slovenije o nas kontakt slovenščina | english vstop iskanje katalog strokovnjakov brskajte po katalogu vpis v katalog vavčerji aktualno novice dogodki baza znanja katalog dobrih praks strokovna gradiva video vsebine razpisi sodelujte ob predsedovanju slovenije svetu eu se predstavite na digitalnem razstavišču tehnologija za ljudi poziv podjetjem k sodelovanju v pozivu - spletne tržnice sps z vavčerji znova podpira digitalizacijo naložbo sofinancirata republika slovenija in evropska unija iz evropskega sklada za regionalni razvoj. brskajte po katalogu strokovnjakov pridobite vavčer za sofinanciranje vpišite se v katalog strokovnjakov novice . apr. priložnosti za digitalizacijo slovenskega gospodarstva v okviru nove finančne perspektive - . apr. oblikovanje predlogov vsebin za študijske programe . mar. z novo pobudo lažje do digitalnih znanj za delovna mesta prihodnosti vse novice dosezite podobne rezultate tudi vi. sodelujte z nami! odkrijte prednosti povezovanja partnerjev v dih slovenije. sodelujte z nami dogodki udeležite se srečanj za digitalno transformacijo. vavčerji do % sofinanciranja na področju digitalizacije. omogočamo digitalno transformacijo. gradimo med-sektorska in multidisciplinarna partnerstva: univerze, raziskovalne in poslovne ustanove, podjetja, ponudniki ikt in podporne organizacije za podjetja, ki predstavljajo ekosistem za trajnostno kratkoročno in dolgoročno podporo tej viziji. povezovanje dih slovenije zagotavlja povezave z vlagatelji, olajša dostop do financiranja digitalne transformacije, poveže uporabnike in ponudnike digitalnih inovacij ter omogoča sinergije med digitalnimi in drugimi ključnimi tehnologijami. kompetence razvoj digitalnih kompetenc in kadrov prihodnosti. podpora digitalni transformaciji skupni razvoj storitev za podporo upravljanju digitalne preobrazbe v podjetjih. inovacije in prototipi spodbujanje odprtega inoviranja, oblikovanje novih poslovnih modelov, eksperimentalnih in pilotnih okolij. internacionalizacija prenos dobrih praks in sodelovanje z drugimi digitalnimi inovacijskimi stičišči v eu. več o dih slovenija strateški partnerji, ki nam pomagajo graditi digitalno prihodnost slovenije sodelujte z nami tudi vi? ostanite na tekočem. prijavite se na enovice! prijavite se na enovice katalog strokovnjakov vavčerji aktualno brskajte po katalogu strokovnjakov pridobite vavčer za sofinanciranje vpišite se v katalog strokovnjakov sodelujte z nami odkrijte prednosti povezovanja partnerjev v dih slovenije. sodelujte z nami dimičeva , ljubljana, slovenija pon.–pet., : - : info@dihslovenia.si podpora naložbo sofinancirata republika slovenija in evropska unija iz evropskega sklada za regionalni razvoj. Članstvo © digital innovation hub slovenia. vse pravice pridržane. pravna obvestila politika zasebnosti piškotki prosimo, potrdite piškotke. na spletni strani dihslovenia.si uporabljamo piškotke z namenom zagotavljanja spletne storitve in funkcionalnosti, ki jih brez njih ne bi mogli nuditi. prosimo vas, da s klikom na spodnji gumb potrdite uporabo piškotkov na naši spletni strani. strinjam se več informacij none github - softwaresaved/habeas-corpus: a corpus of research software used in covid- research. skip to content sign up sign up why github? features → mobile → actions → codespaces → packages → security → code review → project management → integrations → github sponsors → customer stories→ team enterprise explore explore github → learn and contribute topics → collections → trending → learning lab → open source guides → connect with others the readme project → events → community forum → github education → github stars program → marketplace pricing plans → compare plans → contact sales → education → in this repository all github ↵ jump to ↵ no suggested jump to results in this repository all github ↵ jump to ↵ in this organization all github ↵ jump to ↵ in this repository all github ↵ jump to ↵ sign in sign up sign up {{ message }} softwaresaved / habeas-corpus notifications star fork a corpus of research software used in covid- research. mit license stars forks star notifications code issues pull requests actions projects security insights more code issues pull requests actions projects security insights main switch branches/tags branches tags nothing to show {{ refname }} default view all branches nothing to show {{ refname }} default view all tags branches tags go to file code clone https github cli use git or checkout with svn using the web url. work fast with our official cli. learn more. open with github desktop download zip launching github desktop if nothing happens, download github desktop and try again. go back launching github desktop if nothing happens, download github desktop and try again. go back launching xcode if nothing happens, download xcode and try again. go back launching visual studio if nothing happens, download the github extension for visual studio and try again. go back latest commit git stats commits files permalink failed to load latest commit information. type name latest commit message commit time r data docs notebooks .gitignore habeas corpus logo.png license readme.md postbuild requirements.txt view code habeas corpus contributing ✏️ project roadmap 🏁 licensing acknowledgements 👪 references 📚 readme.md habeas corpus this is work done during the hack day at collaborations workshop , to create a corpus of research software used for covid- and coronavirus-related research that will be useful in a number of ways to the research software sustainability community around the software sustainability institute. this is based on and extends the "cord- software mentions" dataset published by the chan zuckerberg institute (doi: https://doi.org/ . /dryad.vmcvdncs ). contributing ✏️ habeas corpus is a collaborative project and we welcome suggestions and contributions. we hope one of the invitations below works for you, but if not, please let us know! 🏃 i'm busy, i only have minute tell a friend about the project! ⏳ i've got minutes - tell me what i should do suggest ideas for how you would like to use habeas corpus 💻 i've got a few hours to work on this take a look at the issues and see if there are any you can contribute to create an analysis using the data and let us know about it 🎉 i really want to help increase the community organise a hackday to use or improve habeas corpus please open a github issue to suggest a new idea or let us know about bugs. project roadmap 🏁 for tasks to work on in the near future, please see open issues. for the bigger picture, please check and contribute to plan.md licensing software code and notebooks from this project are licensed under the open source mit license. project documentation and images are licensed under cc by . . data produced by this project in the data/outputs directory is licensed under cc . other data included in this project from other sources remains licensed under its original license. acknowledgements 👪 this project originated as part of the collaborations workshop . it was based on an original idea by neil chue hong (@npch) and stephan druskat (@sdruskat), incorporated ideas and feedback from michelle barker, daniel s. katz, shoaib sufi, carina haupt and callum rollo, and was developed by alexander konovalov (@alex-konovalov), hao ye (@ha ye), louise chisholm (@louisechisholm), mark turner (@marklturner), neil chue hong (@npch), sammie buzzard (@sammiebuzzard), and stephan druskat (@sdruskat). the data is derived from the "cord- software mentions" dataset published by alex d wade and ivana williams from the chan zuckerberg initiative and released under a cc license. references 📚 softcite dataset v . : du, c., cohoon, j., lopez, p., & howison, j. (forthcoming). softcite dataset: a dataset of software mentions in biomedical and economic research publications. journal of the association for information science and technology. doi: . /asi. . cord- software mentions software in the scientific literature: problems with seeing, finding, and using software mentioned in the biology literature introducing the pid graph about a corpus of research software used in covid- research. topics research-software resources readme license mit license releases no releases published packages no packages published contributors languages jupyter notebook . % other . % © github, inc. terms privacy security status docs contact github pricing api training blog about you can’t perform that action at this time. you signed in with another tab or window. reload to refresh your session. you signed out in another tab or window. reload to refresh your session. none meta interchange meta interchange libraries, computing, metadata, and more trading for images let&# ;s search a koha catalog for something that isn&# ;t at all controversial: what you search for in a library catalog ought to be only between... entering a very brief post to start the new year. i&# ;m not inclined to make elaborate resolutions for the new year other than being very firm... data cleanup as a force for evil a quotidian concern of anybody responsible for a database is the messy data it contains. see a record about a pedro gonzÃ¡lez? bah, the assumption of... on being wrong, wrong, wrong yesterday i gave a lightning talk at the evergreen conference on being wrong. appropriately, i started out the talk on the wrong foot. i intended... fostering a habit of nondisclosure it almost doesn&# ;t need to be said that old-fashioned library checkout cards were terrible for patron privacy. want to know who had checked out a... scaling the annual code lib conference one of the beautiful things about code lib qua banner is that it can be easily taken up by anyway without asking permission. if i wanted... amelia, - last year, i wrote about the blossoming of the mellie-cat, and closed with this line: &# ;sixteen years is not long enough to get to know... mashcat at ala annual + shared notes i&# ;m leaving for chicago tomorrow to attend ala annual (and to eat some real pizza), and while going over the schedule i found some... what makes an anti-librarian? assuming the order gets made and shipped in time (update - - : it did), i&# ;ll be arriving in chicago for ala annual carrying a few tens... imls support for free and open source software the institute of museum and library services is the u.s. government&# ;s primary vehicle for direct federal support of libraries, museums, and archives across the entire... github - docnow/twarc-ids: a plugin for twarc to extract tweet ids from tweet json. skip to content sign up sign up why github? features → mobile → actions → codespaces → packages → security → code review → project management → integrations → github sponsors → customer stories→ team enterprise explore explore github → learn and contribute topics → collections → trending → learning lab → open source guides → connect with others the readme project → events → community forum → github education → github stars program → marketplace pricing plans → compare plans → contact sales → education → in this repository all github ↵ jump to ↵ no suggested jump to results in this repository all github ↵ jump to ↵ in this organization all github ↵ jump to ↵ in this repository all github ↵ jump to ↵ sign in sign up sign up {{ message }} docnow / twarc-ids notifications star fork a plugin for twarc to extract tweet ids from tweet json. mit license star forks star notifications code issues pull requests actions projects security insights more code issues pull requests actions projects security insights main switch branches/tags branches tags nothing to show {{ refname }} default view all branches nothing to show {{ refname }} default view all tags branch tags go to file code clone https github cli use git or checkout with svn using the web url. work fast with our official cli. learn more. open with github desktop download zip launching github desktop if nothing happens, download github desktop and try again. go back launching github desktop if nothing happens, download github desktop and try again. go back launching xcode if nothing happens, download xcode and try again. go back launching visual studio if nothing happens, download the github extension for visual studio and try again. go back latest commit git stats commits files permalink failed to load latest commit information. type name latest commit message commit time test-data .gitignore license readme.md setup.cfg setup.py test_twarc_ids.py twarc_ids.py view code readme.md twarc-ids this module is a simple example of how to create a plugin for twarc. it uses click-plugins to extend the main twarc command, and to manage the command line options. first you need to install twarc and this plugin: pip install twarc pip install twarc-ids now you can collect data using the core twarc utility: twarc search blacklivesmatter > tweets.jsonl and you have a new subcommand ids that is supplied by twarc-ids. twarc ids tweets.jsonl > ids.txt it's good practice to include some tests for your module. see test_twarc_ids.py for an example. you can run it directly with pytest or using: python setup.py test when creating your setup.py make sure you don't forget the entry_points magic so that twarc will find your plugin when it is installed! about a plugin for twarc to extract tweet ids from tweet json. resources readme license mit license releases no releases published packages no packages published languages python . % © github, inc. terms privacy security status docs contact github pricing api training blog about you can’t perform that action at this time. you signed in with another tab or window. reload to refresh your session. you signed out in another tab or window. reload to refresh your session. bethany nowviskie skip to content bethany nowviskie menu bio minor arcana jmu libraries cv search for: search reconstitute the world speculative collections on capacity and care foreword (to the past) posted on october october by bethany nowviskie congratulations to melissa terras and paul gooding on the publication of an important new collection of essays entitled electronic legal deposit: shaping the library collections of the future! this volume takes a global outlook on challenges and successes in preserving digital information, and stems from their digital library futures ahrc project, which first analyzed the impact of electronic legal deposit legislation on academic libraries and their users in the uk. more from melissa here, including “an ark to save learning from deluge? reconceptualising legal deposit after the digital turn,” an oa version of the opening chapter she & paul contributed to the collection. i was honored to be asked to write a foreword to the book, which i share here, under facet publishing’s green oa agreement, as my own author’s last copy of a single chapter from an edited collection. i thought i’d post it, particularly, now — as next week not only marks world digital preservation day, but another highly significant election day in the united states. we are four years on from the moment i describe below… on the morning of november th, , i looked out over a milwaukee ballroom crowded with librarians, archivists, and specialists in digital preservation. some were pensive. many were weeping. others seemed stricken. my audience had gathered for the first joint conference of the digital library federation (dlf, the us-based nonprofit organization i then directed) with its new partner, the national digital stewardship alliance (ndsa)—a cross-industry group that had recently come under dlf’s wing from its place of genesis at the library of congress. we were strangers and friends, largely though not exclusively american, united in a community of practice and the common cause of a dedication to the future of libraries, archives, and their holdings and information services in the digital age. but it suddenly felt as if we didn’t know what information was, and whether—despite all our efforts, expertise, and the shared infrastructure that our memory institutions represented—its future could be made secure. the unexpected outcome of the us presidential election, announced in the wee hours the night before, had cast a pall over this professional audience that crossed party lines. how could so many confident, data-driven predictions have been so wrong? what shared social understandings—built from the seeming common landscape of ubiquitous digital information that we had met to manage and survey—had never, in fact, been shared or were even commonly legible at all? and what evidentiary traces of this time would remain, in a political scene of post-truth posturing, the devaluation of expert knowledge, and the willingness of our new authorities—soon to become as evident on federal websites as in press conferences and cable news punditry—to revise and resubmit the historical record? the weeks and months that followed, for dlf and ndsa members, were filled with action. while the end of term web archive project sprang to its regular work of harvesting us federal domains at moments of presidential transition, reports that trump administration officials had ordered the removal of information on climate change and animal welfare from the websites of the environmental protection agency and us department of agriculture fostered a fear of the widespread deletion of scientific records, and prompted emergency ‘data rescue’ download parties. a new dlf government records transparency and accountability working group was launched. its members began watch-dogging preparations for the us census and highlighting house and senate bills meant to curtail scientific and demographic data creation; scrutinizing proposed changes to the records retention schedules of federal agencies and seeking ways to make the arcanum of their digital preservation workflows more accessible to the general public; and—amid new threats of the deportation of immigrants and the continued rise of violent nationalism—asking crucial questions about what electronic information should be made discoverable and accessible, for the protection of vulnerable persons. the social sciences research council convened a meeting on challenges to the digital preservation of documents of particular value to historians, economists, cultural anthropologists, and other social scientists, and the pegi project—focusing on the preservation of electronic government information—commissioned a wide-ranging report on at-risk, born-digital information meant to be held by us federal depository libraries and other cultural memory institutions for long-term public access and use. over time, reflective, pedagogical, and awareness-raising projects like endangered data week emerged, ties among the ndsa and international organizations like the uk-based digital preservation coalition were strengthened, and conversations on college campuses (fueled by the cambridge analytica scandal and the work of scholars of race, technology, and social media like safiya noble and siva vaidhyanathan) turned more squarely to data ethics and algorithmic literacy. frenetic data rescue parties gave over to the more measured advocacy and storytelling approach of the data refuge movement. and in the uk, an ahrc-funded ‘digital library futures’ project led by paul gooding and melissa terras (the seed of this edited collection) offered a golden opportunity to reflect—in the light of altered global understandings of the preservation and access challenges surrounding digital information—on the parliamentary legal deposit libraries (non print works) regulations of , which extended collecting practices dating to the early modern period to new media formats beyond the book. you hold in your hands (or view on your screens, or listen to through e-readers, or encounter in some other way i can’t yet foresee) an important and timely volume. it is well balanced between reflection-and-outlook and practice-and-method in what our editors call the ‘contested space’ of e-legal deposit—taking on the international and very long-term consequences of our present-day conception, regulation, assembly, positioning, and use of library-held digital collections. in other words, the essays assembled here cross space and time. the editors take a necessarily global view in bringing together a broad array of national approaches to the legal deposit of materials that already circulate in world-wide networks. and while the authors they’ve invited to contribute certainly take a long view of digital information, they also frequently address, head-on, the ways that electronic legal deposit forces our attention not just on posterity, but on the here-and-now of what media consumption means and how it works in the digital age. rather than asking us to rest our imaginations on a far-future prospect in which reading is conducted as it ever was in print (was any such act, as jerome mcgann would ask, self-identical?), the authors of these essays, collectively, assert that the kaleidoscopic mediations of e-legal deposit show us we’ve never really known what reading is. the best thinkers on libraries question the very assumptions that our memory institutions rest upon, while elevating and honoring both their promise and the centuries of labor and careful (if not always disinterested or benign) intent that have made them what they are. melissa terras and paul gooding are among the best, and the perspectives they have assembled here—from publishers, eminent librarians and archivists, technologists, organizers, and scholars—make this edited collection an essential contribution to the literature on digital preservation. it is a necessary book that grapples with legal, practical, technical, and conceptual problems: with the distinctive visions and values of libraries; with the necessarily concomitant development of policies and platforms; and even with the very nature of our documentary heritage, at a moment when print-era logics break down. what i most appreciate is that this book—like the notion of e-legal deposit itself—calls for careful consideration of both present-day services and research possibilities not yet dreamt of. in this, it serves the true mission of legal deposit libraries: to be a stable bridge between a past that is perpetually constructed by our acts of preservation and erasure—and the many futures we may mediate but can barely imagine. posted in higher ed, infrastructure a pledge: self-examination and concrete action in the jmu libraries posted on june june by bethany nowviskie “the beauty of anti-racism is that you don’t have to pretend to be free of racism to be an anti-racist. anti-racism is the commitment to fight racism wherever you find it, including in yourself. and it’s the only way forward.” — ijeoma oluo, author of so you want to talk about race. black lives matter. too long have we allowed acts of racism and deeply ingrained, institutionalized forces of white supremacy to devalue, endanger, and grievously harm black people and members of other minoritized and marginalized groups. state-sanctioned violence and racial terror exist alongside slower and more deep-seated forces of inequality, anti-blackness, colonization, militarization, class warfare, and oppression. as members of the jmu libraries dean’s council and council on diversity, equity, and inclusion, we acknowledge these forces to be both national and local, shaping the daily lived experiences of our students, faculty, staff, and community members. as a blended library and educational technology organization operating within a pwi, the jmu libraries both participates in and is damaged by the whiteness and privilege of our institutions and fields. supporting the james madison university community through a global pandemic has helped us see imbalances, biases, and fault lines of inequality more clearly. we pledge self-examination and concrete action. libraries and educational technology organizations hold power, and can share or even cede it. as we strive to create welcoming spaces and services for all members of our community, we assert the fundamental non-neutrality of libraries and the necessity of taking visible and real action against the forces of racism and oppression that affect bipoc students, faculty, staff, and community members. specifically, and in order to “fight racism wherever [we] find it, including in [ourselves],” we commit to: listen to bipoc and student voices, recognizing that they have long spoken on these issues and have too often gone unheard. educate ourselves and ask questions of all the work we do. (“to what end? to whose benefit? whose comfort is centered? who has most agency and voice? who is silenced, ignored, or harmed? who is elevated, honored, and made to feel safe? who can experience and express joy?”) set public and increasingly measurable goals related to diversity, equity, inclusion, and anti-racism, so that we may be held accountable. continue to examine, revise, and augment our collections, services, policies, spending patterns, and commitments, in order to institutionalize better practices and create offerings with enduring impact. learn from, and do better by, our own colleagues. we are a predominantly white organization and it is likely that we will make mistakes as we try to live up to this pledge. when that happens, we will do the work to learn and rectify. we will apologize, examine our actions and embedded power structures, attempt to mitigate any harm caused by our actions, and we will do better. continue reading “a pledge: self-examination and concrete action in the jmu libraries” posted in higher ed change us, too posted on june may by bethany nowviskie [the following is a brief talk i gave at the opening plenary of rbms , a meeting of the rare books and manuscripts section of the acrl/ala. this year’s theme was “response and responsibility: special collections and climate change,” and my co-panelists were frances beinecke of the national resources defense council and brenda ekwurzel of the union of concerned scientists. many thanks to conference chairs ben goldman and kate hutchens, session chair melissa hubbard, and outgoing rbms chair shannon supple. the talk draws together some of my past writings, all of which are linked to and freely available. images in my slide deck, as here, were by catherine nelson.] six years ago, i began writing about cultural heritage and cultural memory in the context of our ongoing climate disaster. starting to write and talk publicly was a frank attempt to assuage my terror and my grief—my personal grief at past and coming losses in the natural world, and the sense of terror growing inside me, both at the long-term future of the digital and physical collections in my charge, and at the unplanned-for environmental hardships and accelerating social unrest my two young children, then six and nine years old, would one day face. i latched, as people trained as scholars sometimes do, onto a set of rich and varied theoretical frameworks. these were developed by others grappling with the exact same existential dread: some quite recent, some going back to the s, the s, even the s—demonstrating, for me, not just the continuity of scientific agreement on the facts of climate change and the need for collective action (as my co-panelists have demonstrated), but scholarly and artistic agreement on the generative value of responses from what would become the environmental humanities and from practices i might call green speculative design. the concepts and theories i lighted on, however, served another function. they allowed me simultaneously to elevate and to sublimate many of my hardest-hitting feelings. in other words, i put my fears into a linguistic machine labeled “the anthropocene”—engineered to extract angst and allow me to crank out historicized, lyrical melancholy on the other end. since then i’ve also become concerned that, alongside and through the explicit, theoretical frameworks i found in the literature, i leaned unconsciously—as cis-gender white women and other members of dominant groups almost inevitably do—on implicit frameworks of white supremacy, on my gender privilege, and on the settler ideologies that got us here in the first place, all of which uphold and support the kind of emotional and fundamentally self-centered response i was first disposed to make. i see more clearly now that none of this is about my own relatively vastly privileged children and well-tended collections—except insofar as both of them exist within broader networks and collectives of care, as one achingly beloved and all-too-transitory part. please don’t misunderstand me: it remains absolutely vital that we honor our attachments, and acknowledge the complexity and deep reality of our emotional responses to living through the sixth great mass extinction of life on this planet—vital to compassionate teaching and leadership, to responsible stewardship, and to defining value systems that help us become more humane in the face of problems of inhuman scale. grappling with our emotions as librarians and archivists (and as curators, conservators, collectors, community organizers, scholars, and scientists) will be a major part of the work of this conference. it is also vital to doing work that appreciates its own inner standing point, and uses its positionality to promote understanding and effect change. but i’ve felt my own orientation changing. for me, all of this is, every day, less and less about my feelings on special collections and climate change—except to the degree that those feelings drive me toward actions that have systemic impact and are consonant with a set of values we may share. so this is a brief talk that will try to walk you (for what it’s worth) along the intellectual path i’ve taken over the past six years—in the space of about sixteen minutes. continue reading “change us, too” posted in design, infrastructuretagged embodied from the grass roots posted on march june by bethany nowviskie [this is a cleaned-up version of the text from which i spoke at the conference of research libraries uk, held at the wellcome collection in london last week. i’d like to thank my wonderful hosts for an opportunity to reflect on my time at dlf. as i said to the crowd, i hope the talk offers some useful—or at least productively vexing—ideas.] at a meeting in which the status of libraries as “neutral spaces” has been asserted and lauded, i feel obligated to confess: i’m not a believer in dispassionate and disinterested neutrality—not for human beings nor for the institutions that we continually reinforce or reinvent, based on our interactions in and through them. my training as a humanities scholar has shown me all the ways that it is in fact impossible for us to step wholly out of our multiple, layered, subjective positions, interpretive frameworks, and embodied existence. it has also taught me the dangers of assuming—no matter how noble our intentions—that socially constructed institutions might likewise escape their historical and contemporary positioning, and somehow operate as neutral actors in neutral space. happily, we don’t need neutrality to move constructively from independent points of view to shared understandings and collective action. there are models for this. the ones i will focus on today are broadly “dh-adjacent,” and they depend, sometimes uncomfortably, on the vulnerability, subjectivity, and autonomy of the people who engage with them—foregrounding the ways that individual professional roles intersect with personal lives as they come together around shared missions and goals. and as i discuss them, please note that i’ll be referring to the digital humanities and to digital librarianship somewhat loosely—in their cultural lineaments—speaking to the diffuse and socially constructed way both are practiced on the ground. in particular, i’ll reference a dh that is (for my purposes today) relatively unconcerned with technologies, methods, and objects of study. it’s my hope that shifting our focus—after much fruitful discussion, this week, of concrete research support—to a digital humanities that can also be understood as organizational, positional, and intersubjective might prompt some structural attunement to new ways of working in libraries. and i do this here, at a consortial gathering of “the most significant research libraries in the uk and ireland,” because i think that self-consciously expanding our attention in library leadership from the pragmatic provision of data, platforms, skills-teaching, and research support for dh, outward to its larger organizational frame is one way of cracking open serious and opportune contributions by people who would not consider themselves digital humanists at all. this likely includes many of you, your colleagues in university administration across areas and functions, and most members of your libraries’ personnel. such a change in focus invites all of us to be attentive to the deeper and fundamentally different kinds of engagement and transformation we might foster through dh as a vector and perhaps with only simple re-inflections of the resources we already devote to the field. it could also open our organizations up to illuminating partnerships with communities of practice who frankly don’t give a fig about academic disciplinary labels or whether they are or are not “doing dh.” i also speak to library leaders because my call is not for work to be done by individual scholars as researchers and teachers alone, nor even by small teams of librarians laboring in support of the research and cultural heritage enterprise—but rather by our fully-engaged institutions as altered structures of power. continue reading “from the grass roots” posted in administrivia, higher edtagged community-archives, digital humanities, libraries, politics how the light gets in posted on january january by bethany nowviskie i took a chance on a hackberry bowl at a farmer’s market—blue-stained and turned like a drop of water. it’s a good name for it. he had hacked it down at the bottom of his garden. (they’re filling in the timber where the oaks aren’t coming back.) but the craftsman had never worked that kind of wood before, kiln-dried at steamy summer’s height. “will it split?” it did. now it’s winter, and i make kintsukuroi, a golden repair. i found the wax conservators use on gilded picture-frames, and had some mailed from london. it softens in the heat of hands. go on. let the dry air crack you open. you can break and be mended again. posted in infrastructure, past lives posts navigation … next recent travel/talks april , : mcleod memorial lecture, wustl, on “cultural memory and the peri-pandemic library” march -april : speaking/travel hiatus during the pandemic february , : featured talk, aaad : “black temporalities: past, present, and future” july – january : speaking/travel hiatus while starting my new position at james madison university june , : tensions of europe keynote on machine learning & historical understanding, luxembourg june , : rmbs opening plenary on climate change & libraries/archives, baltimore june - , : teaching rare book school in philadelphia: “community archives and digital cultural memory” march , : rluk keynote on dh at the grassroots, london themes themesselect category administrivia design documents geospatial higher ed infrastructure past lives soft circuits & code swinburne twittering unfiltered archives archives select month october june june march january june april march february november october april february november october may march february november july may february january october september august may january november october june april march january november october september june may april january december october september june april march january december october july june may recent posts foreword (to the past) a pledge: self-examination and concrete action in the jmu libraries change us, too from the grass roots how the light gets in reconstitute the world spectra for speculative knowledge design we raise our voices iv. coda: speculative computing ( ) inauguration day open invitations speculative collections alternate futures/usable pasts everywhere, every when oldies but goodies digital humanities in the anthropocene asking for it toward a new deal resistance in the materials too small to fail reality bytes lazy consensus a skunk in the library why, oh why, cc-by? what do girls dig? standard disclaimer this site and its contents are my responsibility alone, and may not reflect the opinions of my employer, colleagues, students, children, or imaginary friends. yours everything here is free to use under a creative commons attribution . international license. twitter linkedin github flickr instagram powered by miniva wordpress theme none github - elichad/software-twilight: software end of project plans skip to content sign up sign up why github? features → mobile → actions → codespaces → packages → security → code review → project management → integrations → github sponsors → customer stories→ team enterprise explore explore github → learn and contribute topics → collections → trending → learning lab → open source guides → connect with others the readme project → events → community forum → github education → github stars program → marketplace pricing plans → compare plans → contact sales → education → in this repository all github ↵ jump to ↵ no suggested jump to results in this repository all github ↵ jump to ↵ in this user all github ↵ jump to ↵ in this repository all github ↵ jump to ↵ sign in sign up sign up {{ message }} elichad / software-twilight notifications star fork software end of project plans view license stars forks star notifications code issues pull requests actions projects security insights more code issues pull requests actions projects security insights main switch branches/tags branches tags nothing to show {{ refname }} default view all branches nothing to show {{ refname }} default view all tags branches tags go to file code clone https github cli use git or checkout with svn using the web url. work fast with our official cli. learn more. open with github desktop download zip launching github desktop if nothing happens, download github desktop and try again. go back launching github desktop if nothing happens, download github desktop and try again. go back launching xcode if nothing happens, download xcode and try again. go back launching visual studio if nothing happens, download the github extension for visual studio and try again. go back latest commit git stats commits files permalink failed to load latest commit information. type name latest commit message commit time __pycache__ .replit codeofconduct.md contributing.md license.md readme.md backend.py decisions.py environment.yml index.ipynb questionnaire.md test_data.py twilight_date_example.svg twilight_plan_example.svg view code software-twilight license introduction available badges question themes running design question format customization of ui further resources known issues readme.md this work is licensed under a creative commons attribution . international license. software-twilight software end of project plans license this project is licensed under the cc-by license. you are free to: share — copy and redistribute the material in any medium or format adapt — remix, transform, and build upon the material for any purpose, even commercially. the licensor cannot revoke these freedoms as long as you follow the license terms. the full text of the license can be found here. introduction development of software under a fixed-term project should consider several aspects of ongoing support after the project's end. there are two main eventualities: the software's development abruptly ends; there is some end-user support, although there will be no new feature development. each of these presents a problem. ending support reduces the sustainability of the environment, while ongoing maintenance requires the dedication of further resources. under the software twilight plan, the project's developer will be aware of necessary considerations. this repository is intended to be used to assess and guide a project maintainer in plans for the software's end of life. we provide a tool to be used, during the active development phase, by a project maintainer to assess and certify support plans for the project once it will no longer be actively developed. on completion of a short questionnaire the user is offered a badge to add to the repository to signal to the community when, and how, the software will go gentle into its good night. available badges we have two badges, as examples, which look look like and mean the following: - we have a (good) plan - twilight is coming up at the specified time question themes the tool covers a number of themes, including: potential funding for ongoing development required levels of future support deployment infrastructure required size of user community size of maintainer group status of ongoing contact with main developer(s)/development group running design the tool is designed in three parts: the front-end is designed with jupyer notebooks. it uses jupyter widgets, appmode package and mybinder.org to display automatically the notebook cells as a web app. the questions and answers are populated by the backend, that provides the appropriate next question based on the answer to the previous one, following a decision tree, until there are no more (relevant) questions to ask. finally, all the answers are processed and one or more badges informing on the end-of-life status of the project are provided in the form of markdown text. a summary of the answers is also provided. this text can be easily pasted into the project readme file. question format the decision tree is populated from the file decisions.py. this file has quite customizable entries in the format described below. this is initially represented by a serialized python dictionary. we have a python object question which has attributes for the question text and a dictionary for the answers (and links to each answer's follow-up question). our input file is like: decision_tree = { : question("is this a question?", {"yes": , "no", }), : question("is it a good question?", {"yes": none, "no", }), : question("really!?", {"yes": none, "no": none}) } decision_tree is an object with (contiguous, [ ,n]?) numeric identifier and a question object with question text and answer dictionary. the answer dictionary keys are answer text (diplayed) and the value the link to the question to follow. none is used to indicate that a decision will be reached with this answer. in this prototype there is no full decision tree. we indicate the path to follow by placing non-supported answers in parentheses. customization of ui if the ui can be readily customized, we describe here that. further resources here we list related resources which may be of interest to the developer of a sustainable project. fairness, etc. known issues this is a proof of concept. it is far from complete. we have a desire that the following features be implemented: improved decision tree input (not deserialization) complete decision tree final badge choice and design about software end of project plans resources readme license view license releases no releases published packages no packages published contributors languages jupyter notebook . % python . % © github, inc. terms privacy security status docs contact github pricing api training blog about you can’t perform that action at this time. you signed in with another tab or window. reload to refresh your session. you signed out in another tab or window. reload to refresh your session. none none collaborations workshop - panel live stream - invidious true invidious log in collaborations workshop - panel live stream video unavailable. watch on youtube show annotations download is disabled. genre: family friendly? no wilson score: . rating: . / engagement: . % softwaresaved subscribe | - shared march , hi! looks like you have javascript turned off. click here to view comments, keep in mind they may take a bit longer to load. play next by default: : : python software carpentry workshop march - version control with git module softwaresaved views : : python software carpentry workshop - nov - building programs with python (part ) softwaresaved views : collaborations workshop - keynotes live stream softwaresaved views : fellowship programme launch webinar softwaresaved views : : python software carpentry workshop - nov - automating tasks with the unix shell softwaresaved views : ssi fellows community call: february softwaresaved views : chris hartgerink keynote talk on "the social model of inaccessibility" softwaresaved views : : r data carpentry workshop - oct - data analysis and visualisation in r softwaresaved views : : python software carpentry workshop march - automating tasks with shell module softwaresaved views : research software camp: q&a with chris hartgerink softwaresaved views : research software camp: chris hartgerink's abstract softwaresaved views : the most elegant key change in all of pop music adam neely . m views released under the agplv by omar roth. btc: dpzymxu ryd yqzjs n kgkwcyry bch: qq ptclkzej eza a et ggc hxsq aylqut npk liberapay view javascript license information. / view privacy policy. current version: . . - ba @ master meta interchange – libraries, computing, metadata, and more skip to content meta interchange libraries, computing, metadata, and more search for submit primary menu about comment policy privacy policy search for submit trading for images posted: february categories: libraries, patron privacy let’s search a koha catalog for something that isn’t at all controversial: what you search for in a library catalog ought to be only between you and the library — and that, only briefly, as the library should quickly forget. of course, between “ought” and “is” lies the devil and his details. let’s poke around with chrome’s devtools: hit control-shift-i (on windows) switch to the network tab. hit control-r to reload the page and get a list of the http requests that the browser makes. we get something like this: there’s a lot to like here: every request was made using https rather than http, and almost all of the requests were made to the koha server. (if you can’t trust the library catalog, who can you trust? well… that doesn’t have an answer as clear as we would like, but i won’t tackle that question here.) however, the two cover images on the result’s page come from amazon: https://images-na.ssl-images-amazon.com/images/p/ . .tzzzzzzz.jpg https://images-na.ssl-images-amazon.com/images/p/ . .tzzzzzzz.jpg what did i trade in exchange for those two cover images? let’s click on the request on and see: :authority: images-na.ssl-images-amazon.com :method: get :path: /images/p/ . .tzzzzzzz.jpg :scheme: https accept: image/webp,image/apng,image/,/*;q= . accept-encoding: gzip, deflate, br accept-language: en-us,en;q= . cache-control: no-cache dnt: pragma: no-cache referer: https://catalog.libraryguardians.com/cgi-bin/koha/opac-search.pl?q=anarchist sec-fetch-dest: image sec-fetch-mode: no-cors sec-fetch-site: cross-site user-agent: mozilla/ . (windows nt . ; win ; x ) applewebkit/ . (khtml, like gecko) chrome/ . . . safari/ . here’s what was sent when i used firefox: host: images-na.ssl-images-amazon.com user-agent: mozilla/ . (windows nt . ; win ; x ; rv: . ) gecko/ firefox/ . accept: image/webp,/ accept-language: en-us,en;q= . accept-encoding: gzip, deflate, br connection: keep-alive referer: https://catalog.libraryguardians.com/cgi-bin/koha/opac-search.pl?q=anarchist dnt: pragma: no-cache amazon also knows what my ip address is. with that, it doesn’t take much to figure out that i am in georgia and am clearly up to no good; after all, one look at the referer header tells all. let’s switch over to using google book’s cover images: https://books.google.com/books/content?id=phzfwaeacaaj&printsec=frontcover&img= &zoom= https://books.google.com/books/content?id=wdgrjqaacaaj&printsec=frontcover&img= &zoom= this time, the request headers are in chrome: :authority: books.google.com :method: get :path: /books/content?id=phzfwaeacaaj&printsec=frontcover&img= &zoom= :scheme: https accept: image/webp,image/apng,image/,/*;q= . accept-encoding: gzip, deflate, br accept-language: en-us,en;q= . cache-control: no-cache dnt: pragma: no-cache referer: https://catalog.libraryguardians.com/ sec-fetch-dest: image sec-fetch-mode: no-cors sec-fetch-site: cross-site user-agent: mozilla/ . (windows nt . ; win ; x ) applewebkit/ . (khtml, like gecko) chrome/ . . . safari/ . x-client-data: cko yqeiilbjaqimtskbcmg yqeiqz kaqi qsobcmuuygeiz /kaqi smobcje ygei bxkaqinusobgkukygeyvrrkaq== and in firefox: host: books.google.com user-agent: mozilla/ . (windows nt . ; win ; x ; rv: . ) gecko/ firefox/ . accept: image/webp,/ accept-language: en-us,en;q= . accept-encoding: gzip, deflate, br connection: keep-alive referer: https://catalog.libraryguardians.com/ dnt: pragma: no-cache cache-control: no-cache on the one hand… the referer now contains only the base url of the catalog. i believe this is due to a difference in how koha figures out the correct image url. when using amazon for cover images, the isbn of the title is normalized and used to construct a url for an tag. koha doesn’t currently set a referrer-policy, so the default of no-referrer-when-downgrade is used and the full referrer is sent. google book’s cover image urls cannot be directly constructed like that, so a bit of javascript queries a web service and gets back the image urls, and for reasons that are unclear to me at the moment, doesn’t send the full url as the referrer. (cover images from openlibrary are fetched in a similar way, but full referer header is sent.) as a side note, the x-client-data header sent by chrome to books.google.com is… concerning. there are some relatively simple things that can be done to limit leaking the full referring url to the likes of google and amazon, including setting the referrer-policy header via web server configuration or meta tag to something like origin or origin-when-cross-origin. setting referrerpolicy for """ % (wordcloud_js.decode('utf '), json.dumps(words, indent= )) sys.stdout.write(output) def text(t): if 'full_text' in t: return t['full_text'] return t['text'] if __name__ == "__main__": main() copy lines copy permalink view git blame reference in new issue go © github, inc. terms privacy security status docs contact github pricing api training blog about you can’t perform that action at this time. you signed in with another tab or window. reload to refresh your session. you signed out in another tab or window. reload to refresh your session. github - lostrses/escape-room: escape room: translating between rses and arts & humanities researchers skip to content sign up sign up why github? features → mobile → actions → codespaces → packages → security → code review → project management → integrations → github sponsors → customer stories→ team enterprise explore explore github → learn and contribute topics → collections → trending → learning lab → open source guides → connect with others the readme project → events → community forum → github education → github stars program → marketplace pricing plans → compare plans → contact sales → education → in this repository all github ↵ jump to ↵ no suggested jump to results in this repository all github ↵ jump to ↵ in this organization all github ↵ jump to ↵ in this repository all github ↵ jump to ↵ sign in sign up sign up {{ message }} lostrses / escape-room notifications star fork escape room: translating between rses and arts & humanities researchers lostrses.github.io/escape-room/ cc-by- . license star forks star notifications code issues pull requests actions projects security insights more code issues pull requests actions projects security insights main switch branches/tags branches tags nothing to show {{ refname }} default view all branches nothing to show {{ refname }} default view all tags branches tags go to file code clone https github cli use git or checkout with svn using the web url. work fast with our official cli. learn more. open with github desktop download zip launching github desktop if nothing happens, download github desktop and try again. go back launching github desktop if nothing happens, download github desktop and try again. go back launching xcode if nothing happens, download xcode and try again. go back launching visual studio if nothing happens, download the github extension for visual studio and try again. go back latest commit git stats commits files permalink failed to load latest commit information. type name latest commit message commit time docs code_of_conduct.md contributing.md license readme.md view code aha: an arts and humanities adventure! welcome! what is aha? problem solution readme.md aha: an arts and humanities adventure! welcome! welcome to ⭐aha: an arts and humanities adventure!⭐ what is aha? aha: an arts and humanities adventure is an interactive game to help 'translate' concepts from computer science, for researchers in the arts and humanities. for researchers in the arts and humanities: this game aims to help you understand some of the ideas, concepts (and jargon) that your research software engineering colleagues have been using. for research software engineers: this will help you explain the ideas and concepts that you use in your work to people who do not have a computer science background. we hope that playing this game will help rses and arts and humanities reserachers work together better and build research software that helps advance research in artss and humanities! this project began at a hackday run as part of software sustainability institute's collaborations workshop . there is a proof-of-concept web version of the game now online! you can see the source for that website in the docs folder. problem researchers in the arts & humanities can benefit greatly from research software, but often don’t have the kind of background in formally-structured design that a physicist or engineer does. this can make developing research software for them challenging- particularly when a&h problems are often defined in ways that are very different from how computational problems are defined. we want to help researchers in a&h and rses to communicate better, so that they can collaborate on building research software more easily. using gamified versions of boring and dry training materials for software development, we want to make learning about software development fun and accessible. solution virtual escape room: solve a set of connected puzzles to escape the virtual game room. in the course of solving the puzzles, the participants will learn key concepts from research software development. our pitch: develop the part of this escape room series: theme: gamified activities to learn the meaning of common jargon words. e.g. api, object, function, sprint, version, agile, automation the escape room will be themed around learning to translate an alien language (software development) expressed in an unusual way, so that the unfamiliar concepts can be understood in the context of our work. for example: which of these flow diagrams is the correct one? what analogy of a rse concept can we find in humanities? format: online, can use existing websites or a github repository with questions and clues to find information. learning journey. aim: the aim is to encourage participants to look for information and find out resources about software development practices and rse related concepts themselves as they find answers to solve the puzzles. outcome of the escape room activity: participants are familiar with concepts/jargon words usually used by software developers. participants are now in a better position to work/interact with research software engineers- or to go on and learn to become digital humanities developers themselves. potential topics and set of activities for escape rooms for part onwards (not proposed for this pitch, but idea for future collaboration): set a repo to teach github / version control (create with long history, ask people to find who did what, and on what days) give a project goal that required chunking down one goal into different tasks and create clues (agile development) create puzzles to teach reproducibility use interesting data table to teach about dataframe and coding using pandas use a visualization tool or shiny app to solve different puzzles about escape room: translating between rses and arts & humanities researchers lostrses.github.io/escape-room/ resources readme license cc-by- . license contributors © github, inc. terms privacy security status docs contact github pricing api training blog about you can’t perform that action at this time. you signed in with another tab or window. reload to refresh your session. you signed out in another tab or window. reload to refresh your session. digital preservation embracing digitality - call for proposals dlf about overview calendar faq foundational principles leadership strategic plan membership join the ndsa member orientation members groups overview interest groups content infrastructure standards and practices active working groups communications and publications conference program fixity survey innovation awards levels of preservation staffing survey publications overview levels of preservation ndsa agenda osf repository conference digipres conference digipres conference cfp past digipres conferences news digital preservation embracing digitality - call for proposals the national digital stewardship alliance (ndsa) invites proposals for digital preservation : embracing digitality (#digipres ) to be held online this year on november th. digitality - the experience of living in a digital culture - has been accelerated by the global pandemic, shifting how we think, work, and exist in digital spaces. digital stewardship professionals have demonstrated that we are able to respond creatively to the preservation, discovery, and access of information beyond the physical environment. how does the advancement of digitality expand the landscape of possibilities for people, systems, the environment, and the world? what opportunities have we gained, and what must we be wary of losing? how can we best position our profession to embrace digitality and intentionally develop strategies, tools, and practices that move us forward as a community? how can we foster partnerships with other professional backgrounds to join us in this effort? please note that proposals do not have to adhere to our conference theme to be considered, but we especially encourage proposals related to embracing digitality, particularly presentations that address: emergent institutional or social/cultural barriers, risks, and opportunities inherent in preserving digitality collaboration and dismantling digital stewardship silos balancing innovation with long-term planning and maintenance critical examination of digital existence(s) and how it impacts the scope of our work envisioning a roadmap for the future of our profession or “where do we go from here, and who are we going with?” because of the virtual format and our interest in minimizing screen fatigue but still facilitating community connection, we will be offering a reduced number of sessions than are typically offered during the in-person digital preservation conference. to make space for as many voices as possible, individuals may present only once on the conference program, though names may be listed more than once in affiliation with awards and/or projects. we will offer additional ways for community members to share content and resources whether conference proposals are accepted or not. proposals are due by monday, may , at : pm est. submission length and format submissions are invited in the following lengths and formats: -minute panel: panels with - speakers on a shared topic, and an emphasis on discussion, will be given -minutes. in line with the rest of the programming, strong preference will be given to panels that are fully inclusive and reflect a wide range of expression and identity. -minute talk/demo: presentations and demonstrations are allocated minutes each, and speakers should reserve time within that allotment ( - minutes) for interactive exchanges on next steps, possible ndsa community action, and discussion or debate. : : lightning talk: share your ideas and/or projects in a lightning talk of six slides, in minutes, using one keyword or picture per slide. solution rooms: looking to connect with colleagues and brainstorm solutions to a preservation problem? propose it for the solution room! these will be -minute breakout rooms in zoom where you can receive peer support on answers to a digital stewardship challenge. submission requirements: proposal title submission format and event: varies by event first and last names, organizational affiliations, and email addresses for all authors / presenters abstract ( words max) proposal ( works max for all formats except for panels, up to words) five keywords for your proposal all submissions will be peer-reviewed by ndsa’s digital preservation program committee. the digipres planning committee will give strong preference to programming that is fully inclusive and reflects a wide range of expression and identity. presenters will be notified of their acceptance in june and guaranteed a registration slot. accepted presentations, panels, and lightning talks will be delivered via pre-recorded video that will “go live” at specific times during the conference, to avoid technology challenges and to provide a more accessible format to all of our attendees. presenters will be expected to be in attendance and available during their presentation time for live q&a. presenters will receive support in the form of tutorials, resources, and individual assistance. proposals are due by monday, may , at : pm est. about the ndsa and digital preservation the ndsa is a consortium of over organizations committed to the long-term preservation and stewardship of digital information and cultural heritage. digital preservation is the major meeting and conference of the ndsa. open to members and non-members alike, it highlights the theory and practice of digital stewardship and preservation, data curation, the digital object lifecycle, and related issues. digital preservation (#digipres ) is held in partnership with our host organization, the council on library and information resources’ (clir) digital library federation. separate calls are being issued for clir+dlf’s events, the dlf forum (november - ) and associated workshop series learn@dlf (november - ). ndsa strives to create a safe, accessible, welcoming, and inclusive event, and adheres to dlf’s code of conduct. questions? feel free to reach out to ndsa-digipres@lists.clir.org and someone will get back to you as soon as possible. ndsa about members groups calendar social twitter itunes youtube news linkedin contact ndsa c/o clir union street suite -pmb alexandria, va e: ndsa@diglib.org ndsa the ndsa is proudly hosted by the digital library federation at clir. all content on this site is available for re-use under a cc by-sa . international license. dlf view this page on github github - robintw/cw-ideas: hack day project from cw working on collating and analysing collaborative ideas and hack day projects from previous collaborations workshops skip to content sign up sign up why github? features → mobile → actions → codespaces → packages → security → code review → project management → integrations → github sponsors → customer stories→ team enterprise explore explore github → learn and contribute topics → collections → trending → learning lab → open source guides → connect with others the readme project → events → community forum → github education → github stars program → marketplace pricing plans → compare plans → contact sales → education → in this repository all github ↵ jump to ↵ no suggested jump to results in this repository all github ↵ jump to ↵ in this user all github ↵ jump to ↵ in this repository all github ↵ jump to ↵ sign in sign up sign up {{ message }} robintw / cw-ideas notifications star fork hack day project from cw working on collating and analysing collaborative ideas and hack day projects from previous collaborations workshops robintw.github.io/cw-ideas/ mit license star forks star notifications code issues pull requests actions projects security insights more code issues pull requests actions projects security insights main switch branches/tags branches tags nothing to show {{ refname }} default view all branches nothing to show {{ refname }} default view all tags branches tags go to file code clone https github cli use git or checkout with svn using the web url. work fast with our official cli. learn more. open with github desktop download zip launching github desktop if nothing happens, download github desktop and try again. go back launching github desktop if nothing happens, download github desktop and try again. go back launching xcode if nothing happens, download xcode and try again. go back launching visual studio if nothing happens, download the github extension for visual studio and try again. go back latest commit git stats commits files permalink failed to load latest commit information. type name latest commit message commit time .github/workflows archetypes content static themes/papermod contributing.md license readme.md config.yml view code exploring previous collaborations workshop ideas (cw-ideas) building locally task split during the hack day hack day presentation readme.md exploring previous collaborations workshop ideas (cw-ideas) this is the repo for a hack day project from collaborations workshop which aims to explore previous ideas from collaborations workshops and provide them in an easily browseable and searchable form. a live version of the website is hosted at https://robintw.github.io/cw-ideas/. the repo consists of markdown versions of the collaborative ideas and hackday pitches, plus code to host a website to view them. to contribute to the repository - either by adding new ideas from previous cws, or to contribute to the code to view the ideas - please see the contributing guide. this repository is licensed under the mit license, and all the ideas themselves are cc-by (this is mentioned at the bottom of each idea). the team creating this was mario antonioletti, heather turner and robin wilson. building locally the repository is automatically built and deployed on every push, but if you want to build locally for testing or debugging purposes, follow the instructions below: install hugo in the root of the repo, run hugo server the site will be built, and served on localhost - see the command-line output for the full url task split during the hack day heather turner: the brains behind the idea robin wilson: the technical guru mario antonioletti: the plodder with superpowers tasks divided orthogonally conversion of past google doc proposals to markdown (mario and robin) configuring and setting up hugo (robin and heather) provisioning a github repo (robin) hack day presentation available here about hack day project from cw working on collating and analysing collaborative ideas and hack day projects from previous collaborations workshops robintw.github.io/cw-ideas/ resources readme license mit license releases no releases published packages no packages published contributors languages html . % css . % javascript . % © github, inc. terms privacy security status docs contact github pricing api training blog about you can’t perform that action at this time. you signed in with another tab or window. reload to refresh your session. you signed out in another tab or window. reload to refresh your session. the andrew w. mellon foundation covid- response & recovery mellon continues to distribute additional funds to help shore up struggling arts and cultural organizations and higher learning institutions. learn more covid- response & recovery mellon continues to distribute additional funds to help shore up struggling arts and cultural organizations and higher learning institutions. learn more about mission history founders andrew w. mellon staff trustees annual reports annual report financials social bond framework investment overview policies code of ethics conflicts of interest and disclosure policy equal opportunity and anti-harassment policy third-party reports of misconduct or misuse of foundation funds whistleblower policy careers contact information programs higher learning research universities and institutes liberal arts colleges mellon mays undergraduate fellowship program new directions fellowships sawyer seminars regranting programs inquiries and guidelines call for proposals: the future of higher learning in prison arts and culture regranting programs inquiries and guidelines art museum staff demographic survey public knowledge publishing preservation access services inquiries and guidelines call for proposals to community-based archives humanities in place initiatives the monuments project monuments faq covid- response & recovery liberation and learning puerto rico just futures research mellon research forum research reports institutional research grants grants database grantmaking policies and guidelines grantmaking policies grant proposal guidelines grant reporting guidelines grant modifications and matching payments guides and forms news & blog events covid- response & recovery mellon continues to distribute additional funds to help shore up struggling arts and cultural organizations and higher learning institutions. learn more covid- response & recovery mellon continues to distribute additional funds to help shore up struggling arts and cultural organizations and higher learning institutions. learn more poets, poems, years of national poetry month. learn more press releases a statement on voting rights april , shared experiences blog a legendary poet's home becomes a sanctuary for young artists april , press releases mellon foundation announces five new proposals funded through the monuments project february , press releases library of congress enriches america’s story by connecting with minority communities, funded by $ m andrew w. mellon foundation grant january , covid- response, press releases andrew w. mellon foundation launches "creatives rebuild new york" january , mellon news with books and new focus, mellon foundation to foster social equity june , all news & blog posts about the andrew w. mellon foundation as the largest supporter of the arts and humanities in the us, the mellon foundation seeks to build just communities where ideas and imagination can thrive. learn more stay connected sign up to stay informed about news and events at the mellon foundation. by signing up, you agree to our privacy policy. *required mellon by the numbers ( - present) continents grants , awarded $ . billion grants database our core programs we believe that the arts and humanities are where we express our complex humanity. that belief is at the core of our grantmaking programs: higher learningenriching our understanding of a complex world, higher learning supports inclusive, multivocal humanities education and diverse learning environments with a focus on historically underserved populations. arts and culturearts and culture celebrates the power of the arts to challenge and activate the human spirit while nurturing a robust and equitable arts and culture ecosystem. public knowledgepublic knowledge supports the creation and preservation of our shared cultural record to help us explore and better understand our intertwined humanity. humanities in placehumanities in place supports a fuller, more complex telling of american histories and lived experiences by deepening the range of how and where our stories are told. created with sketchtool. stay connected sign up to stay informed about news and events at the mellon foundation. by signing up, you agree to our privacy policy. *required sign up below to receive emails from the andrew w mellon foundation. by doing so you agree to our privacy policy. email address* first name last name organization title send me information about (check as many as apply) higher learning arts and culture humanities in place- monuments public knowledge - libraries, archives, publishing, and tech press releases and announcements events subscribe stay connected sign up to stay informed about news and events at the mellon foundation. by signing up, you agree to our privacy policy. *required follow us facebooktwitterlinkedin created with sketchtool.instagram terms of use privacy rss contact us © the andrew w. mellon foundation. github - dokempf/credit-all skip to content sign up sign up why github? features → mobile → actions → codespaces → packages → security → code review → project management → integrations → github sponsors → customer stories→ team enterprise explore explore github → learn and contribute topics → collections → trending → learning lab → open source guides → connect with others the readme project → events → community forum → github education → github stars program → marketplace pricing plans → compare plans → contact sales → education → in this repository all github ↵ jump to ↵ no suggested jump to results in this repository all github ↵ jump to ↵ in this user all github ↵ jump to ↵ in this repository all github ↵ jump to ↵ sign in sign up sign up {{ message }} dokempf / credit-all notifications star fork mit license stars forks star notifications code issues pull requests actions projects wiki security insights more code issues pull requests actions projects wiki security insights master switch branches/tags branches tags nothing to show {{ refname }} default view all branches nothing to show {{ refname }} default view all tags branch tags go to file code clone https github cli use git or checkout with svn using the web url. work fast with our official cli. learn more. open with github desktop download zip launching github desktop if nothing happens, download github desktop and try again. go back launching github desktop if nothing happens, download github desktop and try again. go back launching xcode if nothing happens, download xcode and try again. go back launching visual studio if nothing happens, download the github extension for visual studio and try again. go back latest commit git stats commits files permalink failed to load latest commit information. type name latest commit message commit time creditall .all-contributorsrc .gitignore codeofconduct.md credit-all.odp license.md manifest.in readme.md sandstrom .jpg setup.py view code welcome! thanks for visiting credit all! 😁 what is this project about and why is it important? the problem the solution installation who are we? what does this project need? we need you! how can you get involved? get in touch thank you readme.md welcome! thanks for visiting credit all! 😁 in this document you can find lots of information about this project. you can just scroll down or use the quick links below for each section. welcome! thanks for visiting credit all! 😁 what is this project about and why is it important? the problem the solution installation who are we? what does this project need? we need you! how can you get involved? get in touch thank you what is this project about and why is it important? there is no one size fits all system for capturing all of the contributions during different research projects. this could be a scientific research project, a software development project or an open-source community project. we think it is important that all contributions are recorded and therefore everyone is given credit for their work more fairly. the problem current systems that attribute contributions to authors in academic outputs do not include all of the jobs/roles/tasks that are encompassed in research projects. the current problems include: capturing all roles on a project. capturing all tasks within those roles. how to convert this into the actual authorship or contributions list that can be used for project outputs. how this list can be presented. the solution taking inspiration from malin sandstroms lightning talk at the software sustainability institutes collaboration workshop , in which she proposed to combine the current contributions approaches. slide from malin sandstrom's ssi talk in this project, we propose to: expand current lists to be more inclusive - using current systems such as credit, inria, bids contributors. develop a tool to be used to record these contributions during the project such as within a github repository - we have adapted the all contributor bot for our tool. develop a way that this can be shown on academic papers - lists, table, cinema title page? (look at e.g. brainhack paper w + authors and living with machines). installation you can install the command line tool using pip: python -m pip install git+git://github.com/dokempf/credit-all.git who are we? in alphabetical order: daisy perry (writing a code of conduct, curating data) dominic kempf (initial ideas of the project, writing new code, writing documentation about the code) emma karoune (initial ideas of the project, curating data) malin sandström (initial ideas of the project, curating data) what does this project need? we need you! please review our list of tasks and tell us if something needs to be added. spot a bug and tell us about it! suggest new ways that our contributions list can be presented. if you have any feedback on the work that is going on, then please get in contact. how can you get involved? if you think you can help in any way or just want to suggest something currently not in the project, then please check out the contributor’s guidelines. please note that it’s very important to maintain a positive and supportive environment for everyone who wants to participate. when you join as a collaborator, you must follow the code of conduct in all interactions both on and offline. get in touch please feel free to get in touch with our team: ekaroune@googlemail.com thank you thanks for taking the time to read this project page and do please get involved. about no description, website, or topics provided. resources readme license mit license releases no releases published packages no packages published contributors languages python . % tex . % © github, inc. terms privacy security status docs contact github pricing api training blog about you can’t perform that action at this time. you signed in with another tab or window. reload to refresh your session. you signed out in another tab or window. reload to refresh your session. twarc/urls.py at main · docnow/twarc · github skip to content sign up sign up why github? features → mobile → actions → codespaces → packages → security → code review → project management → integrations → github sponsors → customer stories→ team enterprise explore explore github → learn and contribute topics → collections → trending → learning lab → open source guides → connect with others the readme project → events → community forum → github education → github stars program → marketplace pricing plans → compare plans → contact sales → education → in this repository all github ↵ jump to ↵ no suggested jump to results in this repository all github ↵ jump to ↵ in this organization all github ↵ jump to ↵ in this repository all github ↵ jump to ↵ sign in sign up sign up {{ message }} docnow / twarc notifications star k fork code issues pull requests actions projects wiki security insights more code issues pull requests actions projects wiki security insights permalink main switch branches/tags branches tags nothing to show {{ refname }} default view all branches nothing to show {{ refname }} default view all tags twarc/utils/urls.py / jump to code definitions no definitions found in this file. code navigation not available for this commit go to file go to file t go to line l go to definition r copy path copy permalink cannot retrieve contributors at this time executable file lines ( sloc) bytes raw blame open with desktop view raw view blame #!/usr/bin/env python """ print out the urls in a tweet json stream. """ from __future__ import print_function import json import fileinput for line in fileinput.input(): tweet = json.loads(line) for url in tweet["entities"]["urls"]: if 'unshortened_url' in url: print(url['unshortened_url']) elif url.get('expanded_url'): print(url['expanded_url']) elif url.get('url'): print(url['url']) copy lines copy permalink view git blame reference in new issue go © github, inc. terms privacy security status docs contact github pricing api training blog about you can’t perform that action at this time. you signed in with another tab or window. reload to refresh your session. you signed out in another tab or window. reload to refresh your session. none islandorans unite! it's release time | islandora skip to main content toggle navigation main menu home about events blog contact newsletter support islandora search search you are here : home islandorans unite! it's release time about menu islandora foundation get started community contribute help islandorans unite! it's release time it's that time again everyone! our amazing community contributors have made all sorts of improvements and upgrades to islandora. some have been merged, but some are still hanging out, waiting for the love they need to make it into the code base. we're calling on you - yes you! - to help us get things merged, tested, documented, and released to the world. i would like to kick off this release cycle with a sprint to mop up some the amazing improvements that have unmerged pull requests. did you know that we have pull requests for an advanced search module and a basic batch ingest form just lounging around? and that's not all. there are all kinds of great improvements that just need some time and attention. a little code review and some basic testing by others are all that is needed before we freeze the code and start turning the crank on the release process. here's a rough timetable for the release: april - th: code sprint may rd: code freeze may rd - th: testing, bug fixing, responding to feedback may th - th: documentation sprint may st - june th: more testing, bug fixing, and responding to feedback june st - july nd: testing sprint release! this is, of course, an optimistic plan. if major issues are discovered we will take the time to address them which can affect the timeline. i also plan on liaising with the documentation interest group and folks from the users' call / open meetings for the documentation and testing sprints, and their availabilities may nudge things a week in either direction. an open and transparent release process is one of the hallmarks of our amazing community. if you or your organization have any interest in helping out, please feel free to reach out or sign up for any of the upcoming sprints. there are plenty of opportunities to contribute regardless of your skill set or level of experience with islandora. there's something for everyone! we'll make further announcements for the other sprints, but you can sign up for the code sprint now using our sign up sheet. hope to see you there! submitted by dlamb on mon, / / - : log in to post comments notvisible home about events blog contact newsletter support islandora © copyright islandora foundation. header photo credits. privacy policy. pinboard (items tagged code lib) https://pinboard.in/t:code lib/ ( ) https://twitter.com/rudokemper/status/ /photo/ - - t : : + : https://twitter.com/rudokemper/status/ /photo/ bsscdt rt @rudokemper: floored and honored to have been invited to give a keynote for the #c l #code lib conference next monday. i can't wait to share about our work building open-source tech for communities to map oral histories, and how my journey started in the library + archive space! @code lib c l code lib https://twitter.com/ https://pinboard.in/u:bsscdt/b: a fefac / untitled (https://d keuthy s c .cloudfront.net/static/ems/upload/files/code lib _discogs_blacklight.pdf) - - t : : + : https://d keuthy s c .cloudfront.net/static/ems/upload/files/code lib _discogs_blacklight.pdf rybesh rt @sf : really happy to share, “dynamic integration of discogs data within a blacklight catalog” from now on i’m going to ask myself, “can this talk be a poster?” #code lib code lib https://twitter.com/ https://pinboard.in/u:rybesh/b: d f f/ the code lib journal – advancing arks in the historical ontology space - - t : : + : https://journal.code lib.org/articles/ geephroh code lib digitallibraries digitalpreservation data ontology identifiers digitalhumanities ark computationalarchivalscience cas archives journalarticle https://pinboard.in/ https://pinboard.in/u:geephroh/b: e caf / the code lib journal – managing an institutional repository workflow with gitlab and a folder-based deposit system - - t : : + : https://journal.code lib.org/articles/ aarontay managing an institutional repository workflow with gitlab and a folder-based deposit system by whitney r. johnson-freeman, @vphill, and kristy k. phillips #code lib journal issue . code lib https://twitter.com/ https://pinboard.in/u:aarontay/b: dfc c cda/ listserv . - code lib archives - - t : : + : https://lists.clir.org/cgi-bin/wa?a =code lib;e bc . miaridge rt @kiru: i forgot to post the call earlier: the code lib journal () is looking for volunteers to join its editorial committee. deadline: oct. #code lib code lib https://twitter.com/ https://pinboard.in/u:miaridge/b:e e fb / - c l [ ] future role of libraries in researcher workflows - google slides - - t : : + : https://t.co/jcoe mvhd elibtronic research-lifecycle code lib publish scholarly-communication https://pinboard.in/u:elibtronic/b: b f a/ twitter - - t : : + : https://twitter.com/i/web/status/ aarontay new issue of the the #code lib journal published. some terrific looking papers, including a review of pids for heri… code lib https://twitter.com/ https://pinboard.in/u:aarontay/b: b b d/ ( ) https://journal.code lib.org/ - - t : : + : https://journal.code lib.org/ miaridge rt @kiru: i am very happy to announce the publication of the @code lib journal issue # : webscraping… code lib https://twitter.com/ https://pinboard.in/u:miaridge/b: f c d d c/ the code lib journal – column: we love open source software. no, you can’t have our code - - t : : + : https://journal.code lib.org/articles/ pfhyper librarians are among the strongest proponents of open source software. paradoxically, libraries are also among the least likely to actively contribute their code to open source projects. this article identifies and discusses six main reasons this dichotomy exists and offers ways to get around them. code lib library libt opensource finalproject https://pinboard.in/ https://pinboard.in/u:pfhyper/b: da d a b / the code lib journal – barriers to initiation of open source software projects in libraries - - t : : + : https://journal.code lib.org/articles/ pfhyper libraries share a number of core values with the open source software (oss) movement, suggesting there should be a natural tendency toward library participation in oss projects. however dale askey’s code lib column entitled “we love open source software. no, you can’t have our code,” claims that while libraries are strong proponents of oss, they are unlikely to actually contribute to oss projects. he identifies, but does not empirically substantiate, six barriers that he believes contribute to this apparent inconsistency. in this study we empirically investigate not only askey’s central claim but also the six barriers he proposes. in contrast to askey’s assertion, we find that initiation of and contribution to oss projects are, in fact, common practices in libraries. however, we also find that these practices are far from ubiquitous; as askey suggests, many libraries do have opportunities to initiate oss projects, but choose not to do so. further, we find support for only four of askey’s six oss barriers. thus, our results confirm many, but not all, of askey’s assertions. code lib library libt opensource finalproject https://pinboard.in/ https://pinboard.in/u:pfhyper/b: f d e / twitter - - t : : + : https://twitter.com/i/web/status/ jbfink rt @kiru: the #code lib journal's issue ( / ) has been just published: . worldcat search api, go… code lib https://twitter.com/ https://pinboard.in/u:jbfink/b:d cd f e / twitter - - t : : + : https://twitter.com/i/web/status/ jbfink rt @mjingle: who's excited for the next #code lib conference?! it will be in pittsburgh, pa from march - . is your org interes… code lib https://twitter.com/ https://pinboard.in/u:jbfink/b: defc eb / attempto project - - t : : + : http://attempto.ifi.uzh.ch/site/ blebo nlp basic cnl computationallinguistics controlledlanguage controlled_language code lib compsci english knowledgerepresentation https://pinboard.in/u:blebo/b: a b f a fd/ twitter - - t : : + : https://twitter.com/i/web/status/ danbri when our grandchildren ask about the great #code lib irc battle of the tisane, we will serve them both tea and coff… code lib https://twitter.com/ https://pinboard.in/u:danbri/b: ce a e/ code lib recap – bloggers! - - t : : + : https://saaers.wordpress.com/ / / /code lib- -recap/ geephroh code lib digitallibraries research saa archives https://pinboard.in/ https://pinboard.in/u:geephroh/b: afd / digital technologies development librarian | nc state university libraries - - t : : + : https://www.lib.ncsu.edu/jobs/ehra/dtdl cdmorris we're hiring a digital technologies development librarian @ncsulibraries ! #job #libjobs #code lib #dlf #libtech dlf libtech code lib job libjobs https://twitter.com/ https://pinboard.in/u:cdmorris/b:cf e f / twitter - - t : : + : https://twitter.com/i/web/status/ jbfink ) all the men who want to preserve the idea of a #code lib discussion space as one that's free of such topics as s… code lib https://twitter.com/ https://pinboard.in/u:jbfink/b:d f / google refine cheat sheet (code lib) - - t : : + : https://code libtoronto.github.io/ - - -access/googlerefinecheatsheets.pdf psammead openrefine code lib how-to cheatsheet https://pinboard.in/ https://pinboard.in/u:psammead/b:d c d / untitled (https://www.youtube.com/watch?v=icblvnchpnw) - - t : : + : https://www.youtube.com/watch?v=icblvnchpnw cdmorris code lib southeast happening today! live stream starting at : am eastern. #code libse #code lib code libse code lib https://twitter.com/ https://pinboard.in/u:cdmorris/b:d cf c/ twitter - - t : : + : https://twitter.com/i/web/status/ lbjay it occurs to me the #code lib statement of support for chris bourg, , offers a better model… code lib https://twitter.com/ https://pinboard.in/u:lbjay/b:d d c f/ github - code lib/c l -keynote-statement: code lib community statement in support of chris bourg - - t : : + : https://github.com/code lib/c l -keynote-statement lbjay it occurs to me the #code lib statement of support for chris bourg, , offers a better model… code lib https://twitter.com/ https://pinboard.in/u:lbjay/b: b ef c / twitter - - t : : + : https://twitter.com/i/web/status/ jbfink now that the #code lib discord is up & running, i'm contemplating leaving slack overall, with exception for plannin… code lib https://twitter.com/ https://pinboard.in/u:jbfink/b:c d f ddd d/ ( ) https://twitter.com/palcilibraries/status/ /photo/ - - t : : + : https://twitter.com/palcilibraries/status/ /photo/ cdmorris talking privacy and ra at #c l with dave lacy from @templelibraries #code lib c l code lib https://twitter.com/ https://pinboard.in/u:cdmorris/b: f c c f / scope: an access interface for dips from archivematica - - t : : + : https://github.com/cca-public/dip-access-interface sdellis archives code lib https://pinboard.in/ https://pinboard.in/u:sdellis/b: ef d c / review, appraisal and triage of mail (ratom) - - t : : + : http://ratom.web.unc.edu/ sdellis archives code lib https://pinboard.in/ https://pinboard.in/u:sdellis/b: cdd / national web privacy forum - msu library | montana state university - - t : : + : http://www.lib.montana.edu/privacy-forum/ sdellis privacy analytics code lib https://pinboard.in/ https://pinboard.in/u:sdellis/b: b db e / the code lib journal - - t : : + : https://journal.code lib.org/ ratledge code lib library_technology journal journals_code lib https://pinboard.in/ https://pinboard.in/u:ratledge/b: a f c b / code lib | we are developers and technologists for libraries, museums, and archives who are dedicated to being a diverse and inclusive community, seeking to share ideas and build collaboration. - - t : : + : https://code lib.org/ ratledge code lib https://pinboard.in/ https://pinboard.in/u:ratledge/b: cfc ccb / twitter - - t : : + : https://twitter.com/i/web/status/ verwinv ne'er had the pleasure to attend #code lib myself ... but if you're thinking about it but can't afford to go - ther… code lib https://twitter.com/ https://pinboard.in/u:verwinv/b:f ceb/ twitter - - t : : + : https://twitter.com/justindlc/status/ /photo/ librariesval rt @justindlc: pre-conference meetup at ormsby's for code lib southeast ! #code libse #code lib code lib code libse https://twitter.com/ https://pinboard.in/u:librariesval/b: c ad b / twitter - - t : : + : https://twitter.com/i/web/status/ jbfink thanks @lydia_zv @redlibrarian and jolene (are you on twitter, i can find you?) for a great #code lib day! it was… code lib https://twitter.com/ https://pinboard.in/u:jbfink/b: faa e bad/ twitter - - t : : + : https://twitter.com/i/web/status/ jbfink my slides and speakers notes from #code lib #c ln on ursula franklin's "real world of technology" (which i really… code lib c ln https://twitter.com/ https://pinboard.in/u:jbfink/b:a ed a fc / twitter - - t : : + : https://twitter.com/i/web/status/ jbfink in an unfortunate timing, it appears the code lib wiki is down the first day of #code lib north - there's a cache o… code lib https://twitter.com/ https://pinboard.in/u:jbfink/b: edcfb c/ twitter - - t : : + : https://twitter.com/i/web/status/ jbfink rt @kiru: just off the (word)press: the #code lib journal issue is available: . great articles writ… code lib https://twitter.com/ https://pinboard.in/u:jbfink/b:db c bb a / the code lib journal - - t : : + : http://journal.code lib.org/ jbfink rt @kiru: just off the (word)press: the #code lib journal issue is available: . great articles writ… code lib https://twitter.com/ https://pinboard.in/u:jbfink/b: be / twitter - - t : : + : https://twitter.com/gitwishes/status/ lbjay this is all of #code lib working on @bot lib circa . code lib https://twitter.com/ https://pinboard.in/u:lbjay/b: e b b / twitter - - t : : + : https://twitter.com/gmcharlt/status/ danbri this is fabulous news for the cultural heritage open source world. big ups to @code lib and @clirdlf! #code lib code lib https://twitter.com/ https://pinboard.in/u:danbri/b: cbe ff f / twitter - - t : : + : https://twitter.com/i/web/status/ miaridge rt @achdotorg: we too co-sign the #code lib community statement in support of @mchris duke. we continue to admire an honor our col… code lib https://twitter.com/ https://pinboard.in/u:miaridge/b:cf f d e / code lib/c l -keynote-statement: code lib community statement in support of chris bourg - - t : : + : https://github.com/code lib/c l -keynote-statement jbfink code lib github https://pinboard.in/ https://pinboard.in/u:jbfink/b: b f bd / code lib community statement in support of chris bourg | c l -keynote-statement - - t : : + : https://code lib.github.io/c l -keynote-statement/ wragge rt @clirdlf: we’re proud to stand with the #code lib community in support of #c l keynoter @mchris duke: code lib c l https://twitter.com/ https://pinboard.in/u:wragge/b:d e b e / matthew reidsma : auditing algorithms - - t : : + : https://matthew.reidsrow.com/talks/ malantonio

talks about libraries, technology, and the web by matthew reidsma.

algorithms bias search libraries technology code lib code lib- https://pinboard.in/u:malantonio/b: dd c f / for the love of baby unicorns: my code lib keynote | feral librarian - - t : : + : https://chrisbourg.wordpress.com/ / / /for-the-love-of-baby-unicorns-my-code lib- -keynote/ petej code lib diversity technology libraries inclusion mansplaining https://pinboard.in/ https://pinboard.in/u:petej/b: d e f / jira for archives - google slides - - t : : + : https://docs.google.com/presentation/d/ uwywg -nt qjm-j haavsoh ikzucax efbnlcy /edit#slide=id.g a ccaec_ _ malantonio see https://youtu.be/ cno sernxi?t= h m s for presentation code lib code lib- libraries work-life https://pinboard.in/u:malantonio/b: fc b e / twitter - - t : : + : https://twitter.com/justin_littman/status/ /photo/ aarontay rt @justin_littman: peer review of my #code lib poster on "where to get twitter data for academic research." code lib https://twitter.com/ https://pinboard.in/u:aarontay/b:c c e d/ availability calendar - kalorama guest house - - t : : + : https://secure.rezovation.com/reservations/availabilitycalendar.aspx?s=ut fw wid skorasaurus kalorama guest house code lib https://pinboard.in/ https://pinboard.in/u:skorasaurus/b: f ea / ( ) https://twitter.com/i/web/status/ - - t : : + : https://twitter.com/i/web/status/ docdre rt @nowviskie: icymi: #code lib registration is open! @mmsubram & @mchris duke to keynote, reception in the great hall… code lib https://twitter.com/ https://pinboard.in/u:docdre/b: e f cb/ ( ) https://twitter.com/freethefiles/status/ /photo/ - - t : : + : https://twitter.com/freethefiles/status/ /photo/ verwinv yay! i'm presenting at #code lib. and i can say hello to walter forsberg, @hbmcd and @cristalyze! code lib https://twitter.com/ https://pinboard.in/u:verwinv/b: bf d / ( ) https://twitter.com/i/web/status/ - - t : : + : https://twitter.com/i/web/status/ verwinv registration for #code lib is now open! and its being held in #washingtondc where our #memorylab is - so come visit… washingtondc code lib memorylab https://twitter.com/ https://pinboard.in/u:verwinv/b: bc fa c/ code lib - washington, d.c. - - t : : + : http:// .code lib.org/ verwinv last day to vote #code lib program! don't forget 😓! code lib https://twitter.com/ https://pinboard.in/u:verwinv/b: efcaa db a / presentation voting survey - - t : : + : https://www.surveymonkey.com/r/c l -presentations verwinv vote #code lib proposals rather than the presenters. new anonymity feature! check it: got until / code lib https://twitter.com/ https://pinboard.in/u:verwinv/b: a e b / lodlam challenge winners - - t : : + : https://summit .lodlam.net/ / / /lodlam-challenge-winners/ miaridge rt @lodlam: #lodlam challenge prize winners congrats to dive+ (grand) & warsampo (open data) teams #dh #musetech #code lib dh musetech lodlam code lib https://twitter.com/ https://pinboard.in/u:miaridge/b:c bd / jobboard - - t : : + : https://jobs.code lib.org/ lbjay some heroes don't wear capes, y'all. back online and and better than ever thanks to @ryanwick and @_cb_ #code lib code lib https://twitter.com/ https://pinboard.in/u:lbjay/b:a f f b e/ digital technologies development librarian | ncsu libraries - - t : : + : https://www.lib.ncsu.edu/jobs/ehra/digital-technologies-development-librarian jbfink rt @ronallo: job opening: digital technologies development librarian @ncsulibraries #code lib #libtechwomen know someone? libtechwomen code lib https://twitter.com/ https://pinboard.in/u:jbfink/b: a bff fd/ who's using ipfs in libraries, archives and museums - communities / libraries, archives and museums - discuss.ipfs.io - - t : : + : https://discuss.ipfs.io/t/whos-using-ipfs-in-libraries-archives-and-museums/ sdellis career ipfs libraries code lib https://pinboard.in/ https://pinboard.in/u:sdellis/b:df f bc b/ scott w. h. young on twitter: "slides for my talk on participatory design with underrepresented populations. thank you, #c l :) https://t.co/rvs zdv u" - - t : : + : https://twitter.com/hei_scott/status/ brainwane refers to my code lib keynote on empathy & ux yay code lib https://pinboard.in/ https://pinboard.in/u:brainwane/b: c ef cde / twitter - - t : : + : https://twitter.com/i/web/status/ lbjay have not read the full report but based on the abstract seems useful to those involved in the #code lib incorporati… code lib https://twitter.com/ https://pinboard.in/u:lbjay/b: f f b b / resistanceisfertile - google drive - - t : : + : https://drive.google.com/drive/folders/ b ooqctdnhjmy wn zw htxc pmhswe code lib harlow keynote https://pinboard.in/u:pmhswe/b: c / resistanceisfertile - google drive - - t : : + : https://drive.google.com/drive/folders/ b ooqctdnhjmy wn zw htxc markpbaggett code lib harlow keynote https://pinboard.in/ https://pinboard.in/u:markpbaggett/b:cffeeb e e / google drive cms - - t : : + : https://www.drivecms.xyz/ jju webdev programming tech code lib https://pinboard.in/u:jju/b:f af e a a / code lib | docker presentation - google slides - - t : : + : https://docs.google.com/presentation/d/ p pr p dxikxjwe _sha-rsktax-hzquo-ffz-th /edit#slide=id.p markpbaggett code lib docker https://pinboard.in/ https://pinboard.in/u:markpbaggett/b:bd aec e/ best catalog results page ever - - t : : + : https://www.dropbox.com/s/jbxe jpbdck z/deibel-c l -best-ever.pptx markpbaggett code lib accessibility presentation https://pinboard.in/ https://pinboard.in/u:markpbaggett/b: f b fea a/ participatory user experience design with underrepresented populations: a model for disciplined empathy - - t : : + : http:// .code lib.org/talks/participatory-user-experience-design-with-underrepresented-populations-a-model-for-disciplined-empathy brainwane am honored & humbled to see #c l glad my talk/article was helpful! wish i were at #code lib to thank you in person c l code lib https://twitter.com/ https://pinboard.in/u:brainwane/b: bf ebd d d/ twitter - - t : : + : https://twitter.com/i/web/status/ bsscdt why don't you join us in the #libux slack? sign yourself up: #litaux #ux #code lib… ux libux litaux code lib https://twitter.com/ https://pinboard.in/u:bsscdt/b: f bd a / untitled (http://libux.co/slack?utm_content=buffer f &utm_medium=social&utm_source=twitter.com&utm_campaign=buffer) - - t : : + : http://libux.co/slack bsscdt why don't you join us in the #libux slack? sign yourself up: #litaux #ux #code lib… ux libux litaux code lib https://twitter.com/ https://pinboard.in/u:bsscdt/b: a bf / twitter - - t : : + : https://twitter.com/jschneider/status/ /photo/ jcarletonoh ten principles for user protection: #code lib #privacy #ischoolui ischoolui privacy code lib https://twitter.com/ https://pinboard.in/u:jcarletonoh/b: bf dea b/ technology in hostile states: ten principles for user protection | the tor blog - - t : : + : https://blog.torproject.org/blog/technology-hostile-states-ten-principles-user-protection jcarletonoh ten principles for user protection: #code lib #privacy #ischoolui ischoolui privacy code lib https://twitter.com/ https://pinboard.in/u:jcarletonoh/b: aebf a/ analyzing marc with microxpath, part - u. ogbuji on the s & sies - - t : : + : http://uogbuji.tumblr.com/post/ /analyzing-marc-with-microxpath-part- #_=_ uche analyzing marc with microxpath, part #xml #xpath #libraries #code lib xpath xml libraries code lib https://twitter.com/ https://pinboard.in/u:uche/b: f fca a / library technology jobs - - t : : + : http://librarytechnology.org/jobs/ jbfink rt @yo_bj: / for the #code lib, #lita, and #mashcat crowds, keep an eye out on for #libtech jobs. libtech mashcat lita code lib https://twitter.com/ https://pinboard.in/u:jbfink/b:ad bba / keynote speakers nominations - code lib - - t : : + : http://wiki.code lib.org/ _keynote_speakers_nominations verwinv do you know who should keynote #code lib ? help us out: #c l code lib c l https://twitter.com/ https://pinboard.in/u:verwinv/b:fe b c e/ library of congress lccn permalink sh - - t : : + : https://lccn.loc.gov/sh anneheathen rt @julieswierczek: #code lib #c l - "black lives matter movement" is now a subject heading. . catalogers, make sure you use it! c l code lib https://twitter.com/ https://pinboard.in/u:anneheathen/b: de d ff/ hope for girls & women skip to content facebook instagram twitter linkedin search for: hope for girls & women menu news news from hope upcoming events about us about rhobi about hope about fgm in tanzania background updates from rhobi our supporters awards & articles contact challenges team members marketing material covid- what we do safe houses sponsor a girl sponsored girls community road shows alternative rites of passage film screenings: in the name of your daughter digital champions mapping re-educating cutters donate we provide a safe environment for girls escaping female genital mutilation (fgm) girls often arrive at hope’s safe houses late at night with just the clothes they have run away in. those arriving on foot have to navigate from remote, rural areas in the dark. we also work with local police teams to rescue girls when we are alerted that fgm is going to take place. we provide girls with safety, education and hope. donate to hope sponsor a girl according to the united nations, in the mara region of tanzania, % of women aged between and report having undergone fgm. hope for girls and women was founded by the tanzanian activist rhobi samwelly in . rhobi’s personal experience of being forced to undergo female genital mutilation (fgm) as a child inspired her lifelong commitment to fight for the rights of girls and women. our organisation runs two safe houses in the butiama and serengeti districts of the mara region of tanzania, which shelter and support those fleeing fgm, child marriage, and other forms of gender based violence. read more here. find out more about our important work to provide alternative rites of passage ceremonies here. we’re continually working on raising awareness locally and globally, whilst also raising funds for our safe houses. watch our new film here subscribe here to follow our updates: email address: sign up share this: twitter facebook recent posts / / beccadash human rights detecting pests in maize and cassava with the plantnuru app / / / / beccadash event reports rhobi participates in women’s health talk / / beccadash event reports debating gender-based violence with male villagers in northern tanzania / / / / hopeforgirlsandwomen event reports fighting fgm with maps / / / / beccadash human rights how mapping is helping tanzanian villages source water more posts→ create a website or blog at wordpress.com email (required) name (required) website loading comments... comment × none none none learn@dlf - dlf forum skip to content home about code of conduct coc reporting form thank you resources news affiliated events learn@dlf ndsa’s #digipres sponsors sponsorship opportunities registration cfp search for... search for... toggle navigation toggle navigation home about code of conduct coc reporting form thank you resources news affiliated events learn@dlf ndsa’s #digipres sponsors sponsorship opportunities registration cfp join us for learn@dlf november - , to cultivate creative training and professional development opportunities stemming from our past three successful dlf forum pre-conferences as well as our series of video tutorials from last year’s first-ever virtual dlf forum, we are excited to host learn@dlf the week immediately following the dlf forum and ndsa’s digital preservation on monday-wednesday, november - , . stay tuned for updates on learn@dlf offerings. share your experiences on twitter with #learnatdlf! want forum news? subscribe to our newsletter to stay informed! subscribe sponsorship opportunities about dlf join dlf contact menu sponsorship opportunities about dlf join dlf contact envelope facebook twitter youtube instagram linkedin skip to content open toolbar accessibility tools increase text decrease text grayscale high contrast negative contrast light background links underline readable font reset sitemap call for proposals - dlf forum skip to content home about code of conduct coc reporting form thank you resources news affiliated events learn@dlf ndsa’s #digipres sponsors sponsorship opportunities registration cfp search for... search for... toggle navigation toggle navigation home about code of conduct coc reporting form thank you resources news affiliated events learn@dlf ndsa’s #digipres sponsors sponsorship opportunities registration cfp call for proposals dlf forum & learn@dlf call for proposals clir’s digital library federation invites proposals for the dlf forum (november - ) and learn@dlf (november - ), our workshop series, both held online this year. a separate call will be issued for digital preservation , the annual conference of the ndsa (november ). the forum is a meeting place, a marketplace, and a congress for digital library practitioners from dlf member institutions and the broader community. now that our events will take place virtually for a second time, we look forward to new and better ways to come together—as always, with community at the center. therefore, our guiding focus for this year’s forum is sustaining our community. relentless innovation, disruptive change, and constant demands on our time and energy rarely allow for a pause to assess how we got here. sustenance comes in many forms and while it allows for growth, it is also an end in itself. how can we then shift our focus to prioritize the sustaining and nurturing of ourselves and our communities while still pushing for greater openness and inclusivity? pervasive racism persists and contributes to wrenching inequalities in the united states, especially among our black, indigenous, and people of color (bipoc) communities. clir has long recognized this inequity; diversity, social justice, and broad access to cultural heritage have been integral to our mission. in , we reaffirm our commitment to pursuing greater equity and justice throughout the dlf forum, working with our entire community toward an inclusivity that prizes the chorus of diverse voices needed for systemic change. as such, the planning committee will again prioritize submissions from bipoc people and people working at historically black colleges and universities (hbcus) and other bipoc-centered libraries, archives, and museums. we therefore have self-identification options in the proposal submission form. for all events, we encourage proposals from dlf members and non-members; regulars and newcomers; digital library practitioners and those in adjacent fields such as institutional research and educational technology; and students, early-career professionals and senior staff alike. proposals to more than one event are permitted, though please submit different proposals for each. our events the dlf forum will take place monday, november through wednesday, november , . digital preservation : embracing digitality will take place on thursday, november , . more information on that event can be found here: https://ndsa.org/conference/ learn@dlf is a series of workshops offered the week after the dlf forum, november - , . about presenting accepted presentations and panels will be delivered via pre-recorded video. this format allows for flexible watch times and speeds, captioning, and avoids many technical challenges. videos must be submitted by wednesday, september . presenters will receive support in the form of tutorials, resources, and individual assistance. presenters will be expected to be in attendance and available during their presentation time for live q&a (chat-based or video, format tbd). to make space for as many voices as possible, individuals may present only once on the forum program. the dlf forum is explicitly designed to enact and support the dlf community’s values, and we strive to create a safe, accessible, welcoming, and inclusive event that reflects our code of conduct. submissions & evaluation based on community feedback and the work of our program committee, we welcome submissions geared toward a practitioner audience that: clearly engage with dlf’s mission of advancing research, learning, social justice, and the public good through the creative design and wise application of digital library technologies activate and inspire participants to think, make, and do engage people from different backgrounds, experience levels, and disciplines include clear take-aways that participants can implement in their own work submission formats sessions are invited in the following lengths and formats: at the dlf forum, november - : -minute panels: a panel discussion of three to four speakers on a unified topic, with an emphasis on the discussion. a maximum of four speakers is allowed per submission. proposals with representative and inclusive speaker involvement will be favored by the committee, and all-male-identifying panels will not be accepted. the main goals of the panel format at the dlf forum are to bring together diverse perspectives on a topic and to encourage a community discussion of panelists’ approaches or findings. -minute presentations: a presentation by one to two speakers on a single topic or project. a maximum of two speakers is allowed per submission. presentations will be grouped by the program committee based on overarching themes or ideas. -minute lightning talks: high-profile, high-energy lightning talks held in plenary, with the opportunity to point attendees to contact information and additional materials online. no more than two speakers are allowed per submission. -minute birds of a feather (boaf) sessions: working on a project on which you’d like feedback? have a question you want to ponder with other interested people? new this year, -minute boaf sessions are live video discussion sections where folks can discuss a topic of the proposer’s choice. these are roundtables where ideas can be shared and questions can be asked in the spirit of shared knowledge. at learn@dlf, november - : -minute workshops: live, in-depth, hands-on training sessions on specific tools, techniques, workflows, or concepts. all workshop organizers are asked to provide details on technology needed, participant proficiency level, and learning outcomes for participants. workshops must be interactive and inclusive, and the strongest proposals will demonstrate this clearly. interested in presenting something longer? consider submitting a ‘part i’ (morning session) and ‘part ii’ (afternoon session). - -minute tutorials: pre-recorded training sessions or demonstrations between to minutes in length about specific tools, techniques, workflows, or concepts. proposal requirements proposal title submission format and event: varies by event first and last names, organizational affiliations, and email addresses for all authors / presenters abstract ( words max) proposal ( works max for all formats except for panels and workshops, up to words) five keywords for your proposal submit using our online system: bit.ly/ clircfps. submit your proposal the deadline for all proposals is monday, may , , at : pm eastern time. as in previous years, all submissions will be peer reviewed. broader dlf community input will also be solicited through an open community voting process, which will inform the program committee’s final decisions. selected presenters will be notified over the summer and will have a minimum of four weeks to prepare their recordings. we are still looking for sponsors for this year’s events! if you or someone you know may be interested, check out our sponsorship opportunities or contact us. questions? you can reach us at forum@diglib.org. want forum news? subscribe to our newsletter to stay informed! subscribe sponsorship opportunities about dlf join dlf contact menu sponsorship opportunities about dlf join dlf contact envelope facebook twitter youtube instagram linkedin skip to content open toolbar accessibility tools increase text decrease text grayscale high contrast negative contrast light background links underline readable font reset sitemap none github - hughrun/yawp: command line app for publishing social media posts skip to content sign up sign up why github? features → mobile → actions → codespaces → packages → security → code review → project management → integrations → github sponsors → customer stories→ team enterprise explore explore github → learn and contribute topics → collections → trending → learning lab → open source guides → connect with others the readme project → events → community forum → github education → github stars program → marketplace pricing plans → compare plans → contact sales → education → in this repository all github ↵ jump to ↵ no suggested jump to results in this repository all github ↵ jump to ↵ in this user all github ↵ jump to ↵ in this repository all github ↵ jump to ↵ sign in sign up sign up {{ message }} hughrun / yawp notifications star fork command line app for publishing social media posts agpl- . license stars forks star notifications code issues pull requests actions projects security insights more code issues pull requests actions projects security insights main switch branches/tags branches tags nothing to show {{ refname }} default view all branches nothing to show {{ refname }} default view all tags branch tag go to file code clone https github cli use git or checkout with svn using the web url. work fast with our official cli. learn more. open with github desktop download zip launching github desktop if nothing happens, download github desktop and try again. go back launching github desktop if nothing happens, download github desktop and try again. go back launching xcode if nothing happens, download xcode and try again. go back launching visual studio if nothing happens, download the github extension for visual studio and try again. go back latest commit git stats commits files permalink failed to load latest commit information. type name latest commit message commit time src add more detail to readme apr , .gitignore more readme updates apr , cargo.lock initial code commit apr , cargo.toml minor updates to cargo.toml and readme apr , license initial commit apr , readme.md add yawp definition to readme apr , example.env add more detail to readme apr , view code yawp in brief installation macos or linux from source usage: flags: options: args: environment variables mastodon twitter examples readme.md yawp a command line (cli) app for publishing social media posts. in brief yawp takes some text as an argument and publishes it to the social media accounts of your choice. no need to read the comments, just send your yawp and move on with your day. current options are twitter and mastodon, it's possible more will be added in future (or not). yawp is specifically designed to fit within a broader toolchain: in general terms it tries to follow "the unix philosophy": can take input from stdin (e.g. redirected from a file or another process) outputs the message as plaintext to stdout (i.e. the output is the input) takes all configuration from environment (env) values to enable flexibility installation macos or linux download the relevant binary file from the latest release. save it somewhere in your path, e.g. in /usr/local/bin/. alternatively you can symlink it from wherever you want to save it, like this: ln -s /my/awesome/directory/yawp /usr/local/bin/ from source if you're using another platform or don't trust my binaries you can build your own from source: git clone or download the repository as a zip. cargo build --release usage: yawp [flags] [options] flags: -h, --help prints help information -m, --mastodon send toot -q, --quiet suppress output (error messages will still be sent to stderr) -t, --twitter send tweet -v, --version prints version information options: -e, --env path to env file args: message (post) to send. if using stdin you must provide a hyphen (-) as the argument. however if you do this and are not redirecting stdin from somewhere, yawp will hang your shell unless you supply eof by pressing ctrl + d. (see example below). environment variables yawp requires some environment variables in order to actually publish your message. you can set these in a number of ways depending on your operating system. yawp also allows you to call them in from a file. see example for using a file or example for setting environment values at the same time you call yawp. an example environment variables file is provided at example.env. the possible values are: mastodon for mastodon you need the base url of your instance (server), and an api access token. mastodon_access_token - you can create a token at settings - applications in your mastodon account. you require write:statuses permission. mastodon_base_url - this is the base url of your server. e.g. https://mastodon.social twitter for twitter you need the four tokens provided when you create an app at https://developer.twitter.com/en/apps. twitter_consumer_key twitter_consumer_secret twitter_access_token twitter_access_secret examples provide message on command line: yawp 'hello, world!' -t # output: hello, world! # tweets: hello, world! pipe in message: echo 'hello again, world!' | yawp - -m # output: hello again, world! # toots: hello again, world! read from file # create a file (echo hello fronds; echo " it's me"; echo ...a tree 🌳) > message.txt # run yawp and direct file content into it yawp - output.txt # the message.txt and output.txt files are now identical. read from user input this is not really recommended, but you may find yourself facing a user input prompt if you use a hyphen without providing any redirected input. i.e. if you do this: yawp - # machine awaits user further input from command line don't panic, you can provide the message text by typing it in at the command prompt. there is a catch, however, in that yawp will wait for further input until it reaches eof (end of file). this will not happen when you press enter but can usually be provided by pressing ctrl + d: yawp -t - # machine awaits user further input from command line awoo! [ctrl + d] # output: awoo! # tweets: awoo! provide environment variables from file in some situtations (e.g. when using docker compose) you may have already set environment variables specific to those needed by yawp. if not, you can call them in from a file by providing the filepath using -e or --env: yawp -m --env 'yawp.env' 'i love to toot!' provide environment variables on command line you could also set env settings manually when you call yawp: mastodon_base_url=https://ausglam.space mastodon_access_token=abcd yawp -m '🎺 i am tooting!' about command line app for publishing social media posts resources readme license agpl- . license releases . . latest apr , packages no packages published languages rust . % shell . % © github, inc. terms privacy security status docs contact github pricing api training blog about you can’t perform that action at this time. you signed in with another tab or window. reload to refresh your session. you signed out in another tab or window. reload to refresh your session. orcid skip to main content for full functionality of this site it is necessary to enable javascript. here are the instructions for enabling javascript in your web browser. orcid uses cookies to improve your experience and to help us understand how you use our websites. learn more about how we use cookies. home · carpenpi/docs wiki · github skip to content sign up sign up why github? features → mobile → actions → codespaces → packages → security → code review → project management → integrations → github sponsors → customer stories→ team enterprise explore explore github → learn and contribute topics → collections → trending → learning lab → open source guides → connect with others the readme project → events → community forum → github education → github stars program → marketplace pricing plans → compare plans → contact sales → education → in this repository all github ↵ jump to ↵ no suggested jump to results in this repository all github ↵ jump to ↵ in this organization all github ↵ jump to ↵ in this repository all github ↵ jump to ↵ sign in sign up sign up {{ message }} carpenpi / docs notifications star fork code issues pull requests actions projects wiki security insights more code issues pull requests actions projects wiki security insights home jump to bottom flic anderson edited this page apr , · revisions about carpenpi carpenpi aims to facilitate software carpentry and data carpentry lessons to be taught from a cluster of raspberry pis, to allow them to be run in places with unreliable internet connections. carpenpi was born out of the software sustainability institute's collababorations workshop. the idea was formed by a team during the collaborative ideas session, and the implementation began on the hack day. for more info on the project motivation see the collabw -demo-presentation repository. minimum requirements all raspberry pi's need wifi capability which is built into version and pi's but usb dongles can be included for lower versions. at least two pi's are required for infrastructure and then enough pi's for all attendees. code of conduct we follow the code of conduct outlined by the carpentries architecture see pi-network for an overview. repositories raspberry pi images/setup accesspoint: runs an access point on a pi to set up a local network webserver: runs a web server on a pi to host the carpentries training materials without internet access git-server: runs a git server on a pi to allow course participants to collaborate via git without needing external internet access other repositories traintrainers: carpentry course for trainers who want to use a pi cluster collabw -demopresentation: presentation for the collaborations workshop hackday docs: repository for this wiki workshop-admin: repository for a web app to help administer the courses recent decisions project name: carppi carpenpis carpentpis (with the 't'?) carpentries in a case carpintries the fruit of knowledge pandora's box deliverpi rasppitrain off-grid carpentries raspberry carpenpis carpentryjam logo : combining raspberry pi and carpentries logo. colour scheme close to pi or carpentries? why chose? lets not modify the original images carpentry bolt with raspberry on or replace bolt with raspberry? remove th bolt due to clash of colours fun font or formal? we're too fun to be formal. licence : following the carpentries website we are using the mit licence for code and cc-by for materials. future work see project issues for details for future work. the main areas are: making the pi network auto-configurable updating the training materials the workshop admin area contributors in alphabetical order: abhishek dasgupta alison clarke emily lewis flic anderson irma hafidz jannetta steyn rebecca wilson sam haynes talia caplan pages home pi network clone this wiki locally © github, inc. terms privacy security status docs contact github pricing api training blog about you can’t perform that action at this time. you signed in with another tab or window. reload to refresh your session. you signed out in another tab or window. reload to refresh your session. github - knowledgecaptureanddiscovery/somef-github-action skip to content sign up sign up why github? features → mobile → actions → codespaces → packages → security → code review → project management → integrations → github sponsors → customer stories→ team enterprise explore explore github → learn and contribute topics → collections → trending → learning lab → open source guides → connect with others the readme project → events → community forum → github education → github stars program → marketplace pricing plans → compare plans → contact sales → education → in this repository all github ↵ jump to ↵ no suggested jump to results in this repository all github ↵ jump to ↵ in this organization all github ↵ jump to ↵ in this repository all github ↵ jump to ↵ sign in sign up sign up {{ message }} knowledgecaptureanddiscovery / somef-github-action notifications star fork apache- . license stars forks star notifications code issues pull requests actions projects security insights more code issues pull requests actions projects security insights main switch branches/tags branches tags nothing to show {{ refname }} default view all branches nothing to show {{ refname }} default view all tags branches tag go to file code clone https github cli use git or checkout with svn using the web url. work fast with our official cli. learn more. open with github desktop download zip launching github desktop if nothing happens, download github desktop and try again. go back launching github desktop if nothing happens, download github desktop and try again. go back launching xcode if nothing happens, download xcode and try again. go back launching visual studio if nothing happens, download the github extension for visual studio and try again. go back latest commit git stats commits files permalink failed to load latest commit information. type name latest commit message commit time .github/workflows dockerfile license readme.md action.yml entrypoint.sh view code somef github action basic usage advanced workflow readme.md somef github action this action uses somef to generate a .codemeta file and meet the recommendations from howfairis basic usage in its more basic usage, the github action only uses somef to generate a codemeta.json file. on: [push] jobs: somef_job: runs-on: ubuntu-latest name: run somef steps: # checks-out your repository under $github_workspace, so your job can access it - name: chechout repo uses: actions/checkout@v # use somef generate codemeta.json - name: somef with repo-url input uses: knowledgecaptureanddiscovery/somef-github-action@main with: repo-url: "https://github.com/${{ github.repository }}" advanced workflow a more advanced workflow uses howfairis and create pull request actions to create a howfairis badge and send a pull request with the generated codemeta.json file if necessary: on: [push] jobs: somef_job: runs-on: ubuntu-latest name: test somef steps: # checks-out your repository under $github_workspace, so your job can access it - name: chechout repo uses: actions/checkout@v # run howfairis - name: fair-software uses: fair-software/howfairis-github-action@ . . with: my_repo_url: "https://github.com/${{ github.repository }}" # use somef generate codemeta.json - name: somef with repo-url input uses: knowledgecaptureanddiscovery/somef-github-action@main with: repo-url: "https://github.com/${{ github.repository }}" # create a pr - name: create pull request uses: peter-evans/create-pull-request@v . . with: title: generating codemeta template commit-message: add codemeta.json template committer: github author: ${{ github.actor }} <${{ github.actor }}@users.noreply.github.com> labels: automated pr branch: add-codemeta about no description, website, or topics provided. resources readme license apache- . license releases tags packages no packages published contributors languages shell . % dockerfile . % © github, inc. terms privacy security status docs contact github pricing api training blog about you can’t perform that action at this time. you signed in with another tab or window. reload to refresh your session. you signed out in another tab or window. reload to refresh your session. pull requests · frictionlessdata/frictionless-py · github skip to content sign up sign up why github? features → mobile → actions → codespaces → packages → security → code review → project management → integrations → github sponsors → customer stories→ team enterprise explore explore github → learn and contribute topics → collections → trending → learning lab → open source guides → connect with others the readme project → events → community forum → github education → github stars program → marketplace pricing plans → compare plans → contact sales → education → in this repository all github ↵ jump to ↵ no suggested jump to results in this repository all github ↵ jump to ↵ in this organization all github ↵ jump to ↵ in this repository all github ↵ jump to ↵ sign in sign up sign up {{ message }} frictionlessdata / frictionless-py notifications star fork code issues pull requests actions projects security insights more code issues pull requests actions projects security insights labels milestones labels milestones new pull request new open closed open closed author filter by author author: filter by this user label filter by label use alt + click/return to exclude labels. projects filter by project milestones filter by milestone reviews filter by reviews no reviews review required approved review changes requested assignee filter by who’s assigned sort sort by newest oldest most commented least commented recently updated least recently updated most reactions 👍 👎 😄 🎉 😕 ❤️ 🚀 👀 there aren’t any open pull requests. you could search all of github or try an advanced search. protip! mix and match filters to narrow down what you’re looking for. © github, inc. terms privacy security status docs contact github pricing api training blog about you can’t perform that action at this time. you signed in with another tab or window. reload to refresh your session. you signed out in another tab or window. reload to refresh your session. none strategies for learning from failure subscribe sign in clear suggested topics explore hbr diversity latest the magazine most popular podcasts video store webinars newsletters popular topics managing yourself leadership strategy managing teams gender innovation work-life balance all topics for subscribers the big idea visual library reading lists case selections subscribe my account my library topic feeds orders account settings email preferences log out sign in subscribe diversity latest podcasts video the magazine store webinars newsletters all topics the big idea visual library reading lists case selections my library account settings log out sign in your cart your shopping cart is empty. visit our store guest user subscriber my library topic feeds orders account settings email preferences log out reading list reading lists you have free articles left this month. you are reading your last free article for this month. subscribe for unlimited access. create an account to read more. leadership strategies for learning from failure we are programmed at an early age to think that failure is bad. that belief prevents organizations from effectively learning from their missteps. by amy c. edmondson by amy c. edmondson from the magazine (april ) tweet post share save get pdf buy copies print summary. reprint: r b many executives believe that all failure is bad (although it usually provides lessons) and that learning from it is pretty straightforward. the author, a professor at harvard business school, thinks both beliefs are misguided. in organizational life, she says, some failures are inevitable and some are even good. and successful learning from failure is not simple: it requires context-specific strategies. but first leaders must understand how the blame game gets in the way and work to create an organizational culture in which employees feel safe admitting or reporting on failure. failures fall into three categories: preventable ones in predictable operations, which usually involve deviations from spec; unavoidable ones in complex systems, which may arise from unique combinations of needs, people, and problems; and intelligent ones at the frontier, where “good” failures occur quickly and on a small scale, providing the most valuable information. strong leadership can build a learning culture—one in which failures large and small are consistently reported and deeply analyzed, and opportunities to experiment are proactively sought. executives commonly and understandably worry that taking a sympathetic stance toward failure will create an “anything goes” work environment. they should instead recognize that failure is inevitable in today’s complex work organizations. tweet post share save get pdf buy copies print leer en español the wisdom of learning from failure is incontrovertible. yet organizations that do it well are extraordinarily rare. this gap is not due to a lack of commitment to learning. managers in the vast majority of enterprises that i have studied over the past years—pharmaceutical, financial services, product design, telecommunications, and construction companies; hospitals; and nasa’s space shuttle program, among others—genuinely wanted to help their organizations learn from failures to improve future performance. in some cases they and their teams had devoted many hours to after-action reviews, postmortems, and the like. but time after time i saw that these painstaking efforts led to no real change. the reason: those managers were thinking about failure the wrong way. most executives i’ve talked to believe that failure is bad (of course!). they also believe that learning from it is pretty straightforward: ask people to reflect on what they did wrong and exhort them to avoid similar mistakes in the future—or, better yet, assign a team to review and write a report on what happened and then distribute it throughout the organization. these widely held beliefs are misguided. first, failure is not always bad. in organizational life it is sometimes bad, sometimes inevitable, and sometimes even good. second, learning from organizational failures is anything but straightforward. the attitudes and activities required to effectively detect and analyze failures are in short supply in most companies, and the need for context-specific learning strategies is underappreciated. organizations need new and better ways to go beyond lessons that are superficial (“procedures weren’t followed”) or self-serving (“the market just wasn’t ready for our great new product”). that means jettisoning old cultural beliefs and stereotypical notions of success and embracing failure’s lessons. leaders can begin by understanding how the blame game gets in the way. the blame game failure and fault are virtually inseparable in most households, organizations, and cultures. every child learns at some point that admitting failure means taking the blame. that is why so few organizations have shifted to a culture of psychological safety in which the rewards of learning from failure can be fully realized. executives i’ve interviewed in organizations as different as hospitals and investment banks admit to being torn: how can they respond constructively to failures without giving rise to an anything-goes attitude? if people aren’t blamed for failures, what will ensure that they try as hard as possible to do their best work? this concern is based on a false dichotomy. in actuality, a culture that makes it safe to admit and report on failure can—and in some organizational contexts must—coexist with high standards for performance. to understand why, look at the exhibit “a spectrum of reasons for failure,” which lists causes ranging from deliberate deviation to thoughtful experimentation. which of these causes involve blameworthy actions? deliberate deviance, first on the list, obviously warrants blame. but inattention might not. if it results from a lack of effort, perhaps it’s blameworthy. but if it results from fatigue near the end of an overly long shift, the manager who assigned the shift is more at fault than the employee. as we go down the list, it gets more and more difficult to find blameworthy acts. in fact, a failure resulting from thoughtful experimentation that generates valuable information may actually be praiseworthy. when i ask executives to consider this spectrum and then to estimate how many of the failures in their organizations are truly blameworthy, their answers are usually in single digits—perhaps % to %. but when i ask how many are treated as blameworthy, they say (after a pause or a laugh) % to %. the unfortunate consequence is that many failures go unreported and their lessons are lost. not all failures are created equal a sophisticated understanding of failure’s causes and contexts will help to avoid the blame game and institute an effective strategy for learning from failure. although an infinite number of things can go wrong in organizations, mistakes fall into three broad categories: preventable, complexity-related, and intelligent. preventable failures in predictable operations. most failures in this category can indeed be considered “bad.” they usually involve deviations from spec in the closely defined processes of high-volume or routine operations in manufacturing and services. with proper training and support, employees can follow those processes consistently. when they don’t, deviance, inattention, or lack of ability is usually the reason. but in such cases, the causes can be readily identified and solutions developed. checklists (as in the harvard surgeon atul gawande’s recent best seller the checklist manifesto) are one solution. another is the vaunted toyota production system, which builds continual learning from tiny failures (small process deviations) into its approach to improvement. as most students of operations know well, a team member on a toyota assembly line who spots a problem or even a potential problem is encouraged to pull a rope called the andon cord, which immediately initiates a diagnostic and problem-solving process. production continues unimpeded if the problem can be remedied in less than a minute. otherwise, production is halted—despite the loss of revenue entailed—until the failure is understood and resolved. unavoidable failures in complex systems. a large number of organizational failures are due to the inherent uncertainty of work: a particular combination of needs, people, and problems may have never occurred before. triaging patients in a hospital emergency room, responding to enemy actions on the battlefield, and running a fast-growing start-up all occur in unpredictable situations. and in complex organizations like aircraft carriers and nuclear power plants, system failure is a perpetual risk. although serious failures can be averted by following best practices for safety and risk management, including a thorough analysis of any such events that do occur, small process failures are inevitable. to consider them bad is not just a misunderstanding of how complex systems work; it is counterproductive. avoiding consequential failures means rapidly identifying and correcting small failures. most accidents in hospitals result from a series of small failures that went unnoticed and unfortunately lined up in just the wrong way. intelligent failures at the frontier. failures in this category can rightly be considered “good,” because they provide valuable new knowledge that can help an organization leap ahead of the competition and ensure its future growth—which is why the duke university professor of management sim sitkin calls them intelligent failures. they occur when experimentation is necessary: when answers are not knowable in advance because this exact situation hasn’t been encountered before and perhaps never will be again. discovering new drugs, creating a radically new business, designing an innovative product, and testing customer reactions in a brand-new market are tasks that require intelligent failures. “trial and error” is a common term for the kind of experimentation needed in these settings, but it is a misnomer, because “error” implies that there was a “right” outcome in the first place. at the frontier, the right kind of experimentation produces good failures quickly. managers who practice it can avoid the unintelligent failure of conducting experiments at a larger scale than necessary. leaders of the product design firm ideo understood this when they launched a new innovation-strategy service. rather than help clients design new products within their existing lines—a process ideo had all but perfected—the service would help them create new lines that would take them in novel strategic directions. knowing that it hadn’t yet figured out how to deliver the service effectively, the company started a small project with a mattress company and didn’t publicly announce the launch of a new business. although the project failed—the client did not change its product strategy—ideo learned from it and figured out what had to be done differently. for instance, it hired team members with mbas who could better help clients create new businesses and made some of the clients’ managers part of the team. today strategic innovation services account for more than a third of ideo’s revenues. tolerating unavoidable process failures in complex systems and intelligent failures at the frontiers of knowledge won’t promote mediocrity. indeed, tolerance is essential for any organization that wishes to extract the knowledge such failures provide. but failure is still inherently emotionally charged; getting an organization to accept it takes leadership. building a learning culture only leaders can create and reinforce a culture that counteracts the blame game and makes people feel both comfortable with and responsible for surfacing and learning from failures. (see the sidebar “how leaders can build a psychologically safe environment.”) they should insist that their organizations develop a clear understanding of what happened—not of “who did it”—when things go wrong. this requires consistently reporting failures, small and large; systematically analyzing them; and proactively searching for opportunities to experiment. how leaders can build a psychologically safe environment if an organization’s employees are to help spot existing and pending failures and to learn from them, their leaders must make it safe to speak up. julie morath, the chief operating officer of children’s hospital and clinics of minnesota from to , did just that when she led a highly successful effort to reduce medical errors. here are five practices i’ve identified in my research, with examples of how morath employed them to build a psychologically safe environment. frame the work accurately people need a shared understanding of the kinds of failures that can be expected to occur in a given work context (routine production, complex operations, or innovation) and why openness and collaboration are important for surfacing and learning from them. accurate framing detoxifies failure. in a complex operation like a hospital, many consequential failures are the result of a series of small events. to heighten awareness of this system complexity, morath presented data on u.s. medical error rates, organized discussion groups, and built a team of key influencers from throughout the organization to help spread knowledge and understanding of the challenge. embrace messengers those who come forward with bad news, questions, concerns, or mistakes should be rewarded rather than shot. celebrate the value of the news first and then figure out how to fix the failure and learn from it. morath implemented “blameless reporting”—an approach that encouraged employees to reveal medical errors and near misses anonymously. her team created a new patient safety report, which expanded on the previous version by asking employees to describe incidents in their own words and to comment on the possible causes. soon after the new system was implemented, the rate of reported failures shot up. morath encouraged her people to view the data as good news, because the hospital could learn from failures—and made sure that teams were assigned to analyze every incident. acknowledge limits being open about what you don’t know, mistakes you’ve made, and what you can’t get done alone will encourage others to do the same. as soon as she joined the hospital, morath explained her passion for patient safety and acknowledged that as a newcomer, she had only limited knowledge of how things worked at children’s. in group presentations and one-on-one discussions, she made clear that she would need everyone’s help to reduce errors. invite participation ask for observations and ideas and create opportunities for people to detect and analyze failures and promote intelligent experiments. inviting participation helps defuse resistance and defensiveness. morath set up cross-disciplinary teams to analyze failures and personally asked thoughtful questions of employees at all levels. early on, she invited people to reflect on their recent experiences in caring for patients: was everything as safe as they would have wanted it to be? this helped them recognize that the hospital had room for improvement. suddenly, people were lining up to help. set boundaries and hold people accountable paradoxically, people feel psychologically safer when leaders are clear about what acts are blameworthy. and there must be consequences. but if someone is punished or fired, tell those directly and indirectly affected what happened and why it warranted blame. when she instituted blameless reporting, morath explained to employees that although reporting would not be punished, specific behaviors (such as reckless conduct, conscious violation of standards, failing to ask for help when over one’s head) would. if someone makes the same mistake three times and is then laid off, coworkers usually express relief, along with sadness and concern—they understand that patients were at risk and that extra vigilance was required from others to counterbalance the person’s shortcomings. leaders should also send the right message about the nature of the work, such as reminding people in r&d, “we’re in the discovery business, and the faster we fail, the faster we’ll succeed.” i have found that managers often don’t understand or appreciate this subtle but crucial point. they also may approach failure in a way that is inappropriate for the context. for example, statistical process control, which uses data analysis to assess unwarranted variances, is not good for catching and correcting random invisible glitches such as software bugs. nor does it help in the development of creative new products. conversely, though great scientists intuitively adhere to ideo’s slogan, “fail often in order to succeed sooner,” it would hardly promote success in a manufacturing plant. the slogan “fail often in order to succeed sooner” would hardly promote success in a manufacturing plant. often one context or one kind of work dominates the culture of an enterprise and shapes how it treats failure. for instance, automotive companies, with their predictable, high-volume operations, understandably tend to view failure as something that can and should be prevented. but most organizations engage in all three kinds of work discussed above—routine, complex, and frontier. leaders must ensure that the right approach to learning from failure is applied in each. all organizations learn from failure through three essential activities: detection, analysis, and experimentation. detecting failure spotting big, painful, expensive failures is easy. but in many organizations any failure that can be hidden is hidden as long as it’s unlikely to cause immediate or obvious harm. the goal should be to surface it early, before it has mushroomed into disaster. shortly after arriving from boeing to take the reins at ford, in september , alan mulally instituted a new system for detecting failures. he asked managers to color code their reports green for good, yellow for caution, or red for problems—a common management technique. according to a story in fortune, at his first few meetings all the managers coded their operations green, to mulally’s frustration. reminding them that the company had lost several billion dollars the previous year, he asked straight out, “isn’t anything not going well?” after one tentative yellow report was made about a serious product defect that would probably delay a launch, mulally responded to the deathly silence that ensued with applause. after that, the weekly staff meetings were full of color. that story illustrates a pervasive and fundamental problem: although many methods of surfacing current and pending failures exist, they are grossly underutilized. total quality management and soliciting feedback from customers are well-known techniques for bringing to light failures in routine operations. high-reliability-organization (hro) practices help prevent catastrophic failures in complex systems like nuclear power plants through early detection. electricité de france, which operates nuclear power plants, has been an exemplar in this area: it goes beyond regulatory requirements and religiously tracks each plant for anything even slightly out of the ordinary, immediately investigates whatever turns up, and informs all its other plants of any anomalies. such methods are not more widely employed because all too many messengers—even the most senior executives—remain reluctant to convey bad news to bosses and colleagues. one senior executive i know in a large consumer products company had grave reservations about a takeover that was already in the works when he joined the management team. but, overly conscious of his newcomer status, he was silent during discussions in which all the other executives seemed enthusiastic about the plan. many months later, when the takeover had clearly failed, the team gathered to review what had happened. aided by a consultant, each executive considered what he or she might have done to contribute to the failure. the newcomer, openly apologetic about his past silence, explained that others’ enthusiasm had made him unwilling to be “the skunk at the picnic.” in researching errors and other failures in hospitals, i discovered substantial differences across patient-care units in nurses’ willingness to speak up about them. it turned out that the behavior of midlevel managers—how they responded to failures and whether they encouraged open discussion of them, welcomed questions, and displayed humility and curiosity—was the cause. i have seen the same pattern in a wide range of organizations. a horrific case in point, which i studied for more than two years, is the explosion of the columbia space shuttle, which killed seven astronauts (see “facing ambiguous threats,” by michael a. roberto, richard m.j. bohmer, and amy c. edmondson, hbr november ). nasa managers spent some two weeks downplaying the seriousness of a piece of foam’s having broken off the left side of the shuttle at launch. they rejected engineers’ requests to resolve the ambiguity (which could have been done by having a satellite photograph the shuttle or asking the astronauts to conduct a space walk to inspect the area in question), and the major failure went largely undetected until its fatal consequences days later. ironically, a shared but unsubstantiated belief among program managers that there was little they could do contributed to their inability to detect the failure. postevent analyses suggested that they might indeed have taken fruitful action. but clearly leaders hadn’t established the necessary culture, systems, and procedures. one challenge is teaching people in an organization when to declare defeat in an experimental course of action. the human tendency to hope for the best and try to avoid failure at all costs gets in the way, and organizational hierarchies exacerbate it. as a result, failing r&d projects are often kept going much longer than is scientifically rational or economically prudent. we throw good money after bad, praying that we’ll pull a rabbit out of a hat. intuition may tell engineers or scientists that a project has fatal flaws, but the formal decision to call it a failure may be delayed for months. again, the remedy—which does not necessarily involve much time and expense—is to reduce the stigma of failure. eli lilly has done this since the early s by holding “failure parties” to honor intelligent, high-quality scientific experiments that fail to achieve the desired results. the parties don’t cost much, and redeploying valuable resources—particularly scientists—to new projects earlier rather than later can save hundreds of thousands of dollars, not to mention kickstart potential new discoveries. analyzing failure once a failure has been detected, it’s essential to go beyond the obvious and superficial reasons for it to understand the root causes. this requires the discipline—better yet, the enthusiasm—to use sophisticated analysis to ensure that the right lessons are learned and the right remedies are employed. the job of leaders is to see that their organizations don’t just move on after a failure but stop to dig in and discover the wisdom contained in it. why is failure analysis often shortchanged? because examining our failures in depth is emotionally unpleasant and can chip away at our self-esteem. left to our own devices, most of us will speed through or avoid failure analysis altogether. another reason is that analyzing organizational failures requires inquiry and openness, patience, and a tolerance for causal ambiguity. yet managers typically admire and are rewarded for decisiveness, efficiency, and action—not thoughtful reflection. that is why the right culture is so important. the challenge is more than emotional; it’s cognitive, too. even without meaning to, we all favor evidence that supports our existing beliefs rather than alternative explanations. we also tend to downplay our responsibility and place undue blame on external or situational factors when we fail, only to do the reverse when assessing the failures of others—a psychological trap known as fundamental attribution error. my research has shown that failure analysis is often limited and ineffective—even in complex organizations like hospitals, where human lives are at stake. few hospitals systematically analyze medical errors or process flaws in order to capture failure’s lessons. recent research in north carolina hospitals, published in november in the new england journal of medicine, found that despite a dozen years of heightened awareness that medical errors result in thousands of deaths each year, hospitals have not become safer. fortunately, there are shining exceptions to this pattern, which continue to provide hope that organizational learning is possible. at intermountain healthcare, a system of hospitals that serves utah and southeastern idaho, physicians’ deviations from medical protocols are routinely analyzed for opportunities to improve the protocols. allowing deviations and sharing the data on whether they actually produce a better outcome encourages physicians to buy into this program. (see “fixing health care on the front lines,” by richard m.j. bohmer, hbr april .) motivating people to go beyond first-order reasons (procedures weren’t followed) to understanding the second- and third-order reasons can be a major challenge. one way to do this is to use interdisciplinary teams with diverse skills and perspectives. complex failures in particular are the result of multiple events that occurred in different departments or disciplines or at different levels of the organization. understanding what happened and how to prevent it from happening again requires detailed, team-based discussion and analysis. a team of leading physicists, engineers, aviation experts, naval leaders, and even astronauts devoted months to an analysis of the columbia disaster. they conclusively established not only the first-order cause—a piece of foam had hit the shuttle’s leading edge during launch—but also second-order causes: a rigid hierarchy and schedule-obsessed culture at nasa made it especially difficult for engineers to speak up about anything but the most rock-solid concerns. promoting experimentation the third critical activity for effective learning is strategically producing failures—in the right places, at the right times—through systematic experimentation. researchers in basic science know that although the experiments they conduct will occasionally result in a spectacular success, a large percentage of them ( % or higher in some fields) will fail. how do these people get out of bed in the morning? first, they know that failure is not optional in their work; it’s part of being at the leading edge of scientific discovery. second, far more than most of us, they understand that every failure conveys valuable information, and they’re eager to get it before the competition does. in contrast, managers in charge of piloting a new product or service—a classic example of experimentation in business—typically do whatever they can to make sure that the pilot is perfect right out of the starting gate. ironically, this hunger to succeed can later inhibit the success of the official launch. too often, managers in charge of pilots design optimal conditions rather than representative ones. thus the pilot doesn’t produce knowledge about what won’t work. too often, pilots are conducted under optimal conditions rather than representative ones. thus they can’t show what won’t work. in the very early days of dsl, a major telecommunications company i’ll call telco did a full-scale launch of that high-speed technology to consumer households in a major urban market. it was an unmitigated customer-service disaster. the company missed % of its commitments and found itself confronted with a staggering , late orders. customers were frustrated and upset, and service reps couldn’t even begin to answer all their calls. employee morale suffered. how could this happen to a leading company with high satisfaction ratings and a brand that had long stood for excellence? a small and extremely successful suburban pilot had lulled telco executives into a misguided confidence. the problem was that the pilot did not resemble real service conditions: it was staffed with unusually personable, expert service reps and took place in a community of educated, tech-savvy customers. but dsl was a brand-new technology and, unlike traditional telephony, had to interface with customers’ highly variable home computers and technical skills. this added complexity and unpredictability to the service-delivery challenge in ways that telco had not fully appreciated before the launch. a more useful pilot at telco would have tested the technology with limited support, unsophisticated customers, and old computers. it would have been designed to discover everything that could go wrong—instead of proving that under the best of conditions everything would go right. (see the sidebar “designing successful failures.”) of course, the managers in charge would have to have understood that they were going to be rewarded not for success but, rather, for producing intelligent failures as quickly as possible. designing successful failures perhaps unsurprisingly, pilot projects are usually designed to succeed rather than to produce intelligent failures—those that generate valuable information. to know if you’ve designed a genuinely useful pilot, consider whether your managers can answer yes to the following questions: is the pilot being tested under typical circumstances (rather than optimal conditions)? do the employees, customers, and resources represent the firm’s real operating environment? is the goal of the pilot to learn as much as possible (rather than to demonstrate the value of the proposed offering)? is the goal of learning well understood by all employees and managers? is it clear that compensation and performance reviews are not based on a successful outcome for the pilot? were explicit changes made as a result of the pilot test? in short, exceptional organizations are those that go beyond detecting and analyzing failures and try to generate intelligent ones for the express purpose of learning and innovating. it’s not that managers in these organizations enjoy failure. but they recognize it as a necessary by-product of experimentation. they also realize that they don’t have to do dramatic experiments with large budgets. often a small pilot, a dry run of a new technique, or a simulation will suffice. the courage to confront our own and others’ imperfections is crucial to solving the apparent contradiction of wanting neither to discourage the reporting of problems nor to create an environment in which anything goes. this means that managers must ask employees to be brave and speak up—and must not respond by expressing anger or strong disapproval of what may at first appear to be incompetence. more often than we realize, complex systems are at work behind organizational failures, and their lessons and improvement opportunities are lost when conversation is stifled. savvy managers understand the risks of unbridled toughness. they know that their ability to find out about and help resolve problems depends on their ability to learn about them. but most managers i’ve encountered in my research, teaching, and consulting work are far more sensitive to a different risk—that an understanding response to failures will simply create a lax work environment in which mistakes multiply. this common worry should be replaced by a new paradigm—one that recognizes the inevitability of failure in today’s complex work organizations. those that catch, correct, and learn from failure before others do will succeed. those that wallow in the blame game will not. a version of this article appeared in the april issue of harvard business review. read more on leadership or related topics organizational culture, knowledge management, business processes and experimentation amy c. edmondson is the novartis professor of leadership and management at harvard business school. she is the author of the fearless organization: creating psychological safety in the workplace for learning, innovation, and growth (wiley, ). tweet post share save get pdf buy copies print read more on leadership or related topics organizational culture, knowledge management, business processes and experimentation partner center diversity latest magazine popular topics podcasts video store the big idea visual library case selections subscribe explore hbr the latest most popular all topics magazine archive the big idea reading lists case selections video podcasts webinars visual library my library newsletters hbr press hbr ascend hbr store article reprints books cases collections magazine issues hbr guide series hbr -minute managers hbr emotional intelligence series hbr must reads tools about hbr contact us advertise with us information for booksellers/retailers masthead global editions media inquiries guidelines for authors hbr analytic services copyright permissions manage my account my library topic feeds orders account settings email preferences account faq help center contact customer service follow hbr facebook twitter linkedin instagram your newsreader about us careers privacy policy cookie policy copyright information trademark policy harvard business publishing: higher education corporate learning harvard business review harvard business school copyright © harvard business school publishing. all rights reserved. harvard business publishing is an affiliate of harvard business school. home - research data management - library guides at university of california, santa cruz skip to main content university library hours my account contact us giving search form search menuuniversity library find & borrow research materials search: books, articles & more start your search for research materials course reserves set up reserves or find course materials borrowing policies databases a - z continue your research with more databases interlibrary loan: borrow from other libraries borrow items from libraries worldwide search libraries worldwide (melvyl) search beyond the ucsc library request a purchase help & tutorials recommended resources find the best databases for your classes get research help contact the library with your questions cite your sources get help with citation basics sign in from off-campus access books, articles, and other online materials from off-campus start your research learn how to use library resources collections & scholarly communication our collections digital collections, video games, maps, and more media collection & desk borrow films, music, and digital equipment special collections find and use our unique collections and archives borrow tech & equipment laptops, cameras, mics, and more faculty & teaching support faculty & graduate services learn about how we support your work open access learn about oa policies and publishing online journals locate a journal by its title teaching support consult with us on your next assignment digital scholarship upgrade your digital skills about the library about the library news & events stay up-to-date on library events library computers find and use computer stations at both libraries mchenry library reserve a study room student study center science & engineering library campus maps & directions find our libraries on campus research data management home create your plan (dmptool) preserve & publish (dryad) find data for reuse best practices tools video tutorials researcher to researcher we can help you create a data management plan easily create a data management plan for your next grant proposal using the dmptool preserve & publish your data publish your data in dryad for preservation and discovery manage your paper or data set with a unique persistent identifier. request a doi (digital object identifier) manage your data check out these best practices for file naming, file organization, file formats, archival data storage, metadata creation and data sharing options find data for reuse locate an appropriate data repository research data management lifecycle our goal to assist ucsc faculty, staff and students with strategies and tools for organizing, managing and preserving research data throughout the research data life cycle. request a data consultation scholarly communication & eresearch team email: research@library.ucsc.edu ucsc campus services ucsc its provides a range of research support services, including data backup. next: create your plan (dmptool) >> high streetsanta cruz, ca feedback creative commons attribution . license except where otherwise noted. patrons with disabilities privacy policy staff portal libapps login incident form (staff only) print page edit this page tags: how do i...?, special topics github - code lib/planetcode lib: configuration for https://planet.code lib.org/ skip to content sign up sign up why github? features → mobile → actions → codespaces → packages → security → code review → project management → integrations → github sponsors → customer stories→ team enterprise explore explore github → learn and contribute topics → collections → trending → learning lab → open source guides → connect with others the readme project → events → community forum → github education → github stars program → marketplace pricing plans → compare plans → contact sales → education → in this repository all github ↵ jump to ↵ no suggested jump to results in this repository all github ↵ jump to ↵ in this organization all github ↵ jump to ↵ in this repository all github ↵ jump to ↵ sign in sign up sign up {{ message }} code lib / planetcode lib notifications star fork configuration for https://planet.code lib.org/ stars forks star notifications code issues pull requests actions projects security insights more code issues pull requests actions projects security insights master switch branches/tags branches tags nothing to show {{ refname }} default view all branches nothing to show {{ refname }} default view all tags branch tags go to file code clone https github cli use git or checkout with svn using the web url. work fast with our official cli. learn more. open with github desktop download zip launching github desktop if nothing happens, download github desktop and try again. go back launching github desktop if nothing happens, download github desktop and try again. go back launching xcode if nothing happens, download xcode and try again. go back launching visual studio if nothing happens, download the github extension for visual studio and try again. go back latest commit git stats commits files permalink failed to load latest commit information. type name latest commit message commit time themes venus @ de .gitignore .gitmodules readme.md config.ini test.ini view code planet.code lib.org installation generally installation on the code lib.org server adding (or removing) a feed readme.md planet.code lib.org planet code lib aggregates feeds and blogs of interest to the code lib community. it uses planet venus. installation generally > git clone git@github.com:code lib/planetcode lib.git > cd planetcode lib > git submodule init > git submodule update > ./venus/planet.py --verbose the generated files will be in output/. to test it with one feed, run > ./venus/planet.py --verbose test.ini installation on the code lib.org server downloading and cloning is done over https so it's as generic as possible. no updates are to be made on the server; they should be made locally, pushed to github, then pulled down. > # become the c l user > cd /var/www/code lib.org/planet_new > git clone https://github.com/code lib/planetcode lib.git > cd planetcode lib > git submodule init > git submdule update > ./venus/planet.py --verbose --expunge to update: > # become the c l user > cd /var/www/code lib.org/planet_new/planetcode lib > git pull the relevant line in c l's crontab is: , * * * * cd /var/www/code lib.org/planet_new/planetcode lib; ./venus/planet.py --expunge >& adding (or removing) a feed additions are welcome! email william denton or submit a pull request modifying config.ini. if you're on the list but don't want to be, please do the same, and you'll be removed, no questions asked. about configuration for https://planet.code lib.org/ resources readme releases no releases published packages no packages published contributors + contributors languages xslt . % css . % html . % © github, inc. terms privacy security status docs contact github pricing api training blog about you can’t perform that action at this time. you signed in with another tab or window. reload to refresh your session. you signed out in another tab or window. reload to refresh your session. evergreen downloads – evergreen ils skip to content evergreen – open source library software evergreen – open source library software about us overview annual reports f.a.q. evergreen event code of conduct software freedom conservancy project governance trademark policy documentation official documentation documentation interest group evergreen roadmap evergreen wiki tabular release notes get involved! get involved! committees & interest groups communications mailing lists irc calendar blog jobs proposed development projects merchandise t-shirts and more conference all conferences evergreen international online conference evergreen international online conference event photography policy code of conduct downloads evergreen downloads opensrf downloads home » evergreen downloads evergreen downloads evergreen downloads evergreen depends on the following technologies perl, c, javascript, xml, xpath, xslt, xmpp, opensrf, apache, mod_perl, and postgresql. the latest stable release of a supported linux distribution is recommended for an evergreen installation. for ubuntu, please use the . -bit lts (long term support) server release. currently the latest release from the evergreen . series is recommended for new installations and stable releases are suggested for production systems. note: evergreen servers and staff clients must match. for example, if you are running server version . . , you should use version . . of the staff client. evergreen . . + no longer supports a separate client by default, but building a client remains as an unsupported option. server & staff client downloads . series . series . series status stable stable stable latest release . . . . . . release date - - - - - - release notes release notes release notes release notes tabular release notes summary changelog changelog changelog changelog evergreen installation install instructions install instructions install instructions upgrading notes on upgrading from . . tbd tbd opensrf software . . (md ) . . (md ) . . (md ) server software source (md ) source (md ) source (md ) web staff client extension (“hatch”) windows hatch installer . . (md ) – installation instructions (windows & linux) git repository git location git location git location other evergreen staff clients staff client archive windows staff clients for slightly older stable releases ( . , . ). for mac and linux installing the evergreen client on macs evergreen . . mac staff client [.dmg] evergreen . . mac staff client [.dmg] evergreen . . mac staff client [.zip] evergreen . . mac staff client [.zip] pre-built mac staff client for evergreen . and . – provided by sitka evergreen in action visit the evergreen catalog on our demonstration and development servers, or visit this list of live evergreen libraries. you can also download an evergreen staff client and point it at the evergreen demo or development server (see the community servers page for details). bug reports please report any evergreen bugs/wishlist on launchpad. to submit a vulnerability please email your report to open-ils-security@esilibrary.com. evergreen code museum older versions of evergreen software are available from the evergreen code museum. source code repository a gitweb instance sits atop the git repositories for evergreen and opensrf. you can find both repositories at git.evergreen-ils.org. here is the running change log for the evergreen code repository: watch us work. trac sends code commits to two public evergreen mailing lists: for evergreen commits, subscribe to open-ils-commits for opensrf commits, subscribe to opensrf-commits about evergreen this is the project site for evergreen, a highly-scalable software for libraries that helps library patrons find library materials, and helps libraries manage, catalog, and circulate those materials, no matter how large or complex the libraries. © - gpls and others. evergreen is open source software, freely licensed under gnu gplv or later. the evergreen project is a (c) nonprofit organization. community links evergreen bug tracker evergreen on open hub evergreen wiki git repositories join irc! irc logs official documentation · © evergreen ils · powered by · designed with the customizr theme · none commonplace.net – data. the final frontier. skip to content commonplace.net data. the final frontier. publications a common place all posts about contact infrastructure for heritage institutions – ark pid’s november , november , lukas kosterdata, infrastructure, library in the digital infrastructure program at the library of the university of amsterdam we have reached a first milestone. in my previous post in the infrastructure for heritage institutions series, “change of course“, i mentioned the coming implementation of ark persistent identifiers for our collection objects. since november , , ark pid’s are available for our university library alma catalogue through the primo user interface. implementation of ark pid’s for the other collection description systems […] read more infrastructure for heritage institutions – change of course june , lukas kosterdata, infrastructure, library in july i published the first post about our planning to realise a “coherent and future proof digital infrastructure” for the library of the university of amsterdam. in february i reported on the first results. as frequently happens, since then the conditions have changed, and naturally we had to adapt the direction we are following to achieve our goals. in other words: a change of course, of course. projects i will leave aside the […] read more infrastructure for heritage institutions – first results february , february , lukas kosterdata, infrastructure, library in july i published the post infrastructure for heritage institutions in which i described our planning to realise a “coherent and future proof digital infrastructure” for the library of the university of amsterdam. time to look back: how far have we come? and time to look forward: what’s in store for the near future? ongoing activities i mentioned three “currently ongoing activities”: monitoring and advising on infrastructural aspects of new projects maintaining a structured dynamic overview […] read more infrastructure for heritage institutions july , january , lukas kosterdata, infrastructure, library during my vacation i saw this tweet by liber about topics to address, as suggested by the participants of the liber conference in dublin: it shows a word cloud (yes, a word cloud) containing a large number of terms. i list the ones i can read without zooming in (so the most suggested ones, i guess), more or less grouped thematically: open scienceopen dataopen accesslicensingcopyrightslinked open dataopen educationcitizen science scholarly communicationdigital humanities/dhdigital scholarshipresearch assessmentresearch […] read more ten years linked open data june , february , lukas kosterdata, library this post is the english translation of my original article in dutch, published in meta ( - ), the flemish journal for information professionals. ten years after the term “linked data” was introduced by tim berners-lee it appears to be time to take stock of the impact of linked data for libraries and other heritage institutions in the past and in the future. i will do this from a personal historical perspective, as a library technology professional, […] read more maps, dictionaries and guidebooks august , february , lukas kosterdata interoperability in heterogeneous library data landscapes libraries have to deal with a highly opaque landscape of heterogeneous data sources, data types, data formats, data flows, data transformations and data redundancies, which i have earlier characterized as a “data maze”. the level and magnitude of this opacity and heterogeneity varies with the amount of content types and the number of services that the library is responsible for. academic and national libraries are possibly dealing with more […] read more standard deviations in data modeling, mapping and manipulation june , february , lukas kosterdata or: anything goes. what are we thinking? an impression of elag this year’s elag conference in stockholm was one of many questions. not only the usual questions following each presentation (always elicited in the form of yet another question: “any questions?”). but also philosophical ones (why? what?). and practical ones (what time? where? how? how much?). and there were some answers too, fortunately. this is my rather personal impression of the event. for a […] read more analysing library data flows for efficient innovation november , february , lukas kosterlibrary in my work at the library of the university of amsterdam i am currently taking a step forward by actually taking a step back from a number of forefront activities in discovery, linked open data and integrated research information towards a more hidden, but also more fundamental enterprise in the area of data infrastructure and information architecture. all for a good cause, for in the end a good data infrastructure is essential for delivering high […] read more looking for data tricks in libraryland september , january , lukas kosterlibrary ifla annual world library and information congress lyon – libraries, citizens, societies: confluence for knowledge after attending the ifla library linked data satellite meeting in paris i travelled to lyon for the first three days (august - ) of the ifla annual world library and information congress. this year’s theme “libraries, citizens, societies: confluence for knowledge” was named after the confluence or convergence of the rivers rhône and saône where the city of […] read more library linked data happening august , january , lukas kosterlibrary on august the ifla satellite meeting ‘linked data in libraries: let’s make it happen!’ took place at the national library of france in paris. rurik greenall (who also wrote a very readable conference report) and i had the opportunity to present our paper ‘an unbroken chain: approaches to implementing linked open data in libraries; comparing local, open-source, collaborative and commercial systems’. in this paper we do not go into reasons for libraries to […] read more posts navigation older posts profiles and social @lukask on twitter @lukask on mastodon my orcid my impactstory my zotero my uva profile recent posts infrastructure for heritage institutions – ark pid’s infrastructure for heritage institutions – change of course infrastructure for heritage institutions – first results infrastructure for heritage institutions ten years linked open data maps, dictionaries and guidebooks most popular posts is an e-book a book? ( , views) who needs marc? ( , views) linked data for libraries ( , views) mobile app or mobile web? ( , views) user experience in public and academic libraries ( , views) mainframe to mobile ( , views) (discover and deliver) or else ( , views) recent comments maarten brinkerink on infrastructure for heritage institutions gittaca on infrastructure for heritage institutions libraries & the future of scholarly communication at #btpdf – uc portal on beyond the library tatiana bryant (@bibliotecariat) on analysing library data flows for efficient innovation @bibliotecariat on analysing library data flows for efficient innovation @lizwoolcott on analysing library data flows for efficient innovation tags apps authority files catalog collection conferences cultural heritage data data management developer platforms discovery tools elag exlibris foaf frbr hardware identifiers igelu infrastructure innovation integration interoperability libraries library library . library systems linked data linked open data marc meetings metadata mobile next generation open data open source open stack open systems people persistent identifiers rda rdf semantic web social networking technology uri web . this work is licensed under a creative commons attribution-sharealike . international license. system log in entries feed comments feed wordpress.org top posts & pages explicit and implicit metadata analysing library data flows for efficient innovation privacy & cookies: this site uses cookies. by continuing to use this website, you agree to their use. to find out more, including how to control cookies, see here: cookie policy commonplace.net a common place about all posts contact publications powered by wordpress | theme: astrid by athemes. home - dlf forum skip to content home about code of conduct coc reporting form thank you resources news affiliated events learn@dlf ndsa’s #digipres sponsors sponsorship opportunities registration cfp search for... search for... toggle navigation toggle navigation home about code of conduct coc reporting form thank you resources news affiliated events learn@dlf ndsa’s #digipres sponsors sponsorship opportunities registration cfp a world-class marketplace of ideas for digital glam practitioners since what's the dlf forum? dlf programs stretch year-round, but we are perhaps best known for our signature event, the annual dlf forum. the dlf forum welcomes digital library, archives, and museum practitioners from member institutions and beyond—for whom it serves as a meeting place, marketplace, and congress. learn about the event and plan to attend attend our affiliated events! ndsa's digital preservation november digital preservation is the annual conference of the national digital stewardship alliance. digipres is expected to be a crucial venue for intellectual exchange, community-building, development of best practices, and national-level agenda-setting in the field. learn@dlf november - now in its fourth year, learn@dlf returns in includes engaging, hands-on sessions where attendees will gain experience with new tools and resources, exchange ideas, and develop and share expertise with fellow community members as well as short tutorials about specific tools, techniques, workflows, or concepts. make an impact! sponsor the dlf forum and ndsa's #digipres dlf forum starts in days hours minutes seconds now what makes the dlf forum great? after nearly years of academic library experience and subsequently participating in no less than conferences, i can say that the dlf forum was the most progressive and enlightening conference that i have ever attended. it was downright empowering. ana ndumu dlf forum fellow the thoughtful way the experience was designed was due to the efforts of the organizers...as a first-time participant, i am grateful to have been able to participate in this year’s virtual forum and look forward to continuing to learn from the dlf community! betsy yoon dlf forum community journalist forum updates dlf forum, digipres, and learn@dlf calls for proposals april , we’re delighted to share that it’s cfp season for clir’s annual events. based on community feedback, we’ve made the decision… read more want forum news? subscribe to our newsletter to stay informed! subscribe sponsorship opportunities about dlf join dlf contact menu sponsorship opportunities about dlf join dlf contact envelope facebook twitter youtube instagram linkedin skip to content open toolbar accessibility tools increase text decrease text grayscale high contrast negative contrast light background links underline readable font reset sitemap home - dlf forum skip to content home about code of conduct coc reporting form thank you resources news affiliated events learn@dlf ndsa’s #digipres sponsors sponsorship opportunities registration cfp search for... search for... toggle navigation toggle navigation home about code of conduct coc reporting form thank you resources news affiliated events learn@dlf ndsa’s #digipres sponsors sponsorship opportunities registration cfp a world-class marketplace of ideas for digital glam practitioners since what's the dlf forum? dlf programs stretch year-round, but we are perhaps best known for our signature event, the annual dlf forum. the dlf forum welcomes digital library, archives, and museum practitioners from member institutions and beyond—for whom it serves as a meeting place, marketplace, and congress. learn about the event and plan to attend attend our affiliated events! ndsa's digital preservation november digital preservation is the annual conference of the national digital stewardship alliance. digipres is expected to be a crucial venue for intellectual exchange, community-building, development of best practices, and national-level agenda-setting in the field. learn@dlf november - now in its fourth year, learn@dlf returns in includes engaging, hands-on sessions where attendees will gain experience with new tools and resources, exchange ideas, and develop and share expertise with fellow community members as well as short tutorials about specific tools, techniques, workflows, or concepts. make an impact! sponsor the dlf forum and ndsa's #digipres dlf forum starts in days hours minutes seconds now what makes the dlf forum great? after nearly years of academic library experience and subsequently participating in no less than conferences, i can say that the dlf forum was the most progressive and enlightening conference that i have ever attended. it was downright empowering. ana ndumu dlf forum fellow the thoughtful way the experience was designed was due to the efforts of the organizers...as a first-time participant, i am grateful to have been able to participate in this year’s virtual forum and look forward to continuing to learn from the dlf community! betsy yoon dlf forum community journalist forum updates dlf forum, digipres, and learn@dlf calls for proposals april , we’re delighted to share that it’s cfp season for clir’s annual events. based on community feedback, we’ve made the decision… read more want forum news? subscribe to our newsletter to stay informed! subscribe sponsorship opportunities about dlf join dlf contact menu sponsorship opportunities about dlf join dlf contact envelope facebook twitter youtube instagram linkedin skip to content open toolbar accessibility tools increase text decrease text grayscale high contrast negative contrast light background links underline readable font reset sitemap none marcedit_xslt_files/homosaurus_xml.xsl at master · reeset/marcedit_xslt_files · github skip to content sign up sign up why github? features → mobile → actions → codespaces → packages → security → code review → project management → integrations → github sponsors → customer stories→ team enterprise explore explore github → learn and contribute topics → collections → trending → learning lab → open source guides → connect with others the readme project → events → community forum → github education → github stars program → marketplace pricing plans → compare plans → contact sales → education → in this repository all github ↵ jump to ↵ no suggested jump to results in this repository all github ↵ jump to ↵ in this user all github ↵ jump to ↵ in this repository all github ↵ jump to ↵ sign in sign up sign up {{ message }} reeset / marcedit_xslt_files notifications star fork code issues pull requests actions projects security insights more code issues pull requests actions projects security insights permalink master switch branches/tags branches tags nothing to show {{ refname }} default view all branches nothing to show {{ refname }} default view all tags marcedit_xslt_files/homosaurus_xml.xsl go to file go to file t go to line l copy path copy permalink cannot retrieve contributors at this time lines ( sloc) . kb raw blame open with desktop view raw view blame nz a n

|||anznnbab||||||||||||||a|||||||d

homosaurus

copy lines copy permalink view git blame reference in new issue go © github, inc. terms privacy security status docs contact github pricing api training blog about you can’t perform that action at this time. you signed in with another tab or window. reload to refresh your session. you signed out in another tab or window. reload to refresh your session. none ipfs powers the distributed web ipfs about install docs team blog help ipfs powers the distributed web a peer-to-peer hypermedia protocol designed to make the web faster, safer, and more open. get started how it works view more disable animation the web of tomorrow needs ipfs today ipfs aims to surpass http in order to build a better web for all of us. today's web is inefficient and expensive http downloads files from one computer at a time instead of getting pieces from multiple computers simultaneously. peer-to-peer ipfs saves big on bandwidth — up to % for video — making it possible to efficiently distribute high volumes of data without duplication. today's web can't preserve humanity's history the average lifespan of a web page is days before it's gone forever. it's not good enough for the primary medium of our era to be this fragile. ipfs keeps every version of your files and makes it simple to set up resilient networks for mirroring data. today's web is centralized, limiting opportunity the internet has turbocharged innovation by being one of the great equalizers in human history — but increasing consolidation of control threatens that progress. ipfs stays true to the original vision of an open, flat web by delivering technology to make that vision a reality. today's web is addicted to the backbone ipfs powers the creation of diversely resilient networks that enable persistent availability — with or without internet backbone connectivity. this means better connectivity for the developing world, during natural disasters, or just when you're on flaky coffee shop wi-fi. install ipfs join the future of the web right now — just choose the option that's right for you. store and share files ipfs desktop ipfs for everyone the desktop app offers menubar/tray shortcuts and an easy interface for adding, pinning, and sharing files — plus a full ipfs node ready for heavy-duty hosting and development too. a great choice for devs and non-devs alike. get ipfs desktop command-line install all ipfs, no frills just want ipfs in your terminal? get step-by-step instructions for getting up and running on the command line using the go implementation of ipfs. includes directions for windows, macos, and linux. get the cli ipfs companion add ipfs to your browser get ipfs:// url support and much more in your web browser with this extension. get companion ipfs cluster for servers or big data automatically allocate, replicate, and track your data as pinsets across multiple ipfs nodes. get cluster build with ipfs go implementation the original ipfs, with core implementation, daemon server, cli tooling, and more. get go-ipfs js implementation written entirely in javascript for a world of possibilities in browser implementations. get js-ipfs here's how ipfs works take a look at what happens when you add a file to ipfs. your file, and all of the blocks within it, is given a unique fingerprint called a cryptographic hash. ipfs removes duplications across the network. each network node stores only content it is interested in, plus some indexing information that helps figure out which node is storing what. when you look up a file to view or download, you're asking the network to find the nodes that are storing the content behind that file's hash. you don't need to remember the hash, though — every file can be found by human-readable names using a decentralized naming system called ipns. take a closer look want to dig in? check out the docs hands-on learner? explore protoschool curious where it all began? read the whitepaper ipfs can help here and now no matter what you do with the web, ipfs helps make it better today. archivists ipfs provides deduplication, high performance, and clustered persistence — empowering you to store the world's information for future generations. service providers providing large amounts of data to users? ipfs offers secure, peer-to-peer content delivery — an approach that could save you millions in bandwidth costs. researchers if you're working with or distributing large data sets, ipfs can help provide fast performance and decentralized archiving. developing world high-latency networks are a big barrier for those with poor internet infrastructure. ipfs provides resilient access to data independent of latency or backbone connectivity. blockchains with ipfs, you can address large amounts of data and put immutable, permanent links in transactions — timestamping and securing content without having to put the data itself on-chain. content creators ipfs brings the freedom and independent spirit of the web in full force — and can help you deliver your content at a much lower cost. who's already using ipfs? companies and organizations worldwide are already building amazing things on ipfs. see the list news and more ipfs blog april welcome to ipfs weekly april meet the new ipfs blog & news april storing nfts on ipfs march welcome to ipfs weekly in the media techcrunch why the internet needs ipfs before it’s too late motherboard ipfs wants to create a permanent web makeuseof faster, safer, decentralized internet with ipfs videos why ipfs? developers speak: building on ipfs more videos stay on top of the latest sign up for the ipfs weekly newsletter to get project updates, community news, event details, and more. in your inbox, each tuesday. subscribe protocol labs about join ipfs install github code of conduct docs community help awesome ipfs ipfs cluster team press blog legal protoschool tutorials events filecoin about faq other projects libp p ipld drand multiformats testground twitter facebook youtube © protocol labs | except as noted, content licensed cc-by . . isni | : home page toggle navigation about what is isni? governance our history objectives & policies isni community the isni community isni registration agencies isni members direct data contributors joining isni resources how isni works data quality procedures data inputs & outputs technical documentation training linked data news news & archive isni newsletter help faqs get an isni contact isni search site mailing list get an isni search database search website: about isni isni is the iso certified global standard number for identifying the millions of contributors to creative works and those active in their distribution, including researchers, inventors, writers, artists, visual creators, performers, producers, publishers, aggregators, and more. as iso , it is part of a family of international standard identifiers that includes identifiers of works, recordings, products and right holders in all repertoires, e.g. doi, isan, isbn, isrc, issn, and iswc. the mission of the isni international agency (isni-ia) is to assign to the public name(s) of a researcher, inventor, writer, artist, performer, publisher, etc. a persistent unique identifying number in order to resolve the problem of name ambiguity in search and discovery; and diffuse each assigned isni across all repertoires in the global supply chain so that every published work can be unambiguously attributed to its creator wherever that work is described. by achieving these goals, the isni will act as a bridge identifier across multiple domains and become a critical component in linked data and semantic web applications. key statistics . million isni holds public records of more than million identities . million isni holds public records of over . million individuals (of which . million are researchers) . million isni holds public records of , , organizations sources the isni database is a cross-domain resource with direct contributions from sources news the british library launches its isni portal: a brand new, online service for isni users we are delighted and privileged to announce that the british library has now launched its online, all-in-one service for the international standard name... read more bds builds a new website for isni bdsdigital, the web services and it arm of bds, has built a new isni website, which went live in june . bds also transferred existing content from the... read more music industry isni registrations now free and automated sound credit music credit cloud profile system offers world's only free and automated isni registration service memphis, tenn., october , – every creative work... read more get an isni search database isni international agency (isni-ia) limited registered address: c/o editeur, united house, north road, london, n dp, uk company registration number: follow us privacy terms of use faqs shane lin | scholars' lab home about blog makerspace our work events for students spatial tech accessibility year of blogging charter library people search site map more› home about blog makerspace our work events for students spatial tech accessibility year of blogging charter library people search site map covid- update: the scholars’ lab staff is available, working remotely to support your teaching and research needs. all of our workshops, events, and consultations will be online this semester. contact us at scholarslab@virginia.edu to ask a question or schedule a virtual gis, vr, digital project, or makerspace consultation. for information about other departments, see the library’s status dashboard and faq. people // shane lin senior developer contact contact email:ssl ab@virginia.edu twitter:@shane-et-al about shane writes codes for scholars’ lab, teaches code lab, and co-directs the slab coffee studies program. on the academic side, he works on the history of computing and the impact of digital technology on culture and politics. shane was previously a praxis fellow ( ), makerspace technologist ( - ), digital humanities fellow ( ), and the sole recipient of the scholars’ lab’s prestigious shane lin memorial fellowship ( ). all posts by shane recovering from failure (with g-code) . . the long and messy history of privacy . . bigger nozzles, faster printing . . ninjaflex on the makerbot . . adventures in d printer maintenance . . one day of praxis . . gender and computing (ctd) . . rails is kind of hard to get up and running . . literals . . holy crap . . learning ruby (again) . . crowdsourcing for profit and pleasure . . a practical prism pedagogy proposal . . #!/bin/sh . . back to top how to blog explore home about events connect facebook twitter github email us rss feed contact email:scholarslab@virginia.edu address:p.o. box charlottesville, va - phone: - - twarc-videos · pypi skip to main content switch to mobile version warning some features may not work without javascript. please try enabling it if you encounter problems. search pypi search help sponsors log in register menu help sponsors log in register search pypi search twarc-videos . . pip install twarc-videos copy pip instructions latest version released: mar , a twarc plugin to extract referenced video from tweet data navigation project description release history download files project links homepage statistics github statistics: stars: forks: open issues/prs: view statistics for this project via libraries.io, or by using our public dataset on google bigquery meta author: ed summers requires: python >= . maintainers esummers project description project details release history download files project description twarc-videos this twarc plugin uses youtube_dl to download videos and their metadata from tweets. this is nice because youtube_dl downloads video from many more platforms than youtube including twitter itself. to use twarc-videos first you need to install it: pip install twarc-videos now you can collect data using the core twarc utility. for example this search finds tweets that mention the word "nirvana" and also have native video (twitter video) or a link to youtube: twarc search 'nirvana (has:videos or url:"https://youtu.be")' > nirvana-tweets.jsonl and you have a new subcommand videos that is supplied by twarc-videos. twarc videos nirvana-tweets.jsonl once it is finished you will have a new videos directory that looks something like: videos ├── archive.txt ├── mapping.tsv ├── twitter │ ├── │ │ ├── psychedelia_-_nirvana_-_come_as_you_are.description │ │ ├── psychedelia_-_nirvana_-_come_as_you_are.info.json │ │ └── psychedelia_-_nirvana_-_come_as_you_are.mp │ ├── │ │ ├── rt_your_fav_bands_-_nirvana_come_as_you_are.description │ │ ├── rt_your_fav_bands_-_nirvana_come_as_you_are.info.json │ │ └── rt_your_fav_bands_-_nirvana_come_as_you_are.mp │ ├── │ │ ├── hanna_-_she_s_in_nirvana....description │ │ ├── hanna_-_she_s_in_nirvana....info.json │ │ └── hanna_-_she_s_in_nirvana....mp │ ├── │ │ ├── music_nostalgia_-_nirvana_the_man_who_sold_the_world_..description │ │ ├── music_nostalgia_-_nirvana_the_man_who_sold_the_world_..info.json │ │ └── music_nostalgia_-_nirvana_the_man_who_sold_the_world_..mp │ ├── │ │ ├── take_it_easy_-_abuelo_donde_andas_nirvana.description │ │ ├── take_it_easy_-_abuelo_donde_andas_nirvana.info.json │ │ └── take_it_easy_-_abuelo_donde_andas_nirvana.mp │ ├── │ │ ├── oraetlabora_-_reel_stories_-_dave_grohl_is_on_@bbctwo_this_saturday_at_ . pm...talking_@nirvana_amp_@foofighters_with_dermot_@radioleary_@wearecraftuk.description │ │ ├── oraetlabora_-_reel_stories_-_dave_grohl_is_on_@bbctwo_this_saturday_at_ . pm...talking_@nirvana_amp_@foofighters_with_dermot_@radioleary_@wearecraftuk.info.json │ │ └── oraetlabora_-_reel_stories_-_dave_grohl_is_on_@bbctwo_this_saturday_at_ . pm...talking_@nirvana_amp_@foofighters_with_dermot_@radioleary_@wearecraftuk.mp │ ├── │ ├── │ ├── │ ├── │ └── │ ├── john_-_nirvana_-_in_bloom_live_at_reading_ _@youtube.description │ ├── john_-_nirvana_-_in_bloom_live_at_reading_ _@youtube.info.json │ └── john_-_nirvana_-_in_bloom_live_at_reading_ _@youtube.mp └── youtube ├── x cgfqyjn │ ├── heart-shaped_box_nirvana_music_box.description │ ├── heart-shaped_box_nirvana_music_box.en.vtt │ ├── heart-shaped_box_nirvana_music_box.info.json │ └── heart-shaped_box_nirvana_music_box.mp ├── ahcttcxcryy │ ├── nirvana_-_about_a_girl_mtv_unplugged.description │ ├── nirvana_-_about_a_girl_mtv_unplugged.en.vtt │ ├── nirvana_-_about_a_girl_mtv_unplugged.info.json │ └── nirvana_-_about_a_girl_mtv_unplugged.mp ├── axu-laao_xq │ ├── nirvana_drain_you_lyrics_sub_espanol.description │ ├── nirvana_drain_you_lyrics_sub_espanol.info.json │ └── nirvana_drain_you_lyrics_sub_espanol.mp ├── d dnm f q │ ├── nirvana_-_in_bloom_live_at_reading_ .description │ ├── nirvana_-_in_bloom_live_at_reading_ .info.json │ └── nirvana_-_in_bloom_live_at_reading_ .mp ├── -fh-bqsv e │ ├── becoming_a_minimalist_w_matt_d_avella.description │ ├── becoming_a_minimalist_w_matt_d_avella.en.vtt │ ├── becoming_a_minimalist_w_matt_d_avella.info.json │ └── becoming_a_minimalist_w_matt_d_avella.mp ├── htwkbfoikeg │ ├── nirvana_-_smells_like_teen_spirit_official_music_video.description │ ├── nirvana_-_smells_like_teen_spirit_official_music_video.en.vtt │ ├── nirvana_-_smells_like_teen_spirit_official_music_video.info.json │ └── nirvana_-_smells_like_teen_spirit_official_music_video.mp ├── jwkst g f │ ├── nirvana_healing_centre_overview.description │ ├── nirvana_healing_centre_overview.info.json │ └── nirvana_healing_centre_overview.mp ├── mw e_tngcsy │ ├── everclear_-_santa_monica_official_music_video.description │ ├── everclear_-_santa_monica_official_music_video.info.json │ └── everclear_-_santa_monica_official_music_video.mp ├── n p sitrwy │ ├── nirvana_-_heart-shaped_box.description │ ├── nirvana_-_heart-shaped_box.info.json │ └── nirvana_-_heart-shaped_box.mp ├── oger oqzgts │ ├── nirvana_-_the_man_who_sold_the_world_live_on_mtv_unplugged_ _unedited.description │ ├── nirvana_-_the_man_who_sold_the_world_live_on_mtv_unplugged_ _unedited.en.vtt │ ├── nirvana_-_the_man_who_sold_the_world_live_on_mtv_unplugged_ _unedited.info.json │ └── nirvana_-_the_man_who_sold_the_world_live_on_mtv_unplugged_ _unedited.mp ├── v ry eimcw │ ├── nirvana_-_smells_like_teen_spirit_cover_radio_tapok.description │ ├── nirvana_-_smells_like_teen_spirit_cover_radio_tapok.en.vtt │ ├── nirvana_-_smells_like_teen_spirit_cover_radio_tapok.info.json │ └── nirvana_-_smells_like_teen_spirit_cover_radio_tapok.mp ├── ychvl w _pa │ ├── nirvana_-_where_did_you_sleep_last_night_ d_audio.description │ ├── nirvana_-_where_did_you_sleep_last_night_ d_audio.info.json │ └── nirvana_-_where_did_you_sleep_last_night_ d_audio.mp └── y-lqgqhd xs ├── dodo_tofubeats_-_nirvana_official_music_video.description ├── dodo_tofubeats_-_nirvana_official_music_video.info.json └── dodo_tofubeats_-_nirvana_official_music_video.mp the video/mapping.tsv file is a tab separated value file of video urls found and their corresponding location in disk. testing to run the tests you will need create a .env file that looks like: bearer_token=your_token_here and then: python setup.py test project details project links homepage statistics github statistics: stars: forks: open issues/prs: view statistics for this project via libraries.io, or by using our public dataset on google bigquery meta author: ed summers requires: python >= . maintainers esummers release history release notifications | rss feed this version . . mar , . . mar , . . mar , . . mar , download files download the file for your platform. if you're not sure which to choose, learn more about installing packages. files for twarc-videos, version . . filename, size file type python version upload date hashes filename, size twarc-videos- . . .tar.gz ( . kb) file type source python version none upload date mar , hashes view close hashes for twarc-videos- . . .tar.gz hashes for twarc-videos- . . .tar.gz algorithm hash digest sha acc caea b dabda a af b fd c cf d f b ed caeb copy md ccb fb f d ae f e bb copy blake - edf ad ac b c dabb ced d d e ec de copy close help installing packages uploading packages user guide faqs about pypi pypi on twitter infrastructure dashboard package index name retention our sponsors contributing to pypi bugs and feedback contribute on github translate pypi development credits using pypi code of conduct report security issue privacy policy terms of use status: all systems operational developed and maintained by the python community, for the python community. donate today! © python software foundation site map switch to desktop version english español français 日本語 português (brasil) українська Ελληνικά deutsch 中文 (简体) русский עברית esperanto supported by aws cloud computing datadog monitoring digicert ev certificate facebook / instagram psf sponsor fastly cdn google object storage and download analytics pingdom monitoring salesforce psf sponsor sentry error logging statuspage status page samvera samvera registration now open for samvera virtual connect, april – registration is now open for samvera virtual connect ! samvera virtual connect will take place april th - st from am &# ; pm edt. registration is free and open to anyone with an interest in samvera. this year&# ;s program is packed with presentations and lightning talks of interest to developers, managers, librarians, and other current or... read more &# ; the post registration now open for samvera virtual connect, april &# ; appeared first on samvera. hyku . release includes new customization features hyku . is now available, with new features and improvements. these features add customization options at the institution level, and the improvements provide for easier maintenance of hyku implementations across all adopters. theming improvements now even more theming capability is in the hands of non-technical administrators, offering the ability to create a unique branded repository... read more &# ; the post hyku . release includes new customization features appeared first on samvera. season’s greetings from the samvera community as draws to a close, we want to share our gratitude for each and every person who has helped the samvera community thrive in this difficult year of profound loss and challenge. we could not be the vibrant, welcoming, and valuable open source solution community we are without each and every person who contributed... read more &# ; the post season&# ;s greetings from the samvera community appeared first on samvera. save the date for samvera virtual connect mark your calendar for samvera virtual connect !tuesday, april &# ; wednesday, april , : am &# ; : pm edt / : am – : am pdt / : - : bst / : - : utc watch for more information coming in early including a call for program committee participation and a call for proposals. the post save the date for samvera virtual connect appeared first on samvera. developer resources: bug hunting in hyrax; adding blacklight advanced search to hyku bess sadler from notch has created two excellent guides that may be helpful to developers working in hyrax or hyku applications: bug hunting in hyrax: a well-documented process for finding a bug in a hyrax application adding blacklight_advanced_search to hyku: a how-to guide for adding blacklight advanced search to a hyku application have you or... read more &# ; the post developer resources: bug hunting in hyrax; adding blacklight advanced search to hyku appeared first on samvera. fedora alpha release available for download and testing fedora . alpha- is now available for download and testing. the primary goals for fedora are robust migration support, enhanced digital preservations features, and improved performance and scale. the fedora team will ask the samvera community for testing assistance when the full version is available in early . in the meantime, you can learn... read more &# ; the post fedora alpha release available for download and testing appeared first on samvera. samvera tech : a beginner-friendly overview of samvera samvera connect on-line included an excellent, beginner-friendly overview of “samvera tech ” presented by alisha evans and shana moore, software engineers at notch . evans has turned this presentation into a blog post walking through the technologies used in the samvera community. check out the post on the notch blog: samvera tech the post... read more &# ; the post samvera tech : a beginner-friendly overview of samvera appeared first on samvera. samvera connect presentations if you are not one of the + people who registered to enjoy samvera connect on-line, or if you are but missed a session you wanted to see, links to the recordings and session slide packs are being added to the samvera wiki here. our grateful thanks to all the organizers and speakers who... read more &# ; the post samvera connect presentations appeared first on samvera. things are already happening for samvera connect! samvera connect ‘proper’ is a little over a week away but the poster exhibition is now available here! there is a slack channel #connect-posters for asynchronous comment or discussion and each presenter has a minute video conferencing slot monday th – wednesday st october for live discussion. see sched for details. need a slack... read more &# ; the post things are already happening for samvera connect! appeared first on samvera. samvera connect on-line is nearly here! samvera&# ;s annual connect conference has gone virtual this year, like so many others. nevertheless, we&# ;ve put together an exciting program of workshops, presentations, posters and community social events that we hope will make up for not being able to meet in person. the main events are on friday / and monday &# ; thursday / &# ;... read more &# ; the post samvera connect on-line is nearly here! appeared first on samvera. librarian of things librarian of things weeknote ( ) § zotero pdf reader a new look and functionality for zotero&# ;s pdf reader is still in beta. i can&# ;t wait for this version to be unleashed! § mit d o earlier this week, mit press announced a new open access monograph program. it appears that the transition of scholarly ebooks to another form of subscription product &# ; continue reading "weeknote ( )" weeknote (late) last week i had a week that was more taxing than normal and i had nothing in the tank by friday. so i&# ;m putting together last week&# ;s weeknotes today. also, going forward each section heading has been anchor tagged for your link sharing needs. e.g. § § § § § and § . i say this &# ; continue reading "weeknote (late) " weeknote ( ) today the library is closed as is my place of work&# ;s tradition on the last day of reading week. but as i have three events (helping in a workshop, giving a presentation, participating in a focus group) in my calendar, i&# ;m just going to work the day and bank the time for later. § barbara &# ; continue reading "weeknote ( )" weeknote ( ) another week in which i was doing a lot of behind the scenes work. § duly noted: here&# ;s the article in full. § years ago, i gave a keynote called libraries are for use. and by use, i mean copying that featured the short and sad story of a person who was unable to donate &# ; continue reading "weeknote ( )" weeknote ( ) § last friday i was interviewed for the podcast the grasscast — a game-themed podcast named after the book, the grasshopper: games, life, and utopia. i ramble a little bit in the episode as i tried to be more open and conversational than concise and correct. but i also spoke that way because for some &# ; continue reading "weeknote ( )" weeknote ( ) i don&# ;t have much that i can report in this week&# ;s note. you are just going to have to take my word that this week, a large amount of my time was spent at meetings pertaining to my library department, my union, and anti-black racism work. § last year, around this same time, some colleagues &# ; continue reading "weeknote ( )" weeknote ( ) hey. i missed last week&# ;s weeknote. but we are here now. § this week i gave a class on searching scientific literature to a group of biology masters students. while i was making my slides comparing the advanced search capabilities of web of science and scopus, i discovered this weird behaviour of google scholar: a &# ; continue reading "weeknote ( )" weeknote ( ) this week&# ;s post is not going to capture my ability to be productive while white supremacists appeared to be ushered in and out of the us capitol building by complicit police and covid- continued to ravage my community because our provincial government doesn&# ;t want to spend money on the most vulnerable. instead, i&# ;m just going &# ; continue reading "weeknote ( )" weeknote ( ) § it looks like andromeda yelton is sharing weeknotes (&# ;this week in ai&# ;). i can&# ;t wait to see what she shares with us all in . § earlier this fall, clarivate analytics&# ;announced that it was moving toward a future that calculated the journal impact factor (jif) based on the date of electronic publication and not &# ; continue reading "weeknote ( )" weeknote ( ) § i don&# ;t have much to report in regards to the work i&# ;ve been doing this week. i tried to get our orcid-ojs plugin to work but there is some small strange bug that needs to be squished. luckily, next week i will have the benefit of assistance from the good people of crkn and &# ; continue reading "weeknote ( )" inkdroid inkdroid paper or plastic coincidence? twarc this post was originally published on medium but i spent time writing it so i wanted to have it here too. tl;dr twarc has been redesigned from the ground up to work with the new twitter v api and their academic research track. many thanks for the code and design contributions of betsy alpert, igor brigadir, sam hames, jeff sauer, and daniel verdeer that have made twarc possible, as well as early feedback from dan kerchner, shane lin, miles mccain, 李荣蓬, david thiel, melanie walsh and laura wrubel. extra special thanks to the institute for future environments at queensland university of technology for supporting betsy and sam in their work, and for the continued support of the mellon foundation. back in august of last year twitter announced early access to their new v api, and their plans to sunset the v . api that has been active for almost the last years. over the lifetime of their v . api twitter has become deeply embedded in the media landscape. as magazines, newspapers and television have moved onto the web they have increasingly adopted tweets as a mechanism for citing politicians, celebrities and organizations, while also using them to document current events, generate leads and gather feedback for evolving stories. as a result twitter has also become a popular object of study for humanities and social science researchers looking to understand the world as reflected, refracted and distorted by/in social media. on the surface the v api update seems pretty insignificant since the shape of a tweet, its parts, properties and affordances, aren’t changing at all. tweets with characters of text, images and video will continue to be posted, retweeted and quoted. however behind the scenes the representation of a tweet as data, and the quotas that control the rates at which this data can flow between apps and other third party services will be greatly transformed. needless to say, v represents a big change for the documenting the now project. along with community members we’ve developed and maintained open source tools like twarc that talk directly to the twitter api to help users to search for and collect live tweets that match criteria like hashtags, names and geographic locations. today we’re excited to announce the release of twarc v which has been designed from the ground up to work with the v api and twitter’s new academic research track. clearly it’s extremely problematic having a multi-national corporation act as a gatekeeper for who counts as an academic researcher, and what constitutes academic research. we need look no further than the recent experiences of timnit gebru and margaret mitchell at google for an example of what happens when research questions run up against the business objectives of capital. we only know their stories because gebru and mitchell’s bravely took a principled approach, where many researchers would have knowingly or unknowingly shaped their research to better fit the needs of the company. so it is important for us that twarc still be usable by people with and without access to the academic research track. but we have heard from many users that the academic research track presents new opportunities for twitter data collection that are essential for researchers interested in the observability of social media platforms. twitter is making a good faith effort to work with the academic research community, and we thought twarc should support it, even if big challenges lie ahead. so why are people interested in the academic research track? once your application has been approved you are able to collect data from the full history of tweets, at no cost. this is a massive improvement over the v . access which was limited to a one week window and researchers had to pay for access. access to the full archive means it’s now possible to study events that have happened in the past back to the beginning of twitter in . if you do create any historical datasets we’d love for you to share the tweet identifier datasets in the catalog. however this opening up of access on the one hand comes with a simultaneous contraction in terms of how much data can be collected at one time. the remainder of this post describes some of the details and the design decisions we have made with twarc to address them. if you would prefer to watch a quick introduction to using twarc v please check out this short video: installation if you are familiar with installing twarc nothing is changed. you still install (or upgrade) with pip as you did before: $ pip install --upgrade twarc in fact you will still have full access to the v . api just as you did before. so the old commands will continue to work as they did $ twarc search blacklivesmatter > tweets.jsonl twarc was designed to let you to continue to use twitter’s v . api undisturbed until it is finally turned off by twitter, at which point the functionality will be removed from twarc. all the support for the v api is mediated by a new command line utility twarc . for example to search for blacklivesmatter tweets and write them to a file tweets.jsonl: $ twarc search blacklivesmatter > tweets.jsonl all the usual twarc functionality such as searching for tweets, collecting live tweets from the streaming api endpoint, requesting user timelines and user metadata are all still there, twarc --help gives you the details. but while the interface looks the same there’s quite a bit different going on behind the scenes. representation truth be told, there is no shortage of open source libraries and tools for interacting with the twitter api. in the past twarc has made a bit of a name for itself by catering to a niche group of users who want a reliable, programmable way to collect the canonical json representation of a tweet. javascript object notation (json) is the language of web apis, and twitter has kept its json representation of a tweet relatively stable over the years. rather than making lots of decisions about the many ways you might want to collect, model and analyze tweets twarc has tried to do one thing and do it well (data collection) and get out of the way so that you can use (or create) the tools for putting this data to use. but the json representation of a tweet in the twitter v api is completely burst apart. the v base representation of a tweet is extremely lean and minimal, and just includes the text of the tweet its identifier and a handful of other things. all the details about the user who created the tweet, embedded media, and more are not included. fortunately this information is still available, but the user needs to craft their api request to request tweets using a set of expansions that tell the twitter api what additional entities to include. in addition for each expansion there are a set of field options to include that control what of these expansions is returned. so rather than there being a single json representation of a tweet api users now have the ability to shape the data based on what they need, much like how graphql apis work. this kind of makes you wonder why twitter didn’t make their graphql api available. for specific use cases this customizability is very useful, but the mutability of the representation of a tweet presents challenges when collecting data for future use. if you didn’t request the right expansions or fields when collecting the data then you won’t be able to analyze that data later when doing your research. to solve for this twarc has been designed to collect the richest possible representation for a tweet, by requesting all possible expansions and field combinations for tweets. see the expansions module for the details if you are interested. this takes a significant burden off of users to digest the api documentation, and craft the correct api requests themselves. in addition the twarc community will be monitoring the twitter api documentation going forward to incorporate new expansions and fields as they will inevitably be added in the future. flattening this is diving into the weeds a little bit, but it’s worth noting here that twitter’s introduction of expansions allows data that was once duplicated across multiple tweets (such as user information, media, retweets, etc) to be included once per response from the api. this means that instead of seeing information about the user who created a tweet in the context of their tweet the user will be referenced using an identifier, and this identifier will map to user metadata in the outer envelope of the response. it makes sense why twitter have introduced expansions since it means in a set of tweets from a given user the user information will just be included once rather than repeated times, which means less data, less network traffic and less money. it’s even more significant when consider the large number of possible expansions. however this pass by-reference rather than by-value presents some challenges for stream based processing which expects each tweet to be self-contained. for this reason we’ve introduce the idea of flattening the response data when persisting the json to disk. this means that tools and data pipelines that expect to operate on a stream of tweets can continue to do so. since the representation of a tweet is so dependent on how data is requested we’ve taken the opportunity to introduce a small stanza of twarc specific metadata using the __twarc prefix. this metadata records what api endpoint the data was requested from, and when. this information is critically important when interpreting the data, because some information about a tweet like its retweet and quote counts are constantly changing. data flows as mentioned above you can still collect tweets from the search and streaming api endpoints in a way that seems quite similar to the v api. the big changes however are the quotas associated with these endpoints which govern how much can be collected. these quotas control how many requests can be sent to twitter in minute intervals. in fact these quotas are not much changed, but what’s new are app wide quotas that constrain how many tweets a given application (app) can collect every month. an app in this context is a piece of software (e.g. your twarc software) identified by unique api keys set up in the twitter developer portal. the standard api access sets a , tweet per month limit. this is a huge change considering there were no monthly app limits before. if you get approved for the academic research track your app quota is increased to million per month. this is markedly better but the achievable data volume is still nothing like the v . api, as these graphs attempt to illustrate: twarc will still observe the same rate limits, but once you’ve collected your portion for the month there’s not much that can be done, for that app at least. apart from the quotas twitter’s streaming endpoint in v is substantially changed which impacts how users interact with twarc. previously twarc users would be able to create up to to two connections to the filter stream api. this could be done by simply: twarc filter obama > obama.jsonl however in the twitter v api only apps can connect to the filter stream, and they can only connect once. at first this seems like a major limitation but rather than creating a connection per query the v api allows you to build a set of rules for tweets to match, which in turns controls what tweets are included in the stream. this means you can collect for multiple types of queries at the same time, and the tweets will come back with a piece of metadata indicating what rule caused its inclusion. this translates into a markedly different set of interactions at the command line for collecting from the stream where you first need to set your stream rules and then open a connection to fetch it. twarc stream-rules add blacklivesmatter twarc stream > tweets.jsonl one useful side effect of this is that you can update the stream (add and remove rules) while the stream is in motion: twarc stream-rules add blm while you are limited by the api quota in terms of how many tweets you can collect, tweets are not “dropped on the floor” when the volume gets too high. once upon a time the v . filter stream was rumored to be rate limited when your stream exceeds % of the total volume of new tweets. plugins in addition to twarc helping you collect tweets the github repository has also been a place to collect a set of utilities for working with the data. for example there are scripts for extracting and unshortening urls, identifying suspended/deleted content, extracting videos, buiding wordclouds, putting tweets on maps, displaying network graph visualizations, counting hashtags, and more. these utilities all work like unix filters where the input is a stream of tweets and the output varies depending on what the utility is doing, e.g. a gephi file for a network visualization, or a folder of mp files for video extraction. while this has worked well in general the kitchen sink approach has been difficult to manage from a configuration management perspective. users have to download these scripts manually from github or by cloning the repository. for some users this is fine, but it’s a bit of a barrier to entry for users who have just installed twarc with pip. furthermore these plugins often have their own dependencies which twarc itself does not. this lets twarc can stay pretty lean, and things like youtube_dl, networkx or pandas can be installed by people that want to use utilities that need them. but since there is no way to install the utilities there isn’t a way to ensure that the dependencies are installed, which can lead to users needing to diagnose missing libraries themselves. finally the plugins have typically lacked their own tests. twarc’s test suite has really helped us track changes to the twitter api and to make sure that it continues to operate properly as new functionality has been added. but nothing like this has existed for the utilities. we’ve noticed that over time some of them need updating. also their command line arguments have drifted over time which can lead to some inconsistencies in how they are used. so with twarc we’ve introduced the idea of plugins which extend the functionality of the twarc command, are distributed on pypi separately from twarc, and exist in their own github repositories where they can be developed and tested independently of twarc itself. this is all achieved through twarc ’s use of the click library and specifically click-plugins. so now if you would like to convert your collected tweets to csv you can install the twarc-csv: $ pip install twarc-csv $ twarc search covid > covid .jsonl $ twarc csv covid .jsonl > covid .csv or if you want to extract embedded and referenced videos from tweets you can install twarc-videos which will write all the videos to a directory: $ pip install twarc-videos $ twarc videos covid .jsonl --download-dir covid -videos you can write these plugins yourself and release them as needed. check out the plugin reference implementation tweet-ids for a simple example to adapt. we’re still in the process of porting some of the most useful utilities over and would love to see ideas for new plugins. check out the current list of twarc plugins and use the twarc issue tracker on github to join the discussion. you may notice from the list of plugins that twarc now (finally) has documentation on readthedocs external from the documentation that was previously only available on github. we got by with github’s rendering of markdown documents for a while, but github’s boilerplate designed for developers can prove to be quite confusing for users who aren’t used to selectively ignoring it. readthedocs allows us to manage the command line and api documentation for twarc, and to showcase the work that has gone into the spanish, japanese, portuguese, swedish, swahili and chinese translations. feedback thanks for reading this far! we hope you will give twarc a try. let us know what you think either in comments here, in the docnow slack or over on github. ✨ ✨ happy twarcing! ✨ ✨ ✨ windows users will want to indicate the output file using a second argument rather than redirecting output with >. see this page for details.↩ $ j you may have noticed that i try to use this static website as a journal. but, you know, not everything i want to write down is really ready (or appropriate) to put here. some of these things end up in actual physical notebooks–there’s no beating the tactile experience of writing on paper for some kind of thinking. but i also spend a lot of time on my laptop, and at the command line in some form or another. so i have a directory of time stamped markdown files stored on dropbox, for example: ... /home/ed/dropbox/journal/ - - .md /home/ed/dropbox/journal/ - - .md /home/ed/dropbox/journal/ - - .md /home/ed/dropbox/journal/ - - .md /home/ed/dropbox/journal/ - - .md ... sometimes these notes migrate into a blog post or some other writing i’m doing. i used this technique quite a bit when writing my dissertation when i wanted to jot down things on my phone when an idea arrived. i’ve tried a few different apps for editing markdown on my phone, but mostly settled on ia writer which mostly just gets out of the way. but when editing on my laptop i tend to use my favorite text editor vim with the vim-pencil plugin for making markdown fun and easy. if vim isn’t your thing and you use another text editor keep reading since this will work for you too. the only trick to this method of journaling is that i just need to open the right file. with command completion on the command line this isn’t so much of a chore. but it does take a moment to remember the date, and craft the right path. today while reflecting on how nice it is to still be using unix, it occurred to me that i could create a little shell script to open my journal for that day (or a previous day). so i put this little file j in my path: #!/bin/zsh journal_dir="/home/ed/dropbox/journal" if [ "$ " ]; then date=$ else date=`date +%y-%m-%d` fi vim "$journal_dir/$date.md" so now when i’m in the middle of something else and want to jot a note in my journal i just type j. unix, still crazy after all these years. strengths and weaknesses quoting macey ( ), quoting foucault, quoting nietzsche: one thing is needful. – to ‘give style’ to one’s character – a great and rare art! it is practised by those who survey all the strengths and weaknesses that their nature has to offer and then fit them into an artistic plan until each appears as art and reason and even weaknesses delight the eye. nietzsche, williams, nauckhoff, & del caro ( ), p. this is a generous and lively image of what art does when it is working. art is not perfection. macey, d. ( ). the lives of michel foucault: a biography. verso. nietzsche, f. w., williams, b., nauckhoff, j., & del caro, a. ( ). the gay science: with a prelude in german rhymes and an appendix of songs. cambridge, u.k. ; new york: cambridge university press. data speculation i’ve taken the ill-advised approach of using the coronavirus as a topic to frame the exercises in my computer programming class this semester. i say “ill-advised” because given the impact that covid has been having on students i’ve been thinking they probably need a way to escape news of the virus by way of writing code, rather than diving into it more. it’s late in the semester to modulate things but i think we will shift gears to look at programming through another lens after spring break. that being said, one of the interesting things we’ve been doing is looking at vaccination data that is being released by the maryland department of health through their esri arcgis hub. note: this dataset has since been removed from the web because it has been superseded by a new dataset that includes single dose vaccinations. i guess it’s good that students get a feel for how ephemeral data on the web is, even when it is published by the government. we noticed that this dataset recorded a small number of vaccinations as happening as early as the s up until december , when vaccines were approved for use. i asked students to apply what we have been learning about python (files, strings, loops, and sets) to identify the maryland counties that were responsible for generating this anomalous data. i thought this exercise provided a good demonstration using real, live data that critical thinking about the provenance of data is always important because there is no such thing as raw data (gitelman, ). while we were working with the data to count the number of anomalous vaccinations per county one of my sharp eyed students noticed that the results we were seeing with my version of the dataset (downloaded on february ) were different from what we saw with his (downloaded on march ). we expected to see new rows in the later one because new vaccination data seem to be reported daily–which is cool in itself. but we were surprised to find new vaccination records for dates earlier than december , . why would new vaccinations for these erroneous older dates still be entering the system? for example the second dataset downloaded march acquired new rows: object id vaccination date county daily first dose cumulative first dose daily second dose cumulative second dose / / allegany / / baltimore / / baltimore / / baltimore city / / baltimore / / prince george’s and these rows present in the february version were deleted in the march version: object id vaccination date county daily first dose cumulative first dose daily second dose cumulative second dose / / frederick / / talbot / / baltimore / / caroline / / prince george’s / / anne arundel / / frederick / / wicomico / / frederick i found these additions perplexing at first, because i assumed these outliers were part of an initial load. but it appears that the anomalies are still being generated? the deletions suggest that perhaps the anomalous data is being identified and scrubbed in a live system that is then dumping out the data? or maybe the code that is being used to update the dataset in arcgis hub itself is malfunctioning in some way? if you are interested in toying around with the code and data it is up on github. i was interested to learn about pandas.dataframe.merge which is useful for diffing tables when you use indicator=true. at any rate, having students notice, measure and document anomalies like this seems pretty useful. i also asked them to speculate about what kinds of activities could generate these errors. i meant speculate in the speculative fiction sense of imagining a specific scenario that caused it. i think this made some students scratch their head a bit, because i wasn’t asking them for the cause, but to invent a possible cause. based on the results so far i’d like to incorporate more of these speculative exercises concerned with the functioning of code and data representations into my teaching. i want to encourage students to think creatively about data processing as they learn about the nuts and bolts of how code operates. for example the treatments in how to run a city like amazon, and other fables which use sci-fi to test ideas about how information technologies are deployed in society. another model is the speculative ethics book club which also uses sci-fi to explore the ethical and social consequences of technology. i feel like i need to read up on specualtive research more generally before doing this though (michael & wilkie, ). i’d also like to focus the speculation down at the level of the code or data processing, rather than at the macro super-system level. but that has its place too. another difference is that i was asking students to engage in speculation about the past rather than the future. how did the data end up this way? perhaps this is more of a genealogical approach, of winding things backwards, and tracing what is known. maybe it’s more mystery than sci-fi. the speculative element is important because (in this case) operations at the md dept of health, and their arcgis hub setup are mostly opaque to us. but even when access isn’t a problem these systems they can feel opaque, because rather than there being a dearth of information you are drowning in it. speculation is a useful abductive approach to hypothesis generation and, hopefully, understanding. update - - : over in the fediverse david benque recommended i take a look at matthew stanley’s chapter in (gitelman, ) “where is that moon, anyway? the problem of interpreting historical solar eclipse observations” for the connection to mystery. for the connection to peirce and abduction he also pointed to luciana parisi’s chapter “speculation: a method for the unattainable” in lury & wakeford ( ). definitely things to follow up on! references gitelman, l. (ed.). ( ). “raw data” is an oxymoron. mit press. lury, c., & wakeford, n. ( ). inventive methods: the happening of the social. routledge. michael, m., & wilkie, a. ( ). speculative research. in the palgrave encyclopedia of the possible (pp. – ). cham: springer international publishing. retrieved from https://doi.org/ . / - - - - _ - recovering foucault i’ve been enjoying reading david macey’s biography of michel foucault, that was republished in by verso. macey himself is an interesting figure, both a scholar and an activist who took leave from academia to do translation work and to write this biography and others of lacan and fanon. one thing that struck me as i’m nearing the end of macey’s book is the relationship between foucault and archives. i think foucault has become emblematic of a certain brand of literary analysis of “the archive” that is far removed from the research literature of archival studies, while using “the archive” as a metaphor (caswell, ). i’ve spent much of my life working in libraries and digital preservation, and now studying and teaching about them from the perspective of practice, so i am very sympathetic to this critique. it is perhaps ironic that the disconnect between these two bodies of research is a difference in discourse which foucault himself brought attention to. at any rate, the thing that has struck me while reading this biography is how much time foucault himself spent working in libraries and archives. here’s foucault in his own words talking about his thesis: in histoire de la folie à l’âge classique i wished to determine what could be known about mental illness in a given epoch … an object took shape for me: the knowledge invested in complex systems of institutions. and a method became imperative: rather than perusing … only the library of scientific books, it was necessary to consult a body of archives comprising decrees, rules hospital and prison registers, and acts of jurisprudence. it was in the arsenal or the archives nationales that i undertook the analysis of a knowledge whose visible body is neither scientific nor theoretical discourse, nor literature, but a daily and regulated practice. (macey, , p. ) foucault didn’t simply use archives for his research: understanding the processes and practices of archives were integral to his method. even though the theory and practice of libraries and archives are quite different given their different functions and materials, they are often lumped together as a convenience in the same buildings. macey blurs them a little bit, in sections like this where he talks about how important libraries were to foucault’s work: foucault required access to paris for a variety of reasons, not least because he was also teaching part-time at ens. the putative thesis he had begun at the fondation thiers – and which he now described to polin as being on the philosophy of psychology – meant that he had to work at the bibliothèque nationale and he had already become one of its habitues. for the next thirty years, henri labrouste’s great building in the rue de richelieu, with its elegant pillars and arches of cast iron, would be his primary place of work. his favourite seat was in the hemicycle, the small, raised section directly opposite the entrance, sheltered from the main reading room, where a central aisle separates rows of long tables subdivided into individual reading desks. the hemicycle affords slighty more quiet and privacy. for thirty years, foucault pursued his research here almost daily, with occasional forays to the manuscript department and to other libraries, and contended with the byzantine cataloguing system: two incomplete and dated printed catalogues supplemented by cabinets containing countless index cards, many of them inscribed with copperplate handwriting. libraries were to become foucault’s natural habitat: ‘those greenish institutions where books accumulate and where there grows the dense vegetation of their knowledge’ there’s a metaphor for you: libraries as vegetation :) it kind of reminds me of some recent work looking at decentralized web technologies in terms of mushrooms. but i digress. i really just wanted to note here that the erasure of archival studies from humanities research about “the archive” shouldn’t really be attributed to foucault, whose own practice centered the work of libraries and archives. foucault wasn’t just writing about an abstract archive, he was practically living out of them. as someone who has worked in libraries and archives i can appreciate how power users (pun intended) often knew aspects of the holdings and intricacies of their their management better than i did. archives, when they are working, are always collaborative endeavours, and the important thing is to recognize and attribute the various sides of that collaboration. ps. writing this blog post led me to dig up a few things i want to read (eliassen, ; radford, radford, & lingel, ). references caswell, m. ( ). the archive is not an archives: on acknowledging the intellectual contributions of archival studies. reconstruction, ( ). retrieved from http://reconstruction.eserver.org/issues/ /caswell.shtml eliassen, k. ( ). archives of michel foucualt. in e. røssaak (ed.), the archive in motion, new conceptions of the archive in contemporary thought and new media practices. novus press. macey, d. ( ). the lives of michel foucault: a biography. verso. radford, g. p., radford, m. l., & lingel, j. ( ). the library as heterotopia: michel foucault and the experience of library space. journal of documentation, ( ), – . teaching oop in the time of covid i’ve been teaching a section of the introduction to object oriented programming at the umd college for information studies this semester. it’s difficult for me, and for the students, because we are remote due to the coronavirus pandemic. the class is largely asynchronous, but every week i’ve been holding two synchronous live coding sessions in zoom to discuss the material and the exercises. these have been fun because the students are sharp, and haven’t been shy about sharing their screen and their vscode session to work on the details. but students need quite a bit of self-discipline to move through the material, and probably only about / of the students take advantage of these live sessions. i’m quite lucky because i’m working with a set of lectures, slides and exercises that have been developed over the past couple of years by other instructors: josh westgard, aric bills and gabriel cruz. you can see some of the public facing materials here. having this backdrop of content combined with severance’s excellent (and free) python for everybody has allowed me to focus more on my live sessions, on responsive grading, and to also spend some time crafting additional exercises that are geared to this particular moment. this class is in the college for information studies and not in the computer science department, so it’s important for the students to not only learn how to use a programming language, but to understand programming as a social activity, with real political and material effects in the world. being able to read, understand, critique and talk about code and its documentation is just as important as being able to write it. in practice, out in the “real world” of open source software i think these aspects are arguably more important. one way i’ve been trying to do this in the first few weeks of class is to craft a sequence of exercises that form a narrative around coronavirus testing and data collection to help remind the students of the basics of programming: variables, expressions, conditionals, loops, functions, files. in the first exercise we imagined a very simple data entry program that needed to record results of real-time polymerase chain reaction tests (rt-pcr). i gave them the program and described how it was supposed to work, and asked them describe (in english) any problems that they noticed and to submit a version of the program with problems fixed. i also asked them to reflect on a request from their boss about adding the collection of race, gender and income information. the goal here was to test their ability to read the program and write english about it while also demonstrating a facility for modifying the program. most importantly i wanted them to think about how inputs such as race or gender have questions about categories and standards behind them, and weren’t simply a matter of syntax. the second exercise builds on the first by asking them to adjust the revised program to be able to save the data in a very particular format. yes, in the first exercise the data is stored in memory and printed to the screen in aggregate at the end. the scenario here is that the department of health and human services has assumed the responsibility for covid test data collection from the centers for disease control. of course this really happened, but the data format i chose was completely made up (maybe we will be working with some real data at the end of the semester if i continue with this theme). the goal in this exercise was to demonstrate their ability to read another program and fit a function into it. the students were given a working program that had a save_results() function stubbed out. in addition to submitting their revised code i asked them to reflect on some limitations of the data format chosen, and the data processing pipeline that it was a part of. and in the third exercise i asked them to imagine that this lab they were working in had a scientist who discovered a problem with some of the thresholds for acceptable testing, which required an update to the program from exercise , and also a test suite to make sure the program was behaving properly. in addition to writing the tests i asked them to reflect on what functionality was not being tested that probably should be. this alternation between writing code and writing prose is something i started doing as part of a digital curation class. i don’t know if this dialogical or perhaps dialectical, approach is something others have tried. i should probably do some research to see. in my last class i alternated week by week: one week reading and writing code, the next week reading and writing prose. but this semester i’ve stayed focused on code, but required the reading and writing of code as well as prose about code in the same week. i hope to write more about how this goes, and these exercises as i go. i’m not sure if i will continue with the coronavirus data examples. one thing i’m sensitive to is that my students themselves are experiencing the effects of the coronavirus, and may want to escape it just for a bit in their school work. just writing in the open about it here, in addition to the weekly meetings i’ve had with aric, josh and gabriel has been very useful. speaking of those meetings. i learned today from aric that tomorrow (february th, ) is the th anniversary of python’s first public release! you can see this reflected in this timeline. this v . . release was the first release guido van rossum made outside of cwi and was made on the usenet newsgroup alt.sources where it is split out into chunks that need to be reassembled. back in andrew dalke located a and repackaged these sources in google groups which acquired alt.sources as part of dejanews in . but if you look at the time stamp on the first part of the release you can see that it was made february , (not february ). so i’m not sure if the birthday is actually today. i sent this little note out to my students with this wonderful two part oral history that the computer history museum did with guido van rossum a couple years ago. i turns out both of his parents were atheists and pacifists. his dad went to jail because he refused to be conscripted into the military. that and many more details of his background and thoughts about the evolution of python can be found in these delightful interviews: happy birthday python! gpt- jam one of the joys of pandemic academic life has been a true feast of online events to attend, on a wide variety of topics, some of which are delightfully narrow and esoteric. case in point was today’s reflecting on power and ai: the case of gpt- which lived up to its title. i’ll try to keep an eye out for when the video posts, and update here. the workshop was largely organized around an exploration of whether gpt- , the largest known machine learning language model, changes anything for media studies theory, or if it amounts to just more of the same. so the discussion wasn’t focused so much on what games could be played with gpt- , but rather if gpt- changes the rules of the game for media theory, at all. i’m not sure there was a conclusive answer at the end, but it sounded like the consensus was that current theorization around media is adequate for understanding gpt- , but it matters greatly what theory or theories are deployed. the online discussion after the presentations indicated that attendees didn’t see this as merely a theoretical issue, but one that has direct social and political impacts on our lives. james steinhoff looked at gpt- using a marxist media theory perspective where he told the story of gpt- ’s as a project of openai and as a project of capital. openai started with much fanfare in as a non-profit initiative where the technology, algorithms and models developed would would be kept openly licensed and freely available so that the world could understand the benefits and risks of ai technology. steinhoff described how in the project’s needs for capital (compute power and staff) transitioned it from a non-profit into a capped-profit company, which is now owned, or at least controlled, by microsoft. the code for generating the model as well as the model itself are gated behind a token driven web api run my microsoft. you can get on a waiting list to use it, but apparently a lot of people have been waiting a while, so … being a microsoft employee probably helps. i grabbed a screenshot of the pricing page that steinhoff shared during his presentation: i’d be interested to hear more about how these tokens operate. are they per-request, or are they measured according something else? i googled around a bit during the presentation to try to find some documentation for the web api, and came up empty handed. i did find shreya shankar’s gpt -sandbox project for interacting with the api in your browser (mostly for iteratively crafting text input in order to generate desired output). it depends on the openai python package created by openai themselves. the docs for openai then point at a page on the openai.com website which is behind a login. you can create an account, but you need to be pre-approved (made it through the waitlist) to be able to see the docs. there’s probably some sense that can be made from examining the python client though. all of the presentations in some form or another touched on the billion parameters that were used to generate the model. but the api to the model doesn’t have that many parameters. it allows you to enter text and get text back. but the api surface that the gpt- service provides could be interesting to examine a bit more closely, especially to track how it changes over time. in terms of how this model mediates knowledge and understanding it’ll be important watch. steinhoff’s message seemed to be that, despite the best of intentions, gpt- functions in the service of very large corporations with very particular interests. one dimension that he didn’t explore perhaps because of time, is how the gpt- model itself is fed massive amounts of content from the web, or the commons. indeed % of the data came from the commoncrawl project. gpt- is an example of an extraction project that has been underway at large internet companies for some time. i think the critique of these corporations has often been confined to seeing them in terms of surveillance capitalism rather than in terms of raw resource extraction, or the primitive accumulation of capital. the behavioral indicators of who clicked on what are certainly valuable, but gpt- and sister projects like commoncrawl shows just the accumulation of data with modest amounts of metadata can be extremely valuable. this discussion really hit home for me since i’ve been working with jess ogden and shawn walker using commoncrawl as a dataset for talking about the use of web archives, while also reflecting on the use of web archives as data. commoncrawl provides a unique glimpse into some of the data operations that are at work in the accumulation of web archives. i worry that the window is closing and the commoncrawl itself will be absorbed into microsoft. following steinhoff olya kudina and bas de boer jointly presented some compelling thoughts about how its important to understand gpt- in terms of sociotechnical theory, using ideas drawn from foucault and arendt. i actually want to watch their presentation again because it followed a very specific path that i can’t do justice to here. but their main argument seemed to be that gpt- is an expression of power and that where there is power there is always resistance to power. gpt- can and will be subverted and used to achieve particular political ends of our own choosing. because of my own dissertation research i’m partial to foucault’s idea of governmentality, especially as it relates to ideas of legibility (scott, )–the who, what and why of legibility projects, aka archives. gpt- presents some interesting challenges in terms of legibility because the model is so complex, the results it generates defy deductive logic and auditing. in some ways gpt- obscures more than it makes a population legible, as foucault moved from disciplinary analysis of the subject, to the ways in which populations are described and governed through the practices of pastoral power, of open datasets. again the significance of commoncrawl as an archival project, as a web legibility project, jumps to the fore. i’m not as up on arendt as i should be, so one outcome of their presentation is that i’m going to read her the human condition which they had in a slide. i’m long overdue. references scott, j. c. ( ). seeing like a state: how certain schemes to improve the human condition have failed. yale university press. mimetypes today i learned that python has a mimetypes module, and has ever since guido von rossum added it in . honestly i’m just a bit sheepish to admit this discovery, as someone who has been using python for digital preservation work for about years. but maybe there’s a good reason for that. since the entire version history for python is available on github (which is a beautiful thing in itself) you can see that the mimetypes module started as a guess_type() function built around a pretty simple hard coded mapping of file extensions to mimetypes. the module also includes a little bit of code to look for, and parse, mimetype registries that might be available on the host operating system. the initial mimetype registries used included one from the venerable apache httpd web server, and the netscape web browser, which was about three years old at the time. it makes sense why this function to look up a mimetype for a filename would be useful at that time, since python was being used to serve up files on the nascent web and for sending email, and whatnot. today the module looks much the same, but has a few new functions and about twice as many mimetypes in its internal list. some of the new mimetypes include text/csv, audio.mpeg, application/vnd.ms-powerpoint, application/x-shockwave-flash, application/xml, and application/json. comparing the first commit to the most latest provides a thumbnail sketch of years of web format evolution. i’ll admit, this is is a bit of an esoteric thing to be writing a blog post about. so i should explain. at work i’ve been helping out on a community archiving project which has accumulated a significant amount of photographs, scans, documents of various kinds, audio files and videos. some of these files are embedded in web applications like omeka, some are in cloud storage like google drive, or on the office networked attached storage, and others are on scattered storage devices in people’s desk drawers and closets. we’ve also created new files during community digitization events, and oral history interviews. as part of this work we’ve wanted to start building a place on the web where all these materials live. this has required not only describing the files, but also putting all the files in one place so that access can be provided. in principle this sounds simple. but it turns out that collecting the files from all these diverse locations poses significant challenges, because their context matters. the filenames, and the directories they are found in, are sometimes the only descriptive metadata that exists for this data. in short, the original order matters. but putting this content on the web means that the files need to be brought together and connected with their metadata programmatically. this is how i stumbled across the mimetypes module. i’ve been writing some throwaway code to collect the files together into the same directory structure while preserving their original filenames and locations in an airtable database. i’ve been using the magic module to identify the format of the file, which is used to copy the file into a dropbox storage location. the extension is important because we are expecting this to be a static site serving up the content and we want the files to also be browsable using the dropbox drive. it turns out the mimetypes.guess_extension is pretty useful for turning a mediatype into an file extension. i’m kind of surprised that it took me this long to discover mimetypes, but i’m glad i did. as an aside i think this highlights for me how important git can be as an archive and research method for software studies work. northwest branch cairn here is a short recording and a couple photos from my morning walk along the northwest branch trail with penny. i can’t go every day but at months old she has tons of energy, so it’s generally a good idea for all concerned to go at least every other morning. and it’s a good thing, because the walk is surprisingly peaceful, and it’s such a joy to see her run through the woods. after walking about minutes there is this little cairn that is a reminder for me to turn around. after seeing it grow in size i was sad to see it knocked down one day. but, ever so slowly, it is getting built back up again. spacy · industrial-strength natural language processing in python this app works best with javascript enabled. spacy 💥 out now: spacy v . menu usage models api universe usage models api universe industrial-strength natural language processing in python get things done spacy is designed to help you do real work — to build real products, or gather real insights. the library respects your time, and tries to avoid wasting it. it's easy to install, and its api is simple and productive. get started blazing fast spacy excels at large-scale information extraction tasks. it's written from the ground up in carefully memory-managed cython. if your application needs to process entire web dumps, spacy is the library you want to be using. facts & figures awesome ecosystem in the five years since its release, spacy has become an industry standard with a huge ecosystem. choose from a variety of plugins, integrate with your machine learning stack and build custom components and workflows. read more edit the code & try spacy # pip install -u spacy # python -m spacy download en_core_web_sm import spacy # load english tokenizer, tagger, parser and ner nlp = spacy.load("en_core_web_sm") # process whole documents text = ("when sebastian thrun started working on self-driving cars at " "google in , few people outside of the company took him " "seriously. “i can tell you very senior ceos of major american " "car companies would shake my hand and turn away because i wasn’t " "worth talking to,” said thrun, in an interview with recode earlier " "this week.") doc = nlp(text) # analyze syntax print("noun phrases:", [chunk.text for chunk in doc.noun_chunks]) print("verbs:", [token.lemma_ for token in doc if token.pos_ == "verb"]) # find named entities, phrases and concepts for entity in doc.ents: print(entity.text, entity.label_) features support for + languages trained pipelines for languages multi-task learning with pretrained transformers like bert pretrained word vectors state-of-the-art speed production-ready training system linguistically-motivated tokenization components for named entity recognition, part-of-speech tagging, dependency parsing, sentence segmentation, text classification, lemmatization, morphological analysis, entity linking and more easily extensible with custom components and attributes support for custom models in pytorch, tensorflow and other frameworks built in visualizers for syntax and ner easy model packaging, deployment and workflow management robust, rigorously evaluated accuracy new in v . transformer-based pipelines, new training system, project templates & more spacy v . features all new transformer-based pipelines that bring spacy's accuracy right up to the current state-of-the-art. you can use any pretrained transformer to train your own pipelines, and even share one transformer between multiple components with multi-task learning. training is now fully configurable and extensible, and you can define your own custom models using pytorch, tensorflow and other frameworks. the new spacy projects system lets you describe whole end-to-end workflows in a single file, giving you an easy path from prototype to production, and making it easy to clone and adapt best-practice projects for your own use cases. see what's new from the makers of spacy prodigy: radically efficient machine teaching prodigy is an annotation tool so efficient that data scientists can do the annotation themselves, enabling a new level of rapid iteration. whether you're working on entity recognition, intent detection or image classification, prodigy can help you train and evaluate your models faster. try it out reproducible training for custom pipelines spacy v . introduces a comprehensive and extensible system for configuring your training runs. your configuration file will describe every detail of your training run, with no hidden defaults, making it easy to rerun your experiments and track changes. you can use the quickstart widget or the init config command to get started, or clone a project template for an end-to-end workflow. get started language afrikaans albanian arabic armenian basque bengali bulgarian catalan chinese croatian czech danish dutch english estonian finnish french german greek gujarati hebrew hindi hungarian icelandic indonesian irish italian japanese kannada korean kyrgyz latvian ligurian lithuanian luxembourgish macedonian malayalam marathi multi-language nepali norwegian bokmål persian polish portuguese romanian russian sanskrit serbian setswana sinhala slovak slovenian spanish swedish tagalog tamil tatar telugu thai turkish ukrainian urdu vietnamese yoruba components taggerparsernertextcat hardware cpugpu (transformer) optimize for efficiencyaccuracy # this is an auto-generated partial config. to use it with 'spacy train' # you can run spacy init fill-config to auto-fill all default settings: # python -m spacy init fill-config ./base_config.cfg ./config.cfg [paths] train = null dev = null [system] gpu_allocator = null [nlp] lang = "en" pipeline = [] batch_size = [components] [components.tok vec] factory = "tok vec" [components.tok vec.model] @architectures = "spacy.tok vec.v " [components.tok vec.model.embed] @architectures = "spacy.multihashembed.v " width = ${components.tok vec.model.encode.width} attrs = ["orth", "shape"] rows = [ , ] include_static_vectors = false [components.tok vec.model.encode] @architectures = "spacy.maxoutwindowencoder.v " width = depth = window_size = maxout_pieces = [corpora] [corpora.train] @readers = "spacy.corpus.v " path = ${paths.train} max_length = [corpora.dev] @readers = "spacy.corpus.v " path = ${paths.dev} max_length = [training] dev_corpus = "corpora.dev" train_corpus = "corpora.train" [training.optimizer] @optimizers = "adam.v " [training.batcher] @batchers = "spacy.batch_by_words.v " discard_oversize = false tolerance = . [training.batcher.size] @schedules = "compounding.v " start = stop = compound = . [initialize] vectors = null 🪐get started: pipelines/tagger_parser_ud the easiest way to get started is to clone a project template and run it – for example, this template for training a part-of-speech tagger and dependency parser on a universal dependencies treebank.$python -m spacy project clone pipelines/tagger_parser_ud end-to-end workflows from prototype to production spacy's new project system gives you a smooth path from prototype to production. it lets you keep track of all those data transformation, preprocessing and training steps, so you can make sure your project is always ready to hand over for automation. it features source asset download, command execution, checksum verification, and caching with a variety of backends and integrations. try it out in this free and interactive online course you’ll learn how to use spacy to build advanced natural language understanding systems, using both rule-based and machine learning approaches. it includes exercises featuring videos, slide decks, multiple-choice questions and interactive coding practice in the browser. start the course benchmarks spacy v . introduces transformer-based pipelines that bring spacy's accuracy right up to the current state-of-the-art. you can also use a cpu-optimized pipeline, which is less accurate but much cheaper to run. more results pipeline parser tagger ner en_core_web_trf (spacy v ) . . . en_core_web_lg (spacy v ) . . . en_core_web_lg (spacy v ) . . . full pipeline accuracy on the ontonotes . corpus (reported on the development set). named entity recognition system ontonotes conll ‘ spacy roberta ( ) . . stanza (stanfordnlp) . . flair . . named entity recognition accuracy on the ontonotes . and conll- corpora. see nlp-progress for more results. project template: benchmarks/ner_conll . . qi et al. ( ). . akbik et al. ( ). spacy usage models api reference online course community universe github discussions issue tracker stack overflow connect twitter github youtube blog stay in the loop! receive updates about new releases, tutorials and more. sign up © - explosionlegal / imprint catmandu catmandu catmandu . on may th , nicolas steenlant (our main developer and guru of catmandu) released version . of our catmandu toolkit with some very interesting new features. the main addition is a brand new way how catmandu fix-es can be implemented using the new catmandu::path implementation. this coding by nicolas will make it much easier and [&# ;] lpw : &# ;contrarian perl&# ; &# ; tom hukins at : , tom hukins shares his enthusiasm for catmandu! introducing filestores catmandu is always our tool of choice when working with structured data. using the elasticsearch or mongodb catmandu::store-s it is quite trivial to store and retrieve metadata records. storing and retrieving a yaml, json (and by extension xml, marc, csv,&# ;) files can be as easy as the commands below: $ catmandu import yaml to database [&# ;] catmandu . catmandu . has been released to with some nice new features. there are some new fix routines that were asked by our community: error the &# ;error&# ; fix stops immediately the execution of the fix script and throws an error. use this to abort the processing of a data stream: $ cat myfix.fix unless exists(id) error("no [&# ;] metadata analysis at the command-line i was last week at the elag conference in copenhagen and attended the excellent workshop by christina harlow of cornell university on migrating digital collections metadata to rdf and fedora . one of the important steps required to migrate and model data to rdf is understanding what your data is about. probably old systems need to [&# ;] catmandu . catmandu . has been released today. there has been some speed improvements processing fixes due to switching from the data::util to the ref::util package which has better a support on many perl platforms. for the command line there is now support for preprocessing fix scripts. this means, one can read in variables from the command line into [&# ;] parallel processing with catmandu in this blog post i&# ;ll show a technique to scale out your data processing with catmandu. all catmandu scripts use a single process, in a single thread. this means that if you need to process times as much data , you need times at much time. running a catmandu convert command with the [&# ;] catmandu . after years of programming, minor releases we are finally there: the release of catmandu . ! we have pushed the test coverage of the code to . % and added and cleaned a lot of our documentation. for the new features read our changes file. a few important changes should be noted. &# ; &# ; by default [&# ;] catmandu chat on friday june : cest, we&# ;ll provide a one hour introduction/demo into processing data with catmandu. if you are interested, join us on the event page: https://plus.google.com/hangouts/_/event/c jcknos egjlthk m btha o more instructions on the exact google hangout coordinates for this chat will follow on this web page at friday june : . to enter the chat session, [&# ;] matching authors against viaf identities at ghent university library we enrich catalog records with viaf identities to enhance the search experience in the catalog. when searching for all the books about &# ;chekov&# ; we want to match all name variants of this author. consult viaf http://viaf.org/viaf/ /#chekhov,_anton_pavlovich,_ - and you will see many of them. chekhov Čehov tsjechof txékhov etc any of the these names variants can be [&# ;] nukta africa – making an impact through digital storytelling services our work training events team blog contact menu services our work training events team blog contact subscribe our multimedia and data journalism courses that will help you compete in the global market read more subscribe our multimedia and data journalism courses that will help you compete in the global market read more subscribe our multimedia and data journalism courses that will help you compete in the global market read more subscribe our multimedia and data journalism courses that will help you compete in the global market read more subscribe our multimedia and data journalism courses that will help you compete in the global market read more previous next home of digital & data storytelling learn more how we help journalists realise their career dreams in tanzania in nukta africa through its flagship training program had a lot to celebrate despite the covid- impact on our lives and businesses across the read more how we help journalists realise their career dreams in tanzania in nukta africa through its flagship training program had a lot to celebrate despite the covid- impact on our lives and businesses across the read more our partners and clients mission statement as one of the fastest growing digital media companies, nukta africa aims to transform people’s lives through data and digital tools and content our vision we aim at becoming the leading and most innovative digital media and technology company in sub-saharan africa we provide training and develop digital and data-driven content to improve people’s lives. we are working with journalists, media organisations, ngos and corporates on creating impactful stories what we offer we offer digital and data storytelling trainings we believe in evidence-based journalism and digital storytelling. in our courses you will be able learn emerging techniques and tools on producing data and multimedia projects. on a data journalism course you will be trained on how to produce data-driven stories for print, online, radio and tv while on multimedia storytelling you will learn how to create engaging video, texts, audio and interactive visualisations. we offer training and continuous mentorship to individual journalists, newsrooms, ngos, and companies which want to improve their storytelling skills. our training packages includes:- data journalism courses fact checking courses multimedia storytelling narrative storytelling for advocacy see our upcoming trainings we provide analytical news content through our independent online news portal, www.nukta.co.tz , we offer fresh, analytical and data-driven news stories on business, technology, safari and education. the news stories are not only meant to inform you but also give you evidence-based analyses which can help you make decisions for your daily life, home, work and business. visit nukta.co.tz. provide advertising solutions to businesses businesses can use our news and social media platforms to advertise their products and services to our audience. display ads: you can get all from static images, text, floating banners, popups ads, flash and videos native ads. we provide space for native ads on our site sponsored content: we create impactful content about products and services by answering the so what questions to your customers success stories: we create impactful narratives for csr and success stories for ngos and corporates contact us for more advertising information via sales@nukta.co.tz. we turn complex data into human interesting visualisation has your company spent countless hours and resources collecting data which should be shared with clients and the wider public? we can help you transform this rich data into custom designed infographics with compelling stories. infographics are now a crucial part of storytelling and help to deliver a message as quick as possible to your audience. infographics can help you stand out with engaging report presentations, advertisement and documenting the impact of corporate social responsibility projects. our journey so far + journalists trained on fact checking, digital and data storytelling, multimedia storytelling + training sessions conducted for journalism students, communication professionals and other corporate workers , % online audience growth was recorded within a year in major events on data, digital storytelling and renewable energy were conducted , + articles published in of which % were data driven newly introduced courses to help journalists and communication professionals adopt digital transformation we are committed to provide high quality editorial services while abiding to the best business practices of our media industry. let’s get in touch. privacy policy data policy events training menu privacy policy data policy events training twitter instagram facebook nukta africa . all rights reserved. library hat library hat http://www.bohyunkim.net/blog/ skip to content ↓ bohyunkim.net about publications presentations cv / resume blockchain: merits, issues, and suggestions for compelling use cases jul th, by bohyun (library hat). comments are off for this post * this post was also published in acrl techconnect.*** blockchain holds a great potential for both innovation and disruption. the adoption of blockchain also poses certain risks, and those risks will need to be addressed and mitigated before blockchain becomes mainstream. a lot of people have heard of blockchain at this point. but many are unfamiliar with how this new technology exactly works and unsure about under which circumstances or on what conditions it may be useful to libraries. in this post, i will provide a brief overview of the merits and the issues of blockchain. i will also make some suggestions for compelling use cases of blockchain at the end of this post. what blockchain accomplishes blockchain is the technology that underpins a well-known decentralized cryptocurrency, bitcoin. to simply put, blockchain is a kind of distributed digital ledger on a peer-to-peer (p p) network, in which records are confirmed and encrypted. blockchain records and keeps data in the original state in a secure and tamper-proof manner[ ] by its technical implementation alone, thereby obviating the need for a third-party authority to guarantee the authenticity of the data. records in blockchain are stored in multiple ledgers in a distributed network instead of one central location. this prevents a single point of failure and secures records by protecting them from potential damage or loss. blocks in each blockchain ledger are chained to one another by the mechanism called ‘proof of work.’ (for those familiar with a version control system such as git, a blockchain ledger can be thought of as something similar to a p p hosted git repository that allows sequential commits only.[ ]) this makes records in a block immutable and irreversible, that is, tamper-proof. in areas where the authenticity and security of records is of paramount importance, such as electronic health records, digital identity authentication/authorization, digital rights management, historic materials that may be contested or challenged due to the vested interests of certain groups, and digital provenance to name a few, blockchain can lead to efficiency, convenience, and cost savings. for example, with blockchain implemented in banking, one will be able to transfer funds across different countries without going through banks.[ ] this can drastically lower the fees involved, and the transaction will take effect much more quickly, if not immediately. similarly, adopted in real estate transactions, blockchain can make the process of buying and selling a property more straightforward and efficient, saving time and money.[ ] disruptive potential of blockchain the disruptive potential of blockchain lies in its aforementioned ability to render the role of a third-party authority obsolete, which records and validates transactions and guarantees their authenticity, should a dispute arise. in this respect, blockchain can serve as an alternative trust protocol that decentralizes traditional authorities. since blockchain achieves this by public key cryptography, however, if one loses one’s own personal key to the blockchain ledger holding one’s financial or real estate asset, for example, then that will result in the permanent loss of such asset. with the third-party authority gone, there will be no institution to step in and remedy the situation. issues this is only some of the issues with blockchain. other issues include (a) interoperability between different blockchain systems, (b) scalability of blockchain at a global scale with large amount of data, (c) potential security issues such as the % attack [ ], and (d) huge energy consumption [ ] that a blockchain requires to add a block to a ledger. note that the last issue of energy consumption has both environmental and economic ramifications because it can cancel out the cost savings gained from eliminating a third-party authority and related processes and fees. challenges for wider adoption there are growing interests in blockchain among information professionals, but there are also some obstacles to those interests gaining momentum and moving further towards wider trial and adoption. one obstacle is the lack of general understanding about blockchain in a larger audience of information professionals. due to its original association with bitcoin, many mistake blockchain for cryptocurrency. another obstacle is technical. the use of blockchain requires setting up and running a node in a blockchain network, such as ethereum[ ], which may be daunting to those who are not tech-savvy. this makes a barrier to entry high to those who are not familiar with command line scripting and yet still want to try out and test how a blockchain functions. the last and most important obstacle is the lack of compelling use cases for libraries, archives, and museums. to many, blockchain is an interesting new technology. but even many blockchain enthusiasts are skeptical of its practical benefits at this point when all associated costs are considered. of course, this is not an insurmountable obstacle. the more people get familiar with blockchain, the more ways people will discover to use blockchain in the information profession that are uniquely beneficial for specific purposes. suggestions for compelling use cases of blockchain in order to determine what may make a compelling use case of blockchain, the information profession would benefit from considering the following. what kind of data/records (or the series thereof) must be stored and preserved exactly the way they were created. what kind of information is at great risk to be altered and compromised by changing circumstances. what type of interactions may need to take place between such data/records and their users.[ ] how much would be a reasonable cost for implementation. these will help connecting the potential benefits of blockchain with real-world use cases and take the information profession one step closer to its wider testing and adoption. to those further interested in blockchain and libraries, i recommend the recordings from the library . online mini-conference, “blockchain applied: impact on the information profession,” held back in june. the blockchain national forum, which is funded by imls and is to take place in san jose, ca on august th, will also be livestreamed. notes [ ] for an excellent introduction to blockchain, see “the great chain of being sure about things,” the economist, october , , https://www.economist.com/news/briefing/ -technology-behind-bitcoin-lets-people-who-do-not-know-or-trust-each-other-build-dependable. [ ] justin ramos, “blockchain: under the hood,” thoughtworks (blog), august , , https://www.thoughtworks.com/insights/blog/blockchain-under-hood. [ ] the world food programme, the food-assistance branch of the united nations, is using blockchain to increase their humanitarian aid to refugees. blockchain may possibly be used for not only financial transactions but also the identity verification for refugees. russ juskalian, “inside the jordan refugee camp that runs on blockchain,” mit technology review, april , , https://www.technologyreview.com/s/ /inside-the-jordan-refugee-camp-that-runs-on-blockchain/. [ ] joanne cleaver, “could blockchain technology transform homebuying in cook county — and beyond?,” chicago tribune, july , , http://www.chicagotribune.com/classified/realestate/ct-re- -blockchain-homebuying- -story.html. [ ] “ % attack,” investopedia, september , , https://www.investopedia.com/terms/ / -attack.asp. [ ] sherman lee, “bitcoin’s energy consumption can power an entire country — but eos is trying to fix that,” forbes, april , , https://www.forbes.com/sites/shermanlee/ / / /bitcoins-energy-consumption-can-power-an-entire-country-but-eos-is-trying-to-fix-that/# ff aa bc . [ ] osita chibuike, “how to setup an ethereum node,” the practical dev, may , , https://dev.to/legobox/how-to-setup-an-ethereum-node- a . [ ] the interaction can also be a self-executing program when certain conditions are met in a blockchain ledger. this is called a “smart contract.” see mike orcutt, “states that are passing laws to govern ‘smart contracts’ have no idea what they’re doing,” mit technology review, march , , https://www.technologyreview.com/s/ /states-that-are-passing-laws-to-govern-smart-contracts-have-no-idea-what-theyre-doing/. posted in: coding, library, technology. tagged: bitcoin · blockchain · distributed ledger technology · dlt taking diversity to the next level dec th, by bohyun (library hat). comments are off for this post ** this post was also published in acrl techconnect on dec. , .*** “building bridges in a divisive climate: diversity in libraries, archives, and museums,” panel discussion program held at the university of rhode island libraries on thursday november , . getting minorities on board i recently moderated a panel discussion program titled “building bridges in a divisive climate: diversity in libraries, archives, and museums.” participating in organizing this program was interesting experience. during the whole time, i experienced my perspective constantly shifting back and forth as (i) someone who is a woman of color in the us who experiences and deals with small and large daily acts of discrimination, (ii) an organizer/moderator trying to get as many people as possible to attend and participate, and (iii) a mid-career librarian who is trying to contribute to the group efforts to find a way to move the diversity agenda forward in a positive and inclusive way in my own institution. in the past, i have participated in multiple diversity-themed programs either as a member of the organizing committee or as an attendee and have been excited to see colleagues organize and run such programs. but when asked to write or speak about diversity myself, i always hesitated and declined. this puzzled me for a long time because i couldn’t quite pinpoint where my own resistance was coming from. i am writing about this now because i think it may shed some light on why it is often difficult to get minorities on board with diversity-related efforts. a common issue that many organizers experience is that often these diversity programs draw many allies who are already interested in working on the issue of diversity, equity, and inclusion but not necessarily a lot of those who the organizers consider to be the target audience, namely, minorities. what may be the reason? perhaps i can find a clue for the answer to this question from my own resistance regarding speaking or writing about diversity, preferring rather to be in the audience with a certain distance or as an organizer helping with logistics behind the scene. to be honest, i always harbored a level of suspicion about how much of the sudden interests in diversity is real and how much of it is simply about being on the next hot trend. trends come and go, but issues lived through many lives of those who belong to various systematically disadvantaged and marginalized groups are not trends. although i have been always enthusiastic about participating in diversity-focused programs as attendees and was happy to see diversity, equity, and inclusion discussed in articles and talks, i wasn’t ready to sell out my lived experience as part of a hot trend, a potential fad. to be clear, i am not saying that any of the diversity-related programs or events were asking speakers or authors to be a sell-out. i am only describing how things felt to me and where my own resistance was originating. i have been and am happy to see diversity discussed even as a one-time fad. better a fad than no discussion at all. one may argue that that diversity has been actively discussed for quite some time now. a few years, maybe several, or even more. some of the prominent efforts to increase diversity in librarianship i know, for example, go as far back as when oregon state university libraries sponsored two scholarships to the code lib conference, one for women and the other for minorities, which have continued from then on as the code lib diversity scholarship. but if one has lived the entire life as a member of a systematically disadvantaged group either as a woman, a person of color, a person of certain sexual orientation, a person of a certain faith, a person with a certain disability, etc., one knows better than expecting some sudden interests in diversity to change the world we live in and most of the people overnight. i admit i have been watching the diversity discussion gaining more and more traction in librarianship with growing excitement and concern at the same time. for i felt that all of what is being achieved through so many people’s efforts may get wiped out at any moment. the more momentum it accrues, i worried, the more serious backlash it may come to face. for example, it was openly stated that seeking racial/ethnic diversity is superficial and for appearance’s sake and that those who appear to belong to “team diversity” do not work as hard as those in “team mainstream.” people make this type of statements in order to create and strengthen a negative association between multiple dimensions of diversity that are all non-normative (such as race/ethnicity, religion, sexual orientation, immigration status, disability) and unfavorable value judgements (such as inferior intellectual capacity or poor work ethic). according to this kind of flawed reasoning, a tech company whose entire staff consists of twenty-something white male programmers with a college degree, may well have achieved a high level of diversity because the staff might have potentially (no matter how unlikely) substantial intellectual and personal differences in their thinking, background, and experience, and therefore their clear homogeneity is no real problem. that’s just a matter of trivial “appearance.” the motivation behind this kind of intentional misdirection is to derail current efforts towards expanding diversity, equity, and inclusion by taking people’s attention away from the real issue of systematic marginalization in our society. of course, the ultimate goal of all diversity efforts should be not the mere inclusion of minorities but enabling them to have agency as equal as the agency those privileged already possess. but note that objections are being raised against mere inclusion. anti-diversity sentiment is real, and people will try to rationalize it in any way they can. then of course, the other source of my inner resistance to speaking or writing about diversity has been the simple fact that thinking about diversity, equity, and inclusion does not take me to a happy place. it reminds me of many bad experiences accumulated over time that i would rather not revisit. this is why i admire those who have spoken and written about their lived experience as a member of a systematically discriminated and marginalized group. their contribution is a remarkably selfless one. i don’t have a clear answer to how this reflection on my own resistance against actively speaking or writing about diversity will help future organizers. but clearly, being asked to join many times had an effect since i finally did accept the invitation to moderate a panel and wrote this article. so, if you are serious about getting more minorities – whether in different religions, genders, disabilities, races, etc. – to speak or write on the issue, then invite them and be ready to do it over and over again even if they decline. don’t expect that they will trust you at the first invitation. understand that by accepting such an invitation, minorities do risk far more than non-minorities will ever do. the survey i ran for the registrants of the “building bridges in a divisive climate: diversity in libraries, archives, and museums” panel discussion program showed several respondents expressing their concern about the backlash at their workplaces that did or may result from participating in diversity efforts as a serious deterrent. if we would like to see more minorities participate in diversity efforts, we must create a safe space for everyone and take steps to deal with potential backlash that may ensue afterwards. a gentle intro or a deep dive? another issue that many organizers of diversity-focused events, programs, and initiatives struggle with is two conflicting expectations from their audience. on one hand, there are those who are familiar with diversity, equity, and inclusion issues and want to see how institutions and individuals are going to take their initial efforts to the next level. these people often come from organizations that already implemented certain pro-diversity measures such as search advocates for the hiring process. and educational programs that familiarize the staff with the topic of diversity, equity, and inclusion. on the other hand, there are still many who are not quite sure what diversity, equity, and inclusion exactly mean in a workplace or in their lives. those people would continue to benefit from a gentle introduction to things such as privilege, microaggression, and unconscious biases. the feedback surveys collected after the “building bridges in a divisive climate: diversity in libraries, archives, and museums” panel discussion program showed these two different expectations. some people responded that they deeply appreciated the personal stories shared by the panelists, noting that they did not realize how often minorities are marginalized even in one day’s time. others, however, said they would be like to hear more about actionable items and strategies that can be implemented to further advance the values of diversity, equity, and inclusion that go beyond personal stories. balancing these two different demands is a hard act for organizers. however, this is a testament to our collective achievement that more and more people are aware of the importance of continuing efforts to improve diversity, equity, and inclusion in libraries, archives, and museums. i do think that we need to continue to provide a general introduction to diversity-related issues, exposing people to everyday experience of marginalized groups such as micro-invalidation, impostor syndrome, and basic concepts like white privilege, systematic oppression, colonialism, and intersectionality. one of the comments we received via the feedback survey after our diversity panel discussion program was that the program was most relevant in that it made “having colleagues attend with me to hear what i myself have never told them” possible. general programs and events can be an excellent gateway to more open and less guarded discussion. at the same time, it seems to be high time for us in libraries, museums, and archives to take a deep dive into different realms of diversity, equity, and inclusion as well. diversity comes in many dimensions such as age, disability, religion, sexual orientation, race/ethnicity, and socioeconomic status. many of us feel more strongly about one issue than others. we should create opportunities for ourselves to advocate for specific diversity issues that we care most. the only thing i would emphasize is that one specific dimension of diversity should not be used as an excuse to neglect others. exploring socioeconomic inequality issues without addressing how they work combined with the systematic oppression of marginalized groups such as native americans, women, or immigrants at the same time can be an example of such a case. all dimensions of diversity are closely knitted with one another, and they do not exist independently. for this reason, a deep dive into different realms of diversity, equity, and inclusion must be accompanied by the strong awareness of their intersectionality. recommendations and resources for future organizers organizing a diversity-focused program takes a lot of effort. while planning the “building bridges in a divisive climate: diversity in libraries, archives, and museums” panel discussion program at the university of rhode island libraries, i worked closely with my library dean, karim boughida, who originally came up with the idea of having a panel discussion program at the university of rhode island libraries, and renee neely in the libraries’ diversity initiatives for approximately two months. for panelists, we decided to recruit as many minorities from diverse institutions and backgrounds. we were fortunate to find panelists from a museum, an archive, both a public and an academic library with varying degrees of experience in the field from only a few years to over twenty-five years, ranging from a relatively new archivist to an experienced museum and a library director. our panel consisted of one-hundred percent people of color. the thoughts and perspectives that those panelists shared were, as a result, remarkably diverse and insightful. for this reason, i recommend spending some time to get the right speakers for your program if your program will have speakers. discussion at the “building bridges in a divisive climate: diversity in libraries, archives, and museums,” at the university of rhode island libraries another thing i would like to share is the questions that i created for the panel discussion. even though we had a whole hour, i was able to cover only several of them. but since i discussed all these questions in advance with the panelists and they helped me put a final touch on some of those, i think these questions can be useful to future organizers who may want to run a similar program. they can be utilized for a panel discussion, an unconference, or other types of programs. i hope this is helpful and save time for other organizers. sample questions for the diversity panel discussion why should libraries, archives, museums pay attention to the issues related to diversity, equity, and inclusion? in what ways do you think the lack of diversity in our profession affects the perception of libraries, museums, and archives in the communities we serve? do you have any personal or work-related stories that you would like to share that relate to diversity, equity, and inclusion issues? how did you get interested in diversity, equity, and inclusion issues? suppose you discovered that your library’s, archive’s or museum’s collection includes prejudiced information, controversial objects/ documents, or hate-inducing material. what would you do? suppose a group of your library / archive / museum patrons want to use your space to hold a local gathering that involves hate speech. what would you do? what would you be mostly concerned about, and what would the things that you would consider to make a decision on how you will respond? do you think libraries, archives, and museums are a neutral place? what do you think neutrality means to a library, an archive, a museum in practice in a divisive climate such as now? what are some of the areas in libraries, museums, and archives where you see privileges and marginalization function as a barrier to achieving our professional values – equal access and critical thinking? what can we do to remove those barriers? could you tell us how colonialist thinking and practice are affecting libraries, museums, and archives either consciously or unconsciously? since not everyone is familiar with what colonialism is, please begin with first your brief interpretation of what colonialist thinking or practice look like in libraries, museums, and archives first? what do you think libraries, archives, and museums can do more to improve critical thinking in the community that we serve? although libraries, archives, museums have been making efforts to recruit, hire, and retain diverse personnel in recent years, the success rate has been relatively low. for example, in librarianship, it has been reported that often those hired through these efforts experienced backlash at their own institutions, were subject to unrealistic expectations, and met with unsupportive environment, which led to burnout and a low retention rate of talented people. from your perspective – either as a manager hiring people or a relatively new librarian who looked for jobs – what do you think can be done to improve this type of unfortunate situation? many in our profession express their hesitation to actively participate in diversity, equity, and inclusion-related discussion and initiatives at their institutions because of the backlash from their own coworkers. what do you think we can do to minimize such backlash? some people in our profession express strong negative feelings regarding diversity, equity, and inclusion-related initiatives. how much of this type of anti-diversity sentiment do you think exist in your field? some worry that this is even growing faster in the current divisive and intolerant climate. what do you think we can do to counter such anti-diversity sentiment? there are many who are resistant to the values of diversity, equity, and inclusion. have you taken any action to promote and advance these values facing such resistance? if so, what was your experience like, and what would be some of the strategies you may recommend to others working with those people? many people in our profession want to take our diversity, equity, and inclusion initiatives to the next level, beyond offering mere lip service or simply playing a numbers game for statistics purpose. what do you think that next level may be? lastly, i felt strongly about ensuring that the terms and concepts often thrown out in diversity/equity/inclusion-related programs and events – such as intersectionality, white privilege, microaggression, patriarchy, colonialism, and so on – are not used to unintentionally alienate those who are unfamiliar with them. these concepts are useful and convenient shortcuts that allow us to communicate a large set of ideas previously discussed and digested, so that we can move our discussion forward more efficiently. they should not make people feel uncomfortable nor generate any hint of superiority or inferiority. to this end, i create a pre-program survey which all program registrants were encouraged to take. my survey simply asked people how familiar and how comfortable they are with a variety of terms. at the panel discussion program, we also distributed the glossary of these terms, so that people can all become familiar with them. also, videos can quickly bring all attendees up-to-speed with some basic concepts and phenomena in diversity discussion. for example, in the beginning of our panel discussion program, i played two short videos, “life of privilege explained in a $ race” and “what if we treated white coworkers the way we treat minority coworkers?”, which were well received by the attendees. i am sharing the survey questions, the video links, and the glossary in the hope that they may be helpful as a useful tool for future organizers. for example, one may decide to provide a glossary like this before the program or run an unconference that aims at unpacking the meanings of these terms and discussing how they relate to people’s daily lives. in closing: diversity, libraries, technology, and our own biases disagreements on social issues are natural. but the divisiveness that we are currently experiencing seems to be particularly intense. this deeply concerns us, educators and professionals working in libraries, archives, and museums. libraries, archives, and museums are public institutions dedicated to promoting and advancing civic values. diversity, equity, and inclusion are part of those core civic values that move our society forward. this task, however, has become increasingly challenging as our society moves in a more and more divisive direction. to make matters even more complicated, libraries, archives, museums in general lack diversity in their staff composition. this homogeneity can impede achieving our own mission. according to the recent report from ithaka s+r released this august, we do not appear to have gotten very far. their report “inclusion, diversity, and equity: members of the association of research (arl) libraries – employee demographics and director perspectives,” shows that libraries and library leadership/administration are both markedly white-dominant ( % and % white non-hispanic respectively). also, while librarianship in general are female dominant ( %), the technology field in libraries is starkly male ( %) along with makerspace ( %), facilities ( %), and security ( %) positions. the survey results in the report show that while the majority of library directors say there are barriers to achieving more diversity in their library, they attribute those barriers to external rather than internal factors such as the library’s geographic location and the insufficiently diverse application pool resulting from the library’s location. what is fascinating, however, is that this directly conflicts with the fact that libraries do show little variation in the ratio of white staff based on degree of urbanization. equally interesting is that the staff in more homogeneous and less diverse (over % white non-hispanic) libraries think that their libraries are much more equitable than the library community ( % vs %) and that library directors (and staff) consider their own library to be more equitable, diverse, and inclusive than the library community with respect to almost every category such as race/ethnicity, gender, lgbtq, disabilities, veterans, and religion. while these findings in the ithaka s+r report are based upon the survey results from arl libraries, similar staff composition and attitudes can be assumed to apply to libraries in general. there is a great need for both the library administration and the staff to understand their own unconscious and implicit biases, workplace norms, and organizational culture that may well be thwarting their own diversity efforts. diversity, equity, and inclusion have certainly been a topic of active discussion in the recent years. many libraries have established a committee or a task force dedicated to improving diversity. but how are those efforts paying out? are they going beyond simply paying a lip service? is it making a real difference to everyday experience of minority library workers? can we improve, and if so where and how? where do we go from here? those would be the questions that we will need to examine in order to take our diversity efforts in libraries, archives, and museums to the next level. notes the program description is available at https://web.uri.edu/library/ / / /building-bridges-in-a-divisive-climate-diversity-in-libraries-archives-and-museums/ ↩ carol bean, ranti junus, and deborah mouw, “conference report: code libcon ,” the code lib journal, no. (march , ), http://journal.code lib.org/articles/ . ↩ note that this kind of biased assertions often masquerades itself as an objective intellectual pursuit in academia when in reality, it is a direct manifestation of an existing prejudice reflecting the limited and shallow experience of the person posting the question him/herself. a good example of this is found in the remark in made by larry summers, the former harvard president. he suggested that one reason for relatively few women in top positions in science may be “issues of intrinsic aptitude” rather than widespread indisputable everyday discrimination against women. he resigned after the harvard faculty of arts and sciences cast a vote of no confidence. see scott jaschik, “what larry summers said,” inside higher ed, february , , https://www.insidehighered.com/news/ / / /summers _ . ↩ our pre-program survey questions can be viewed at https://docs.google.com/forms/d/e/ faipqlscp-nqnkhaqli_ pvdidw-dqzraflycdikutu dzjqm f ra/viewform. ↩ for this purpose, asking all participants to respect one another’s privacy in advance can be a good policy. in addition to this, we specifically decided not to stream or record our panel discussion program, so that both panelists and attendees can freely share their experience and thoughts. ↩ a good example is the search advocate program from oregon state university. see http://searchadvocate.oregonstate.edu/. ↩ for an example, see the workshops offered by the office of community, equity, and inclusion of the university of rhode island at https://web.uri.edu/diversity/ced-inclusion-courses-overview/. ↩ for the limitations of the mainstream diversity discussion in lis (library and information science) with the focus on inclusion and cultural competency, see david james hudson, “on ‘diversity’ as anti-racism in library and information studies: a critique,” journal of critical library and information studies , no. (january , ), https://doi.org/https://doi.org/ . /jclis.v i . . ↩ you can see our glossary at https://drive.google.com/file/d/ uci huuytrelgny-dbnsoxf_ilpm n/view?usp=sharing; this glossary was put together by renee neely. ↩ for the nitty-gritty logistical details for organizing a large event with a group of local and remote volunteers, check the organizer’s toolkit created by the #critlib unconference organizers at https://critlib .wordpress.com/organizers-toolkit/. ↩ roger schonfeld and liam sweeney, “inclusion, diversity, and equity: members of the association of research libraries,” ithaka s+r, august , , http://www.sr.ithaka.org/publications/inclusion-diversity-and-equity-arl/. ↩ for the early discussion of diversity-focused recruitment in library technology, see jim hahn, “diversity recruitment in library information technology,” acrl techconnect blog, august , , https://acrl.ala.org/techconnect/post/diversity-recruitment-in-library-information-technology. ↩ see april hathcock, “white librarianship in blackface: diversity initiatives in lis,” in the library with the lead pipe, october , , http://www.inthelibrarywiththeleadpipe.org/ /lis-diversity/ and angela galvan, “soliciting performance, hiding bias: whiteness and librarianship,” in the library with the lead pipe (blog), june , , http://www.inthelibrarywiththeleadpipe.org/ /soliciting-performance-hiding-bias-whiteness-and-librarianship. ↩ posted in: diversity. tagged: equity · inclusion · resources from need to want: how to maximize social impact for libraries, archives, and museums oct th, by bohyun (library hat). comments are off for this post at the ndp at three event organized by imls yesterday, sayeed choudhury on the “open scholarly communications” panel suggested that libraries think about return on impact in addition to return on investment (roi). he further elaborated on this point by proposing a possible description of such impact. his description was that when an object or resource created through scholarly communication efforts is being used by someone we don’t know and is interpreted correctly without contacting us (=libraries, archives, museums etc.), that is an impact; to push that further, if someone uses the object or the resource in a way we didn’t anticipate, that’s an impact; if it is integrated into someone’s workflow, that’s also an impact. this emphasis on impact as a goal for libraries, archives, and museums (or non-profit organizations in general to apply broadly) resonated with me particularly because i gave a talk just a few days ago to a group of librarians at the iolug conference about how libraries can and should maximize their social impact in the context of innovation in the way many social entrepreneurs have been already doing for quite some time. in this post, i would like to revisit one point that i made in that talk. it is a specific interpretation of the idea of maximizing social impact as a conscious goal for libraries, archives, and museums (lam). hopefully, this will provide a useful heuristic for lam institutions in mapping out the future efforts. considering that roi is a measure of cost-effectiveness, i believe impact is a much better goal than roi for lam institutions. we often think that to collect, organize, provide equitable access to, and preserve information, knowledge, and cultural heritage is the goal of a library, an archive, and a museum. but doing that well doesn’t mean simply doing it cost-effectively. our efforts no doubt aim at achieving better-collected, better-organized, better-accessed, and better-preserved information, knowledge, and cultural heritage. however, our ultimate end-goal is attained only when such information, knowledge, and cultural heritage is better used by our users. not simply better accessed, but better used in the sense that the person gets to leverage such information, knowledge, and cultural heritage to succeed in whatever endeavor that s/he was making, whether it be career success, advanced education, personal fulfillment, or private business growth. in my opinion, that’s the true impact that lam institutions should aim at. if that kind of impact were a destination, cost-effectiveness is simply one mode of transportation, preferred one maybe but not quite comparable to the destination in terms of importance. but what does “better used” exactly mean? “integrated into people’s workflow” is a hint; “unanticipated use” is another clue. if you are like me and need to create and design that kind of integrated or unanticipated use at your library, archive, or museum, how will you go about that? this is the same question we ask over and over again. how do you plan and implement innovation? yes, we will go talk to our users, ask what they would like to see, meet with our stakeholders and find out their interests and concerns are, discuss ourselves what we can do to deliver things that our users want, and go from there to another wonderful project we work hard for. then after all that, we reach a stage where we stop and wonder where that “greater social impact” went in almost all our projects. and we frantically look for numbers. how many people accessed what we created? how many downloads? what does the satisfaction survey say? in those moments, how does the “impact” verbiage help us? how does that help us in charting our actual path to creating and maximizing our social impact more than the old-fashioned “roi” verbiage? at least roi is quantifiable and measurable. this, i believe, is why we need a more concrete heuristic to translate the lofty “impact” to everyday “actions” we can take. maybe not quite as specific as to dictate what exactly those actions are at each project level but a bit more specific to enable us to frame the value we are attempting to create and deliver at our lam institutions beyond cost-effectiveness. i think the heuristic we need is the conversion of need to demand. what is an untapped need that people are not even aware of in the realm of information, knowledge, and cultural heritage? when we can identify any such need in a specific form and successfully convert that need to a demand, we make an impact. by “demand,” i mean the kind of user experience that people will desire and subsequently fulfill by using that object, resource, tool, service, etc., we create at our library, archive, and museum. (one good example of such desirable ux that comes to my mind is nypl photo booth: https://www.nypl.org/blog/ / / /snapshots-nypl.) when we create a demand out of such an untapped need, when the fulfillment of that kind of demand effectively creates, strengthens, and enriches our society in the direction of information, knowledge, evidence-based decisions, and truth being more valued, promoted, and equitably shared, i think we get to maximize our social impact. in the last “going forward” panel where the information discovery was discussed, loretta parham pointed out that in the corporate sector, information finds consumers, not the other way. by contrast, we (by which i mean all of us working at lam institutions) still frame our value in terms of helping and supporting users access and use our material, resources, and physical and digital objects and tools. this is a mistake in my opinion, because it is a self-limiting value proposition for libraries, archives, and museums. what is the point of us lam institutions, working so hard to get the public to use their resources and services? the end goal is so that we can maximize our social impact through such use. the rhetoric of “helping and supporting people to access and use our resources” does not adequately convey that. businesses want their clients to use their goods and services, of course. but their real target is the making of profit out of those uses, aka purchases. similarly, but far more importantly, the real goal of libraries, archives and museums is to move the society forward, closer in the direction of knowledge, evidence-based decisions, and truth being more valued, promoted, and equitably shared. one person at a time, yes, but the ultimate goal reaching far beyond individuals. the end goal is maximizing our impact on this side of the public good. posted in: librarianship, library, management, usability, user experience. tagged: archives · change · d d · design thinking · digital collection · goal · impact · innovation · libraries · museums · ndpthree · social entrepreneurship · ux how to price d printing service fees may nd, by bohyun (library hat). comments are off for this post ** this post was originally published in acrl techconnect on may. , .*** many libraries today provide d printing service. but not all of them can afford to do so for free. while free d printing may be ideal, it can jeopardize the sustainability of the service over time. nevertheless, many libraries tend to worry about charging service fees. in this post, i will outline how i determined the pricing schema for our library’s new d printing service in the hope that more libraries will consider offering d printing service if having to charge the fee is a factor stopping them. but let me begin with libraries’ general aversion to fees. a d printer in action at the health sciences and human services library (hs/hsl), univ. of maryland, baltimore service fees are not your enemy charging fees for the library’s service is not something librarians should regard as a taboo. we live in the times in which a library is being asked to create and provide more and more new and innovative services to help users successfully navigate the fast-changing information landscape. a makerspace and d printing are certainly one of those new and innovative services. but at many libraries, the operating budget is shrinking rather than increasing. so, the most obvious choice in this situation is to aim for cost-recovery. it is to be remembered that even when a library aims for cost-recovery, it will be only partial cost-recovery because there is a lot of staff time and expertise that is spent on planning and operating such new services. libraries should not be afraid to introduce new services requiring service fees because users will still benefit from those services often much more greatly than a commercial equivalent (if any). think of service fees as your friend. without them, you won’t be able to introduce and continue to provide a service that your users need. it is a business cost to be expected, and libraries will not make profit out of it (even if they try). still bothered? almost every library charges for regular (paper) printing. should a library rather not provide printing service because it cannot be offered for free? library users certainly wouldn’t want that. determining your service fees what do you need in order to create a pricing scheme for your library’s d printing service? (a) first, you need to list all cost-incurring factors. those include (i) the equipment cost and wear and tear, (ii) electricity, (iii) staff time & expertise for support and maintenance, and (iv) any consumables such as d print filament, painter’s tape. remember that your new d printer will not last forever and will need to be replaced by a new one in - years. also, some of these cost-incurring factors such as staff time and expertise for support is fixed per d print job. on the other hand, another cost-incurring factor, d print filament, for example, is a cost factor that increases in proportion to the size/density of a d model that is printed. that is, the larger and denser a d print model is, the more filament will be used incurring more cost. (b) second, make sure that your pricing scheme is readily understood by users. does it quickly give users a rough idea of the cost before their d print job begins? an obscure pricing scheme can confuse users and may deter them from trying out a new service. that would be bad user experience. also in d printing, consider if you will also charge for a failed print. perhaps you do. perhaps you don’t. maybe you want to charge a fee that is lower than a successful print. whichever one you decide on, have that covered since failed prints will certainly happen. (c) lastly, the pricing scheme should be easily handled by the library staff. the more library staff will be involved in the entire process of a library patron using the d printing service from the beginning to the end, the more important this becomes. if the pricing scheme is difficult for the staff to work with when they need charge for and process each d print job, the new d printing service will increase their workload significantly. which staff will be responsible for which step of the new service? what would be the exact tasks that the staff will need to do? for example, it may be that several staff at the circulation desk need to learn and handle new tasks involving the d printing service, such as labeling and putting away completed d models, processing the payment transaction, delivering the model, and marking the job status for the paid d print job as ‘completed’ in the d printing staff admin portal if there is such a system in place. below is the screenshot of the hs/hsl d printing staff admin portal developed in-house by the library it team. the hs/hsl d printing staff admin portal, university of maryland, baltimore examples – d printing service fees it’s always helpful to see how other libraries are doing when you need to determine your own pricing scheme. here are some examples that shows ten libraries’ d printing pricing scheme changed over the recent three years. unr delamare library https://guides.library.unr.edu/ dprinting – $ . per cubic inch of modeling material (raised to $ . starting july, ). – uprint – model material: $ . per cubic inch (= . gm= . lb) – uprint – support materials: $ . per cubic inch ncsu hunt library https://www.lib.ncsu.edu/do/ d-printing - uprint d printer: $ per cubic inch of material (abs), with a $ minimum – makerbot d printer: $ . per gram of material (pla), with a $ minimum – uprint – $ per cubic inch of material, $ minimum – f – $ . per gram of material, $ minimum southern illinois university library http://libguides.siue.edu/ d/request – originally $ per hour of printing time; reduced to $ as the demand grew. – lulzbot taz , luzbot mini – $ . per hour of printing time. byu library http://guides.lib.byu.edu/c.php?g= &p= – – makerbot replicator / ultimaker extended $ . per gram for standard ( . mm) resolution; $ . per gram for high ( . mm) resolution. university of michigan library the cube d printer checkout is no longer offered. – cost for professional d printing service; open access d printing is free. gvsu library https://www.gvsu.edu/techshowcase/makerspace- .htm – $ . per gram with a $ . minimum – free (ultimaker +, makerbot replicator , , x) university of tennessee, chattanooga library http://www.utc.edu/library/services/studio/ d-printing/index.php – – makerbot th, th – $ . per gram port washington public library http://www.pwpl.org/ d-printing/ d-printing-guidelines/ – makerbot – $ per hour of printing time miami university – $ . per gram of the finished print; – ? ucla library, dalhousie university library ( ) free types of d printing service fees from the examples above, you will notice that many d printing service fee schemes are based upon the weight of a d-print model. this is because these libraries are trying recover the cost of the d filament, and the amount of filament used is most accurately reflected in the weight of the resulting d-printed model. however, there are a few problems with the weight-based d printing pricing scheme. first, it is not readily calculable by a user before the print job, because to do so, the user will have to weigh a model that s/he won’t have until it is d-printed. also, once d-printed, the staff will have to weigh each model and calculate the cost. this is time-consuming and not very efficient. for this reason, my library considered an alternative pricing scheme based on the size of a d model. the idea was that we will have roughly three different sizes of an empty box – small, medium, and large – with three different prices assigned. whichever box into which a user’s d printed object fits will determine how much the user will pay for her/his d-printed model. this seemed like a great idea because it is easy to determine how much a model will cost to d-print to both users and the library staff in comparison to the weight-based pricing scheme. unfortunately, this size-based pricing scheme has a few significant flaws. a smaller model may use more filament than a larger model if it is denser (meaning the higher infill ratio). second, depending on the shape of a model, a model that fits in a large box may use much less filament than the one that fits in a small box. think about a large tree model with think branches. then compare that with a % filled compact baseball model that fits into a smaller box than the tree model does. thirdly, the resolution that determines a layer height may change the amount of filament used even if what is d-printed is a same model. different infill ratios – image from https://www.packtpub.com/sites/default/files/article-images/ os_ _ .png charging based upon the d printing time so we couldn’t go with the size-based pricing scheme. but we did not like the problems of the weight-based pricing scheme, either. as an alternative, we decided to go with the time-based pricing scheme because printing time is proportionate to how much filament is used, but it does not require that the staff weigh the model each time. a d-printing software gives an estimate of the printing time, and most d printers also display actual printing time for each model printed. first, we wanted to confirm the hypothesis that d printing time and the weight of the resulting model are proportionate to each other. i tested this by translating the weight-based cost to the time-based cost based upon the estimated printing time and the estimated weight of several cube models. here is the result i got using the makerbot replicator x. . gm/ min= . gm per min. . gm/ min= . gm per min. . gm/ min= . gm per min. . gm/ min= . gm per min. . gm/ min= . gm per min. . gm/ min= . gm per min. there is some variance, but the hypothesis holds up. based upon this, now let’s calculate the d printing cost by time. d plastic filament is $ for abs/pla and $ for the dissolvable per . kg (= . lb) from makerbot. that means that filament cost is $ . per gram for abs/pla and $ . per gram for the dissolvable. so, d filament cost is cents per gram on average. finalizing the service fee for d printing for an hour of d printing time, the amount of filament used would be . gm (= . x min). this gives us the filament cost of cents per hour of d printing (= . gm x cents). so, for the cost-recovery of filament only, i get roughly $ per hour of d printing time. earlier, i mentioned that filament is only one of the cost-incurring factors for the d printing service. it’s time to bring in those other factors, such as hardware wear/tear, staff time, electricity, maintenance, etc., plus “no-charge-for-failed-print-policy,” which was adopted at our library. those other factors will add an additional amount per d print job. and at my library, this came out to be about $ . (i will not go into details about how these have been determined because those will differ at each library.) so, the final service fee for our new d printing service was set to be $ up to hour of d printing + $ per additional hour of d printing. the $ is broken down to $ per hour of d printing that accounts for the filament cost and $ fixed cost for every d print job. to help our users to quickly get an idea of how much their d print job will cost, we have added a feature to the hs/hsl d print job submission form online. this feature automatically calculates and displays the final cost based upon the printing time estimate that a user enters. the hs/hsl d print job submission form, university of maryland, baltimore don’t be afraid of service fees i would like to emphasize that libraries should not be afraid to set service fees for new services. as long as they are easy to understand and the staff can explain the reasons behind those service fees, they should not be a deterrent to a library trying to introduce and provide a new innovative service. there is a clear benefit in running through all cost-incurring factors and communicating how the final pricing scheme was determined (including the verification of the hypothesis that d printing time and the weight of the resulting model are proportionate to each other) to all library staff who will be involved in the new d printing service. if any library user inquire about or challenges the service fee, the staff will be able to provide a reasonable explanation on the spot. i implemented this pricing scheme at the same time as the launch of my library’s makerspace (the hs/hsl innovation space at the university of maryland, baltimore – http://www.hshsl.umaryland.edu/services/ispace/) back in april . we have been providing d printing service and charging for it for more than two years. i am happy to report that during that entire duration, we have not received any complaint about the service fee. no library user expected our new d printing service to be free, and all comments that we received regarding the service fee were positive. many expressed a surprise at how cheap our d printing service is and thanked us for it. to summarize, libraries should be willing to explore and offer new innovating services even when they require charging service fees. and if you do so, make sure that the resulting pricing scheme for the new service is (a) sustainable and accountable, (b) readily graspable by users, and (c) easily handled by the library staff who will handle the payment transaction. good luck and happy d printing at your library! an example model with the d printing cost and the filament info displayed at the hs/hsl, university of maryland, baltimore posted in: library, management, technology, user experience. tagged: d printer · d printing · budget · charge · cost · funding · makerspace · service fees · sustainability · user experience · ux post-election statements and messages that reaffirm diversity nov th, by bohyun (library hat). comments are off for this post these are statements and messages sent out publicly or internally to re-affirm diversity, equity, and inclusion by libraries or higher ed institutions. i have collected these – some myself and many others through my fellow librarians. some of them were listed on my blog post, “finding the right words in post-election libraries and higher ed.” so there are some duplicates. if you think that your organization is already so much pro-diversity that there is no need to confirm or re-affirm diversity, you can’t be farther from the everyday reality that minorities experience. sometimes, saying isn’t much. but right now, saying it out loud can mean everything. if you support those who belong to minority groups but don’t say it out loud, how would they know it? right now, nothing is obvious other than there is a lot of hate and violence towards minorities. feel free to use these as your resource to craft a similar message. feel free to add if you have similar messages you have received or created in the comments section. if you haven’t heard from the organization you belong to, please ask for a message reaffirming and committing to diversity, equity, and inclusion. [update / / : statements from ala and lita have been released. i have added them below.] i will continue to add additional statements as i find them. if you see anything missing, please add below in the comment or send it via twitter @bohyunkim. thanks! from librarians but i know that there will be libraries librarian zoe fisher to other librarians care for one another director chris bourg to the mit libraries staff finding the right words in post-election libraries and higher ed (my e-mail sent to the it team at university of maryland, baltimore health sciences and human services library) with a a pin and a prayer dean k. g. schneider to the sonoma state university library staff from library associations lita ala pla arl dlf code lib [draft in github] from libraries james madison university libraries northwestern university libraries university of oregon libraries from higher ed institutions clarke university cuny duke universitymit loyola university, maryland northwestern university penn state university the catholic university of america university of california university of michigan university of nebraska, lincoln university of nevada, reno university of oregon university of rochester and rochester institute of technology university of florida addressing racially charged flyers on the campus marshall university president jerome a. gilbert’s statement regarding post-election tweet drexel university moving on as a community after the election dear members of the drexel community, it is heartening to me to see the drexel community come together over the last day to digest the news of the presidential election — and to do so in the spirit of support and caring that is so much a part of this university. we gathered family-style, meeting in small, informal groups in several places across campus, including the student center for inclusion and culture, our residence halls, and as colleagues over a cup of coffee. many student leaders, particularly from our multicultural organizations, joined the conversation. this is not a process that can be completed in just one day, of course. so i hope these conversations will continue as long as students, faculty and professional staff feel they are needed, and i want to assure you that our professional staff in student life, human resources, faculty affairs, as well as our colleagues in the lindy center for civic engagement, will be there for your support. without question, many members of our community were deeply concerned by the inflammatory rhetoric and hostility on the campaign trail that too often typified this bitter election season. as i wrote over the summer, the best response to an uncertain and at times deeply troubling world is to remain true to our values as an academic community. in the context of a presidential election, it is vital that we understand and respect that members of our broadly diverse campus can hold similarly diverse political views. the expression of these views is a fundamental element of the free exchange of ideas and intellectual inquiry that makes drexel such a vibrant institution. at the same time, drexel remains committed to ensuring a welcoming, inclusive, and respectful environment. those tenets are more important than ever. while we continue to follow changes on the national scene, it is the responsibility of each of us at drexel to join together to move ahead, unified in our commitment to open dialogue, civic engagement and inclusion. i am grateful for all you do to support drexel as a community that welcomes and encourages all of its members. lane community college good morning, colleagues, i am in our nation’s capital today. i’d rather be at home! like me, i am guessing that many of you were glued to the media last night to find out the results of the election. though we know who our next president will be, this transition still presents a lot of uncertainty. it is not clear what our future president’s higher education policies will be but we will be working with our national associations to understand and influence where we can. during times like this there is an opening for us to decide how we want to be with each other. moods will range from joy to sadness and disbelief. it seems trite but we do need to work together, now more than ever. as educators we have a unique responsibility to create safe learning environments where every student can learn and become empowered workers and informed citizens. this imperative seems even more important today. our college values of equity and inclusion have not changed and will not change and it is up to each of us to assure that we live out our values in every classroom and in each interaction. preparing ourselves and our students for contentious discussions sparked by the election is work we must do. it is quite likely that some of our faculty, staff and students may be feeling particularly vulnerable right now. can we reach out to each other and let each other know that we all belong at lane? during my inservice remarks i said that “we must robustly reject the calculated narrative of cynicism, division and despair. instead of letting this leak into our narratives, together we can bet on hope not fear, respect not hate, unity not division.” at lane we have the intellect (and proud of it) and wherewithal to do this. i am attaching a favorite reading from meg wheatley which is resonating with me today and will end with gary snyder’s words from to the children …..stay together learn the flowers go light. maryland institute college of art post-election community forums and support dear campus community, no matter how each of us voted yesterday, most of us likely agree that the presidential campaign has been polarizing on multiple fronts. as a result, today is a difficult day for our nation and our campus community. in our nation, regardless of how one has aligned with a candidate, half of our country feels empowered and the other half sad and perhaps angry. because such dynamics and feelings need to be addressed and supported on campus, this memo outlines immediate resources for our community of students, faculty and staff, and describes opportunities for fashioning dialogues and creative actions going forward. before sharing the specifics, let me say unambiguously that mica will always stand firm in our commitment to diversity and inclusion. this morning’s presidential task force on diversity, inclusion, equity, and globalization meeting discussed measures to ensure that, as a creative community, we will continue to build a culture where everyone is honored and supported for success. the impact of exhibitions such as the current baltimore rising show remains as critical as ever, and mica fosters an educational environment that is welcoming of all. in the short term our focus is to support one another. whether you are happy or distressed with the results, there has been sufficient feedback to indicate that our campus community is struggling with how to make sense of such a divisive election process. you may find the following services helpful and are encouraged to take advantage of them: for students: student counseling maintains walk-in hours from : – : pm every day. students are welcome to stop by the student counseling center ( mt. royal avenue) during that time or call - - and enter x once the recording begins to schedule an appointment. for faculty and staff: the employee assistance program (eap) is available to provide free, confidential support hours a day. the eap can be reached by calling - - - or visiting healthadvocate.com/members and providing the username “maryland institute college of art”. for all mica community members: mica’s chaplain, the rev, maintains standing hours every monday and can be reached in the reflection room (meyerhoff house) or by calling the office of diversity and intercultural development at - - . there are three events this week that can provide a shared space for dialogue; all are welcome: the “after the baltimore uprising: still waiting for change” community forum attached to the baltimore rising exhibition takes place tonight from : pm to : pm in the lazarus center. an open space for all mica community members will be hosted by the black student union tonight at : pm in the meyerhoff house underground. in partnership with our student nami group, mica will host a “messages of hope” event for the entire mica community that will allow for shared space and reflection. this event will be on friday, november th, and will begin at : pm in cohen plaza. in various upcoming meetings we look forward to exploring with campus members other appropriate activities that can be created to facilitate expressions and dialogues. a separate communication is coming from provost david bogen to the faculty regarding classroom conversations with students regarding the election. northwestern university women’s center dear northwestern students, faculty, staff and community members: the women’s center is open today. our staff members are all here and available to talk, to provide resources and tools, or to help however you might need it. most importantly, the space itself is available for whatever you need, whether that is to gather as a group, to sit alone somewhere comfortable and quiet, or to talk to someone who will listen. we are still here, and we are here for all people as an intentionally intersectional space. you are welcome to drop by physically, make a call to our office, or send an email. know that this space is open and available to you. portland community college to the pcc staff as someone who spent the last several years in washington d.c. working to advance community colleges, i feel a special poignancy today hearing so many students, colleagues, and friends wonder and worry about the future—and about their futures. we must acknowledge that this political season has highlighted deep divisions in our society. today i spent time with cabinet speaking about how we can assert our shared values and take positive action as a pcc community to deepen our commitment to equity, inclusion and civic engagement. pcc will always welcome students and colleagues who bring a rich array of perspectives and experiences. that diversity is among our greatest strengths. today it is imperative that we stand by faculty, staff and students who may be experiencing fear or uncertainty—affirming with our words and deeds that pcc is about equitable student success and educational opportunity for all. never has this mission been more powerful or more essential. i have only been here a few months, but have already learned that pcc is a remarkable and caring community. much is happening right now in real time, and i appreciate the efforts of all. for my part, i promise to communicate often as we continue to plan for our shared future. p.s. today and in the days ahead, we will be holding space for people to be together in community. here are a few of the opportunities identified so far. portland community college to students dear students: as someone who spent the last several years working in washington d.c., i feel a special poignancy this week hearing many of you express worry and uncertainty about the future. there is little doubt that this political season has highlighted some deep divisions in our society. both political candidates have acknowledged as much. at the same time, people representing the full and diverse spectrum of our country come to our nation’s community colleges in hopes of a better life. pcc is such a place – where every year thousands of students find their path and pursue their dreams. all should find opportunity here, and all should feel safe and welcome. the rich diversity of pcc offers an amazing opportunity for dialogue across difference, and for developing skills that are the foundation of our democratic society. let this moment renew your passion for making a better life for yourself, your community and your country and for becoming the kind of leader you want to follow. rutgers university aaup-aft (american association of university professors – american federation of teachers) resisting donald trump we are shocked and horrified that donald trump, who ran on a racist, xenophobic, misogynist platform, is now the president of the us. in response to this new political landscape, the administrative heads of several universities have issued statements embracing their diverse student, faculty, and staff bodies and offering support and protection. (see statements from the university of california and the california state university). president barchi has yet to address the danger to the rutgers community and its core mission. this afternoon, our faculty union and the rutgers one coalition held an emergency meeting of students, faculty, and community activists in new brunswick. we discussed means of responding to the attacks that people may experience in the near future. most immediately, we approved the following statement by acclamation at the -strong meeting: “rutgers one, a coalition of faculty, staff, students and community members, calls upon the rutgers administration to join us in condemning all acts of bigotry on this campus and refuse to tolerate any attacks on immigrants, women, arabs, muslims, people of color, lgbtq people and all others in our diverse community. we demand that president barchi and his administration provide sanctuary, support, and protection to those who are already facing attacks on our campuses. we need concrete action that can ensure a safe environment for all. further, we commit ourselves to take action against all attempts by the trump administration to target any of our students, staff or faculty. we are united in resistance to bigotry of every kind and welcome all to join us in solidarity.” we also resolved to take the following steps: we will be holding weekly friday meetings at pm in our union office in new brunswick to bring together students, faculty and staff to organize against the trump agenda. we hope to expand these to camden and newark as well. (if you are willing to help organize this, please email back.) we will be creating a list serve to coordinate our work. if you want to join this list, please reply to this email. we are making posters and stickers which declare sanctuaries from racism, xenophobia, sexism, bigotry, religious intolerance, and attacks on unions. once these materials are ready we will write to you so that you may post them on windows, office doors, cars etc. in the meantime, we urge you to talk to your students and colleagues of color as well as women and offer them your support and solidarity. as you may recall, the executive committee issued a denunciation of donald trump on october , . now our slogan, one from the labor movement, is “don’t mourn. organize!” that is where we are now – all the more poignantly because of donald trump’s appeal to workers. let us organize, and let us also expand our calling of education. in your classrooms, your communities, and your families, find the words and sentiments that will redeem all of us from tuesday’s disgrace. university of chicago message from president and provost early in the fall quarter, we sent a message welcoming each of you to the new academic year and affirming our strong commitment to two foundational values of the university – fostering an environment of free expression and open discourse; and ensuring that diversity and inclusion are essential features of the fabric of our campus community and our interactions beyond campus. recent national events have generated waves of disturbing, exclusionary and sometimes threatening behavior around the country, particularly concerning gender and minority status. as a result, many individuals are asking whether the nation and its institutions are entering a period in which supporting the values of diversity and inclusion, as well as free expression and open discourse, will be increasingly challenging. as the president and provost of the university of chicago, we are writing to reaffirm in the strongest possible terms our unwavering commitment to these values, and to the importance of the university as a community acting on these values every day. fulfilling our highest aspirations with respect to these values and their mutual reinforcement will always demand ongoing attention and work on the part of all of us. the current national environment underscores the importance of this work. it means that we need to manifest these values more rather than less, demand more of ourselves as a community, and together be forthright and bold in demonstrating what our community aspires to be. we ask all of you for your help and commitment to the values of diversity and inclusion, free expression, and open discourse and what they mean for each of us working, learning, and living in this university community every day. university of illinois, chicago dear students, faculty, and staff, the events of the past week have come with mixed emotions for many of you. we want you to know that uic remains steadfast in its commitment to creating and sustaining a community that recognizes and values the inherent worth and dignity of every person, while fostering an environment of mutual respect among all members. today, we reaffirm the university’s commitment to access, equity, inclusion and nondiscrimination. critical to this commitment is the work of several offices on campus that provide resources to help you be safe and successful. if you have questions, need someone to talk to, or a place to express yourself, you should consider contacting these offices: office for access and equity (oae). oae is responsible for assuring campus compliance in matters of equal opportunity, affirmative action, and nondiscrimination in the academic and work environment. oae also offers dispute resolution services (drs) to assist with conflict in the workplace not involving unlawful discrimination matters. uic counseling center. the uic counseling center is a primary resource providing comprehensive mental health services that foster personal, interpersonal, academic, and professional thriving for uic students. student legal services. uic’s student legal services (sls) is a full-service law office dedicated to providing legal solutions for currently enrolled students. office of diversity. the office of diversity leads strategic efforts to advance access, equity, and inclusion as fundamental principles underpinning all aspects of university life. it initiates programs that promote an inclusive university climate, partner with campus units to formulate systems of accountability, and develop links with the local community and alumni groups. centers for cultural understanding and social change. the centers for cultural understanding and social change (ccusc) are a collaborative group of seven centers with distinct histories, missions, and locations that promote the well-being of and cultural awareness about underrepresented and underserved groups at uic. uic dialogue initiative. the uic dialogue initiative seeks to build an inclusive campus community where students, faculty, and staff feel welcomed in their identities, valued for their contributions, and feel their identities can be openly expressed. through whatever changes await us, as a learning community we have a special obligation to ensure that our conversations and dialogues over the next weeks and months respect our varied backgrounds and beliefs. university of maryland, baltimore to the umb community: last week, we elected a new president for our country. i think most will agree that the campaign season was long and divisive, and has left many feeling separated from their fellow citizens. in the days since the election, i’ve heard from the leaders of umb and of the university of maryland medical center and of the many programs we operate that serve our neighbors across the city and state. these leaders have relayed stories of students, faculty, staff, families, and children who feel anxious and unsettled, who feel threatened and fearful. it should be unnecessary to reaffirm umb’s commitment to diversity, inclusion, and respect — these values are irrevocable — but when i hear that members of our family are afraid, i must reiterate that the university will not tolerate incivility of any kind, and that the differences we celebrate as a diverse community include not just differences of race, religion, nationality, gender, and sexual identity, but also of experience, opinion, and political affiliation and ideology. if you suffer any harassment, please contact your supervisor or your student affairs dean. in the months ahead, we will come together as a university community to talk about how the incoming administration might influence the issues we care about most: health care access and delivery; education; innovation; social justice and fair treatment for all. we will talk about the opportunities that lay ahead to shape compassionate policy and to join a national dialogue on providing humane care and services that uplift everyone in america. for anyone who despairs, we will talk about building hope. should you want to share how you’re feeling post-election, counselors are available. please contact the student counseling center or the employee assistance program to schedule an appointment. i look forward to continuing this conversation about how we affirm our fundamental mission to improve the human condition and serve the public good. like the values we uphold, this mission endures — irrespective of the person or party in political power. it is our binding promise to the leaders of this state and, even more importantly, to the citizens we serve together. university of west georgia dear colleagues, as we head into the weekend concluding a week, really several weeks, of national and local events, i am reminded of the incredible opportunity of reflection and discourse we have as a nation and as an institution of higher learning. this morning, we held on campus a moving ceremony honoring our veterans–those who have served and who have given the ultimate sacrifice to uphold and protect our freedoms. it is those freedoms that provide the opportunity to elect a president and those freedoms that provide an environment of civil discourse and opinion. clearly, the discourse of this election cycle has tested the boundaries. this is an emotional time for many of our faculty, staff, and students. i ask that as a campus community we hold true to the intended values of our nation and those who sacrificed to protect those values and the core values of our institution–caring, collaboration, inclusiveness, and wisdom. we must acknowledge and allow the civil discourse and opinion of all within a safe environment. that is what should set us apart. it is part of our dna in higher education to respect and encourage variance and diversity of belief, thought, and culture. i call on your professionalism during these times and so appreciate your passion and care for each other and our students. virginia commonwealth university to staff election message dear vcu and vcu health communities, yesterday, we elected new leaders for our city, commonwealth and nation. i am grateful to those of you who made your voice heard during the electoral process, including many of our students who voted for the first time. whether or not your preferred candidate won, you were a part of history and a part of the process that moves our democracy forward. thank you. i hope you will always continue to make your voice heard, both as voters and as well-educated leaders in our society. as with any election, some members of our community are enthusiastic about the winners, others are not. for many, this election cycle was notably emotional and difficult. now is the time, then, to demonstrate the values that make virginia commonwealth university such a remarkable place. we reaffirm our commitment to working together across boundaries of discipline or scholarship, as members of one intellectual community, to achieve what’s difficult. we reaffirm our commitment to inclusion, to ensuring that every person who comes to vcu is respected and emboldened to succeed. we reaffirm that we will always be a place of the highest integrity, accountability, and we will offer an unyielding commitment to serving those who need us. history changes with every election. what does not change are the commitments we share as one community that is relentlessly focused on advancing the human experience for all people. you continue to inspire me. and i know you will continue to be a bright light for richmond, virginia, our nation and our world. virginia commonwealth university school of education to students election message dear students, on tuesday we elected new leaders for our city, our commonwealth and our nation. although leadership will be changing, i echo dr. rao’s message below in that our mission outlined by the quest for distinction to support student success, advance knowledge and strengthen our communities remains steadfast. at the vcu school of education, we work to create safe spaces where innovation, inclusion and collaboration can thrive. we actively work across boundaries and disciplines to address the complex challenges facing our communities, schools and families. the election of new leaders provides new opportunities for our students, faculty and staff to build bridges that help us reach our goal of making an impact in urban and high need environments. i encourage you to engage in positive dialogues with one another as the city, commonwealth and nation adjust to the change in leadership, vision and strategy. virginia commonwealth university division of student affairs dear students, we are writing to you, collectively, as leaders in the division of student affairs. we acknowledge that this election season was stressful for many individuals in our vcu community, culminating with the election of the next president. some members of our campus community have felt disrespected, attacked and further marginalized by political rhetoric during the political process. we want to affirm support of all of our students while also recognizing the unique experiences and concerns of individuals. we want all students to know that we are here to support you, encourage you and contribute to your success. we now live in a space of uncertainty as we transition leadership in our nation. often, with this uncertainty comes a host of thoughts and feelings. we hope that you will take advantage of some of the following services and programs we offer through our division to support your well-being, including: office of multicultural student affairs, self-care space, university counseling services , the wellness resource center, trans lives matter panel and survivor solidarity support, recreational sports, restorative yoga and mind & body classes. we encourage students to express their concerns and engage in conversations that further the core values articulated in quest, the vcu strategic plan. we continue to have an opportunity to make individual and collective choices about how we work to bridge differences in a manner that builds up our community. our staff will have a table each day next week on the vcu compass from noon to : p.m. to receive your concerns, suggestions and just listen. please stop by to meet us. we want you to know you have our full support. other organizations aclu joint statement from california legislative leaders on result of presidential election posted in: diversity, librarianship, library, management. tagged: college · communication · diversity · election · equity · higher ed · inclusion · library · university finding the right words in post-election libraries and higher ed nov th, by bohyun (library hat). comments are off for this post ** this post was originally published in acrl techconnect on nov. , .*** this year’s election result has presented a huge challenge to all of us who work in higher education and libraries. usually, libraries, universities, and colleges do not comment on presidential election result and we refrain from talking about politics at work. but these are not usual times that we are living in. a black female student was shoved off the sidewalk and called the ‘n’ word at baylor university. the ku klux klan is openly holding a rally. west virginia officials publicly made a racist comment about the first lady. steve bannon’s prospective appointment as the chief strategist and senior counsel to the new president is being praised by white nationalist leaders and fiercely opposed by civil rights groups at the same time. bannon is someone who calls for an ethno-state, openly calls martin luther king a fraud, and laments white dispossession and the deconstruction of occidental civilization. there are people drawing a swastika at a park. the ‘whites only’ and ‘colored’ signs were put up over water fountains in a florida school. a muslim student was threatened with a lighter. asian-american women are being assaulted. hostile acts targeting minority students are taking place on college campuses. libraries and educational institutions exist because we value knowledge and science. knowledge and science do not discriminate. they grow across all different races, ethnicities, religions, nationalities, sexual identities, and disabilities. libraries and educational institutions exist to enable and empower people to freely explore, investigate, and harness different ideas and thoughts. they support, serve, and belong to ‘all’ who seek knowledge. no matter how naive it may sound, they are essential to the betterment of human lives, and they do so by creating strength from all our differences, not likeness. this is why diversity, equity, inclusion are non-negotiable and irrevocable values in libraries and educational institutions. how do we reconcile these values with the president-elect who openly dismissed and expressed hostility towards them? his campaign made remarks and promises that can be interpreted as nothing but the most blatant expressions of racism, sexism, intolerance, bigotry, harassment, and violence. what will we do to address the concerns of our students, staff, and faculty about their physical safety on campus due to their differences in race, ethnicity, religion, nationality, gender, and sexual identity? how do we assure them that we will continue to uphold these values and support everyone regardless of what they look like, how they identify their gender, what their faiths are, what disabilities they may have, who they love, where they come from, what languages they speak, or where they live? how? we say it. explicitly. clearly. and repeatedly. if you think that your organization is already very much pro-diversity that there is no need to confirm or reaffirm diversity, you can’t be farther from the everyday life minorities experience. sometimes, saying isn’t much. but right now, saying it out loud can mean everything. if you support those who belong to minority groups but don’t say it out loud, how would they know it? right now, nothing is obvious other than there is a lot of hate and violence towards minorities. the entire week after the election, i agonized about what to say to my small team of it people whom i supervise at work. as a manager, i felt that it was my responsibility to address the anxiety and uncertainty that some of my staff – particularly those in minority groups – would be experiencing due to the election result. i also needed to ensure that whatever dialogue takes place regarding the differences of opinions between those who were pleased and those who were distressed with the election result, those dialogues remain civil and respectful. crafting an appropriate message was much more challenging than i anticipated. i felt very strongly about the need to re-affirm the unwavering support and commitment to diversity, equity, and inclusion particularly in relation to libraries and higher education, no matter how obvious it may seem. i also felt the need to establish (within the bounds of my limited authority) that we will continue to respect, value, and celebrate diversity in interacting with library users as well as other library and university staff members. employees are held to the standard expectations of their institutions, such as diversity, equity, inclusion, tolerance, civil dialogue, and no harassment or violence towards minorities, even if their private opinions conflict with them. at the same time, i wanted to strike a measured tone and neither scare nor upset anyone, whichever side they were on in the election. as a manager, i have to acknowledge that everyone is entitled to their private opinions as long as they do not harm others. i suspect that many of us – either a manager or not – want to say something similar about the election result. not so much about who was and should have been as about what we are going to do now in the face of these public incidences of anger, hatred, harassment, violence, and bigotry directed at minority groups, which are coming out at an alarming pace because it affects all of us, not just minorities. finding the right words, however, is difficult. you have to carefully consider your role, audience, and the message you want to convey. the official public statement from a university president is going to take a tone vastly different from an informal private message a supervisor sends out to a few members of his or her team. a library director’s message to library patrons assuring the continued service for all groups of users with no discrimination will likely to be quite different from the one she sends to her library staff to assuage their anxiety and fear. for such difficulty not to delay and stop us from what we have to and want to say to everyone we work with and care for, i am sharing the short message that i sent out to my team last friday, days after the election. (n.b. ‘cats’ stands for ‘computing and technology services’ and umb refers to ‘university of maryland, baltimore.’) this is a customized message to address my own team. i am sharing this as a potential template for you to craft your own message. i would like to see more messages that reaffirm diversity, equity, and inclusion as non-negotiable values, explicitly state that we will not step backwards, and make a commitment to continued unwavering support for them. dear cats, this year’s close and divisive election left a certain level of anxiety and uncertainty in many of us. i am sure that we will hear from president perman and the university leadership soon. in the meantime, i want to remind you of something i believe to be very important. we are all here – just as we have been all along – to provide the most excellent service to our users regardless of what they look like, what their faiths are, where they come from, what languages they speak, where they live, and who they love. a library is a powerful place where people transform themselves through learning, critical thinking, and reflection. a library’s doors have been kept open to anyone who wants to freely explore the world of ideas and pursue knowledge. libraries are here to empower people to create a better future. a library is a place for mutual education through respectful and open-minded dialogues. and, we, the library staff and faculty, make that happen. we get to make sure that people’s ethnicity, race, gender, disability, socio-economic backgrounds, political views, or religious beliefs do not become an obstacle to that pursuit. we have a truly awesome responsibility. and i don’t have to tell you how vital our role is as a cats member in our library’s fulfilling that responsibility. whichever side we stood on in this election, let’s not forget to treat each other with respect and dignity. let’s use this as an opportunity to renew our commitment to diversity, one of the umb’s core values. inclusive excellence is one of the themes of the umb - strategic plan. each and every one of us has a contribution to make because we are stronger for our differences. we have much work ahead of us! i am out today, but expect lots of donuts monday. have a great weekend, bohyun monday, i brought in donuts of many different kinds and told everyone they were ‘diversity donuts.’ try it. i believe it was successful in easing some stress and tension that was palpable in my team after the election. photo from flickr: https://www.flickr.com/photos/vnysia/ before crafting your own message, i recommend re-reading your institution’s core values, mission and vision statements, and the most recent strategic plan. most universities, colleges, and libraries include diversity, equity, inclusion, or something equivalent to these somewhere. also review all public statements or internal messages that came from your institution that reaffirms diversity, equity, and inclusion. you can easily incorporate those into your own message. make sure to clearly state your (and your institution’s) continued commitment to and unwavering support for diversity and inclusion and explicitly oppose bigotry, intolerance, harassment, and acts of violence. encourage civil discourse and mutual respect. it is very important to reaffirm the values of diversity, equity, and inclusion ‘before’ listing any resources and help that employees or students may seek in case of harassment or assault. without the assurance from the institution that it indeed upholds those values and will firmly stand by them, those resources and help mean little. below i have also listed messages, notes, and statements sent out by library directors, managers, librarians, and university presidents that reaffirm the full support for and commitment to diversity, equity, and inclusion. i hope to see more of these come out. if you have already received or sent out such a message, i invite you to share in the comments. if you have not, i suggest doing so as soon as possible. send out a message if you are in a position where doing so is appropriate. don’t forget to ask for a message addressing those values if you have not received any from your organization. director chris bourg to the mit libraries staff https://chrisbourg.wordpress.com/ / / /care-for-one-another/ dean k. g. schneider to the sonoma state university library staff http://freerangelibrarian.com/ / / /pin-and-a-prayer/ librarian zoe fisher to other librarians https://quickaskzoe.com/ / / /but-i-know-that-there-will-be-libraries/ university of california statement on presidential election results https://www.universityofcalifornia.edu/press-room/university-california-statement-election university of nevada, reno http://www.unr.edu/president/communications/ - - -election university of michigan http://president.umich.edu/news-communications/letters-to-the-community/ -election-message/ university of rochester and rochester institute of technology http://wxxinews.org/post/ur-presidents-post-election-letter-strikes-sour-note-some duke university https://today.duke.edu/ / /statement-president-brodhead-following- -election clarke university http://www.clarke.edu/page.aspx?id= mit https://news.mit.edu/ /letter-mit-community-new-administration-washington- northwestern university https://news.northwestern.edu/stories/ / /president-schapiro-on-the-election-and-the-university/ “post-election statements and messages that reaffirm diversity” (a list of more post-election statements and messages that reaffirm diversity) posted in: diversity, librarianship, library, management. tagged: diversity · election · equity · inclusion · message · post-election · statement · template · tolerance say it out loud – diversity, equity, and inclusion nov th, by bohyun (library hat). comments are off for this post i usually and mostly talk about technology. but technology is so far away from my thought right now. i don’t feel that i can afford to worry about internet surveillance or how to protect privacy at this moment. not that they are unimportant. such a worry is real and deserves our attention and investigation. but at a time like this when there are so many reports of public incidences of hatred, bigotry, harassment, and violence reported on university and college campuses, on streets, and in many neighborhoods coming in at an alarming pace, i don’t find myself reflecting on how we can use technology to deal with this problem. for the problem is so much bigger. there are people drawing a swastika at a park. the ‘whites only’ and ‘colored’ signs were put up over water fountains in a florida school. a muslim student was threatened with a lighter. asian-american women are being assaulted. hostile acts targeting minority students are taking place on college campuses. a black female student was shoved off the sidewalk and called the ‘n’ word at baylor university. newt gingrich called for a house committee for un-american activities. the ku klux klan is openly holding a rally. the list goes on and on. photo from http://www.wftv.com/news/local/investigation-underway-after- -racist-signs-posted-above-water-fountains-at-first-coast-high-school/ we are justified to be freaking out. i suspect this is a deal breaker to not just democrats, not just clinton supporters, but a whole lot more people. not everyone who voted for donald trump endorse the position that women, people of color, muslims, lgbt, and all other minority groups deserve and should be deprived of the basic human right to be not publicly threatened, harassed, and assaulted, i hope. i am sure that many who voted for donald trump do support diversity, equity, and inclusion as important and non-negotiable values. i believe that many who voted for donald trump do not want a society where some of their family, friends, colleagues, and neighbors have to live in constant fear for their physical safety at minimum. there are very many white people who absolutely condemn bigotry, threat, hatred, discrimination, harassment, and violence directed at minorities and give their unwavering support to diversity, equity, and inclusion. the problem is that i don’t hear it said loudly enough, clearly enough, publicly enough. i realized that we – myself included – do not say this enough. one of my fellow librarians, steve, wrote this on his facebook wall after the election. i am a year old white guy. … i go out into the world today and i’m trying to hold a look on my face that says i don’t hate you black people, hispanic people, gay people, muslim people. i mean you no harm. i don’t want to deport you or imprison you. you are my brothers and sisters. i want for you all of the benefits, the rights, the joys (such as they are) that are afforded to everybody else in our society. i don’t think this look on my face is effective. why should they trust me? you can never appear to be doing the right thing. it requires doing the right thing. of course, steve doesn’t want to harm me because i am not white, i know. i am % positive that he wouldn’t assault me because i am female. but by stating this publicly (i mean as far as his fb friends can see the post), he made a difference to me. steve is not republican. but i would feel so much better if people i know tell me the same thing whether they are democrat or republican. and i think it will make a huge difference to others when we all say this together. sometimes, saying isn’t much. but right now, saying it aloud can mean everything. if you support those who belong to minority groups but don’t say it out loud, how would they know it? because right now, nothing is obvious other than there is a lot of hate and violence towards minorities. at this point, which candidate you voted for doesn’t matter. what matters is whether you will condone open hatred and violence towards minorities and women, thereby making it acceptable in our society. there is a lot at stake here, and this goes way beyond party politics. publicly confirming our continued support for and unwavering commitment to diversity is a big deal. people who are being insulted, threatened, harassed, and assaulted need to hear it. and when we say this together loudly enough, clearly enough, explicitly enough, it will deafen the voice of hatred, bigotry, and intolerance and chase it away to the margins of our society again. so i think i am going to say this whenever i have a chance whether formally or informally whether it is in a written form or in a conversation. if you are a librarian, you should say this to your library users. if you are a teacher, you should say this to your students. if you run a business, you need to say this to your employees and customers. if you manage a team at work, tell your team. say this out loud to your coworkers, friends, family, neighbors, and everyone you interact with. “i support all minorities and stand for diversity, equity, and inclusion.” “i object to and will not condone the acts of harassment, violence, hatred, and threats directed at minorities.” “i will not discriminate anyone based upon their ethnicity, race, sexual orientation, disability, political views, socio-economic backgrounds, or religious beliefs.” we cannot allow diversity, equity, and inclusion to become minority opinions. and it is up to us to keep it mainstream and to make it prevail. say it aloud and act on it. in times like this, many of us look to institutions that we belong to, the organizations we work for, professionally participate in, or personally support. we expect them to reconfirm the very basic values of diversity, equity, and inclusion. since i work for a university, i have been looking up and reading statements from higher education institutions. so far, not a great number of universities have made public statements confirming their continued support for diversity. i am sure more are on the way. but i expected more of them would come out more promptly. this is unfortunate because many of them openly expressed their support for diversity and even include diversity in their values, mission, and goals. if your organization hasn’t already confirmed their support for these values and expressed their commitment to provide safety for all minorities, ask for it. you may even be in a position to actually craft and issue one. for those in need of right words to express your intention clearly, here are some good examples below. “the university of california is proud of being a diverse and welcoming place for students, faculty, and staff with a wide range of backgrounds, experiences and perspectives. diversity is central to our mission. we remain absolutely committed to supporting all members of our community and adhering to uc’s principles against intolerance. as the principles make clear, the university ‘strives to foster an environment in which all are included’ and ‘all are given an equal opportunity to learn and explore.’ the university of california will continue to pursue and protect these principles now and in the future, and urges our students, faculty, staff, and all others associated with the university to do so as well.” – university of california “our responsibility is to remain committed to education, discovery and intellectual honesty – and to diversity, equity and inclusion. we are at our best when we come together to engage respectfully across our ideological differences; to support all who feel marginalized, threatened or unwelcome; and to pursue knowledge and understanding, as we always have, as the students, faculty and staff of the university of michigan.” – university of michigan “northwestern is committed to being a welcoming and inclusive community for all, regardless of their beliefs, and i assure you that will not change.” – northwestern university “as a catholic university, clarke will not step away from its many efforts to heighten our awareness of the individuals and groups who are exclude and marginalized in so many ways and to take action for their protection and inclusion. today, i call on us as a community to step up our efforts to promote understanding and inclusion and to reach out to those among us who are feeling further disenfranchised, fearful and confused as a result of the election.” – clarke university “as president, i need to represent all of rit, and i therefore do not express preferences for political candidates. i do feel it important, however, to represent and reinforce rit’s shared commitment to the value of inclusive diversity. i have heard from many in our community that the result of the recent election has raised concerns from those in our minority populations, those who come from immigrant families, those from countries outside of the u.s., those in our lgbtqia+ community, those who practice islam, and even those in our female population about whether they should be concerned for their safety and well-being as a result of the horrific discourse that accompanied the presidential election process and some of the specific views and proposals presented. at rit, we have treasured the diverse contributions of members of these groups to our campus community, and i want to reassure all that one of rit’s highest priorities is to demonstrate the extraordinary value of inclusive diversity and that we will continue to respect, appreciate, and benefit from the contributions of all. anyone who feels unsafe here should make their feelings known to me and to others in a position to address their concerns. concerned members of our community can also take advantage of opportunities to engage in open discourse about the election in the mosaic center and at tomorrow’s grey matter discussion.” – rochester institute of technology please go ahead and say these out loud to people around you if you mean them. no matter how obvious and cheesy they sound, i assure you, they are not obvious and cheesy to those who are facing open threats, harassment, and violence. let’s boost the signal; let’s make it loud; let’s make it overwhelming. “i support all minorities and stand for diversity, equity, and inclusion.” “i object to and will not condone the acts of harassment, violence, hatred, and threats directed at minorities.” “i will not discriminate anyone based upon their ethnicity, race, sexual orientation, disability, political views, socio-economic backgrounds, or religious beliefs.” posted in: diversity. tagged: · election · hate crime · racism cybersecurity, usability, online privacy, and digital surveillance may th, by bohyun (library hat). comments are off for this post ** this post was originally published in acrl techconnect on may. , .*** cybersecurity is an interesting and important topic, one closely connected to those of online privacy and digital surveillance. many of us know that it is difficult to keep things private on the internet. the internet was invented to share things with others quickly, and it excels at that job. businesses that process transactions with customers and store the information online are responsible for keeping that information private. no one wants social security numbers, credit card information, medical history, or personal e-mails shared with the world. we expect and trust banks, online stores, and our doctor’s offices to keep our information safe and secure. however, keeping private information safe and secure is a challenging task. we have all heard of security breaches at j.p morgan, target, sony, anthem blue cross and blue shield, the office of personnel management of the u.s. federal government, university of maryland at college park, and indiana university. sometimes, a data breach takes place when an institution fails to patch a hole in its network systems. sometimes, people fall for a phishing scam, or a virus in a user’s computer infects the target system. other times, online companies compile customer data into personal profiles. the profiles are then sold to data brokers and on into the hands of malicious hackers and criminals. image from flickr – https://www.flickr.com/photos/topgold/ cybersecurity vs. usability to prevent such a data breach, institutional it staff are trained to protect their systems against vulnerabilities and intrusion attempts. employees and end users are educated to be careful about dealing with institutional or customers’ data. there are systematic measures that organizations can implement such as two-factor authentication, stringent password requirements, and locking accounts after a certain number of failed login attempts. while these measures strengthen an institution’s defense against cyberattacks, they may negatively affect the usability of the system, lowering users’ productivity. as a simple example, security measures like a captcha can cause an accessibility issue for people with disabilities. or imagine that a university it office concerned about the data security of cloud services starts requiring all faculty, students, and staff to only use cloud services that are soc type ii certified as an another example. soc stands for “service organization controls.” it consists of a series of standards that measure how well a given service organization keeps its information secure. for a business to be soc certified, it must demonstrate that it has sufficient policies and strategies that will satisfactorily protect its clients’ data in five areas known as “trust services principles.” those include the security of the service provider’s system, the processing integrity of this system, the availability of the system, the privacy of personal information that the service provider collects, retains, uses, discloses, and disposes of for its clients, and the confidentiality of the information that the service provider’s system processes or maintains for the clients. the soc type ii certification means that the business had maintained relevant security policies and procedures over a period of at least six months, and therefore it is a good indicator that the business will keep the clients’ sensitive data secure. the dropbox for business is soc certified, but it costs money. the free version is not as secure, but many faculty, students, and staff in academia use it frequently for collaboration. if a university it office simply bans people from using the free version of dropbox without offering an alternative that is as easy to use as dropbox, people will undoubtedly suffer. some of you may know that the usps website does not provide a way to reset the password for users who forgot their usernames. they are instead asked to create a new account. if they remember the account username but enter the wrong answers to the two security questions more than twice, the system also automatically locks their accounts for a certain period of time. again, users have to create a new account. clearly, the system that does not allow the password reset for those forgetful users is more secure than the one that does. however, in reality, this security measure creates a huge usability issue because average users do forget their passwords and the answers to the security questions that they set up themselves. it’s not hard to guess how frustrated people will be when they realize that they entered a wrong mailing address for mail forwarding and are now unable to get back into the system to correct because they cannot remember their passwords nor the answers to their security questions. to give an example related to libraries, a library may decide to block all international traffic to their licensed e-resources to prevent foreign hackers who have gotten hold of the username and password of a legitimate user from accessing those e-resources. this would certainly help libraries to avoid a potential breach of licensing terms in advance and spare them from having to shut down compromised user accounts one by one whenever those are found. however, this would make it impossible for legitimate users traveling outside of the country to access those e-resources as well, which many users would find it unacceptable. furthermore, malicious hackers would probably just use a proxy to make their ip address appear to be located in the u.s. anyway. what would users do if their organization requires them to reset passwords on a weekly basis for their work computers and several or more systems that they also use constantly for work? while this may strengthen the security of those systems, it’s easy to see that it will be a nightmare having to reset all those passwords every week and keeping track of them not to forget or mix them up. most likely, they will start using less complicated passwords or even begin to adopt just one password for all different services. some may even stick to the same password every time the system requires them to reset it unless the system automatically detects the previous password and prevents the users from continuing to use the same one. ill-thought-out cybersecurity measures can easily backfire. security is important, but users also want to be able to do their job without being bogged down by unwieldy cybersecurity measures. the more user-friendly and the simpler the cybersecurity guidelines are to follow, the more users will observe them, thereby making a network more secure. users who face cumbersome and complicated security measures may ignore or try to bypass them, increasing security risks. image from flickr – https://www.flickr.com/photos/topgold/ cybersecurity vs. privacy usability and productivity may be a small issue, however, compared to the risk of mass surveillance resulting from aggressive security measures. in , the guardian reported that the communication records of millions of people were being collected by the national security agency (nsa) in bulk, regardless of suspicion of wrongdoing. a secret court order prohibited verizon from disclosing the nsa’s information request. after a cyberattack against the university of california at los angeles, the university of california system installed a device that is capable of capturing, analyzing, and storing all network traffic to and from the campus for over days. this security monitoring was implemented secretly without consulting or notifying the faculty and those who would be subject to the monitoring. the san francisco chronicle reported the it staff who installed the system were given strict instructions not to reveal it was taking place. selected committee members on the campus were told to keep this information to themselves. the invasion of privacy and the lack of transparency in these network monitoring programs has caused great controversy. such wide and indiscriminate monitoring programs must have a very good justification and offer clear answers to vital questions such as what exactly will be collected, who will have access to the collected information, when and how the information will be used, what controls will be put in place to prevent the information from being used for unrelated purposes, and how the information will be disposed of. we have recently seen another case in which security concerns conflicted with people’s right to privacy. in february , the fbi requested apple to create a backdoor application that will bypass the current security measure in place in its ios. this was because the fbi wanted to unlock an iphone c recovered from one of the shooters in san bernadino shooting incident. apple ios secures users’ devices by permanently erasing all data when a wrong password is entered more than ten times if people choose to activate this option in the ios setting. the fbi’s request was met with strong opposition from apple and others. such a backdoor application can easily be exploited for illegal purposes by black hat hackers, for unjustified privacy infringement by other capable parties, and even for dictatorship by governments. apple refused to comply with the request, and the court hearing was to take place in march . the fbi, however, withdrew the request saying that it found a way to hack into the phone in question without apple’s help. now, apple has to figure out what the vulnerability in their ios if it wants its encryption mechanism to be foolproof. in the meanwhile, ios users know that their data is no longer as secure as they once thought. around the same time, the senate’s draft bill titled as “compliance with court orders act of ,” proposed that people should be required to comply with any authorized court order for data and that if that data is “unintelligible” – meaning encrypted – then it must be decrypted for the court. this bill is problematic because it practically nullifies the efficacy of any end-to-end encryption, which we use everyday from our iphones to messaging services like whatsapp and signal. because security is essential to privacy, it is ironic that certain cybersecurity measures are used to greatly invade privacy rather than protect it. because we do not always fully understand how the technology actually works or how it can be exploited for both good and bad purposes, we need to be careful about giving blank permission to any party to access, collect, and use our private data without clear understanding, oversight, and consent. as we share more and more information online, cyberattacks will only increase, and organizations and the government will struggle even more to balance privacy concerns with security issues. why libraries should advocate for online privacy? the fact that people may no longer have privacy on the web should concern libraries. historically, libraries have been strong advocates of intellectual freedom striving to keep patron’s data safe and protected from the unwanted eyes of the authorities. as librarians, we believe in people’s right to read, think, and speak freely and privately as long as such an act itself does not pose harm to others. the library freedom project is an example that reflects this belief held strongly within the library community. it educates librarians and their local communities about surveillance threats, privacy rights and law, and privacy-protecting technology tools to help safeguard digital freedom, and helped the kilton public library in lebanon, new hampshire, to become the first library to operate a tor exit relay, to provide anonymity for patrons while they browse the internet at the library. new technologies brought us the unprecedented convenience of collecting, storing, and sharing massive amount of sensitive data online. but the fact that such sensitive data can be easily exploited by falling into the wrong hands created also the unparalleled level of potential invasion of privacy. while the majority of librarians take a very strong stance in favor of intellectual freedom and against censorship, it is often hard to discern a correct stance on online privacy particularly when it is pitted against cybersecurity. some even argue that those who have nothing to hide do not need their privacy at all. however, privacy is not equivalent to hiding a wrongdoing. nor do people keep certain things secrets because those things are necessarily illegal or unethical. being watched / will drive any person crazy whether s/he is guilty of any wrongdoing or not. privacy allows us safe space to form our thoughts and consider our actions on our own without being subject to others’ eyes and judgments. even in the absence of actual massive surveillance, just the belief that one can be placed under surveillance at any moment is sufficient to trigger self-censorship and negatively affects one’s thoughts, ideas, creativity, imagination, choices, and actions, making people more conformist and compliant. this is further corroborated by the recent study from oxford university, which provides empirical evidence that the mere existence of a surveillance state breeds fear and conformity and stifles free expression. privacy is an essential part of being human, not some trivial condition that we can do without in the face of a greater concern. that’s why many people under political dictatorship continue to choose death over life under mass surveillance and censorship in their fight for freedom and privacy. the electronic frontier foundation states that privacy means respect for individuals’ autonomy, anonymous speech, and the right to free association. we want to live as autonomous human beings free to speak our minds and think on our own. if part of a library’s mission is to contribute to helping people to become such autonomous human beings through learning and sharing knowledge with one another without having to worry about being observed and/or censored, libraries should advocate for people’s privacy both online and offline as well as in all forms of communication technologies and devices. posted in: library, technology, usability, user experience, web. tagged: data security · digital freedom · encryption · internet · password · soc · tor three recent talks of mine on ux, data visualization, and it management apr th, by bohyun (library hat). comments are off for this post i have been swamped at work and pretty quiet here in my blog. but i gave a few talks recently. so i wanted to share those at least. i presented about how to turn the traditional library it department and its operation that is usually behind the scene into a more patron-facing unit at the recent american library association midwinter meeting back in january. this program was organized by the lita heads of it interest group. in march, i gave a short lightning talk at the code lib conference about the data visualization project of library data at my library. i was also invited to speak at the usmai (university system of maryland and affiliated institutions) ux unconference and gave a talk about user experience, personas, and the idea of applying library personas to library strategic planning. here are those three presentation slides for those interested! strategically ux oriented with personas from bohyun kim visualizing library data from bohyun kim turning the it dept. outward from bohyun kim posted in: ala, library, presentation, technology, usability, user experience. tagged: code lib · data visualization · it · management · ux near us and libraries, robots have arrived oct th, by bohyun (library hat). comments are off for this post ** this post was originally published in acrl techconnect on oct. , .*** the movie, robot and frank, describes the future in which the elderly have a robot as their companion and also as a helper. the robot monitors various activities that relate to both mental and physical health and helps frank with various house chores. but frank also enjoys the robot’s company and goes on to enlist the robot into his adventure of breaking into a local library to steal a book and a greater heist later on. people’s lives in the movie are not particularly futuristic other than a robot in them. and even a robot may not be so futuristic to us much longer either. as a matter of fact, as of june , there is now a commercially available humanoid robot that is close to performing some of the functions that the robot in the movie ‘frank and robot’ does. pepper robot, image from aldebaran, https://www.aldebaran.com/en/a-robots/who-is-pepper a japanese company, softbank robotics corp. released a humanoid robot named ‘pepper’ to the market back in june. the pepper robot is feet tall, pounds, speaks languages and is equipped with an array of cameras, touch sensors, accelerometer, and other sensors in his “endocrine-type multi-layer neural network,” according to the cnn report. the pepper robot was priced at ¥ , ($ , ). the pepper owners are also responsible for an additional ¥ , ($ ) monthly data and insurance fee. while the pepper robot is not exactly cheap, it is surprisingly affordable for a robot. this means that the robot industry has now matured to the point where it can introduce a robot that the mass can afford. robots come in varying capabilities and forms. some robots are as simple as a programmable cube block that can be combined with one another to be built into a working unit. for example, cubelets from modular robotics are modular robots that are used for educational purposes. each cube performs one specific function, such as flash, battery, temperature, brightness, rotation, etc. and one can combine these blocks together to build a robot that performs a certain function. for example, you can build a lighthouse robot by combining a battery block, a light-sensor block, a rotator block, and a flash block. a variety of cubelets available from the modular robotics website. by contrast, there are advanced robots such as those in the form of an animal developed by a robotics company, boston dynamics. some robots look like a human although much smaller than the pepper robot. nao is a -cm tall humanoid robot that moves, recognizes, hears and talks to people that was launched in . nao robots are an interactive educational toy that helps students to learn programming in a fun and practical way. noticing their relevance to stem education, some libraries are making robots available to library patrons. westport public library provides robot training classes for its two nao robots. chicago public library lends a number of finch robots that patrons can program to see how they work. in celebration of the national robotics week back in april, san diego public library hosted their first robot day educating the public about how robots have impacted the society. san diego public library also started a weekly robotics club inviting anyone to join in to help build or learn how to build a robot for the library. haslet public library offers the robotics camp program for th to th graders who want to learn how to build with lego mindstorms ev kits. school librarians are also starting robotics clubs. the robotics club at new rochelle high school in new york is run by the school’s librarian, ryan paulsen. paulsen’s robotics club started with faculty, parent, and other schools’ help along with a grant from nasa and participated in a first robotics competition. organizations such as the robotics academy at carnegie mellon university provides educational outreach and resources. image from aldebaran website at https://www.aldebaran.com/en/humanoid-robot/nao-robot there are also libraries that offer coding workshops often with arduino or raspberry pi, which are inexpensive computer hardware. ames free library offers raspberry pi workshops. san diego public library runs a monthly arduino enthusiast meetup. arduinos and raspberry pis can be used to build digital devices and objects that can sense and interact the physical world, which are close to a simple robot. we may see more robotics programs at those libraries in the near future. robots can fulfill many other functions than being educational interactive toys, however. for example, robots can be very useful in healthcare. a robot can be a patient’s emotional companion just like the pepper. or it can provide an easy way to communicate for a patient and her/his caregiver with physicians and others. a robot can be used at a hospital to move and deliver medication and other items and function as a telemedicine assistant. it can also provide physical assistance for a patient or a nurse and even be use for children’s therapy. humanoid robots like pepper may also serve at a reception desk at companies. and it is not difficult to imagine them as sales clerks at stores. robots can be useful at schools and other educational settings as well. at a workplace, teleworkers can use robots to achieve more active presence. for example, universities and colleges can offer a similar telepresence robot to online students who want to virtually experience and utilize the campus facilities or to faculty who wish to offer the office hours or collaborate with colleagues while they are away from the office. as a matter of fact, the university of texas, arlington, libraries recently acquired several telepresence robots to lend to their faculty and students. not all robots do or will have the humanoid form as the pepper robot does. but as robots become more and more capable, we will surely get to see more robots in our daily lives. references alpeyev, pavel, and takashi amano. “robots at work: softbank aims to bring pepper to stores.” bloomberg business, june , . http://www.bloomberg.com/news/articles/ - - /robots-at-work-softbank-aims-to-bring-pepper-to-stores. “boston dynamics.” accessed september , . http://www.bostondynamics.com/. boyer, katie. “robotics clubs at the library.” public libraries online, june , . http://publiclibrariesonline.org/ / /robotics-clubs-at-the-library/. “finch robots land at cpl altgeld.” chicago public library, may , . https://www.chipublib.org/news/finch-robots-land-at-cpl/. mcnickle, michelle. “ medical robots that could change healthcare – informationweek.” informationweek, december , . http://www.informationweek.com/mobile/ -medical-robots-that-could-change-healthcare/d/d-id/ . singh, angad. “‘pepper’ the emotional robot, sells out within a minute.” cnn.com, june , . http://www.cnn.com/ / / /tech/pepper-robot-sold-out/. tran, uyen. “sdpl labs: arduino aplenty.” the library incubator project, april , . http://www.libraryasincubatorproject.org/?p= . “ut arlington library to begin offering programming robots for checkout.” university of texas arlington, march , . https://www.uta.edu/news/releases/ / /library-robots- .php. waldman, loretta. “coming soon to the library: humanoid robots.” wall street journal, september , , sec. new york. http://www.wsj.com/articles/coming-soon-to-the-library-humanoid-robots- . posted in: library, technology. tagged: education · libraries · robotics · robots · stem ← earlier posts subscribe to our feed via rss search about libraryhat is a blog written by bohyun kim, cto & associate professor at the university of rhode island libraries (bohyun.kim.ois [at] gmail [dot] com; @bohyunkim). most popular - libraries meet the second machine age - future? libraries? what now? – after the ala summit on the future of libraries - query a google spreadsheet like a database with google visualization api query language - enabling the research ‘flow’ and serendipity in today’s digital library environment - research librarianship in crisis: mediate when, where, and how? - why not grow coders from the inside of libraries? - do you feel inadequate? for hard-working overachievers - redesigning the item record summary view in a library catalog and a discovery interface - fear no longer regular expressions - using git with bitbucket: basic commands – pull, add, commit, push - aaron swartz and too-comfortable research libraries - common misconceptions about library job search: what i have learned from the other side of the table - applying game dynamics to library services - how to make your writing less terrible - netflix and libraries: you are what “your users” think you are, not what you think you are archives july ( ) december ( ) october ( ) may ( ) november ( ) may ( ) april ( ) october ( ) september ( ) july ( ) march ( ) february ( ) september ( ) june ( ) may ( ) march ( ) december ( ) november ( ) october ( ) september ( ) july ( ) april ( ) march ( ) february ( ) january ( ) december ( ) november ( ) october ( ) september ( ) august ( ) july ( ) june ( ) may ( ) march ( ) february ( ) january ( ) october ( ) september ( ) august ( ) july ( ) june ( ) may ( ) march ( ) february ( ) january ( ) december ( ) november ( ) october ( ) september ( ) august ( ) july ( ) june ( ) april ( ) february ( ) january ( ) december ( ) november ( ) october ( ) september ( ) august ( ) july ( ) tags acrl ala api change codeyear coding communication conference continuing education design election emerging technologies equity inclusion interview with brand-new librarians it javascript job job search jquery kindle libcodeyear librarian libraries library library day in the life lis lita makerspace management mls mobile new librarians post-mls presentation programming publication technology tips tweet-up twitter usability ux web © library hat | powered by wordpress a wordpress theme by ravi varma islandora islandora what's new what's new manez tue, / / - : body our website has been overhauled in a big way. we have moved to drupal , changed our look, and shifted content around to make it easier to find the islandora information and resources that you need. can't find something you expect from the old site? let us know and we'll get it fixed. islandora open meeting: april , islandora open meeting: april , agriffith tue, / / - : body we are happy to announce the date of our next open meeting! join us on april , any time between : - : pm edt. the open meetings are drop-in style sessions where users of all levels and abilities gather to ask questions, share use cases and get updates on islandora. there will be experienced islandora users on hand to answer questions or give demos. we would love for your to join us any time during the -hour window, so feel free to pop by any time! more details about the open meeting, and the zoom link to join, are in this google doc. registration is not required. if you would like a calendar invite as a reminder, please let us know at community@islandora.ca. upcoming dig sprint upcoming dig sprint agriffith thu, / / - : body the islandora documentation interest group is holding a sprint! to support the upcoming release of islandora, the dig has planned a -week documentation, writing-and-updating sprint to occur as part of the release process. to prepare for that effort, we’re going to spend april – th on an auditing sprint, where volunteers will review existing documentation and complete this spreadsheet, providing a solid overview of the current status of our docs so we know where to best deploy our efforts during the release. this sprint will run alongside the upcoming pre-release code sprint, so if you’re not up for coding, auditing docs is a great way to contribute during sprint season! we are looking for volunteers to sign up to take on two sprint roles: auditor: review a page of documentation and fill out a row in the spreadsheet indicating things like the current status (‘good enough’ or ‘needs work’) , the goal for that particular page (e.g., “explain how to create an object,” or “compare islandora concepts to islandora concepts”), and the intended audience (beginners, developers, etc.). reviewer: read through a page that has been audited and indicate if you agree with the auditor’s assessment, add additional notes or suggestions as needed; basically, give a second set of eyes on each page. you can sign up for the sprint here, and sign up for individual pages here. community announcement community announcement agriffith wed, / / - : body as you know, the islandora foundation has recently updated its governance structure to remain compliant with canadian non-profit regulations. islandora foundation members approved these changes at the annual general meeting in early march. a summary of these changes is provided here, as well as our emerging roadmap for moving forward. a newly formed “leadership group”, composed of representatives from our partner-level member organizations, replaces the pre-existing board of directors, and a smaller board of directors remains responsible for islandora’s administrative and fiscal responsibilities. this leadership group met for the first time on friday, march th to begin to discuss their goals going forward, and the ways the leadership group will interact with the other governance structures of the islandora community. the leadership group immediately affirmed their commitment to transparent communication and collaboration with the vibrant, robust islandora community and will be creating a terms of reference over the next month. the terms of reference will be written with agility and transformation in mind, as we work together to secure a strong future for both the community and codebase. in the meantime, please let us know if you have any questions regarding the formation of the leadership group, and stay tuned to hear more about the initial goals of this group. islandorans unite! it's release time islandorans unite! it's release time dlamb mon, / / - : body it's that time again everyone! our amazing community contributors have made all sorts of improvements and upgrades to islandora. some have been merged, but some are still hanging out, waiting for the love they need to make it into the code base. we're calling on you - yes you! - to help us get things merged, tested, documented, and released to the world. i would like to kick off this release cycle with a sprint to mop up some the amazing improvements that have unmerged pull requests. did you know that we have pull requests for an advanced search module and a basic batch ingest form just lounging around? and that's not all. there are all kinds of great improvements that just need some time and attention. a little code review and some basic testing by others are all that is needed before we freeze the code and start turning the crank on the release process. here's a rough timetable for the release: april - th: code sprint may rd: code freeze may rd - th: testing, bug fixing, responding to feedback may th - th: documentation sprint may st - june th: more testing, bug fixing, and responding to feedback june st - july nd: testing sprint release! this is, of course, an optimistic plan. if major issues are discovered we will take the time to address them which can affect the timeline. i also plan on liaising with the documentation interest group and folks from the users' call / open meetings for the documentation and testing sprints, and their availabilities may nudge things a week in either direction. an open and transparent release process is one of the hallmarks of our amazing community. if you or your organization have any interest in helping out, please feel free to reach out or sign up for any of the upcoming sprints. there are plenty of opportunities to contribute regardless of your skill set or level of experience with islandora. there's something for everyone! we'll make further announcements for the other sprints, but you can sign up for the code sprint now using our sign up sheet. hope to see you there! islandora open meeting: march , islandora open meeting: march , agriffith wed, / / - : body we will be holding our next open meeting on tuesday, march from : am to : pm eastern. full details, and the zoom link to join, are in this google doc. the meeting is drop-in and will be free form, with experienced islandora users on hand to answer questions or give demos on request. we would love for your to join us any time during the -hour window, so feel free to pop by any time. registration is not required. if you would like a calendar invite as a reminder, please let us know at community@islandora.ca. isle: now with islandora isle: now with islandora dlamb tue, / / - : body the islandora foundation is pleased to announce that isle for islandora has gone alpha and is now available! what is isle? isle (short for islandora enterprise), is "dockerized" islandora, and seeks to create community managed infrastructure, streamlining the installation and maintenance of an islandora repository. with isle, the bulk of your repository's infrastructure is managed for you, and updates are as easy as pulling in new docker images. system administrators are only responsible for maintaining and updating their drupal site, and can rely on isle to handle fedora, solr, the triplestore, and all the other services we use to run a digital repository. the project began as a mellon grant funded initiative by the islandora collaboration group back in for islandora . then in january , the icg, born digital, lyrasis, cwrc, and the islandora foundation got together and started working on a version for islandora . this version would be a full community project, worked on in the open and residing in the islandora-devops github organization. what are the benefits of using isle? on top of being easier to install, run, and update, there are many awesome reasons to use isle for running islandora. first and foremost: speed. simply put, isle is fast! installation time is simply the amount of time it takes to download the images from dockerhub. for those who are building the images themselves, isle takes advantage of docker's buildkit feature for blazing fast builds. a complete rebuild of the entire stack consistently takes less than ten minutes on my laptop. and for small tweaks to the environment, builds often take seconds to make a change. compared to our ansible playbook, which usually takes around minutes for me, this is a significant boost to productivity when testing/deploying changes! because it's so quick, it lends itself well to automation using ci/cd tools like github actions and gitlab. the islandora foundation is "dogfooding" with isle, putting it at the center of its deployment strategy for future.islandora.ca and release testing. isle is also cross-platform. it is the first and only community supported way to run islandora on a windows machine. any windows computer with wsl can build and run isle. isle also supports arm builds, and can be run on cheaper cloud resources, newer macs with m chips, and even (theoretically) raspberry pis. how can i get isle? docker images for islandora are automatically pushed to dockerhub and are available here. if you want to run them using docker-compose, you can use isle-dc to build yourself a sandbox or a local development environment. upcoming sprint: metadata upcoming sprint: metadata dlamb wed, / / - : body our very own metadata interest group is running a sprint from march th to the th, and everyone's invited to participate. we'll be auditing the default metadata fields that we ship with and comparing them to the excellent metadata profile the mig has worked so hard to create for us. the goal of the sprint is just to find out where the gaps are so we know the full scope of work needed to implement their recommendations. if you can navigate the drupal fields ui (or just want to learn!), contributing is easy and would be super helpful to us. no programming required. and if you don't have an islandora instance to work on (or are having a hard time installing one), we're making a fresh sandbox just for the sprint. also, islandora foundation staff (a.k.a. me) and representatives from the mig will be on hand to help out and answer any questions you may have. you can sign up for the sprint here, and choose a metadata field to audit in this spreadsheet. as always, commit to as much or as little as you like. it only takes a couple minutes to check out a field and its settings to see if they line up with the recommendations. if we get enough folks to sign up, then many hands will make light work of this task! this is yet another sign of the strength of our awesome community. an interest group is taking it upon themselves to run a sprint to help achieve their goals, and the islandora foundation couldn't be happier to help. if you're a member of an interest group and want help engaging the community to make your goals happen, please feel free to reach out on slack or email me (dlamb@islandora.ca). islandora open meeting: february , islandora open meeting: february , manez wed, / / - : body we will be holding another open drop-in session on tuesday, february from : am to : pm eastern. full details, and the zoom link to join, are in this google doc. the meeting is free form, with experienced islandora users on hand to answer questions or give demos on request. please drop in at any time during the four-hour window. registration is not required. if you would like a calendar invite as a reminder, please let us know at community@islandora.ca. islandora open meeting: january , islandora open meeting: january , manez thu, / / - : body we will be holding another open drop-in session on january th from : am to : pm eastern. full details, and the zoom link to join, are in this google doc. the meeting is free form, with experienced islandora users on hand to answer questions or give demos on request. please drop in at any time during the four-hour window. registration is not required. if you would like a calendar invite as a reminder, please let us know at community@islandora.ca. nick ruest search nick ruest home c.v. posts presentations publications projects visualizations music contact nick ruest associate librarian york university biography nick ruest is an associate librarian in the digital scholarship infrastructure department at york university, co-principal investigator of the andrew w. mellon foundation funded the archives unleashed project, co-principal investigator of the sshrc grant “a longitudinal analysis of the canadian world wide web as a historical resource, - ”, and co-principal investigator of the compute canada research platforms and portals web archives for longitudinal knowledge. at york university, he oversees the libraries’ preservation initiatives, along with creating and implementing systems that support the capture, description, delivery, and preservation of digital objects having significant content of enduring value. he was previously active in the islandora and fedora communities, serving as project director for the islandora claw project, member of the islandora foundation’s roadmap committee and board of directors, and contributed code to the project. he has also served as the release manager for islandora and fedora, the moderator for the ocul digital curation community, the president of the ontario library and technology association, and president of mcmaster university academic librarians’ association. interests web archives data analytics distributed systems information retrieval digital preservation education mlis, wayne state university bachelor of arts political science, minor in history, university of michigan-dearborn recent publications more publications fostering community engagement through datathon events: the archives unleashed experience samantha fritz, ian milligan, nick ruest, jimmy lin pdf from archive to analysis: accessing web archives at scale through a cloud-based interface nick ruest, samantha fritz, ryan deschamps, jimmy lin, ian milligan pdf building community at distance: a datathon during covid- samantha fritz, ian milligan, nick ruest, jimmy lin pdf project content-based exploration of archival images using neural networks tobi adewoye, xiao han, nick ruest, ian milligan, samantha fritz, jimmy lin pdf project video the archives unleashed project: technology, process, and community to improve scholarly access to web archives nick ruest, jimmy lin, ian milligan, samantha fritz pdf project video we could, but should we? ethical considerations for providing access to geocities and other historical digital collections jimmy lin, ian milligan, douglas w. oard, nick ruest, katie shilton pdf the archives unleashed project: technology, process, and community to improve scholarly access to web archives nick ruest, jimmy lin, ian milligan, samantha fritz pdf solr integration in the anserini information retrieval toolkit ryan clancy, toke eskildsen, nick ruest, jimmy lin pdf dataset project building community and tools for analyzing web archives through datathons ian milligan, nathalie casemajor, samantha fritz, jimmy lin, nick ruest, matthew s. weber, nicholas worby pdf project slides scalable content-based analysis of images in web archives with tensorflow and the archives unleashed toolkit hsiu-wei yang, linqing liu, ian milligan, nick ruest, jimmy lin pdf dataset project poster recent & upcoming talks more talks lowering the barrier to access: the archives unleashed cloud project jun , the web that was: archives, traces, reflections resaw slides project sustainability and research platforms: the archives unleashed cloud project jun , international internet preservation consortium web archiving conference slides see a little warclight: building an open-source web archive portal with project blacklight jun , international internet preservation consortium web archiving conference slides web archives analysis at scale with the archives unleashed cloud (with ian milligan) apr , cni spring membership meeting slides oh, i get by with a little help from my friends: interdisciplinary web archive collaboration. feb , workshop on quantitative analysis and the digital turn in historical studies slides make it walk! may , archives association of ontario slides hot tips to boost your interdisciplinary web archive collaboration! apr , lewis & ruth sherman centre for digital scholarship speak series slides the world is a beautiful and terrible place mar , national forum on ethics and archiving the web slides video boosting your interdisciplinary web archive collaboration feb , bc research libraries group lecture series slides twitter and web archive analysis at scale feb , data love-in : a day of data management planning and conversations slides recent posts more posts four fucking years of donald trump nearly four years ago i decided to start collecting tweets to donald trump out of morbid curiosity. if i was a real archivist, i would … jan , min read enhancing archives unleashed toolkit usability with spark-submit originally posted here. over the last month, we have put out several toolkit releases. the primary focus of the releases has been … may , min read cloud-hosted web archive data: the winding path to web archive collections as data originally posted here. web archives are hard to use, and while the past activities of archives unleashed has helped to lower these … feb , min read twut. wait, wut? twut? originally posted here. introduction a few of the archives unleashed team members have a pretty in-depth background of working with … dec , min read exploring #elxn twitter data introduction a few years ago library archives canada, ian milligan and i collected tweets from the nd canadian federal election. ian … nov , min read projects archives unleashed project archives unleashed aims to make petabytes of historical internet content accessible to scholars and others interested in researching the recent past. supported by a grant from the andrew w. mellon foundation, we will be developing web archive search and data analysis tools to enable scholars and librarians to access, share, and investigate recent history since the early days of the world wide web. web archives for historical research our research focuses on both web histories - writing about the recent past as reflected in web archives - as well as methodological approaches to understanding these repositories. islandora claw islandora claw is the next generation of islandora. fedora repository fedora is the flexible, modular, open source repository platform with native linked data support. visualizations , #elxn images dear donald; may - january . happy new year to everyone, including the haters and the fake news media! totally clears the president. thank you! , , audio cover images from the internet archive , , #womensmarch images a month of tweets at @realdonaldtrump islandora claw development , , #panamapapers images , #ymmfire images , #thehip, #hipinkingston images , , #elxn images , #makedonalddrumpfagain images #elxn wordclouds by day anon development visualization islandora .x- . development music unfavourable offerings sloppy a-sides, last of the worst strange delights the humans soundtrackpro unlimited the achievements the potions - regular release ep sloppy b-sides, first of the worst surnom de gorille the potions @ the lifton matterwave foci . audio wardrobe's jacuzzi contact ruestn@yorku.ca cc-by · powered by the academic theme for hugo. cite × copy download tutela – learning network vai al contenuto tutela learning network menu home network what we do contact a space for critical thinking and horizontal learning ci hanno accusato di voler dividere le famiglie, ma noi volevamo tutt’altro. abbiamo cercato di far capire che l’associazione si proponeva come un appoggio forte alle famiglie. anche per la mia famiglia è stato difficile far comprendere perché volevo questo tipo di organizzazione. ho dovuto tranquillizzarle/i che la mia intenzione non era dar vita ad una ribellione all’interno della comunità, ma aprire una strada diversa per appoggiare le donne rom. (anna várnai, ungheria) la sociedad tiene que educar a sus chicas y mujeres a pensar de manera independiente. en lugar de enfatizar tanto la virtud del sacrificio, tendría que animarlas para que se den prioridad. tendríamos que enseñarles que reclamar sus derechos es justo y normal. igualmente, la sociedad tendría que entender que es normal que las mujeres que viven en relaciones abusivas se sientan vulnerables y débiles. las chicas jóvenes y las mujeres deben saber que solo cuando cuidamos de nosotras mismas y nos valoramos, logramos encontrar nuestro verdadero ser y nuestra fuerza interior, para así aceptar quienes somos. (swati kamble) voices asociación la colectiva articoli recenti resonancias: primer encuentro marzo, common place: talking about co-housing and communities of sharing marzo, rassegna ‘this is not an altas’ febbraio, home network what we do contact blog su wordpress.com. none none inkdroid inkdroid paper or plastic coincidence? twarc this post was originally published on medium but i spent time writing it so i wanted to have it here too. tl;dr twarc has been redesigned from the ground up to work with the new twitter v api and their academic research track. many thanks for the code and design contributions of betsy alpert, igor brigadir, sam hames, jeff sauer, and daniel verdeer that have made twarc possible, as well as early feedback from dan kerchner, shane lin, miles mccain, 李荣蓬, david thiel, melanie walsh and laura wrubel. extra special thanks to the institute for future environments at queensland university of technology for supporting betsy and sam in their work, and for the continued support of the mellon foundation. back in august of last year twitter announced early access to their new v api, and their plans to sunset the v . api that has been active for almost the last years. over the lifetime of their v . api twitter has become deeply embedded in the media landscape. as magazines, newspapers and television have moved onto the web they have increasingly adopted tweets as a mechanism for citing politicians, celebrities and organizations, while also using them to document current events, generate leads and gather feedback for evolving stories. as a result twitter has also become a popular object of study for humanities and social science researchers looking to understand the world as reflected, refracted and distorted by/in social media. on the surface the v api update seems pretty insignificant since the shape of a tweet, its parts, properties and affordances, aren’t changing at all. tweets with characters of text, images and video will continue to be posted, retweeted and quoted. however behind the scenes the representation of a tweet as data, and the quotas that control the rates at which this data can flow between apps and other third party services will be greatly transformed. needless to say, v represents a big change for the documenting the now project. along with community members we’ve developed and maintained open source tools like twarc that talk directly to the twitter api to help users to search for and collect live tweets that match criteria like hashtags, names and geographic locations. today we’re excited to announce the release of twarc v which has been designed from the ground up to work with the v api and twitter’s new academic research track. clearly it’s extremely problematic having a multi-national corporation act as a gatekeeper for who counts as an academic researcher, and what constitutes academic research. we need look no further than the recent experiences of timnit gebru and margaret mitchell at google for an example of what happens when research questions run up against the business objectives of capital. we only know their stories because gebru and mitchell’s bravely took a principled approach, where many researchers would have knowingly or unknowingly shaped their research to better fit the needs of the company. so it is important for us that twarc still be usable by people with and without access to the academic research track. but we have heard from many users that the academic research track presents new opportunities for twitter data collection that are essential for researchers interested in the observability of social media platforms. twitter is making a good faith effort to work with the academic research community, and we thought twarc should support it, even if big challenges lie ahead. so why are people interested in the academic research track? once your application has been approved you are able to collect data from the full history of tweets, at no cost. this is a massive improvement over the v . access which was limited to a one week window and researchers had to pay for access. access to the full archive means it’s now possible to study events that have happened in the past back to the beginning of twitter in . if you do create any historical datasets we’d love for you to share the tweet identifier datasets in the catalog. however this opening up of access on the one hand comes with a simultaneous contraction in terms of how much data can be collected at one time. the remainder of this post describes some of the details and the design decisions we have made with twarc to address them. if you would prefer to watch a quick introduction to using twarc v please check out this short video: installation if you are familiar with installing twarc nothing is changed. you still install (or upgrade) with pip as you did before: $ pip install --upgrade twarc in fact you will still have full access to the v . api just as you did before. so the old commands will continue to work as they did $ twarc search blacklivesmatter > tweets.jsonl twarc was designed to let you to continue to use twitter’s v . api undisturbed until it is finally turned off by twitter, at which point the functionality will be removed from twarc. all the support for the v api is mediated by a new command line utility twarc . for example to search for blacklivesmatter tweets and write them to a file tweets.jsonl: $ twarc search blacklivesmatter > tweets.jsonl all the usual twarc functionality such as searching for tweets, collecting live tweets from the streaming api endpoint, requesting user timelines and user metadata are all still there, twarc --help gives you the details. but while the interface looks the same there’s quite a bit different going on behind the scenes. representation truth be told, there is no shortage of open source libraries and tools for interacting with the twitter api. in the past twarc has made a bit of a name for itself by catering to a niche group of users who want a reliable, programmable way to collect the canonical json representation of a tweet. javascript object notation (json) is the language of web apis, and twitter has kept its json representation of a tweet relatively stable over the years. rather than making lots of decisions about the many ways you might want to collect, model and analyze tweets twarc has tried to do one thing and do it well (data collection) and get out of the way so that you can use (or create) the tools for putting this data to use. but the json representation of a tweet in the twitter v api is completely burst apart. the v base representation of a tweet is extremely lean and minimal, and just includes the text of the tweet its identifier and a handful of other things. all the details about the user who created the tweet, embedded media, and more are not included. fortunately this information is still available, but the user needs to craft their api request to request tweets using a set of expansions that tell the twitter api what additional entities to include. in addition for each expansion there are a set of field options to include that control what of these expansions is returned. so rather than there being a single json representation of a tweet api users now have the ability to shape the data based on what they need, much like how graphql apis work. this kind of makes you wonder why twitter didn’t make their graphql api available. for specific use cases this customizability is very useful, but the mutability of the representation of a tweet presents challenges when collecting data for future use. if you didn’t request the right expansions or fields when collecting the data then you won’t be able to analyze that data later when doing your research. to solve for this twarc has been designed to collect the richest possible representation for a tweet, by requesting all possible expansions and field combinations for tweets. see the expansions module for the details if you are interested. this takes a significant burden off of users to digest the api documentation, and craft the correct api requests themselves. in addition the twarc community will be monitoring the twitter api documentation going forward to incorporate new expansions and fields as they will inevitably be added in the future. flattening this is diving into the weeds a little bit, but it’s worth noting here that twitter’s introduction of expansions allows data that was once duplicated across multiple tweets (such as user information, media, retweets, etc) to be included once per response from the api. this means that instead of seeing information about the user who created a tweet in the context of their tweet the user will be referenced using an identifier, and this identifier will map to user metadata in the outer envelope of the response. it makes sense why twitter have introduced expansions since it means in a set of tweets from a given user the user information will just be included once rather than repeated times, which means less data, less network traffic and less money. it’s even more significant when consider the large number of possible expansions. however this pass by-reference rather than by-value presents some challenges for stream based processing which expects each tweet to be self-contained. for this reason we’ve introduce the idea of flattening the response data when persisting the json to disk. this means that tools and data pipelines that expect to operate on a stream of tweets can continue to do so. since the representation of a tweet is so dependent on how data is requested we’ve taken the opportunity to introduce a small stanza of twarc specific metadata using the __twarc prefix. this metadata records what api endpoint the data was requested from, and when. this information is critically important when interpreting the data, because some information about a tweet like its retweet and quote counts are constantly changing. data flows as mentioned above you can still collect tweets from the search and streaming api endpoints in a way that seems quite similar to the v api. the big changes however are the quotas associated with these endpoints which govern how much can be collected. these quotas control how many requests can be sent to twitter in minute intervals. in fact these quotas are not much changed, but what’s new are app wide quotas that constrain how many tweets a given application (app) can collect every month. an app in this context is a piece of software (e.g. your twarc software) identified by unique api keys set up in the twitter developer portal. the standard api access sets a , tweet per month limit. this is a huge change considering there were no monthly app limits before. if you get approved for the academic research track your app quota is increased to million per month. this is markedly better but the achievable data volume is still nothing like the v . api, as these graphs attempt to illustrate: twarc will still observe the same rate limits, but once you’ve collected your portion for the month there’s not much that can be done, for that app at least. apart from the quotas twitter’s streaming endpoint in v is substantially changed which impacts how users interact with twarc. previously twarc users would be able to create up to to two connections to the filter stream api. this could be done by simply: twarc filter obama > obama.jsonl however in the twitter v api only apps can connect to the filter stream, and they can only connect once. at first this seems like a major limitation but rather than creating a connection per query the v api allows you to build a set of rules for tweets to match, which in turns controls what tweets are included in the stream. this means you can collect for multiple types of queries at the same time, and the tweets will come back with a piece of metadata indicating what rule caused its inclusion. this translates into a markedly different set of interactions at the command line for collecting from the stream where you first need to set your stream rules and then open a connection to fetch it. twarc stream-rules add blacklivesmatter twarc stream > tweets.jsonl one useful side effect of this is that you can update the stream (add and remove rules) while the stream is in motion: twarc stream-rules add blm while you are limited by the api quota in terms of how many tweets you can collect, tweets are not “dropped on the floor” when the volume gets too high. once upon a time the v . filter stream was rumored to be rate limited when your stream exceeds % of the total volume of new tweets. plugins in addition to twarc helping you collect tweets the github repository has also been a place to collect a set of utilities for working with the data. for example there are scripts for extracting and unshortening urls, identifying suspended/deleted content, extracting videos, buiding wordclouds, putting tweets on maps, displaying network graph visualizations, counting hashtags, and more. these utilities all work like unix filters where the input is a stream of tweets and the output varies depending on what the utility is doing, e.g. a gephi file for a network visualization, or a folder of mp files for video extraction. while this has worked well in general the kitchen sink approach has been difficult to manage from a configuration management perspective. users have to download these scripts manually from github or by cloning the repository. for some users this is fine, but it’s a bit of a barrier to entry for users who have just installed twarc with pip. furthermore these plugins often have their own dependencies which twarc itself does not. this lets twarc can stay pretty lean, and things like youtube_dl, networkx or pandas can be installed by people that want to use utilities that need them. but since there is no way to install the utilities there isn’t a way to ensure that the dependencies are installed, which can lead to users needing to diagnose missing libraries themselves. finally the plugins have typically lacked their own tests. twarc’s test suite has really helped us track changes to the twitter api and to make sure that it continues to operate properly as new functionality has been added. but nothing like this has existed for the utilities. we’ve noticed that over time some of them need updating. also their command line arguments have drifted over time which can lead to some inconsistencies in how they are used. so with twarc we’ve introduced the idea of plugins which extend the functionality of the twarc command, are distributed on pypi separately from twarc, and exist in their own github repositories where they can be developed and tested independently of twarc itself. this is all achieved through twarc ’s use of the click library and specifically click-plugins. so now if you would like to convert your collected tweets to csv you can install the twarc-csv: $ pip install twarc-csv $ twarc search covid > covid .jsonl $ twarc csv covid .jsonl > covid .csv or if you want to extract embedded and referenced videos from tweets you can install twarc-videos which will write all the videos to a directory: $ pip install twarc-videos $ twarc videos covid .jsonl --download-dir covid -videos you can write these plugins yourself and release them as needed. check out the plugin reference implementation tweet-ids for a simple example to adapt. we’re still in the process of porting some of the most useful utilities over and would love to see ideas for new plugins. check out the current list of twarc plugins and use the twarc issue tracker on github to join the discussion. you may notice from the list of plugins that twarc now (finally) has documentation on readthedocs external from the documentation that was previously only available on github. we got by with github’s rendering of markdown documents for a while, but github’s boilerplate designed for developers can prove to be quite confusing for users who aren’t used to selectively ignoring it. readthedocs allows us to manage the command line and api documentation for twarc, and to showcase the work that has gone into the spanish, japanese, portuguese, swedish, swahili and chinese translations. feedback thanks for reading this far! we hope you will give twarc a try. let us know what you think either in comments here, in the docnow slack or over on github. ✨ ✨ happy twarcing! ✨ ✨ ✨ windows users will want to indicate the output file using a second argument rather than redirecting output with >. see this page for details.↩ $ j you may have noticed that i try to use this static website as a journal. but, you know, not everything i want to write down is really ready (or appropriate) to put here. some of these things end up in actual physical notebooks–there’s no beating the tactile experience of writing on paper for some kind of thinking. but i also spend a lot of time on my laptop, and at the command line in some form or another. so i have a directory of time stamped markdown files stored on dropbox, for example: ... /home/ed/dropbox/journal/ - - .md /home/ed/dropbox/journal/ - - .md /home/ed/dropbox/journal/ - - .md /home/ed/dropbox/journal/ - - .md /home/ed/dropbox/journal/ - - .md ... sometimes these notes migrate into a blog post or some other writing i’m doing. i used this technique quite a bit when writing my dissertation when i wanted to jot down things on my phone when an idea arrived. i’ve tried a few different apps for editing markdown on my phone, but mostly settled on ia writer which mostly just gets out of the way. but when editing on my laptop i tend to use my favorite text editor vim with the vim-pencil plugin for making markdown fun and easy. if vim isn’t your thing and you use another text editor keep reading since this will work for you too. the only trick to this method of journaling is that i just need to open the right file. with command completion on the command line this isn’t so much of a chore. but it does take a moment to remember the date, and craft the right path. today while reflecting on how nice it is to still be using unix, it occurred to me that i could create a little shell script to open my journal for that day (or a previous day). so i put this little file j in my path: #!/bin/zsh journal_dir="/home/ed/dropbox/journal" if [ "$ " ]; then date=$ else date=`date +%y-%m-%d` fi vim "$journal_dir/$date.md" so now when i’m in the middle of something else and want to jot a note in my journal i just type j. unix, still crazy after all these years. strengths and weaknesses quoting macey ( ), quoting foucault, quoting nietzsche: one thing is needful. – to ‘give style’ to one’s character – a great and rare art! it is practised by those who survey all the strengths and weaknesses that their nature has to offer and then fit them into an artistic plan until each appears as art and reason and even weaknesses delight the eye. nietzsche, williams, nauckhoff, & del caro ( ), p. this is a generous and lively image of what art does when it is working. art is not perfection. macey, d. ( ). the lives of michel foucault: a biography. verso. nietzsche, f. w., williams, b., nauckhoff, j., & del caro, a. ( ). the gay science: with a prelude in german rhymes and an appendix of songs. cambridge, u.k. ; new york: cambridge university press. data speculation i’ve taken the ill-advised approach of using the coronavirus as a topic to frame the exercises in my computer programming class this semester. i say “ill-advised” because given the impact that covid has been having on students i’ve been thinking they probably need a way to escape news of the virus by way of writing code, rather than diving into it more. it’s late in the semester to modulate things but i think we will shift gears to look at programming through another lens after spring break. that being said, one of the interesting things we’ve been doing is looking at vaccination data that is being released by the maryland department of health through their esri arcgis hub. note: this dataset has since been removed from the web because it has been superseded by a new dataset that includes single dose vaccinations. i guess it’s good that students get a feel for how ephemeral data on the web is, even when it is published by the government. we noticed that this dataset recorded a small number of vaccinations as happening as early as the s up until december , when vaccines were approved for use. i asked students to apply what we have been learning about python (files, strings, loops, and sets) to identify the maryland counties that were responsible for generating this anomalous data. i thought this exercise provided a good demonstration using real, live data that critical thinking about the provenance of data is always important because there is no such thing as raw data (gitelman, ). while we were working with the data to count the number of anomalous vaccinations per county one of my sharp eyed students noticed that the results we were seeing with my version of the dataset (downloaded on february ) were different from what we saw with his (downloaded on march ). we expected to see new rows in the later one because new vaccination data seem to be reported daily–which is cool in itself. but we were surprised to find new vaccination records for dates earlier than december , . why would new vaccinations for these erroneous older dates still be entering the system? for example the second dataset downloaded march acquired new rows: object id vaccination date county daily first dose cumulative first dose daily second dose cumulative second dose / / allegany / / baltimore / / baltimore / / baltimore city / / baltimore / / prince george’s and these rows present in the february version were deleted in the march version: object id vaccination date county daily first dose cumulative first dose daily second dose cumulative second dose / / frederick / / talbot / / baltimore / / caroline / / prince george’s / / anne arundel / / frederick / / wicomico / / frederick i found these additions perplexing at first, because i assumed these outliers were part of an initial load. but it appears that the anomalies are still being generated? the deletions suggest that perhaps the anomalous data is being identified and scrubbed in a live system that is then dumping out the data? or maybe the code that is being used to update the dataset in arcgis hub itself is malfunctioning in some way? if you are interested in toying around with the code and data it is up on github. i was interested to learn about pandas.dataframe.merge which is useful for diffing tables when you use indicator=true. at any rate, having students notice, measure and document anomalies like this seems pretty useful. i also asked them to speculate about what kinds of activities could generate these errors. i meant speculate in the speculative fiction sense of imagining a specific scenario that caused it. i think this made some students scratch their head a bit, because i wasn’t asking them for the cause, but to invent a possible cause. based on the results so far i’d like to incorporate more of these speculative exercises concerned with the functioning of code and data representations into my teaching. i want to encourage students to think creatively about data processing as they learn about the nuts and bolts of how code operates. for example the treatments in how to run a city like amazon, and other fables which use sci-fi to test ideas about how information technologies are deployed in society. another model is the speculative ethics book club which also uses sci-fi to explore the ethical and social consequences of technology. i feel like i need to read up on specualtive research more generally before doing this though (michael & wilkie, ). i’d also like to focus the speculation down at the level of the code or data processing, rather than at the macro super-system level. but that has its place too. another difference is that i was asking students to engage in speculation about the past rather than the future. how did the data end up this way? perhaps this is more of a genealogical approach, of winding things backwards, and tracing what is known. maybe it’s more mystery than sci-fi. the speculative element is important because (in this case) operations at the md dept of health, and their arcgis hub setup are mostly opaque to us. but even when access isn’t a problem these systems they can feel opaque, because rather than there being a dearth of information you are drowning in it. speculation is a useful abductive approach to hypothesis generation and, hopefully, understanding. update - - : over in the fediverse david benque recommended i take a look at matthew stanley’s chapter in (gitelman, ) “where is that moon, anyway? the problem of interpreting historical solar eclipse observations” for the connection to mystery. for the connection to peirce and abduction he also pointed to luciana parisi’s chapter “speculation: a method for the unattainable” in lury & wakeford ( ). definitely things to follow up on! references gitelman, l. (ed.). ( ). “raw data” is an oxymoron. mit press. lury, c., & wakeford, n. ( ). inventive methods: the happening of the social. routledge. michael, m., & wilkie, a. ( ). speculative research. in the palgrave encyclopedia of the possible (pp. – ). cham: springer international publishing. retrieved from https://doi.org/ . / - - - - _ - recovering foucault i’ve been enjoying reading david macey’s biography of michel foucault, that was republished in by verso. macey himself is an interesting figure, both a scholar and an activist who took leave from academia to do translation work and to write this biography and others of lacan and fanon. one thing that struck me as i’m nearing the end of macey’s book is the relationship between foucault and archives. i think foucault has become emblematic of a certain brand of literary analysis of “the archive” that is far removed from the research literature of archival studies, while using “the archive” as a metaphor (caswell, ). i’ve spent much of my life working in libraries and digital preservation, and now studying and teaching about them from the perspective of practice, so i am very sympathetic to this critique. it is perhaps ironic that the disconnect between these two bodies of research is a difference in discourse which foucault himself brought attention to. at any rate, the thing that has struck me while reading this biography is how much time foucault himself spent working in libraries and archives. here’s foucault in his own words talking about his thesis: in histoire de la folie à l’âge classique i wished to determine what could be known about mental illness in a given epoch … an object took shape for me: the knowledge invested in complex systems of institutions. and a method became imperative: rather than perusing … only the library of scientific books, it was necessary to consult a body of archives comprising decrees, rules hospital and prison registers, and acts of jurisprudence. it was in the arsenal or the archives nationales that i undertook the analysis of a knowledge whose visible body is neither scientific nor theoretical discourse, nor literature, but a daily and regulated practice. (macey, , p. ) foucault didn’t simply use archives for his research: understanding the processes and practices of archives were integral to his method. even though the theory and practice of libraries and archives are quite different given their different functions and materials, they are often lumped together as a convenience in the same buildings. macey blurs them a little bit, in sections like this where he talks about how important libraries were to foucault’s work: foucault required access to paris for a variety of reasons, not least because he was also teaching part-time at ens. the putative thesis he had begun at the fondation thiers – and which he now described to polin as being on the philosophy of psychology – meant that he had to work at the bibliothèque nationale and he had already become one of its habitues. for the next thirty years, henri labrouste’s great building in the rue de richelieu, with its elegant pillars and arches of cast iron, would be his primary place of work. his favourite seat was in the hemicycle, the small, raised section directly opposite the entrance, sheltered from the main reading room, where a central aisle separates rows of long tables subdivided into individual reading desks. the hemicycle affords slighty more quiet and privacy. for thirty years, foucault pursued his research here almost daily, with occasional forays to the manuscript department and to other libraries, and contended with the byzantine cataloguing system: two incomplete and dated printed catalogues supplemented by cabinets containing countless index cards, many of them inscribed with copperplate handwriting. libraries were to become foucault’s natural habitat: ‘those greenish institutions where books accumulate and where there grows the dense vegetation of their knowledge’ there’s a metaphor for you: libraries as vegetation :) it kind of reminds me of some recent work looking at decentralized web technologies in terms of mushrooms. but i digress. i really just wanted to note here that the erasure of archival studies from humanities research about “the archive” shouldn’t really be attributed to foucault, whose own practice centered the work of libraries and archives. foucault wasn’t just writing about an abstract archive, he was practically living out of them. as someone who has worked in libraries and archives i can appreciate how power users (pun intended) often knew aspects of the holdings and intricacies of their their management better than i did. archives, when they are working, are always collaborative endeavours, and the important thing is to recognize and attribute the various sides of that collaboration. ps. writing this blog post led me to dig up a few things i want to read (eliassen, ; radford, radford, & lingel, ). references caswell, m. ( ). the archive is not an archives: on acknowledging the intellectual contributions of archival studies. reconstruction, ( ). retrieved from http://reconstruction.eserver.org/issues/ /caswell.shtml eliassen, k. ( ). archives of michel foucualt. in e. røssaak (ed.), the archive in motion, new conceptions of the archive in contemporary thought and new media practices. novus press. macey, d. ( ). the lives of michel foucault: a biography. verso. radford, g. p., radford, m. l., & lingel, j. ( ). the library as heterotopia: michel foucault and the experience of library space. journal of documentation, ( ), – . teaching oop in the time of covid i’ve been teaching a section of the introduction to object oriented programming at the umd college for information studies this semester. it’s difficult for me, and for the students, because we are remote due to the coronavirus pandemic. the class is largely asynchronous, but every week i’ve been holding two synchronous live coding sessions in zoom to discuss the material and the exercises. these have been fun because the students are sharp, and haven’t been shy about sharing their screen and their vscode session to work on the details. but students need quite a bit of self-discipline to move through the material, and probably only about / of the students take advantage of these live sessions. i’m quite lucky because i’m working with a set of lectures, slides and exercises that have been developed over the past couple of years by other instructors: josh westgard, aric bills and gabriel cruz. you can see some of the public facing materials here. having this backdrop of content combined with severance’s excellent (and free) python for everybody has allowed me to focus more on my live sessions, on responsive grading, and to also spend some time crafting additional exercises that are geared to this particular moment. this class is in the college for information studies and not in the computer science department, so it’s important for the students to not only learn how to use a programming language, but to understand programming as a social activity, with real political and material effects in the world. being able to read, understand, critique and talk about code and its documentation is just as important as being able to write it. in practice, out in the “real world” of open source software i think these aspects are arguably more important. one way i’ve been trying to do this in the first few weeks of class is to craft a sequence of exercises that form a narrative around coronavirus testing and data collection to help remind the students of the basics of programming: variables, expressions, conditionals, loops, functions, files. in the first exercise we imagined a very simple data entry program that needed to record results of real-time polymerase chain reaction tests (rt-pcr). i gave them the program and described how it was supposed to work, and asked them describe (in english) any problems that they noticed and to submit a version of the program with problems fixed. i also asked them to reflect on a request from their boss about adding the collection of race, gender and income information. the goal here was to test their ability to read the program and write english about it while also demonstrating a facility for modifying the program. most importantly i wanted them to think about how inputs such as race or gender have questions about categories and standards behind them, and weren’t simply a matter of syntax. the second exercise builds on the first by asking them to adjust the revised program to be able to save the data in a very particular format. yes, in the first exercise the data is stored in memory and printed to the screen in aggregate at the end. the scenario here is that the department of health and human services has assumed the responsibility for covid test data collection from the centers for disease control. of course this really happened, but the data format i chose was completely made up (maybe we will be working with some real data at the end of the semester if i continue with this theme). the goal in this exercise was to demonstrate their ability to read another program and fit a function into it. the students were given a working program that had a save_results() function stubbed out. in addition to submitting their revised code i asked them to reflect on some limitations of the data format chosen, and the data processing pipeline that it was a part of. and in the third exercise i asked them to imagine that this lab they were working in had a scientist who discovered a problem with some of the thresholds for acceptable testing, which required an update to the program from exercise , and also a test suite to make sure the program was behaving properly. in addition to writing the tests i asked them to reflect on what functionality was not being tested that probably should be. this alternation between writing code and writing prose is something i started doing as part of a digital curation class. i don’t know if this dialogical or perhaps dialectical, approach is something others have tried. i should probably do some research to see. in my last class i alternated week by week: one week reading and writing code, the next week reading and writing prose. but this semester i’ve stayed focused on code, but required the reading and writing of code as well as prose about code in the same week. i hope to write more about how this goes, and these exercises as i go. i’m not sure if i will continue with the coronavirus data examples. one thing i’m sensitive to is that my students themselves are experiencing the effects of the coronavirus, and may want to escape it just for a bit in their school work. just writing in the open about it here, in addition to the weekly meetings i’ve had with aric, josh and gabriel has been very useful. speaking of those meetings. i learned today from aric that tomorrow (february th, ) is the th anniversary of python’s first public release! you can see this reflected in this timeline. this v . . release was the first release guido van rossum made outside of cwi and was made on the usenet newsgroup alt.sources where it is split out into chunks that need to be reassembled. back in andrew dalke located a and repackaged these sources in google groups which acquired alt.sources as part of dejanews in . but if you look at the time stamp on the first part of the release you can see that it was made february , (not february ). so i’m not sure if the birthday is actually today. i sent this little note out to my students with this wonderful two part oral history that the computer history museum did with guido van rossum a couple years ago. i turns out both of his parents were atheists and pacifists. his dad went to jail because he refused to be conscripted into the military. that and many more details of his background and thoughts about the evolution of python can be found in these delightful interviews: happy birthday python! gpt- jam one of the joys of pandemic academic life has been a true feast of online events to attend, on a wide variety of topics, some of which are delightfully narrow and esoteric. case in point was today’s reflecting on power and ai: the case of gpt- which lived up to its title. i’ll try to keep an eye out for when the video posts, and update here. the workshop was largely organized around an exploration of whether gpt- , the largest known machine learning language model, changes anything for media studies theory, or if it amounts to just more of the same. so the discussion wasn’t focused so much on what games could be played with gpt- , but rather if gpt- changes the rules of the game for media theory, at all. i’m not sure there was a conclusive answer at the end, but it sounded like the consensus was that current theorization around media is adequate for understanding gpt- , but it matters greatly what theory or theories are deployed. the online discussion after the presentations indicated that attendees didn’t see this as merely a theoretical issue, but one that has direct social and political impacts on our lives. james steinhoff looked at gpt- using a marxist media theory perspective where he told the story of gpt- ’s as a project of openai and as a project of capital. openai started with much fanfare in as a non-profit initiative where the technology, algorithms and models developed would would be kept openly licensed and freely available so that the world could understand the benefits and risks of ai technology. steinhoff described how in the project’s needs for capital (compute power and staff) transitioned it from a non-profit into a capped-profit company, which is now owned, or at least controlled, by microsoft. the code for generating the model as well as the model itself are gated behind a token driven web api run my microsoft. you can get on a waiting list to use it, but apparently a lot of people have been waiting a while, so … being a microsoft employee probably helps. i grabbed a screenshot of the pricing page that steinhoff shared during his presentation: i’d be interested to hear more about how these tokens operate. are they per-request, or are they measured according something else? i googled around a bit during the presentation to try to find some documentation for the web api, and came up empty handed. i did find shreya shankar’s gpt -sandbox project for interacting with the api in your browser (mostly for iteratively crafting text input in order to generate desired output). it depends on the openai python package created by openai themselves. the docs for openai then point at a page on the openai.com website which is behind a login. you can create an account, but you need to be pre-approved (made it through the waitlist) to be able to see the docs. there’s probably some sense that can be made from examining the python client though. all of the presentations in some form or another touched on the billion parameters that were used to generate the model. but the api to the model doesn’t have that many parameters. it allows you to enter text and get text back. but the api surface that the gpt- service provides could be interesting to examine a bit more closely, especially to track how it changes over time. in terms of how this model mediates knowledge and understanding it’ll be important watch. steinhoff’s message seemed to be that, despite the best of intentions, gpt- functions in the service of very large corporations with very particular interests. one dimension that he didn’t explore perhaps because of time, is how the gpt- model itself is fed massive amounts of content from the web, or the commons. indeed % of the data came from the commoncrawl project. gpt- is an example of an extraction project that has been underway at large internet companies for some time. i think the critique of these corporations has often been confined to seeing them in terms of surveillance capitalism rather than in terms of raw resource extraction, or the primitive accumulation of capital. the behavioral indicators of who clicked on what are certainly valuable, but gpt- and sister projects like commoncrawl shows just the accumulation of data with modest amounts of metadata can be extremely valuable. this discussion really hit home for me since i’ve been working with jess ogden and shawn walker using commoncrawl as a dataset for talking about the use of web archives, while also reflecting on the use of web archives as data. commoncrawl provides a unique glimpse into some of the data operations that are at work in the accumulation of web archives. i worry that the window is closing and the commoncrawl itself will be absorbed into microsoft. following steinhoff olya kudina and bas de boer jointly presented some compelling thoughts about how its important to understand gpt- in terms of sociotechnical theory, using ideas drawn from foucault and arendt. i actually want to watch their presentation again because it followed a very specific path that i can’t do justice to here. but their main argument seemed to be that gpt- is an expression of power and that where there is power there is always resistance to power. gpt- can and will be subverted and used to achieve particular political ends of our own choosing. because of my own dissertation research i’m partial to foucault’s idea of governmentality, especially as it relates to ideas of legibility (scott, )–the who, what and why of legibility projects, aka archives. gpt- presents some interesting challenges in terms of legibility because the model is so complex, the results it generates defy deductive logic and auditing. in some ways gpt- obscures more than it makes a population legible, as foucault moved from disciplinary analysis of the subject, to the ways in which populations are described and governed through the practices of pastoral power, of open datasets. again the significance of commoncrawl as an archival project, as a web legibility project, jumps to the fore. i’m not as up on arendt as i should be, so one outcome of their presentation is that i’m going to read her the human condition which they had in a slide. i’m long overdue. references scott, j. c. ( ). seeing like a state: how certain schemes to improve the human condition have failed. yale university press. mimetypes today i learned that python has a mimetypes module, and has ever since guido von rossum added it in . honestly i’m just a bit sheepish to admit this discovery, as someone who has been using python for digital preservation work for about years. but maybe there’s a good reason for that. since the entire version history for python is available on github (which is a beautiful thing in itself) you can see that the mimetypes module started as a guess_type() function built around a pretty simple hard coded mapping of file extensions to mimetypes. the module also includes a little bit of code to look for, and parse, mimetype registries that might be available on the host operating system. the initial mimetype registries used included one from the venerable apache httpd web server, and the netscape web browser, which was about three years old at the time. it makes sense why this function to look up a mimetype for a filename would be useful at that time, since python was being used to serve up files on the nascent web and for sending email, and whatnot. today the module looks much the same, but has a few new functions and about twice as many mimetypes in its internal list. some of the new mimetypes include text/csv, audio.mpeg, application/vnd.ms-powerpoint, application/x-shockwave-flash, application/xml, and application/json. comparing the first commit to the most latest provides a thumbnail sketch of years of web format evolution. i’ll admit, this is is a bit of an esoteric thing to be writing a blog post about. so i should explain. at work i’ve been helping out on a community archiving project which has accumulated a significant amount of photographs, scans, documents of various kinds, audio files and videos. some of these files are embedded in web applications like omeka, some are in cloud storage like google drive, or on the office networked attached storage, and others are on scattered storage devices in people’s desk drawers and closets. we’ve also created new files during community digitization events, and oral history interviews. as part of this work we’ve wanted to start building a place on the web where all these materials live. this has required not only describing the files, but also putting all the files in one place so that access can be provided. in principle this sounds simple. but it turns out that collecting the files from all these diverse locations poses significant challenges, because their context matters. the filenames, and the directories they are found in, are sometimes the only descriptive metadata that exists for this data. in short, the original order matters. but putting this content on the web means that the files need to be brought together and connected with their metadata programmatically. this is how i stumbled across the mimetypes module. i’ve been writing some throwaway code to collect the files together into the same directory structure while preserving their original filenames and locations in an airtable database. i’ve been using the magic module to identify the format of the file, which is used to copy the file into a dropbox storage location. the extension is important because we are expecting this to be a static site serving up the content and we want the files to also be browsable using the dropbox drive. it turns out the mimetypes.guess_extension is pretty useful for turning a mediatype into an file extension. i’m kind of surprised that it took me this long to discover mimetypes, but i’m glad i did. as an aside i think this highlights for me how important git can be as an archive and research method for software studies work. northwest branch cairn here is a short recording and a couple photos from my morning walk along the northwest branch trail with penny. i can’t go every day but at months old she has tons of energy, so it’s generally a good idea for all concerned to go at least every other morning. and it’s a good thing, because the walk is surprisingly peaceful, and it’s such a joy to see her run through the woods. after walking about minutes there is this little cairn that is a reminder for me to turn around. after seeing it grow in size i was sad to see it knocked down one day. but, ever so slowly, it is getting built back up again. none none simple mail transfer protocol - wikipedia simple mail transfer protocol from wikipedia, the free encyclopedia jump to navigation jump to search internet protocol used for relaying e-mails "smtp" redirects here. for the email delivery company, see smtp (company). for short message transfer protocol, see gsm . . internet protocol suite application layer bgp dhcp dns ftp http https imap ldap mgcp mqtt nntp ntp pop ptp onc/rpc rtp rtsp rip sip smtp snmp ssh telnet tls/ssl xmpp more... transport layer tcp udp dccp sctp rsvp more... internet layer ip ipv ipv icmp icmpv ecn igmp ipsec more... link layer arp ndp ospf tunnels l tp ppp mac ethernet wi-fi dsl isdn fddi more... v t e the simple mail transfer protocol (smtp) is an internet standard communication protocol for electronic mail transmission. mail servers and other message transfer agents use smtp to send and receive mail messages. user-level email clients typically use smtp only for sending messages to a mail server for relaying, and typically submit outgoing email to the mail server on port or per rfc . for retrieving messages, imap and pop are standard, but proprietary servers also often implement proprietary protocols, e.g., exchange activesync. since smtp's introduction in , it was updated, modified and extended multiple times. the protocol version in common use today has extensible structure with various extensions for authentication, encryption, binary data transfer, internationalized email addresses. smtp servers commonly use the transmission control protocol on port number (for plaintext) and (for encrypted communications). contents history . predecessors to smtp . original smtp . modern smtp mail processing model protocol overview . smtp vs mail retrieval . remote message queue starting outgoing mail smtp server . outgoing mail server access restrictions . . restricting access by location . . client authentication . ports smtp transport example smtp extensions . extension discovery mechanism . binary data transfer . mail delivery mechanism extensions . on-demand mail relay . internationalization extension . extensions . . bitmime . . smtp-auth . . smtputf . security extensions . . starttls or "opportunistic tls" . . smtp mta strict transport security . . smtp tls reporting spoofing and spamming implementations related requests for comments see also notes references external links history[edit] predecessors to smtp[edit] various forms of one-to-one electronic messaging were used in the s. users communicated using systems developed for specific mainframe computers. as more computers were interconnected, especially in the u.s. government's arpanet, standards were developed to permit exchange of messages between different operating systems. smtp grew out of these standards developed during the s. smtp traces its roots to two implementations described in : the mail box protocol, whose implementation has been disputed,[ ] but is discussed in rfc and other rfcs, and the sndmsg program, which, according to rfc , ray tomlinson of bbn invented for tenex computers to send mail messages across the arpanet.[ ][ ][ ] fewer than hosts were connected to the arpanet at this time.[ ] further implementations include ftp mail[ ] and mail protocol, both from .[ ] development work continued throughout the s, until the arpanet transitioned into the modern internet around . original smtp[edit] in , jon postel published rfc which proposed the mail transfer protocol as a replacement of the use of the file transfer protocol (ftp) for mail. rfc of may removed all references to ftp and allocated port for tcp and udp.[citation needed], an allocation that has since been removed by iana). in november , postel published rfc "simple mail transfer protocol". the smtp standard was developed around the same time as usenet, a one-to-many communication network with some similarities.[citation needed] smtp became widely used in the early s. at the time, it was a complement to the unix to unix copy program (uucp), which was better suited for handling email transfers between machines that were intermittently connected. smtp, on the other hand, works best when both the sending and receiving machines are connected to the network all the time. both used a store and forward mechanism and are examples of push technology. though usenet's newsgroups were still propagated with uucp between servers,[ ] uucp as a mail transport has virtually disappeared[ ] along with the "bang paths" it used as message routing headers.[ ] sendmail, released with . cbsd in , soon after rfc was published in november , was one of the first mail transfer agents to implement smtp.[ ] over time, as bsd unix became the most popular operating system on the internet, sendmail became the most common mta (mail transfer agent).[ ] the original smtp protocol supported only unauthenticated unencrypted -bit ascii text communications, susceptible to trivial man-in-the-middle attack, spoofing, and spamming, and requiring any binary data to be encoded to readable text before transmission. due to absence of proper authentication mechanism, by design every smtp server was an open mail relay. the internet mail consortium (imc) reported that % of mail servers were open relays in ,[ ] but less than % in .[ ] because of spam concerns most email providers blocklist open relays,[ ] making original smtp essentially impractical for general use on the internet. modern smtp[edit] in november , rfc defined extended simple mail transfer protocol (esmtp), which established a general structure for all existing and future extensions which aimed to add-in the features missing from the original smtp. esmtp defines consistent and manageable means by which esmtp clients and servers can be identified and servers can indicate supported extensions. message submission ( rfc ) and smtp-auth ( rfc ) were introduced in and , both describing new trends in email delivery. originally, smtp servers were typically internal to an organization, receiving mail for the organization from the outside, and relaying messages from the organization to the outside. but as time went on, smtp servers (mail transfer agents), in practice, were expanding their roles to become message submission agents for mail user agents, some of which were now relaying mail from the outside of an organization. (e.g. a company executive wishes to send email while on a trip using the corporate smtp server.) this issue, a consequence of the rapid expansion and popularity of the world wide web, meant that smtp had to include specific rules and methods for relaying mail and authenticating users to prevent abuses such as relaying of unsolicited email (spam). work on message submission ( rfc ) was originally started because popular mail servers would often rewrite mail in an attempt to fix problems in it, for example, adding a domain name to an unqualified address. this behavior is helpful when the message being fixed is an initial submission, but dangerous and harmful when the message originated elsewhere and is being relayed. cleanly separating mail into submission and relay was seen as a way to permit and encourage rewriting submissions while prohibiting rewriting relay. as spam became more prevalent, it was also seen as a way to provide authorization for mail being sent out from an organization, as well as traceability. this separation of relay and submission quickly became a foundation for modern email security practices. as this protocol started out purely ascii text-based, it did not deal well with binary files, or characters in many non-english languages. standards such as multipurpose internet mail extensions (mime) were developed to encode binary files for transfer through smtp. mail transfer agents (mtas) developed after sendmail also tended to be implemented -bit-clean, so that the alternate "just send eight" strategy could be used to transmit arbitrary text data (in any -bit ascii-like character encoding) via smtp. mojibake was still a problem due to differing character set mappings between vendors, although the email addresses themselves still allowed only ascii. -bit-clean mtas today tend to support the bitmime extension, permitting some binary files to be transmitted almost as easily as plain text (limits on line length and permitted octet values still apply, so that mime encoding is needed for most non-text data and some text formats). in , the smtputf extension was created to support utf- text, allowing international content and addresses in non-latin scripts like cyrillic or chinese. many people contributed to the core smtp specifications, among them jon postel, eric allman, dave crocker, ned freed, randall gellens, john klensin, and keith moore. mail processing model[edit] blue arrows depict implementation of smtp variations email is submitted by a mail client (mail user agent, mua) to a mail server (mail submission agent, msa) using smtp on tcp port . most mailbox providers still allow submission on traditional port . the msa delivers the mail to its mail transfer agent (mail transfer agent, mta). often, these two agents are instances of the same software launched with different options on the same machine. local processing can be done either on a single machine, or split among multiple machines; mail agent processes on one machine can share files, but if processing is on multiple machines, they transfer messages between each other using smtp, where each machine is configured to use the next machine as a smart host. each process is an mta (an smtp server) in its own right. the boundary mta uses dns to look up the mx (mail exchanger) record for the recipient's domain (the part of the email address on the right of @). the mx record contains the name of the target mta. based on the target host and other factors, the sending mta selects a recipient server and connects to it to complete the mail exchange. message transfer can occur in a single connection between two mtas, or in a series of hops through intermediary systems. a receiving smtp server may be the ultimate destination, an intermediate "relay" (that is, it stores and forwards the message) or a "gateway" (that is, it may forward the message using some protocol other than smtp). per rfc section . , each hop is a formal handoff of responsibility for the message, whereby the receiving server must either deliver the message or properly report the failure to do so. once the final hop accepts the incoming message, it hands it to a mail delivery agent (mda) for local delivery. an mda saves messages in the relevant mailbox format. as with sending, this reception can be done using one or multiple computers, but in the diagram above the mda is depicted as one box near the mail exchanger box. an mda may deliver messages directly to storage, or forward them over a network using smtp or other protocol such as local mail transfer protocol (lmtp), a derivative of smtp designed for this purpose. once delivered to the local mail server, the mail is stored for batch retrieval by authenticated mail clients (muas). mail is retrieved by end-user applications, called email clients, using internet message access protocol (imap), a protocol that both facilitates access to mail and manages stored mail, or the post office protocol (pop) which typically uses the traditional mbox mail file format or a proprietary system such as microsoft exchange/outlook or lotus notes/domino. webmail clients may use either method, but the retrieval protocol is often not a formal standard. smtp defines message transport, not the message content. thus, it defines the mail envelope and its parameters, such as the envelope sender, but not the header (except trace information) nor the body of the message itself. std and rfc define smtp (the envelope), while std and rfc define the message (header and body), formally referred to as the internet message format. protocol overview[edit] smtp is a connection-oriented, text-based protocol in which a mail sender communicates with a mail receiver by issuing command strings and supplying necessary data over a reliable ordered data stream channel, typically a transmission control protocol (tcp) connection. an smtp session consists of commands originated by an smtp client (the initiating agent, sender, or transmitter) and corresponding responses from the smtp server (the listening agent, or receiver) so that the session is opened, and session parameters are exchanged. a session may include zero or more smtp transactions. an smtp transaction consists of three command/reply sequences: mail command, to establish the return address, also called return-path,[ ] reverse-path,[ ] bounce address, mfrom, or envelope sender. rcpt command, to establish a recipient of the message. this command can be issued multiple times, one for each recipient. these addresses are also part of the envelope. data to signal the beginning of the message text; the content of the message, as opposed to its envelope. it consists of a message header and a message body separated by an empty line. data is actually a group of commands, and the server replies twice: once to the data command itself, to acknowledge that it is ready to receive the text, and the second time after the end-of-data sequence, to either accept or reject the entire message. besides the intermediate reply for data, each server's reply can be either positive ( xx reply codes) or negative. negative replies can be permanent ( xx codes) or transient ( xx codes). a reject is a permanent failure and the client should send a bounce message to the server it received it from. a drop is a positive response followed by message discard rather than delivery. the initiating host, the smtp client, can be either an end-user's email client, functionally identified as a mail user agent (mua), or a relay server's mail transfer agent (mta), that is an smtp server acting as an smtp client, in the relevant session, in order to relay mail. fully capable smtp servers maintain queues of messages for retrying message transmissions that resulted in transient failures. a mua knows the outgoing mail smtp server from its configuration. a relay server typically determines which server to connect to by looking up the mx (mail exchange) dns resource record for each recipient's domain name. if no mx record is found, a conformant relaying server (not all are) instead looks up the a record. relay servers can also be configured to use a smart host. a relay server initiates a tcp connection to the server on the "well-known port" for smtp: port , or for connecting to an msa, port . the main difference between an mta and an msa is that connecting to an msa requires smtp authentication. smtp vs mail retrieval[edit] smtp is a delivery protocol only. in normal use, mail is "pushed" to a destination mail server (or next-hop mail server) as it arrives. mail is routed based on the destination server, not the individual user(s) to which it is addressed. other protocols, such as the post office protocol (pop) and the internet message access protocol (imap) are specifically designed for use by individual users retrieving messages and managing mail boxes. to permit an intermittently-connected mail server to pull messages from a remote server on demand, smtp has a feature to initiate mail queue processing on a remote server (see remote message queue starting below). pop and imap are unsuitable protocols for relaying mail by intermittently-connected machines; they are designed to operate after final delivery, when information critical to the correct operation of mail relay (the "mail envelope") has been removed. remote message queue starting[edit] remote message queue starting enables a remote host to start processing of the mail queue on a server so it may receive messages destined to it by sending a corresponding command. the original turn command was deemed insecure and was extended in rfc with the etrn command which operates more securely using an authentication method based on domain name system information.[ ] outgoing mail smtp server[edit] an email client needs to know the ip address of its initial smtp server and this has to be given as part of its configuration (usually given as a dns name). this server will deliver outgoing messages on behalf of the user. outgoing mail server access restrictions[edit] server administrators need to impose some control on which clients can use the server. this enables them to deal with abuse, for example spam. two solutions have been in common use: in the past, many systems imposed usage restrictions by the location of the client, only permitting usage by clients whose ip address is one that the server administrators control. usage from any other client ip address is disallowed. modern smtp servers typically offer an alternative system that requires authentication of clients by credentials before allowing access. restricting access by location[edit] under this system, an isp's smtp server will not allow access by users who are outside the isp's network. more precisely, the server may only allow access to users with an ip address provided by the isp, which is equivalent to requiring that they are connected to the internet using that same isp. a mobile user may often be on a network other than that of their normal isp, and will then find that sending email fails because the configured smtp server choice is no longer accessible. this system has several variations. for example, an organisation's smtp server may only provide service to users on the same network, enforcing this by firewalling to block access by users on the wider internet. or the server may perform range checks on the client's ip address. these methods were typically used by corporations and institutions such as universities which provided an smtp server for outbound mail only for use internally within the organisation. however, most of these bodies now use client authentication methods, as described below. where a user is mobile, and may use different isps to connect to the internet, this kind of usage restriction is onerous, and altering the configured outbound email smtp server address is impractical. it is highly desirable to be able to use email client configuration information that does not need to change. client authentication[edit] modern smtp servers typically require authentication of clients by credentials before allowing access, rather than restricting access by location as described earlier. this more flexible system is friendly to mobile users and allows them to have a fixed choice of configured outbound smtp server. smtp authentication, often abbreviated smtp auth, is an extension of the smtp in order to log in using an authentication mechanism. ports[edit] communication between mail servers generally uses the standard tcp port designated for smtp. mail clients however generally don't use this, instead using specific "submission" ports. mail services generally accept email submission from clients on one of: (submission), as formalized in rfc (previously rfc ) this port was deprecated after rfc , until the issue of rfc . port and others may be used by some individual providers, but have never been officially supported. many internet service providers now block all outgoing port traffic from their customers. mainly as an anti-spam measure,[ ] but also to cure for the higher cost they have when leaving it open, perhaps by charging more from the few customers that requires it open. smtp transport example[edit] a typical example of sending a message via smtp to two mailboxes (alice and theboss) located in the same mail domain (example.com or localhost.com) is reproduced in the following session exchange. (in this example, the conversation parts are prefixed with s: and c:, for server and client, respectively; these labels are not part of the exchange.) after the message sender (smtp client) establishes a reliable communications channel to the message receiver (smtp server), the session is opened with a greeting by the server, usually containing its fully qualified domain name (fqdn), in this case smtp.example.com. the client initiates its dialog by responding with a helo command identifying itself in the command's parameter with its fqdn (or an address literal if none is available).[ ] s: smtp.example.com esmtp postfix c: helo relay.example.com s: smtp.example.com, i am glad to meet you c: mail from: s: ok c: rcpt to: s: ok c: rcpt to: s: ok c: data s: end data with . c: from: "bob example" c: to: alice example c: cc: theboss@example.com c: date: tue, jan : : - c: subject: test message c: c: hello alice. c: this is a test message with header fields and lines in the message body. c: your friend, c: bob c: . s: ok: queued as c: quit s: bye {the server closes the connection} the client notifies the receiver of the originating email address of the message in a mail from command. this is also the return or bounce address in case the message cannot be delivered. in this example the email message is sent to two mailboxes on the same smtp server: one for each recipient listed in the to and cc header fields. the corresponding smtp command is rcpt to. each successful reception and execution of a command is acknowledged by the server with a result code and response message (e.g., ok). the transmission of the body of the mail message is initiated with a data command after which it is transmitted verbatim line by line and is terminated with an end-of-data sequence. this sequence consists of a new-line (), a single full stop (period), followed by another new-line. since a message body can contain a line with just a period as part of the text, the client sends two periods every time a line starts with a period; correspondingly, the server replaces every sequence of two periods at the beginning of a line with a single one. such escaping method is called dot-stuffing. the server's positive reply to the end-of-data, as exemplified, implies that the server has taken the responsibility of delivering the message. a message can be doubled if there is a communication failure at this time, e.g. due to a power shortage: until the sender has received that reply, it must assume the message was not delivered. on the other hand, after the receiver has decided to accept the message, it must assume the message has been delivered to it. thus, during this time span, both agents have active copies of the message that they will try to deliver.[ ] the probability that a communication failure occurs exactly at this step is directly proportional to the amount of filtering that the server performs on the message body, most often for anti-spam purposes. the limiting timeout is specified to be minutes.[ ] the quit command ends the session. if the email has other recipients located elsewhere, the client would quit and connect to an appropriate smtp server for subsequent recipients after the current destination(s) had been queued. the information that the client sends in the helo and mail from commands are added (not seen in example code) as additional header fields to the message by the receiving server. it adds a received and return-path header field, respectively. some clients are implemented to close the connection after the message is accepted ( ok: queued as ), so the last two lines may actually be omitted. this causes an error on the server when trying to send the reply. smtp extensions[edit] extension discovery mechanism[edit] clients learn a server's supported options by using the ehlo greeting, as exemplified below, instead of the original helo. clients fall back to helo only if the server does not support ehlo greeting.[ ] modern clients may use the esmtp extension keyword size to query the server for the maximum message size that will be accepted. older clients and servers may try to transfer excessively sized messages that will be rejected after consuming network resources, including connect time to network links that is paid by the minute.[ ] users can manually determine in advance the maximum size accepted by esmtp servers. the client replaces the helo command with the ehlo command. s: smtp .example.com esmtp postfix c: ehlo bob.example.com s: -smtp .example.com hello bob.example.org [ . . . ] s: -size s: -pipelining s: help thus smtp .example.com declares that it can accept a fixed maximum message size no larger than , , octets ( -bit bytes). in the simplest case, an esmtp server declares a maximum size immediately after receiving an ehlo. according to rfc , however, the numeric parameter to the size extension in the ehlo response is optional. clients may instead, when issuing a mail from command, include a numeric estimate of the size of the message they are transferring, so that the server can refuse receipt of overly-large messages. binary data transfer[edit] original smtp supports only a single body of ascii text, therefore any binary data needs to be encoded as text into that body of the message before transfer, and then decoded by the recipient. binary-to-text encodings, such as uuencode and binhex were typically used. the bitmime command was developed to address this. it was standardized in as rfc [ ] it facilitates the transparent exchange of e-mail messages containing octets outside the seven-bit ascii character set by encoding them as mime content parts, typically encoded with base . mail delivery mechanism extensions[edit] on-demand mail relay[edit] main article: on-demand mail relay on-demand mail relay (odmr) is an smtp extension standardized in rfc that allows an intermittently-connected smtp server to receive email queued for it when it is connected. internationalization extension[edit] main article: international email original smtp supports email addresses composed of ascii characters only, which is inconvenient for users whose native script is not latin based, or who use diacritic not in the ascii character set. this limitation was alleviated via extensions enabling utf- in address names. rfc introduced experimental[ ] utf smtp command and later was superseded by rfc that introduced smtputf command. these extensions provide support for multi-byte and non-ascii characters in email addresses, such as those with diacritics and other language characters such as greek and chinese.[ ] current support is limited, but there is strong interest in broad adoption of rfc and the related rfcs in countries like china that have a large user base where latin (ascii) is a foreign script. extensions[edit] like smtp, esmtp is a protocol used to transport internet mail. it is used as both an inter-server transport protocol and (with restricted behavior enforced) a mail submission protocol. the main identification feature for esmtp clients is to open a transmission with the command ehlo (extended hello), rather than helo (hello, the original rfc standard). a server will respond with success (code ), failure (code ) or error (code , , , , or ), depending on its configuration. an esmtp server returns the code ok in a multi-line reply with its domain and a list of keywords to indicate supported extensions. a rfc compliant server returns error code , allowing esmtp clients to try either helo or quit. each service extension is defined in an approved format in subsequent rfcs and registered with the internet assigned numbers authority (iana). the first definitions were the rfc optional services: send, soml (send or mail), saml (send and mail), expn, help, and turn. the format of additional smtp verbs was set and for new parameters in mail and rcpt. some relatively common keywords (not all of them corresponding to commands) used today are: bitmime – bit data transmission, rfc atrn – authenticated turn for on-demand mail relay, rfc auth – authenticated smtp, rfc chunking – chunking, rfc dsn – delivery status notification, rfc (see variable envelope return path) etrn – extended version of remote message queue starting command turn, rfc help – supply helpful information, rfc pipelining – command pipelining, rfc size – message size declaration, rfc starttls – transport layer security, rfc ( ) smtputf – allow utf- encoding in mailbox names and header fields, rfc utf smtp – allow utf- encoding in mailbox names and header fields, rfc (deprecated[ ]) the esmtp format was restated in rfc (superseding rfc ) and updated to the latest definition in rfc in . support for the ehlo command in servers became mandatory, and helo designated a required fallback. non-standard, unregistered, service extensions can be used by bilateral agreement, these services are indicated by an ehlo message keyword starting with "x", and with any additional parameters or verbs similarly marked. smtp commands are case-insensitive. they are presented here in capitalized form for emphasis only. an smtp server that requires a specific capitalization method is a violation of the standard.[citation needed] bitmime[edit] at least the following servers advertise the bitmime extension: apache james (since . . a )[ ] citadel (since . ) courier mail server gmail[ ] icewarp iis smtp service kerio connect lotus domino microsoft exchange server (as of exchange server ) novell groupwise opensmtpd oracle communications messaging server postfix sendmail (since . ) the following servers can be configured to advertise bitmime, but do not perform conversion of -bit data to -bit when connecting to non- bitmime relays: exim and qmail do not translate eight-bit messages to seven-bit when making an attempt to relay -bit data to non- bitmime peers, as is required by the rfc.[ ] this does not cause problems in practice, since virtually all modern mail relays are -bit clean.[ ] microsoft exchange server advertises bitmime by default, but relaying to a non- bitmime peer results in a bounce. this is allowed by rfc section . smtp-auth[edit] main article: smtp authentication the smtp-auth extension provides an access control mechanism. it consists of an authentication step through which the client effectively logs into the mail server during the process of sending mail. servers that support smtp-auth can usually be configured to require clients to use this extension, ensuring the true identity of the sender is known. the smtp-auth extension is defined in rfc . smtp-auth can be used to allow legitimate users to relay mail while denying relay service to unauthorized users, such as spammers. it does not necessarily guarantee the authenticity of either the smtp envelope sender or the rfc "from:" header. for example, spoofing, in which one sender masquerades as someone else, is still possible with smtp-auth unless the server is configured to limit message from-addresses to addresses this authed user is authorized for. the smtp-auth extension also allows one mail server to indicate to another that the sender has been authenticated when relaying mail. in general this requires the recipient server to trust the sending server, meaning that this aspect of smtp-auth is rarely used on the internet.[citation needed] smtputf [edit] supporting servers include: postfix (version . and later)[ ] momentum (versions . [ ] and . . , and later) sendmail (under development) exim (experimental as of the . release) communigate pro as of version . . [ ] courier-mta as of version . [ ] halon as of version . [ ] microsoft exchange server as of protocol revision . [ ] haraka and other servers.[ ] oracle communications messaging server as of release . . .[ ] security extensions[edit] mail delivery can occur both over plain text and encrypted connections, however the communicating parties might not know in advance of other party's ability to use secure channel. starttls or "opportunistic tls"[edit] main articles: opportunistic tls and email encryption the starttls extensions enables supporting smtp servers to notify connecting clients that it supports tls encrypted communication and offers the opportunity for clients to upgrade their connection by sending the starttls command. servers supporting the extension do not inherently gain any security benefits from its implementation on its own, as upgrading to a tls encrypted session is dependent on the connecting client deciding to exercise this option, hence the term opportunistic tls. starttls is effective only against passive observation attacks, since the starttls negotiation happens in plain text and an active attacker can trivially remove starttls commands. this type of man-in-the-middle attack is sometimes referred to as striptls, where the encryption negotiation information sent from one end never reaches the other. in this scenario both parties take the invalid or unexpected responses as indication that the other does not properly support starttls, defaulting to traditional plain-text mail transfer.[ ] note that starttls is also defined for imap and pop in other rfcs, but these protocols serve different purposes: smtp is used for communication between message transfer agents, while imap and pop are for end clients and message transfer agents. electronic frontier foundation maintains a "starttls everywhere" list that similarly to "https everywhere" list allows relying parties to discover others supporting secure communication without prior communication.[ ] rfc officially declared plain text obsolete and recommend always using tls, adding ports with implicit tls. smtp mta strict transport security[edit] a newer rfc called "smtp mta strict transport security (mta-sts)" aims to address the problem of active adversary by defining a protocol for mail servers to declare their ability to use secure channels in specific files on the server and specific dns txt records. the relying party would regularly check existence of such record, and cache it for the amount of time specified in the record and never communicate over insecure channels until record expires.[ ] note that mta-sts records apply only to smtp traffic between mail servers while communications between end client and the mail server are protected by https, http strict transport security. in april google mail announced support for mta-sts.[ ] smtp tls reporting[edit] a number of protocols allows secure delivery of messages, but they can fail due to misconfigurations or deliberate active interference, leading to undelivered messages or delivery over unencrypted or unauthenticated channels. rfc "smtp tls reporting" describes a reporting mechanism and format for sharing statistics and specific information about potential failures with recipient domains. recipient domains can then use this information to both detect potential attacks and diagnose unintentional misconfigurations. in april google mail announced support for smtp tls reporting.[ ] spoofing and spamming[edit] main articles: anti-spam techniques and email authentication the original design of smtp had no facility to authenticate senders, or check that servers were authorized to send on their behalf, with the result that email spoofing is possible, and commonly used in email spam and phishing. occasional proposals are made to modify smtp extensively or replace it completely. one example of this is internet mail , but neither it, nor any other has made much headway in the face of the network effect of the huge installed base of classic smtp. instead, mail servers now use a range of techniques, such as stricter enforcement of standards such as rfc ,[ ][ ] domainkeys identified mail, sender policy framework and dmarc, dnsbls and greylisting to reject or quarantine suspicious emails.[ ] implementations[edit] there is also smtp proxy implementation as for example nginx.[ ] main articles: list of mail server software and comparison of mail servers related requests for comments[edit] rfc – requirements for internet hosts—application and support (std ) rfc – smtp service extension for message size declaration (оbsoletes: rfc ) rfc – anti-spam recommendations for smtp mtas (bcp ) rfc – simple mail transfer protocol rfc – smtp service extension for command pipelining (std ) rfc – smtp service extensions for transmission of large and binary mime messages rfc – smtp service extension for secure smtp over transport layer security (obsoletes rfc ) rfc – smtp service extension for delivery status notifications (obsoletes rfc ) rfc – enhanced status codes for smtp (obsoletes rfc , updated by rfc ) rfc – an extensible message format for delivery status notifications (obsoletes rfc ) rfc – message disposition notification (updates rfc ) rfc – recommendations for automatic responses to electronic mail rfc – smtp operational experience in mixed ipv /v environments rfc – overview and framework for internationalized email (updated by rfc ) rfc – smtp service extension for authentication (obsoletes rfc , updates rfc , updated by rfc ) rfc – email submission operations: access and accountability requirements (bcp ) rfc – a registry for smtp enhanced mail system status codes (bcp ) (updates rfc ) rfc – the simple mail transfer protocol (obsoletes rfc aka std , rfc , rfc , rfc , updates rfc ) rfc – internet message format (obsoletes rfc aka std , and rfc ) rfc – downgrading mechanism for email address internationalization rfc – message submission for mail (std ) (obsoletes rfc , rfc ) rfc – the multipart/report content type for the reporting of mail system administrative messages (obsoletes rfc , and in turn rfc ) rfc – smtp extension for internationalized email addresses (updates rfc , rfc , rfc , and rfc ) rfc – cleartext considered obsolete: use of transport layer security (tls) for email submission and access see also[edit] bounce address cram-md (a sasl mechanism for esmtpa) rfc email email encryption dkim ident list of mail server software list of smtp server return codes pop before smtp / smtp after pop internet message access protocol binary content extension rfc sender policy framework (spf) simple authentication and security layer (sasl) rfc smtp authentication variable envelope return path comparison of email clients for information about smtp support notes[edit] ^ the history of electronic mail, tom van vleck: "it is not clear this protocol was ever implemented" ^ the first network email, ray tomlinson, bbn ^ picture of "the first email computer" by dan murphy, a pdp- ^ dan murphy's tenex and tops- papers archived november , , at the wayback machine ^ rfc ^ rfc – network mail meeting summary ^ rfc – a proposed mail protocol ^ tldp.org ^ draft-barber-uucp-project-conclusion- – the conclusion of the uucp mapping project ^ the article about sender rewriting contains technical background info about the early smtp history and source routing before rfc . ^ eric allman ( ), sendmail – an internetwork mail router (pdf), bsd unix documentation set, berkeley: university of california, retrieved june , ^ craig partridge ( ), the technical development of internet email (pdf), ieee annals of the history of computing, , ieee computer society, pp. – , doi: . /mahc. . , s cid , archived from the original (pdf) on may , ^ paul hoffman (february , ). "allowing relaying in smtp: a survey". internet mail consortium. retrieved may , . cs maint: discouraged parameter (link) ^ paul hoffman (august ). "allowing relaying in smtp: a series of surveys". internet mail consortium. archived from the original on january , . retrieved may , . cs maint: discouraged parameter (link) ^ "in unix, what is an open mail relay? - knowledge base". web.archive.org. june , . retrieved march , . ^ "the mail, rcpt, and data verbs", [d. j. bernstein] ^ rfc section- . ^ systems, message. "message systems introduces latest version of momentum with new api-driven capabilities". www.prnewswire.com. retrieved july , . ^ cara garretson ( ). "isps pitch in to stop spam". pc world. retrieved january , . last month, the anti-spam technical alliance, formed last year by yahoo, america online, earthlink, and microsoft, issued a list of antispam recommendations that includes filtering port . ^ rfc , simple mail transfer protocol, j. klensin, the internet society (october ) ^ rfc ^ rfc #section- . . . . ^ john klensin; ned freed; marshall t. rose; einar a. stefferud; dave crocker (november ). smtp service extensions. ietf. doi: . /rfc . rfc . ^ "mail parameters". iana. retrieved april , . ^ which was obsoleted in by rfc corresponding to the then new std ^ "mail parameters". november , . ^ jiankang yao (december , ). "chinese email address". eai (mailing list). ietf. retrieved may , . ^ "smtp service extension parameters". iana. retrieved november , . ^ james server - changelog. james.apache.org. retrieved on - - . ^ bitmime service advertised in response to ehlo on gmail-smtp-in.l.google.com port , checked november ^ qmail bugs and wishlist. home.pages.de. retrieved on - - . ^ the bitmime extension. cr.yp.to. retrieved on - - . ^ "postfix smtputf support is enabled by default", february , , postfix.org ^ "message systems introduces latest version of momentum with new api-driven capabilities" (press release). ^ "version . revision history". communigate.com. ^ sam varshavchik (september , ). "new releases of courier packages". courier-announce (mailing list). ^ changelog ^ "ms-oxsmtp: simple mail transfer protocol (smtp) extensions". july , . ^ "eai readiness in tlds" (pdf). february , . ^ "communications messaging server release notes". oracle.com. october . ^ a b "introducing mta strict transport security (mta-sts) | hardenize blog". www.hardenize.com. retrieved april , . ^ "starttls everywhere". eff. retrieved august , . ^ a b cimpanu, catalin. "gmail becomes first major email provider to support mta-sts and tls reporting". zdnet. retrieved april , . ^ message non compliant with rfc ^ message could not be delivered. please ensure the message is rfc compliant. ^ why are the emails sent to microsoft account rejected for policy reasons? ^ "nginx docs | configuring nginx as a mail proxy server". references[edit] hughes, l ( ). internet e-mail: protocols, standards and implementation. artech house publishers. isbn - - - - . hunt, c ( ). sendmail cookbook. o'reilly media. isbn - - - - . johnson, k ( ). internet email protocols: a developer's guide. addison-wesley professional. isbn - - - - . loshin, p ( ). essential email standards: rfcs and protocols made practical. john wiley & sons. isbn - - - - . rhoton, j ( ). programmer's guide to internet mail: smtp, pop, imap, and ldap. elsevier. isbn - - - - . wood, d ( ). programming internet mail. o'reilly. isbn - - - - . external links[edit] iana registry of mail parameters includes service extension keywords rfc smtp service extensions rfc simple mail transfer protocol rfc smtp service extension for authentication (obsoletes rfc ) rfc smtp and lmtp transmission types registration (with esmtpa) rfc message submission for mail (obsoletes rfc , which obsoletes rfc ) v t e email clients free software current alpine balsa citadel/ux claws mail cleancode email cone evolution fetchmail fdm geary getmail gnumail gnus gnuzilla imp kmail mahogany mailpile mailx mailx (heirloom project) modest mozilla thunderbird mulberry mutt nmh / mh offlineimap roundcube seamonkey squirrelmail sylpheed trojitá yam zimbra discontinued arachne beonex communicator blitzmail classilla columbia mm elm fossamail hula mailody mozilla mail & newsgroups nylas n spicebird proprietary freeware em client emailtray foxmail i.scribe mailbird opera mail spark spike touchmail retail hiri bloomba/wordperfect mail newton ibm notes inscribe apple mail mail (windows) microsoft outlook novell groupwise airmail postbox shareware becky! forté agent gyazmail the bat! donationware pegasus mail discontinued cc:mail claris emailer courier cyberdog cyberjack embrowser eudora (discontinued in , moved to open-source in ) mailbox microsoft entourage microsoft internet mail and news microsoft mail minuet netscape mail netscape messenger nextmail outlook express pine pocomail popmail sparrow turnpike webspyder windows live mail windows mail windows messaging related technologies smtp imap jmap lmtp pop push-imap smap smtp uucp related topics email unicode and email category comparison retrieved from "https://en.wikipedia.org/w/index.php?title=simple_mail_transfer_protocol&oldid= " categories: internet mail protocols hidden categories: webarchive template wayback links cs maint: discouraged parameter articles with short description short description matches wikidata use mdy dates from october all articles with unsourced statements articles with unsourced statements from march articles with unsourced statements from april articles with unsourced statements from october navigation menu personal tools not logged in talk contributions create account log in namespaces article talk variants views read edit view history more search navigation main page contents current events random article about wikipedia contact us donate contribute help learn to edit community portal recent changes upload file tools what links here related changes upload file special pages permanent link page information cite this page wikidata item print/export download as pdf printable version in other projects wikimedia commons languages العربية azərbaycanca भोजपुरी Български bosanski català Čeština dansk deutsch eesti Ελληνικά español esperanto euskara فارسی français galego 한국어 Հայերեն हिन्दी hrvatski bahasa indonesia Íslenska italiano עברית kurdî latviešu lëtzebuergesch lietuvių magyar Македонски മലയാളം bahasa melayu nederlands 日本語 norsk bokmål norsk nynorsk Олык марий polski português română Русский shqip simple english slovenčina slovenščina Српски / srpski srpskohrvatski / српскохрватски suomi svenska ไทย türkçe Українська tiếng việt 吴语 yorùbá 中文 edit links this page was last edited on april , at : (utc). text is available under the creative commons attribution-sharealike license; additional terms may apply. by using this site, you agree to the terms of use and privacy policy. wikipedia® is a registered trademark of the wikimedia foundation, inc., a non-profit organization. privacy policy about wikipedia disclaimers contact wikipedia mobile view developers statistics cookie statement catmandu catmandu about download tutorial may , catmandu . on may th , nicolas steenlant (our main developer and guru of catmandu) released version . of our catmandu toolkit with some very interesting new features. the main addition is a brand new way how catmandu fix-es can be implemented using the new catmandu::path implementation. this coding by nicolas will make it much easier and straightforward to implement any kind of fixes in perl. in the previous versions of catmandu there were only two options to create new fixes: create a perl package in the catmandu::fix namespace which implements a fix method. this was very easy: update the $data hash you got as first argument, return the updated $data and you were done. then disadvantage was that accessing fields in a deeply nested record was tricky and slow to code. create a perl package in the catmandu::fix namespace which implemented emit functions. these were functions that generate perl code on the fly. using emit functions it was easier to get fast access to deeply nested data. but, to create fix packages was pretty complex. in catmandu . there is now support for a third and easy way to create new fixes using the catmandu::fix::builder and catmandu::fix::path class. let me give an simple example of a skeleton fix that does nothing: package catmandu::fix::rot ; use catmandu::sane; use moo; use catmandu::util::path qw(as_path); use catmandu::fix::has; with 'catmandu::fix::builder'; has path => (fix_arg => ); sub _build_fixer { my ($self) = @_; sub { my $data = $_[ ]; # ..do some magic here ... $data; } } ; in the code above we start implementing a rot (path) fix that should read a string on a json path and encrypt it using the rot algorithm. this fix is only the skeleton which doesn’t do anything. what we have is: we import the as_path method be able to easily access data on json paths/ we import catmandu::fix::has to be able to use has path constructs to read in arguments for our fix. we import catmandu::fix::builder to use the new catmandu . builder class provides a _build_fixermethod. the builder is nothing more than a closure that reads the data, does some action on the data and return the data. we can use this skeleton builder to implement our rot algorithm. add these lines instead of the # do some magic part: # on the path update the string value... as_path($self->path)->updater( if_string => sub { my $value = shift; $value =~ tr{n-za-mn-za-m}{a-za-z}; $value; }, )->($data); the as_path method receives a json path string an creates an object which you can use to manipulate data on that path. one can update the values found with the updater method, or read data at that path with the getter method or create a new path with the creator method. in our example, we update the string found at the json path using if_string condition. the updaterhas many conditions: if_string needs a closure what should happen when a string is found on the json path. if_array_ref needs a closure what should happen when an array is found on the json path. if_hash_refneeds a closure what should happen when a hash is found on the json path. in our case we are only interested in transforming strings using our rot (path) fix. the rot algorithm is very easy and only switched the order of some characters. when we execute this fix on some sample data we get this result: $ catmandu -i lib convert null to yaml --fix 'add_field(demo,hello);rot v (demo)' --- demo: uryyb ... in this case the fix can be written much shorter when we know that every catmandu::path method return a closure (hint: look at the ->($data) in the code. the complete fix can look like: package catmandu::fix::rot ; use catmandu::sane; use moo; use catmandu::util::path qw(as_path); use catmandu::fix::has; with 'catmandu::fix::builder'; has path => (fix_arg => ); sub _build_fixer { my ($self) = @_; # on the path update the string value... as_path($self->path)->updater( if_string => sub { my $value = shift; $value =~ tr{n-za-mn-za-m}{a-za-z}; $value; }, ); } ; this is as easy as it can get to manipulate deeply nested data with your own perl tools. all the code is in perl, there is no limit on the number of external cpan packages one can include in these builder fixes. we can’t wait what catmandu extensions you will create. written by hochstenbach leave a comment posted in advanced, updates tagged with catmandu, fix language, perl april , lpw : “contrarian perl” – tom hukins at : , tom hukins shares his enthusiasm for catmandu! written by hochstenbach leave a comment posted in uncategorized june , introducing filestores catmandu is always our tool of choice when working with structured data. using the elasticsearch or mongodb catmandu::store-s it is quite trivial to store and retrieve metadata records. storing and retrieving a yaml, json (and by extension xml, marc, csv,…) files can be as easy as the commands below: $ catmandu import yaml to database < input.yml $ catmandu import json to database < input.json $ catmandu import marc to database < marc.data $ catmandu export database to yaml > output.yml a catmandu.yml configuration file is required with the connection parameters to the database: $ cat catmandu.yml --- store: database: package: elasticsearch options: client: ' _ ::direct' index_name: catmandu ... given these tools to import and export and even transform structured data, can this be extended to unstructured data? in institutional repositories like librecat we would like to manage metadata records and binary content (for example pdf files related to the metadata). catmandu . introduces the catmandu::filestore as an extension to the already existing catmandu::store to manage binary content. a catmandu::filestore is a catmandu::store where each catmandu::bag acts as a “container” or a “folder” that can contain zero or more records describing file content. the files records themselves contain pointers to a backend storage implementation capable of serialising and streaming binary files. out of the box, one catmandu::filestore implementation is available catmandu::store::file::simple, or short file::simple, which stores files in a directory. some examples. to add a file to a filestore, the stream command needs to be executed: $ catmandu stream /tmp/myfile.pdf to file::simple --root /data --bag --id myfile.pdf in the command above: /tmp/myfile.pdf is the file up be uploaded to the file::store. file::simple is the name of the file::store implementation which requires one mandatory parameter, --root /data which is the root directory where all files are stored. the--bag is the “container” or “folder” which contains the uploaded files (with a numeric identifier ). and the --id myfile.pdf is the identifier for the new created file record. to download the file from the file::store, the stream command needs to be executed in opposite order: $ catmandu stream file::simple --root /data --bag --id myfile.pdf to /tmp/file.pdf or $ catmandu stream file::simple --root /data --bag --id myfile.pdf > /tmp/file.pdf on the file system the files are stored in some deep nested structure to be able to spread out the file::store over many disks: /data `--/ `--/ `--/ `--/myfile.pdf a listing of all “containers” can be retreived by requesting an export of the default (index) bag of the file::store: $ catmandu export file::simple --root /data to yaml _id: ... a listing of all files in the container “ ” can be done by adding the bag name to the export command: $ catmandu export file::simple --root /data --bag to yaml _id: myfile.pdf _stream: !!perl/code '{ "dummy" }' content_type: application/pdf created: md : '' modified: size: ... each file::store implementation supports at least the fields presented above: _id: the name of the file _stream: a callback function to retrieve the content of the file (requires an io::handle as input) content_type: the mime-type of the file created: a timestamp when the file was created modified: a timestamp when the file was last modified size: the byte length of the file md : optional a md checksum we envision in catmandu that many implementations of filestores can be created to be able to store files in github, bagits, fedora commons and more backends. using the catmandu::plugin::sidecar catmandu::filestore-s and catmandu::store-s can be combined as one endpoint. using catmandu::store::multi and catmandu::store::file::multi many different implementations of stores and filestores can be combined. this is a short introduction, but i hope you will experiment a bit with the new functionality and provide feedback to our project. written by hochstenbach leave a comment posted in uncategorized march , catmandu . catmandu . has been released to with some nice new features. there are some new fix routines that were asked by our community: error the “error” fix stops immediately the execution of the fix script and throws an error. use this to abort the processing of a data stream: $ cat myfix.fix unless exists(id) error("no id found?!") end $ catmandu convert json --fix myfix.fix < data.json valid the “valid” fix condition can be used to validate a record (or part of a record) against a jsonschema. for instance we can select only the valid records from a stream: $ catmandu convert json --fix 'select valid('', jsonschema, schema:myschema.json)' < data.json or, create some logging: $ cat myfix.fix unless valid(author, jsonschema, schema:authors.json) log("errors in the author field") end $ catmandu convert json --fix myfix.fix < data.json rename the “rename” fix can be used to recursively change the names of fields in your documents. for example, when you have this json input: { "foo.bar": " ", "my.name": "patrick" } you can transform all periods (.) in the key names to underscores with this fix: rename('','\.','_') the first parameter is the fields “rename” should work on (in our case it is an empty string, meaning the complete record). the second and third parameters are the regex search and replace parameters. the result of this fix is: { "foo_bar": " ", "my_name": "patrick" } the “rename” fix will only work on the keys of json paths. for example, given the following path: my.deep.path.x.y.z the keys are: my deep path x y z the second and third argument search and replaces these seperate keys. when you want to change the paths as a whole take a look at the “collapse()” and “expand()” fixes in combination with the “rename” fix: collapse() rename('',"my\.deep","my.very.very.deep") expand() now the generated path will be: my.very.very.deep.path.x.y.z of course the example above could be written more simple as “move_field(my.deep,my.very.very.deep)”, but it serves as an example that powerful renaming is possible. import_from_string this fix is a generalisation of the “from_json” fix. it can transform a serialised string field in your data into an array of data. for instance, take the following yaml record: --- foo: '{"name":"patrick"}' ... the field ‘foo’ contains a json fragment. you can transform this json into real data using the following fix: import_from_string(foo,json) which creates a ‘foo’ array containing the deserialised json: --- foo: - name: patrick the “import_from_string” look very much like the “from_json” string, but you can use any catmandu::importer. it always created an array of hashes. for instance, given the following yaml record: --- foo: "name;hobby\nnicolas;drawing\npatrick;music" you can transform the csv fragment in the ‘foo’ field into data by using this fix: import_from_string(foo,csv,sep_char:";") which gives as result: --- foo: - hobby: drawing name: nicolas - hobby: music name: patrick ... i the same way it can process marc, xml, rdf, yaml or any other format supported by catmandu. export_to_string the fix “export_to_string” is the opposite of “import_from_string” and is the generalisation of the “to_json” fix. given the yaml from the previous example: --- foo: - hobby: drawing name: nicolas - hobby: music name: patrick ... you can create a csv fragment in the ‘foo’ field with the following fix: export_to_string(foo,csv,sep_char:";") which gives as result: --- foo: "name;hobby\nnicolas;drawing\npatrick;music" search_in_store the fix “search_in_store” is a generalisation of the “lookup_in_store” fix. the latter is used to query the “_id” field in a catmandu::store and return the first hit. the former, “search_in_store” can query any field in a store and return all (or a subset) of the results. for instance, given the yaml record: --- foo: "(title:abc or author:dave) and not year: " ... then the following fix will replace the ‘foo’ field with the result of the query in a solr index: search_in_store('foo', store:solr, url: 'http://localhost: /solr/catalog') as a result, the document will be updated like: --- foo: start: , limit: , hits: [...], total: ... where start: the starting index of the search result limit: the number of result per page hits: an array containing the data from the result page total: the total number of search results every catmandu::solr can have another layout of the result page. look at the documentation of the catmandu::solr implementations for the specific details. thanks for all your support for catmandu and keep on data converting 🙂 written by hochstenbach leave a comment posted in uncategorized june , metadata analysis at the command-line i was last week at the elag conference in copenhagen and attended the excellent workshop by christina harlow of cornell university on migrating digital collections metadata to rdf and fedora . one of the important steps required to migrate and model data to rdf is understanding what your data is about. probably old systems need to be converted for which little or no documentation is available. instead of manually processing large xml or marc dumps, tools like metadata breakers can be used to find out which fields are available in the legacy system and how they are used. mark phillips of the university of north texas wrote recently in code lib a very inspiring article how this could be done in python. in this blog post i’ll demonstrate how this can be done using a new catmandu tool: catmandu::breaker. to follow the examples below, you need to have a system with catmandu installed. the catmandu::breaker tools can then be installed with the command: $ sudo cpan catmandu::breaker a breaker is a command that transforms data into a line format that can be easily processed with unix command line tools such as grep, sort, uniq, cut and many more. if you need an introduction into unix tools for data processing please follow the examples johan rolschewski of berlin state library and i presented as an elag bootcamp. as a simple example lets create a yaml file and demonstrate how this file can be analysed using catmandu::breaker: $ cat test.yaml --- name: john colors: - black - yellow - red institution: name: acme years: - - - - this example has a combination of simple name/value pairs a list of colors and a deeply nested field. to transform this data into the breaker format execute the command: $ catmandu convert yaml to breaker < test.yaml colors[] black colors[] yellow colors[] red institution.name acme institution.years[] institution.years[] institution.years[] institution.years[] name john the breaker format is a tab-delimited output with three columns: an record identifier: read from the _id field in the input data, or a counter when no such field is present. a field name. nested fields are seperated by dots (.) and list are indicated by the square brackets ([]) a field value when you have a very large json or yaml field and need to find all the values of a deeply nested field you could do something like: $ catmandu convert yaml to breaker < data.yaml | grep "institution.years" using catmandu you can do this analysis on input formats such as json, yaml, xml, csv, xls (excell). just replace the yaml by any of these formats and run the breaker command. catmandu can also connect to oai-pmh, z . or databases such as mongodb, elasticsearch, solr or even relational databases such as mysql, postgres and oracle. for instance to get a breaker format for an oai-pmh repository issue a command like: $ catmandu convert oai --url http://lib.ugent.be/oai to breaker if your data is in a database you could issue an sql query like: $ catmandu convert dbi --dsn 'dbi:oracle' --query 'select * from table where ...' --user 'user/password' to breaker some formats, such as marc, doesn’t provide a great breaker format. in catmandu, marc files are parsed into a list of list. running a breaker on a marc input you get this: $ catmandu convert marc to breaker < t/camel.usmarc | head fol record[][] ldr fol record[][] _ fol record[][] cam a fol record[][] fol record[][] _ fol record[][] fol fol record[][] fol record[][] fol record[][] fol record[][] a the marc fields are part of the data, not part of the field name. this can be fixed by adding a special ‘marc’ handler to the breaker command: $ catmandu convert marc to breaker --handler marc < t/camel.usmarc | head fol ldr cam a fol fol fol imchf fol . fol s nyua eng fol a fol a (paper/cd-rom : alk. paper) fol a dlc fol c dlc fol d dlc now all the marc subfields are visible in the output. you can use this format to find, for instance, all unique values in a marc file. lets try to find all unique values: $ catmandu convert marc to breaker --handler marc < camel.usmarc | grep "\t " | cut -f | sort -u s nyua eng s mau eng s njua eng s cau b eng s caua eng s mau eng s mau eng s mau eng s mau eng s cau eng nam a catmandu::breaker doesn’t only break input data in a easy format for command line processing, it can also do a statistical analysis on the breaker output. first process some data into the breaker format and save the result in a file: $ catmandu convert marc to breaker --handler marc < t/camel.usmarc > result.breaker now, use this file as input for the ‘catmandu breaker’ command: $ catmandu breaker result.breaker | name | count | zeros | zeros% | min | max | mean | median | mode | variance | stdev | uniq | entropy | |------|-------|-------|--------|-----|-----|------|--------|--------|----------|-------|------|---------| | | | | . | | | | | | | | | . / . | | | | | . | | | | | | | | | . / . | | | | | . | | | | | | | | | . / . | | | | | . | | | | | | | | | . / . | | a | | | . | | | | | | | | | . / . | | a | | | . | | | . | | | . | . | | . / . | | a | | | . | | | | | | | | | . / . | | c | | | . | | | | | | | | | . / . | | d | | | . | | | . | . | [ , ] | . | . | | . / . | | a | | | . | | | | | | | | | . / . | | a | | | . | | | | | | | | | . / . | | b | | | . | | | | | | | | | . / . | | | | | . | | | | | | | | | . / . | | a | | | . | | | | | | | | | . / . | | a | | | . | | | . | | | . | . | | . / . | | d | | | . | | | . | | | . | . | | . / . | | q | | | . | | | . | | | . | . | | . / . | | a | | | . | | | . | | | . | . | | . / . | | c | | | . | | | . | | | . | . | | . / . | | d | | | . | | | . | | | . | . | | . / . | | a | | | . | | | | | | | | | . / . | | b | | | . | | | . | | | . | . | | . / . | | c | | | . | | | . | | | . | . | | . / . | | a | | | . | | | . | | | . | . | | . / . | | a | | | . | | | | | | | | | . / . | | b | | | . | | | | | | | | | . / . | | c | | | . | | | | | | | | | . / . | | a | | | . | | | . | | | . | . | | . / . | | a | | | . | | | | | | | | | . / . | | b | | | . | | | . | | | . | . | | . / . | | c | | | . | | | . | | | . | . | | . / . | | e | | | . | | | . | | | . | . | | . / . | | a | | | . | | | . | | | . | . | | . / . | | a | | | . | | | . | | | . | . | | . / . | | a | | | . | | | . | | | . | . | | . / . | | a | | | . | | | . | | | . | . | | . / . | | v | | | . | | | . | | | . | . | | . / . | | a | | | . | | | . | | | . | . | | . / . | | ldr | | | . | | | | | | | | | . / . as a result you get a table listing the usage of subfields in all the input records. from this output we can learn: the ‘ ’ field is available in records (see: count) one record doesn’t contain a ‘ a’ subfield (see: zeros) the ‘ a’ is available in all records at least once at most times (see: min, max) only out of ‘ a’ subfields have unique values (see: uniq) the last column ‘entropy’ provides a number how interesting the field is for search engines. the higher the entropy, the more uniq content can be found. i hope this tools are of some use in your projects! written by hochstenbach comments posted in uncategorized may , catmandu . catmandu . has been released today. there has been some speed improvements processing fixes due to switching from the data::util to the ref::util package which has better a support on many perl platforms. for the command line there is now support for preprocessing fix scripts. this means, one can read in variables from the command line into a fix script. for instance, when processing data you might want to keep some provenance data about your data sources in the output. this can be done with the following commands: $ catmandu convert marc --fix myfixes.fix --var source=publisher --var date= - < data.mrc with a myfixes.fix like: add_field(my_source,{{source}}) add_field(my_data,{{date}}) marc_field( ,title) marc_field( ,issn) . . . etc . . your json output will now contain the clean ‘title’ and ‘issn’ fields but also for each record a ‘my_source’ with value ‘publisher ’ and a ‘my_date’ with value ‘ - ’. by using the text::hogan compiler full support of the mustache language is available. in this new catmandu version there have been also some new fix functions you might want to try out, see our fixes cheat sheet for a full overview. written by hochstenbach leave a comment posted in updates april , parallel processing with catmandu in this blog post i’ll show a technique to scale out your data processing with catmandu. all catmandu scripts use a single process, in a single thread. this means that if you need to process times as much data , you need times at much time. running a catmandu convert command with the -v option will show you the speed of a typical conversion: $ catmandu convert -v marc to json --fix heavy_load.fix < input.marc > output.json added ( /sec) added ( /sec) added ( /sec) added ( /sec) added ( /sec) added ( /sec) added ( /sec) added ( /sec) added ( /sec) added ( /sec) in the example above we process an ‘input.marc’ marc file into a ‘output.json’ json file with some difficult data cleaning in the ‘heave_load.fix’ fix script. using a single process we can reach about records per second. it would take . hours to process one million records and hours to process ten million records. can we make this any faster? when you buy a computer they are all equipped with multiple processors. using a single process, only one of these processors are used for calculations. one would get much ‘bang for the buck’ if all the processors could be used. one technique to do that is called ‘parallel processing’. to check the amount of processors available on your machine use the file ‘/proc/cpuinfo’: on your linux system: $ cat /proc/cpuinfo | grep processor processor : processor : the example above shows two lines: i have two cores available to do processing on my laptop. in my library we have servers which contain , , or more processors. this means that if we could do our calculations in a smart way then our processing could be , , or times as fast (in principle). to check if your computer is using all that calculating power, use the ‘uptime’ command: $ uptime : : up days, : , users, load average: . , . , . in the example above i ran did ‘uptime’ on one of our servers with processors. it shows a load average of about . to . . this means that in the last minutes between and processors where being used and the other two did nothing. if the load average is less than the number of cores ( in our case) it means: the server is waiting for input. if the load average is equal to the number of cores it means: the server is using all the cpu power available. if the load is bigger than the number of cores, then there is more work available than can be executed by the machine, some processes need to wait. now you know some unix commands we can start using the processing power available on your machine. in my examples i’m going to use a unix tool called ‘gnu parallel’ to run catmandu scripts on all the processors in my machine in the most efficient way possible. to do this you need to install gnu parallel: sudo yum install parallel the second ingredient we need is a way to cut our input data into many parts. for instance if we have a processor machine we would like to create equal chunks of data to process in parallel. there are very many ways to cut your data in to many parts. i’ll show you a trick we use in at ghent university library with help of a mongodb installation. first install, mongodb and the mongodb catmandu plugins (these examples are taken from our centos documentation): $ sudo cat > /etc/yum.repos.d/mongodb.repo < part $ catmandu export mongodb --database_name -q '{"part.rand ": }' > part we are going to use these catmandu commands in a bash script which makes use of gnu parallel run many conversions simultaneously. #!/bin/bash # file: parallel.sh cpu=$ if [ "${cpu}" == "" ]; then /usr/bin/parallel -u $ {} < result.${cpu}.json fi this example script above shows how a conversion process could run on a -processor machine. the lines with ‘/usr/bin/parallel’ show how gnu parallel is used to call this script with two arguments ‘ ’ and ‘ ’ (for the -processor example). in the lines with ‘catmandu export’ shows how chunks of data are read from the database and processed with the ‘heavy_load.fix’ fix script. if you have a -processor machine, you would need to provide parallel an input which contains the numbers , , to and change the query to ‘part.rand ’. gnu parallel is a very powerfull command. it gives the opportunity to run many processes in parallel and even to spread out the load over many machines if you have a cluster. when all these machines have access to your mongodb database then all can receive chunks of data to be processed. the only task left is to combine all results which can be as easy as a simple ‘cat’ command: $ cat result.*.json > final_result.json written by hochstenbach comments posted in advanced tagged with catmandu, json path, library, linux, marc, parallel procesing, perl february , catmandu . after years of programming, minor releases we are finally there: the release of catmandu . ! we have pushed the test coverage of the code to . % and added and cleaned a lot of our documentation. for the new features read our changes file. a few important changes should be noted. by default catmandu will read and write valid json files. in previous versions the default input format was (new)line delimited json records as in: {"record":" "} {"record":" "} {"record":" "} instead of the valid json array format: [{"record":" "},{"record":" "},{"record":" "}] the old format can still be used as input but will be read much faster when using the –line_delimited option on the command line. thus, write: # fast $ catmandu convert json --line_delimited < lines.json.txt instead of: # slow $ catmandu convert json < lines.json.txt by default catmandu will export in the valid json-array format. if you still need to use the old format, then provide the –line_delimited option on the command line: $ catmandu convert yaml to json --line_delimited < data.yaml we thank all contributors for these wonderful four years of open source coding and we wish you all four new hacking years. our thanks goes to: nicolas steenlant christian pietsch dave sherohman dries moreels friedrich summann jakob voss johann rolschewski jonas smedegaard jörgen eriksson magnus enger maria hedberg mathias lösch najko jahn nicolas franck patrick hochstenbach petra kohorst robin sheat snorri briem upasana shukla vitali peil deutsche forschungsgemeinschaft for providing us the travel funds lund university library , ghent university library and bielefeld university library to provide us a very welcome environment for open source collaboration. written by hochstenbach leave a comment posted in uncategorized june , catmandu chat on friday june : cest, we’ll provide a one hour introduction/demo into processing data with catmandu. if you are interested, join us on the event page: https://plus.google.com/hangouts/_/event/c jcknos egjlthk m btha o more instructions on the exact google hangout coordinates for this chat will follow on this web page at friday june : . to enter the chat session, a working version of the catmandu virtualbox needs to be running on your system: https://librecatproject.wordpress.com/get-catmandu/ written by hochstenbach leave a comment posted in events june , matching authors against viaf identities at ghent university library we enrich catalog records with viaf identities to enhance the search experience in the catalog. when searching for all the books about ‘chekov’ we want to match all name variants of this author. consult viaf http://viaf.org/viaf/ /#chekhov,_anton_pavlovich,_ - and you will see many of them. chekhov Čehov tsjechof txékhov etc any of the these names variants can be available in the catalog data if authority control is not in place (or not maintained). searching any of these names should result in results for all the variants. in the past it was a labor intensive, manual job for catalogers to maintain an authority file. using results from linked data fragments research by ruben verborgh (iminds) and the catmandu-rdf tools created by jakob voss (gbv) and rdf-ldf by patrick hochstenbach, ghent university started an experiment to automatically enrich authors with viaf identities. in this blog post we will report on the setup and results of this experiment which will also be reported at elag . context three ingredients are needed to create a web of data: a scalable way to produce data. the infrastructure to publish data. clients accessing the data and reusing them in new contexts. on the production site there doesn’t seem to be any problem creating huge datasets by libraries. any transformation of library data to linked data will quickly generate an enormous number of rdf triples. we see this in the size of public available datasets: ugent academic bibliography: . . triples libris catalog: . . triples gallica: . . triples dbpedia: . . triples viaf: . . triples europeana: . . triples the european library: . . . triples pubchem: . . . triples also for accessing data, from a consumers perspective the “easy” part seems to be covered. instead of thousands of apis available and many documents formats for any dataset, sparql and rdf provide the programmer a single protocol and document model. the claim of the linked data fragments researchers is that on the publication side, reliable queryable access to public linked data datasets largely remains problematic due to the low availability percentages of public sparql endpoints [ref]. this is confirmed by the study by researchers from pontificia universidad católica in chili and national university of ireland where more than half of the public sparql endpoints seem to be offline . days per month. this gives an availability rate of less than % [ref]. the source of this high rate of inavailability can be traced back to the service model of linked data where two extremes exists to publish data (see image below). from: http://www.slideshare.net/rubenverborgh/dbpedias-triple-pattern-fragments at one side, data dumps (or dereferencing of urls) can be made available which requires a simple http server and lots of processing power on the client side. at the other side, an open sparql endpoint can be provided which requires a lot of processing power (hence, hardware investment) on the serverside. with sparql endpoints, clients can demand the execution of arbitrarily complicated queries. furthermore, since each client requests unique, highly specific queries, regular caching mechanisms are ineffective, since they can only optimized for repeated identical requests. this situation can be compared with providing a database sql dump to endusers or open database connection on which any possible sql statement can be executed. to a lesser extent libraries are well aware of the different modes of operation between running oai-pmh services and z . /sru services. linked data fragment researchers provide a third way, triple pattern fragments, to publish data which tries to provide the best of both worlds: access to a full dump of datasets while providing a queryable and cachable interface. for more information on the scalability of this solution i refer to the report presented at the th international usewod workshop. the experiment viaf doesn’t provide a public sparql endpoint, but a complete dump of the data is available at http://viaf.org/viaf/data/. in our experiments we used the viaf (virtual international authority file), which is made available under the odc attribution license. from this dump we created a hdt database. hdt provides a very efficient format to compress rdf data while maintaining browser and search functionality. using command line tools rdf/xml, turtle and ntriples can be compressed into a hdt file with an index. this standalone file can be used to without the need of a database to query huge datasets. a viaf conversion to hdt results in a gb file and a gb index. using the linked data fragments server by ruben verborgh, available at https://github.com/linkeddatafragments/server.js, this hdt file can be published as a nodejs application. for a demonstration of this server visit the iminds experimental setup at: http://data.linkeddatafragments.org/viaf using triple pattern fragments a simple rest protocol is available to query this dataset. for instance it is possible to download the complete dataset using this query: $ curl -h "accept: text/turtle" http://data.linkeddatafragments.org/viaf if we only want the triples concerning chekhov (http://viaf.org/viaf/ ) we can provide a query parameter: $ curl -h "accept: text/turtle" http://data.linkeddatafragments.org/viaf?subject=http://viaf.org/viaf/ likewise, using the predicate and object query any combination of triples can be requested from the server. $ curl -h "accept: text/turtle" http://data.linkeddatafragments.org/viaf?object="chekhov" the memory requirements of this server are small enough to run a copy of the viaf database on a macbook air laptop with gb ram. using specialised triple pattern fragments clients, sparql queries can be executed against this server. for the catmandu project we created a perl client rdf::ldf which is integrated into catmandu-rdf. to request all triples from the endpoint use: $ catmandu convert rdf --url http://data.linkeddatafragments.org/viaf --sparql 'select * {?s ?p ?o}' or, only those triples that are about “chekhov”: $ catmandu convert rdf --url http://data.linkeddatafragments.org/viaf --sparql 'select * {?s ?p "chekhov"}' in the ghent university experiment a more direct approach was taken to match authors to viaf. first, as input a marc dump from the catalog is being streamed into a perl program using a catmandu iterator. then, we extract the and fields which contain $a (name) and $d (date) subfields. these two fields are combined in a search query, as if we would search: chekhov, anton pavlovich, - if there is exactly one hit in our local viaf copy, then the result is reported. a complete script to process marc files this way is available at a github gist. to run the program against a marc dump execute the import_viaf.pl command: $ ./import_viaf.pl --type usmarc file.mrc - l $$aedwards, everett eugene,$$d - http://viaf.org/viaf/ - l $$aclelland, marjorie bolton,$$d - http://viaf.org/viaf/ - l $$aschein, edgar h. - l $$akilbridge, maurice d.,$$d - http://viaf.org/viaf/ - l $$awiseman, frederick. - l $$amiller, wilhelm,$$d - http://viaf.org/viaf/ - l $$ahazlett, thomas c.,$$d - http://viaf.org/viaf/ [edit: - - an updated version of the code is available as a git project https://github.com/librecat/marc rdf ] all the authors in the marc dump will be exported. if there is exactly one single match against viaf it will be added to the author field. we ran this command for one night in a single thread against . authors containing a date and found . exact matches in viaf (= %). in a quite recent follow up of our experiments, we investigated how ldf clients can be used in a federated setup. when combining in the ldf algorithm the triples result from many ldf servers, one sparql query can be run over many machines. these results are demonstrated at the iminds demo site where a single sparql query can be executed over the combined viaf and dbpedia datasets. a perl implementation of this federated search is available in the latest version of rdf-ldf at github. we strongly believe in the success of this setup and the scalability of this solution as demonstrated by ruben verborgh at the usewod workshop. using linked data fragments a range of solutions are available to publish data on the web. from simple data dumps to a full sparql endpoint any service level can be provided given the resources available. for more than a half year dbpedia has been running an ldf server with . % availability on a cpu , gb ram amazon server with . million requests. scaling out, services such has the lod laundromat cleans . datasets and provides access to them using a single fat ldf server ( gb ram). for more information on the federated searches with linked data fragments visit the blog post of ruben verborgh at: http://ruben.verborgh.org/blog/ / / /federated-sparql-queries-in-your-browser/ written by hochstenbach leave a comment posted in advanced tagged with ldf, linked data, marc, perl, rdf, sparql, triple pattern fragments, viaf older posts recent posts catmandu . lpw : “contrarian perl” – tom hukins introducing filestores catmandu . metadata analysis at the command-line catmandu . parallel processing with catmandu catmandu . catmandu chat matching authors against viaf identities preprocessing catmandu fixes earthquake in kathmandu importing files from a hotfolder directory librecat/memento hackathon day : merry christmas! day : exporting rdf data with catmandu day : importing rdf data with catmandu day : marc to dublin core day : set up your own oai data service day : harvest data with oai-pmh day : index your data with elasticsearch day : store your data in mongodb day : working with csv and excel files day : processing marc with catmandu day : processing json data from webservices day : catmandu json paths day : introduction into catmandu day : editing text with nano day : grep, less and wc day : bash basics create a free website or blog at wordpress.com. catmandu create a free website or blog at wordpress.com. email (required) name (required) website loading comments... comment × privacy & cookies: this site uses cookies. by continuing to use this website, you agree to their use. to find out more, including how to control cookies, see here: cookie policy javascript is not available. we’ve detected that javascript is disabled in this browser. please enable javascript or switch to a supported browser to continue using twitter.com. you can see a list of supported browsers in our help center. help center terms of service privacy policy cookie policy imprint ads info © twitter, inc. something went wrong, but don’t fret — let’s give it another shot. . . updated: / / * enhancement: marceditor: added a button to provide quick access to the available task list. * enhancement: marceditor: code is in place to begin allowing users to show/hide menu/toolbar buttons. this should be available in a near term update. . . updated: / / * bug fix: internet archive => hathitrust plugin updates to correct debug link generation. * update: file assoc. updates * update: installer - file extensions will now assign to . .x . . updated: / / * enhancement: z . -- users can add more than criteria. * update: plugin -- internet archive => hathitrust plugin updated to allow for multiple date type searches. * update: z . ui changes to make it easier to prevent data from being hidden on high zoom * update: in the preferences, the task location can now allow environment variables in the file path (example: %appdata%) * update: updated json/rdf components * bug fix: validate headings window was freezing when using some of the new linked data rule options. * enhancement: custom reports -- added a ui validation to ensure required data is provided (this wasn't previously the case). * enhancement: marcvalidator -- added some updated language in the error changing. * bug fix: marcvalidator -- make sure that all file handles are closed (there was a case where one of the handles was remaining opened and could, potentially, result in a locked process). . . updated: / / * enhancement: marceditor global edit functions -- a new preview option has been added (replace all, add field, delete field, copy field, edit indicators, edit field, edit subfield, swap field) * enhancement: ui enhancement to ensure that a status message is present so users know the process is running (replace all, add field, delete field, copy field, edit indicators, edit field, edit subfield, swap field) * enhancement: marcengine -- added json => xml translation * enhancement: xml/json profile wizard - added support for json-ld formatted data. * enhancement: xslt -- including xslt for the homosaurus vocabulary * enhancement: oclc api -- surfacing more debugging information to make it easier to see when an issue is occuring * bug fix: marcvalidator -- ensured all file handles are closing and released * behavior change: kbart marc plugin - tool will preference isbn if present (currently, it selects the last isbn if multiples of the same type are present) * bug fix: installer -- cleaned up some old files * behavior change: oclc has discontinued providing work id information in worldcat.org. i've shifted to using the classify api till a better option is found. * clean-up: ui clean up in the migration wizard * clean-up: ui clean up of the main window * bug fix/clean-up: corrected ui to add back missing icons (for example, in the extract selected records form) . . updated: / / * enhancement: updated plugin manager * enhancement: oclc connexion plugin added/converted * enhancement: internet archive => hathitrust packager added/converted * enhancement: marc => kbart converter added/converted * enhancement: make check digit added/converted * enhancement: microlif => mnemonic converted added/converted * ris => marc plugin added/converted * enhancement: installer evaluates for the -bit access database engine ( ) on bit systems * enhancement: installer evaluates for the c++ runtime required by the access database engine on bit systems * behavior change: restart as -bit program has been hidden * enhancement: marc sql explorer has been folded into the primary marcedit application [results in a reduction of dependencies] * bug fix: clustering tools -- beta build wasn't allowing the clustering tools to function correctly. * enhancement: oclc search -- batch searching has been allowed * enhancement: oclc integration -- new session diagnostics option added for debugging processes * bug fix: integration settings import -- if no settings have ever been set and the initial file hasn’t been created, import will say it’s completed, but it won’t. * bug fix: oclc integration -- if the expires_at element is null or fails to parse, it can throw an error. this is now trapped and will attempt to reauthorize. * bug fix: console: added process to consume event processing for validate and split tasks. . . updated: / / * bug fix: installer throws an error when attempting to install per user * bug fix: marceditor -- marcedit will be deprecating legacy page loading. this option is now ignored if set and will be removed entirely in future builds. . . updated: / / * change: allow os to manage supported supported security protocol types. * change: remove com.sun dependency related to dns and httpserver * change: changed appdata path * change: first install automatically imports settings from marcedit . - .x * change: field count - simplify ui (consolidate elements) * change: windows -- update help urls to oclc * change: generate fast headings -- update help urls * change: .net changes thread stats queuing. updating thread processing on forms: * generate fast headings * batch process records * build links * main window * rda helper * delete selected records * marc tools * check url tools * marcvalidator * marcengine * task manager * z . * ils integration processing * character conversions * format handing (delimited text, openrefine, etc.) * change: xml function list -- update process for opening urls * change: z . preferences window - update process for opening urls * change: about windows -- new information, updated how version information is calculated. * change: catalog calculator window -- update process for opening urls * change: generate call numbers -- update process for opening urls * change: generate material formats -- update process for opening urls * change: tab delimiter -- remove context windows * change: tab delimiter -- new options ui * change: tab delimiter -- normalization changes * change: remove old help html page * change: remove old hex editor page * change: updated hex editor to integrate into main program * change: main window -- remove custom scheduler dependency * change: ui update to allow more items * change: main window -- new icon * change: main window -- update process for opening urls * change: main window -- removed context menus * change: main window -- upgrade changes to new executable name * change: main window -- updated the following menu items: * edit linked data tools * removed old help menu item * added new application shortcut * change: oclc bulk downloader -- new ui elements to correspond to new oclc api * change: oclc search page -- new ui elements to correspond to new oclc api * change: preferences -- updates related to various preference changes: * hex editor * integrations * editor * other * change: rda helper -- update process for opening urls * change: rda helper -- opening files for editing * change: removed the script maker * change: templates for perl and vbscripts includes * change: removed find/search xml in the xml editor and consolidated in existing windows * change: delete selected records: exposed the form and controls to the marceditor * change: sparql browser -- update process for opening urls * change: sparql browser -- removed context menus * change: troubleshooting wizard -- added more error codes and kb information to the wizard * change: unimarc utility -- controls change, configurable transform selections * change: marc utilities -- removed the context menu * change: first run wizard -- new options, new agent images * change: xml editor -- delete block addition * change: xml editor -- xquery transform support * change: xml profile wizard -- option to process attributes * change: marceditor -- status bar control doesn't exist in net . . control has changed. * change: marceditor -- improved page loading * change: marceditor -- file tracking updated to handle times when the file opened is a temp record * change: marceditor -- removed ~ k of old code * change: marceditor -- added delete selected records option * change: removed helper code used by installer * change: removed office menu formatting code * change: consolidated extensions into new class (removed files) * change: removed calls marshalled to the windows api -- replaced with managed code * change: openrefine format handler updated to capture changes between openrefine versions * change: marcengine -- namespace update to * change: wizard -- missing unicode font options more obvious * change: wizard install puts font in program directory so that additional users can simply copy (not download) the font on use * change: checkurls: removed support for insecure crypto-types * change: checkurls: additional heuristics to respond dynamically to http status codes * change: all components -- .net . includes a new codepages library that allows for extended codepage support beyond the default framework. added across the project. * change: marcvalidator -- new rules process that attempts to determine if records are too long for processing when validating rules or structure. * change: command-line -- batch process switch has been added to the tasks processing function * change: options -- allow user path to be reset. * bug fix: main window -- corrects process for determining version for update * bug fix: main window -- updated image * bug fix: when doing first run, wizard not showing in some cases. * bug fix: main window -- last tool used sometimes shows duplicates * bug fix: rda helper -- $e processing * bug fix: rda helper -- punctuation in the $e * bug fix: xml profile wizard -- when the top element is selected, it's not viewed for processing (which means not seeing element data or attribute data) * bug fix: marceditor -- page processing correct to handle invalid formatted data better * bug fix: installation wizard -- if a unicode font was installed during the first run process, it wouldn't be recognized. * bug fix: marcvalidator fails when attempting to process a .mrk file from outside the marceditor * bug fix: linked data processing: when processing services with multiple redirects -- process may stop pre-maturely. (example: lc's id.loc.gov xx processing) * bug fix: edit field -- find fields with just spaces are trimmed, causing the field data to process improperly. * bug fix: rda helper will fail if ldr length is incorrect when attempting to determine character encoding literary machines literary machines digital libraries, books, archives archiviiify a short guide to download digitized books from internet archive and rehost on your own infrastructure using iiif with full-text search. pywb . - docker quickstart four years have passed since i first wrote of pywb: it was a young tool at the time, but already usable and extremely simple to deploy. since then a lot of works has been done by ilya kreymer (and others), resulting in all the new features available with the . release. also, some very big webarchiving initiatives have moved and used pywb in these years: webrecorder itself, rhizome, perma, arquivo pt in portugal, the italian national library in florence (italy), (others i’m missing). anonymous webarchiving webarchiving activities, as any other activity where an http client is involved, leave marks of their steps: the web server you are visiting or crawling will save your ip address in its logs (or even worse it can decide to ban your ip). this is usually not a problem, there are plenty of good reasons for a webserver to keep logs of its visitors. but sometimes you may need to protect your own identity when you are visiting or saving something from a website, and there a lot of sensitive careers that need this protection: activists, journalist, political dissidents. tor has been invented for this, and today offer a good protection to browse anonymously the web. can we also archive the web through tor? open bni il maggio viene annunciato il rilascio libero della bibliografia nazionale italiana (bni). viene apprezzata l’apertura di questo catalogo (anche se con i limiti dei soli pdf), e da profano di biblioteconomia faccio anche una domanda sull’effettivo caso d’uso della bni. il agosto viene annunciato il rilascio delle annate e anche in formato unimarc e marcxml. incuriosito dal catalogo inizio ad esplorarlo, per pensare a possibili trasformazioni (triple rdf) o arricchimenti con/verso altri dati (wikidata). epub linkrot linkrot also affects epub files (who would have thought! :)). how to check the health of external links in epub books (required tools: a shell, atool, pup, gnu parallel). skos nuovo soggettario, api e autocomplete come creare una api per un form con autocompletamento usando i termini del nuovo soggettario, con i sorted sets di redis e nginx+lua. serve deepzoom images from a zip archive with openseadragon vips is a fast image processing system. version higher than . can generate static tiles of big images in deepzoom format, saving them directly into a zip archive. a wayback machine (pywb) on a cheap, shared host for a long time the only free (i’m unaware of commercial ones) implementation of a web archival replay software has been the wayback machine (now openwayback). it’s a stable and mature software, with a strong community behind. to use it you need to be confident with the deploy of a java web application; not so difficult, and documentation is exaustive. but there is a new player in the game, pywb, developed by ilya kramer, a former internet archive developer. built in python, relatively simpler than wayback, and now used in a pro archiving project at rhizome. opendata dell’anagrafe biblioteche come usare gli opendata dell’anagrafe delle biblioteche italiane e disegnare su una mappa web gli indirizzi delle biblioteche. api json dell’opac sbn alcuni mesi fa è stata rilasciata da iccu una app mobile per consultare l’opac sbn. anche se graficamente poco accattivante l’app funziona bene, e trovo molto utili le funzioni di ricerca di un libro scansionando il codice a barre con la camera del telefonino, e la possibilità di bookmarkare dei preferiti. incuriosito dal funzionamento ho pensato di analizzarne il traffico http. the odi – open data institute search the odi submit search suggestion hello, this is a suggestion suggestion hello, this is a suggestion topic hello, this is a topic topic hello, this is a topic the week in data membership jobs knowledge & opinion explainers guides reports news blog topics case studies worknotes odi inside business series the week in data projects & services projects data as culture become an odi member startups & fostering innovation research & development tools & resources consultancy & advice courses and training events courses talks webinars members events past events book a speaker for your event our community odi partners odi nodes odi trainers our startups odi members about the odi our strategy tender opportunities odi team contact us terms of use, privacy and policies jobs brand annual reports and financial statements the data spectrum “we want a world where data works for everyone” jeni tennison, vice president and chief strategy advisor what we do > our services make better decisions using data ...and manage any harmful impacts. we work with companies and governments to build an open, trustworthy data ecosystem. our services what's new > explore all data on teachers’ lives during the pandemic our new report looks at how the pandemic has affected teachers and pupils across the country reflections on the commission for race and ethnic disparities report dr jeni tennison obe and milly zimeta look at the data used in the sewell report by the commission for race and ethnic disparities highlights from our innovate uk-funded r&d programme – take a look at how we’ve supported innovation, improved data infrastructure and encouraged ethical data sharing across the uk over the past four years explore all new report > explore all data about teachers' lives in the pandemic the impact of the pandemic on teachers’ and pupils’ lives, through the lens of new data made available to the odi explore all our current projects > explore all transforming agriculture in south asia and sub-saharan africa – how a new data toolkit promises to transform agriculture and secure food supplies for growing communities helping bridge the data divide – the odi and microsoft are working together to help address the looming ‘data divide’ data innovation for the uk – our r&d projects for innovate uk cut across themes of data sharing and trust, supporting innovation and upgrading data infrastructure explore all work with us whether you’re in the private, public or third sectors, we can help with your data strategy. share your ideas with us, and we’ll be in touch to find out more. get in touch latest blogposts > explore all evaluation of the odi's r&d programme, funded by innovate uk invitation to tender (itt) - evaluating the data assurance market the weird and the wonderful: reflections on the commission for race and ethnic disparities report objective data? reflections on the commission for race and ethnic disparities report explore all explainers > explore all what is a digital twin? what is an identifier? what is a computer model? spot the difference – explaining the covid- apps explore all data ethics canvas > view the canvas free tool to help identify and manage ethical issues the data ethics canvas is a free, downloadable tool for anyone who collects, shares or uses data. it helps identify and manage ethical issues – at the start of a project that uses data, and throughout. view the canvas free tools > explore all the data ethics canvas open standards for data data ecosystem mapping datopolis board game explore all upcoming talks > explore all odi fridays: artist everest pipkin on the making of shell song odi fridays: open data and china – a ten year review odi fridays: data for tackling non-communicable diseases odi fridays: making local deliveries safer, cleaner, and healthier explore all upcoming events > explore all open data in a day: online applying machine learning and ai techniques to data anonymisation is for everyone anonymisation is for everyone explore all the open data institute works with companies and governments to build an open, trustworthy data ecosystem, where people can make better decisions using data and manage any harmful impacts. find out more about what we do open data institute, th floor, kings place, york way, london n ag information wants to be free information wants to be free a librarian, writer and educator reflecting on the profession and the tools we use to serve our patrons drop the ball when i visited my parents in december of , they asked me to go through a box of old stuff they wanted to get rid of. my mother had kept basically all the art we did, the bajillion songs and poems i wrote, everything we did for school, etc. it was surprising how much she&# ;d ... in all the bad&# ; some good things wow, this has been a hard year. no one&# ;s life has been untouched by between the pandemic and unrelenting proof that the social safety net has been dismantled by late-stage capitalism, the state-sanctioned murders of black and brown people and ensuing protests, the horrendous wildfires that felt like horsemen of the coming climate apocalypse, and a stressful election. it&# ;s horrifying. ... making customizable interactive tutorials with google forms in september, i gave a talk at oregon state university&# ;s instruction librarian get-together about the interactive tutorials i built at pcc last year that have been integral to our remote instructional strategy. i thought i&# ;d share my slides and notes here in case others are inspired by what i did and to share the amazing ... the crushing expectations on working women and where&# ;s my fucking village? on friday and saturday, my twitter feed was full of anger and frustration over a blog post on the alsc (association for library services to children) blog. entitled &# ;how motherhood has influenced me as a children’s librarian,&# ; the post was problematic because it suggested (probably unintentionally) that childless children&# ;s librarians could not connect with patrons as much or have ... recognition doesn&# ;t have to be a zero sum game as usual, the week the library journal movers and shakers were announced, i saw plenty of complaints about the award and, in some cases, awardees. i’ve been reading this sort of hurtful negativity since when i was named a mover and shaker (and a friend of mine wrote a blog comment calling us “the ... thoughts on work, well-being, solidarity, and advocacy in our current&# ; situation i have been wanting to blog for weeks. i have several blog posts i started that i just couldn&# ;t get through. my attention span reminds me of my son&# ;s at age when his teacher delicately suggested we should have him assessed for adhd. it rapidly jumps between various tasks at hand, my family, my ... #lismentalhealth: that time my brain and job tried to kill me happy lis mental health week friends! i want to start this post by recognizing someone who has done a great deal to support library workers&# ; mental health in the face of toxic workplaces, kaetrena davis kendrick. kaetrena has done some incredibly valuable research on low morale and toxic workplaces in librarianship and has created an awesome ... my year in books (and podcasts) this was a pretty good year for me. nothing particularly amazing or wonderful or eventful happened to me, though my son has been such a source of pride and light for me that i sometimes can&# ;t believe i&# ;m his mom. i still live in the same messed up world we all do. my migraines have actually ... when libraries and librarians pretend to be neutral, they often cause harm two recent events made me think (again) about the toxic nature of &# ;library neutrality&# ; and the fact that, more often than not, neutrality is whiteness/patriarchy/cis-heteronormativity/ableism/etc. parading around as neutrality and causing harm to folks from historically marginalized groups. the insidious thing about whiteness and these other dominant paradigms is that they are largely invisible to ... thoughts at mid-career part : where to from here? this is the fifth in a series of essays. you can access the rest here, though it’s not necessary to read them all or in order. &# ;to me, the only habit worth ‘designing for’ is the habit of questioning one’s habitual ways of seeing” -jenny odell, how to do nothing &# ;we have to fight for this world, but we ... maisonbisson menu close home search subscribe ☰menu maisonbisson a bunch of stuff i would have emailed you about scroll downpage of older posts → every journalist ryu spaeth on the dirty job of journalism: [e]very journalist […] at some point will have to face the morally indefensible way we go about our business: namely, using other people to tell a story about the world. not everyone dupes their subjects into trusting them, but absolutely everyone robs other people of their stories to tell their own. every journalist knows this flushed feeling, a mix of triumph and guilt, of securing the story that will redound glory unto them, not the subject. some subjects who have no outlet, who are voiceless, approve of this arrangement, since they have no other way of getting their story heard. but even they will not wholly recognize their own depiction in the newspaper, by virtue of the fact that it was told by someone else with their own agenda. this is what jonathan franzen has called the “inescapable shame of being a storyteller”—that it involves stealing from another person, much in the way some people believe a photograph steals a bit of the sitter’s soul. casey bisson on #journalism, #reporting, #storytelling, dec the three tribes of the internet authors primavera de filippi, juan ortiz freuler, and joshua tan outline three competing narratives that have shaped the internet: libertarian, corporate, and nationalist. “[these narratives] emerged from a community of shared interests; each calls for a set of institutional arrangements; each endures in today’s politics.” » about words casey bisson on #internet, #hyperspace, #law, #governance, #libertarian, #corporate, #nationalist, #berkman klein center, #harvard berkman center, nov happy d.b. cooper day d.b. cooper day is celebrated on this day, the saturday following thanksgiving, every year. casey bisson on #agent smith, #aircraft hijacking, #aviation accidents and incidents, #d.b. cooper, #fbi, #federal bureau of investigation, #festival, #hijackers, #hijacking, #mysteries, #skyjacking, nov vitaminwater's #nophoneforayear contest back in the before times, vitaminwater invited applicants to a contest to go a full year without a smartphone or tablet. it was partly in response to rising concerns over the effect of all those alerts on our brains. over , people clamored for the chance, but author elana a. mugdan’s entry stood out with an amusing video, and in february the company took away her iphone s and handed her a kyocera flip phone. » about words casey bisson on #vitaminwater, #nophoneforayear, #scrollfreeforayear, #smartphones, #ethical technology, #humane technology, nov membership-driven news media from the membership guide’s handbook/manifesto: journalism is facing both a trust crisis and a sustainability crisis. membership answers to both. it is a social contract between a news organization and its members in which members give their time, money, energy, expertise, and connections to support a cause that they believe in. in exchange, the news organization offers transparency and opportunities to meaningfully contribute to both the sustainability and impact of the organization. elsewhere it continues: membership is not subscription by another name, nor a brand campaign that can be toggled on and off. …and: memberful routines are workflows that connect audience members to journalism and the people producing it. routines are the basis for a strong membership strategy. notice that audience members are specified here, which is likely a wider group than your members. casey bisson on #membership, #journalism, #monetization, #publishers, #news organizations, #media, oct political bias in social media algorithms and media monetization models new reports reveal yet more structural political biases in consumption and monetization models. » about wordscasey bisson on #politics, #media, #algorithms, #monetization, #bias, #journalism, #social media, #news organizations, oct media monetization vs. internet advertising media face structural, regulatory, and technical hurdles to effectively monetizing with ads on the internet, but there are some solutions that are working. » about words casey bisson on #advertising, #ads, #media monetization, #monetization models, #media, #journalism, #news organizations, aug the argument against likes: aim for deeper, more genuine interactions sweet pea on the state of social media and dating apps: “we are not creating a healthy society when we’re telling millions of young people that the key to happy relationships is photo worthy of an impulsive right swipe.” » about words casey bisson on #likes, #social media, #dating apps, #social software, #signal, aug paid reactions: virtual awards and tipping reddit and twitch both allow members to pay for the privilege of reacting to other member's content with special awards or stickers. » about words casey bisson on #social media, #reactions, #paid reactions, #virtual awards, #tipping, #revenue, #reddit, #twitch, aug reactions facebook introduced reactions with an emphasis on both the nuance they enabled and the mobile convenience: “[i]f you are sharing something that is sad [...] it might not feel comfortable to like that post.” later: “commenting might afford nuanced responses, but composing those responses on a [mobile] keypad takes too much time.” » about words casey bisson on #reactions, #likes, #social media, #facebook, #instagram, aug “likes” vs. “faves” twitter switched from faves to likes in . “you might like a lot of things, but not everything can be your *favorite*,” they explained. weeks after the change, liking activity for existing users was up % and % for new users. » about words casey bisson on #likes, #faves, #social media, #twitter, #facebook, #microcopy, aug honey cocktails: eau de lavender liquor.com’s recipe for eau de lavender, from a larger collection of cocktails with honey. they all look and sound delightful, but i can vouch for the eau de lavender. ingredients / oz tequila / oz fresh lemon juice / oz honey syrup egg white dash scrappy’s lavender bitters garnish: lavender sprig steps add all ingredients into a shaker and dry-shake (without ice). add ice and shake again to emulsify thoroughly. strain into a chilled coupe glass. garnish with a lavender sprig. honey syrup: add / cup honey and / cup water to a small saucepan over medium heat. (you can experiment and decide how much of a honey flavor you want in your syrup. the more honey you use, the thicker the syrup and stronger in flavor it will be.) stir until blended. strain into a jar and seal tightly with a lid. will keep for month in the refrigerator. ↩︎ casey bisson on #cocktails, #mixology, #honey, may satellite tracking if you’re not reading skyriddles blog, then you’re not tracking the sky above. and you might have missed the re-discovery of a satellite launched in and lost for nearly years. as it turns out, there’s a lot of stuff that’s been forgotten up there, and quite a bit that some are trying to hide. the blog is an entertaining view into the world satellites, including communication, spy, weather, research, and the occasional probe going further afield. casey bisson on #satellite tracking, #space, apr i'm missing restaurants now @nakedlunchsf was notable for having both a strong contender for the best burger in the city, _and_... casey bisson on #photo, #photoblog, #stayhome, #supportlocalbusiness, mar when unzip fails on macos with utf unzip can fail on macos when utf- chars are in the archive. the solution is to use ditto. via a github issue: ditto -v -x -k --sequesterrsrc --rsrc filename.zip destinationdirectory casey bisson on #zip, #unzip, #macos, #utf , feb tiktok vs. instagram zuckerberg describes tiktok as “almost like the explore tab that we have on instagram,” but connie chan suggests he's missing the deeper value of ai, and techcrunch's josh constantine suggests zuck is missing the bigger difference in intent on tiktok. » about words casey bisson on #tiktok, #instagram, #social media, #social software, #social networks, #social signals, #artificial intelligence, #ai, jan swipegram template benjamin lee’s instructions and downloadable template to make panoramic carousel instagrams (aka #swipegram), as illustrated via his animation above. » about words casey bisson on #instagram, #template, #swipegram, dec “it is clear that the books owned the shop... “it is clear that the books owned the shop rather than the other way about. everywhere they... casey bisson on #photo, #photoblog, #lovemaine, #portlandmaine, #mustbevancouver, #penderstreet, #downtownvancouver, dec “life is like riding a bicycle... “life is like riding a bicycle. to keep your balance, you must keep moving.” —wisdom by albert... casey bisson on #photo, #photoblog, #forahappymoment, #voreskbh, #visitcopenhagen, #buyfilmnotmegapixels, #ig_denmark, #fujipro h, #ishootfilm, #travelog, #filmisnotdead, #visitdenmark, #mytinyatlas, #pro h, #fuji, #believeinfilm, #københavn, #analoguepeople, #instapassport, #staybrokeshootfilm, #hasselblad, #igerscopenhagen, #flashesofdelight, #exploringtheglobe, nov notes about spotify creator features spotify often gets bashed by top creators. the service pays just $ . per stream, but with million users listening to an average of hours per month, those streams can add up for creators who can get the listener’s attention. spotify verifies artists who then get additional benefits on the platform. some artists find success the traditional route, some optimize their work for the system, others work the system…and some really work it. relevance to other network/aggregation platforms: tiny payments add up, and given a platform, creators will find a way to get and maximize value from it. the critical component is customers. casey bisson on #spotify, #creators, #social networks, #revenue, #aggregation, nov exiftool examples i use for encoding analog camera details i’m a stickler for detail and love to add exif metadata for my film cameras to my scanned images. these are my notes to self about the data i use most often. i only wish exif had fields to record the film details too. » about wordscasey bisson on #exiftool, #photography, #exif, #metadata, nov random notes on instagram delete your old photos, rebrand your page, and delete it entirely are all common advice. plus some tools and traps to be aware of. » about words casey bisson on #instagram, #social media, #photography, oct every media has its tastemakers and influencers every media, network, or platform has would-be influencers or promoters who can help connect consumers with creators. don’t mistake the value of these tastemakers, and be sure to find a place for them to create new value for your platform. » about wordscasey bisson on #spotify, #instagram, #social media, #social networks, #influencers, #tastemakers, oct storehouse: the most wonderful story sharing flop ever storehouse shuttered in summer , just a couple years after they launched, but the app and website introduced or made beautiful a few features that remain interesting now. » about wordscasey bisson on #storehouse, #photo sharing, #story sharing, #microblogging, #blogging, #social media, #user-generated content, #ugc, oct page of older posts →maisonbisson javascript is not available. we’ve detected that javascript is disabled in this browser. please enable javascript or switch to a supported browser to continue using twitter.com. you can see a list of supported browsers in our help center. help center terms of service privacy policy cookie policy imprint ads info © twitter, inc. something went wrong, but don’t fret — let’s give it another shot. islandora open meeting: april , | islandora skip to main content toggle navigation main menu home about events blog contact newsletter support islandora search search you are here : home islandora open meeting: april , about menu islandora foundation get started community contribute help islandora open meeting: april , we are happy to announce the date of our next open meeting! join us on april , any time between : - : pm edt. the open meetings are drop-in style sessions where users of all levels and abilities gather to ask questions, share use cases and get updates on islandora. there will be experienced islandora users on hand to answer questions or give demos. we would love for your to join us any time during the -hour window, so feel free to pop by any time! more details about the open meeting, and the zoom link to join, are in this google doc. registration is not required. if you would like a calendar invite as a reminder, please let us know at community@islandora.ca. submitted by agriffith on tue, / / - : log in to post comments notvisible home about events blog contact newsletter support islandora © copyright islandora foundation. header photo credits. privacy policy. javascript is not available. we’ve detected that javascript is disabled in this browser. please enable javascript or switch to a supported browser to continue using twitter.com. you can see a list of supported browsers in our help center. help center terms of service privacy policy cookie policy imprint ads info © twitter, inc. something went wrong, but don’t fret — let’s give it another shot. hectic pace hectic pace a view on libraries, the library business, and the business of libraries my pre-covid things authors note: these parodies are always about libraries and always based on christmas songs, stories, or poems. being what it is, this year is an exception to both&# ;that&# ;s right, i&# ;m siding with my family and admitting that my favorite things is not a christmas song. (sung to the tune of “my favorite things&# ;) [click the youtube link to listen while you sing along.] eating in restaurants and movies on big screenspeople who don&# ;t doubt the virtue of vaccines.inspiring leaders who don&# ;t act like kings.these were a few of my pre-covid things. live music venues and in-person classes.no masks or ... sitting in the reading room all day (sung to the tune of “walking in a winter wonderland”) [click the youtube link to listen while you sing along.] people shhhhhh, are you listening? in the stacks, laptops glistening the reading light&# ;s bright the library&# ;s right for sitting in the reading room all day. gone away are the book stacks here to stay, the only town&# ;s fax. we share all our books without judgy looks. sitting in the reading room all day. in the lobby we could build a book tree. readers guide is green and they stack well. i&# ;ll say &# ;do we have &# ;em?&# ; you&# ;ll say, &# ;yeah man.&# ; ... it’s the best library time of the year (sung to the tune of “it&# ;s the most wonderful time of the year&# ;) press play to sing along with the instrumental track! it&# ;s the best library time of the year with no more children yelling and no one is telling you &# ;get it in gear!&# ; it&# ;s the best library time of the year it&# ;s the qui-quietest season at school only smile-filled greetings and no more dull meetings where bosses are cruel it&# ;s the qui-quietest season at school there&# ;ll be books for re-stocking vendor end-of-year-hawking and overdue fine cash for beer send the word out to pre-schools drag queen visit ... maybe it’s books we need [i figured this was a song in desperate need of some new lyrics. sung to the tune of baby it&# ;s cold outside. you&# ;re gonna want to grab a singing partner and use the instrumental track for this one!] (listen to the track while you sing!) i really must binge (but maybe it&# ;s books we need) you mustn&# ;t infringe (it&# ;s definitely books we need) this season has been (reading will make you grin) so fun to watch (i&# ;ll hold the remote, you hold my scotch) my netflix queue scrolls forever (mystery, poems, whichever) and stranger things won&# ;t just watch itself (grab ... being a better ally: first, believe warning: i might make you uncomfortable. i’m uncomfortable. but it comes from an earnest place. i was recently lucky enough to participate with my oclc membership &# ; research division colleagues in deetta jones &# ; associates’ cultural competency training. this day-long session has a firm spot in the top of my professional development experiences. (not coincidentally, one of the others in that top was deetta’s management training i took part in when she was with the association of research libraries). a week later, i&# ;m still processing this incredible experience. and i&# ;m very grateful to oclc for sponsoring the workshop! ... fake news forever! librarians were among the first to join the call to arms and combat the onslaught of fake news that has permeated our political discussions for the last several months. frankly, it seems hard for anyone to be on the other side of this issue. but is it? not long after the effort to stop fake news in its tracks, a group of librarians began to consider the long-term implications of eradicating an entire body of content from history. thus began a concerted effort to preserve all the fake news that a vigilant group of librarians could gather up. building on ... how will you be remembered? my grandfather had a sizable library when he passed away, and his son (my father) would wind up with roughly half of it. i remember shelves and shelves of books of quotations. he was a criminal lawyer with a love of quotes. i either inherited this love or caught it through the osmosis of being surrounded by these books throughout my childhood. most of the books were ruined over the years by mold and silverfish and a dose of neglect. but i managed to save a few handfuls of eclectic titles. their smell still transports me to the basement of ... seeking certainty “uncertain times” is a phrase you hear a lot these days. it was actually in the title of the ala town hall that took place in atlanta last month (ala town hall: library advocacy and core values in uncertain times). political turmoil, uncertainty, divisiveness, and vitriol have so many of us feeling a bit unhinged. when i feel rudderless, adrift, even completely lost at sea, i tend to seek a safer port. i’ve exercised this method personally, geographically, and professionally and it has always served me well. for example, the stability and solid foundation provided by my family gives me solace ... no not google search box, just you (to the tune of “all i want for christmas is you”) (if you need a karaoke track, try this one) i don’t need a lot for freedom, peace, or love, democracy, and i don’t care about the congress or their failed bureaucracy i just want a li-brar-y filled with places just for me a librarian or two no not google search box, just you i don’t want a lot of features search results are too grotesque i don’t care about the systems back behind your reference desk i don’t need to download e-books on the de-vice of my choice noisy ... we are ala i’ve been thinking a lot about governance lately. that said, i will avoid the topic of the recent u.s. election as much as possible, even though it is a factor in what makes me think about governance. instead, i will focus on library governance and what makes it work and not work. spoiler alert: active participation. i am an admitted governance junky, an unapologetic lover of robert’s rules of order, and someone who tries to finds beauty in bureaucratic process. i blame my heritage. i come from a long line of federal government employees, all of us born in the ... planet code lib http://planet.code lib.org planet code lib - http://planet.code lib.org ed summers: https://inkdroid.org/ / / /coincidence/

coincidence?

- - t : : + : digital library federation: the #dlfteach toolkit: recommending epubs for accessibility https://www.diglib.org/the-dlfteach-toolkit-recommending-epubs-for-accessibility/

this post was written by hal hinderliter, as part of practitioner perspectives: developing, adapting, and contextualizing the #dlfteach toolkit, a blog series from dlf’s digital library pedagogy group highlighting the experiences of digital librarians and archivists who utilize the #dlfteach toolkit and are new to teaching and/or digital tools.

the digital library pedagogy working group, also known as #dlfteach, is a grassroots community of practice, empowering digital library practitioners to see themselves as teachers and equip teaching librarians to engage learners in how digital library technologies shape our knowledge infrastructure. the group is open to anyone interested in learning about or collaborating on digital library pedagogy. join our google group to get involved.

for this blog post, i’ve opted to provide some background information on the topic of my #dlfteach toolkit entry: the epub (not an acronym) format, used for books and other documents. librarians, instructors, instructional designers and anyone else who needs to select file formats for content distribution should be aware of what epub has to offer!

electronic books: the fight over formats

the production and circulation of books, journals, and other long-form texts has been radically impacted by the growth of computer-mediated communication. electronic books (“e-books”) first emerged near a half-century ago as text-only ascii files, but are now widely available in a multitude of different file formats. most notably, three competing options have been competing for market dominance: pdf files, kf files (for amazon’s kindle devices), and the open-source epub format. the popularity of handheld kindle devices has created a devoted fan base for kf e-books, but in academia the ubiquitous pdf file remains the most common way to distribute self-contained digital documents. in contrast to these options, a growing movement is urging that libraries and schools eschew kindles and abandon their reliance on pdfs in favor of the epub electronic book format.

the epub file format preserves documents as self-contained packages that manage navigation and presentation separately from the document’s reflowable content, allowing users to alter font sizes, typefaces, and color schemes to suit their individual preferences. e-books saved in the epub format are compatible with apple’s ipads and iphones as well as sony’s reader, barnes & nobles nook, and an expansive selection of software applications for desktop, laptop, and tablet computers. increasingly, that list includes screen reader software such as voice dream and vitalsource bookshelf, meaning that a single file format – epub – can be readily accessed by both sighted and visually impaired audiences.

the lineage of epub can be traced back to the digital audio-based information system (daisy), developed in under the direction of the swedish library of talking books and braille. today, epub is an open-source standard that is managed by the international digital publishing forum, part of the w c. in contrast to the proprietary origins of both pdf and kf e-books, modifications to the open epub standard have always been subject to public input and debate.

accessibility in academia: epub versus pdf

proponents of universal design principles recommend the use of documents that are fully accessible to everyone, including users of assistive technologies, e.g., screen readers and refreshable braille displays. the dtbook format, a precursor to epub, was specifically referenced by rose et al. ( ) in their initial delineation of universal design for learning (udl) as part of udl’s requirement for multiple means of presentation. at the time, the assumption was that dtbooks would be distributed only to students who needed accessible texts, with either printed copies or pdf files for sighted learners. today, however, it is no longer necessary to provide multiple formats, since epub (the accessibility community’s preferred replacement for dtbooks) can be used with equal efficacy by all types of students.

in contrast, pdf files can range from completely inaccessible to largely accessible, depending on the amount of effort the publisher expended during the remediation process. pdf files generated from word processing programs (e.g., microsoft word) are not accessible by default, but instead require additional tweaks that necessitate the use of adobe’s acrobat pro software (the version of acrobat that retails for $ per year). users of assistive technologies have no recourse but to attempt opening a pdf file before often finding that the document lacks structure (needed for navigation), alt tags, metadata, or other crucial features. even for sighted learners, pdfs downloaded from their university’s online repository will be difficult to view on smartphones, since pdf’s fixed page dimensions will require endless zooming and scrolling to display each column of text at an adequate font size.

the superior accessibility of epub has inspired major publishers to establish academic repositories of articles in epub format, e.g., abc-clio, acls humanities, ebsco e-books, proquest’s ebrary, elsevier’s sciencedirect, taylor & francis. many digital-only journals offer their editions as epubs. for example, trude eikebrokk, editor of professions & professionalism, investigated the advantages of publishing in the epub format as described in this excerpt from the online journal code{ }lib:

there are two important reasons why we wanted to replace pdf as our primary e-journal format. pdf is a print format. it will never be the best choice for reading on tablets (e.g. ipad) or smartphones, and it is challenging to read pdf files on e-book readers … we wanted to replace or supplement the pdf format with epub to better support digital reading. our second reason for replacing pdf with epub was to alleviate accessibility challenges. pdf is a format that can cause many barriers, especially for users of screen readers (synthetic speech or braille). for example, excel tables are converted into images, which makes it impossible for screen readers to access the table content. pdf documents might also lack search and navigation support, due to either security restrictions, a lack of coded structure in text formats, or the use of pdf image formats. this can make it difficult for any reader to use the document effectively and impossible for screen reader users. on the other hand, correct use of xhtml markup and css style sheets in an epub file will result in search and navigation functionalities, support for text-to-speech/braille and speech recognition technologies. accessibility is therefore an essential aspect of publishing e-journals: we must consider diverse user perspectives and make universal design a part of the publishing process.

the future of epub

a robust community of accessibility activists, publishers, and e-book developers continues to advance the epub specification. the update to epub added synchronized audio narration, embedded video, mathml equations, html animations, and javascript-based interactivity to the format’s existing support for metadata, hyperlinks, embedded fonts, text (saved as xhtml files) and illustrations in both scalable vector graphic (svg) and pixel-based formats. next up: the recently announced upgrade to epub . , which embraces documents created under the . standard while improving support for accessible rich internet applications (aria) and other forms of rich media. if you’re ready to join this revolution, have a run through the #dlfteach toolkit’s epub makerspace lesson plan!

the post the #dlfteach toolkit: recommending epubs for accessibility appeared first on dlf.

- - t : : + : gayle hangingtogether: nederlandse ronde tafel sessie over next generation metadata: denk groter dan naco en worldcat http://feedproxy.google.com/~r/hangingtogetherorg/~ /n-abc qabia/

met dank aan ellen hartman, oclc, voor het vertalen van de oorspronkelijke engelstalige blogpost.

op maart werd een nederlandse ronde tafel discussie georganiseerd als onderdeel van de oclc research discussieserie over next generation metadata.

bibliothecarissen, met achtergronden in metadata, bibliotheeksystemen, de nationale bibliografie en back-office processen, namen deel aan deze sessie. hierbij werd een mooie variatie aan academische en erfgoed instellingen in nederland en belgië vertegenwoordigd. de deelnemers waren geëngageerd, eerlijk en leverden met hun kennis en inzicht constructieve bijdragen aan een prettige uitwisseling van kennis.

in kaart brengen van initiatieven

net als in de andere ronde tafel sessies werden de deelnemers gevraagd om in kaart te helpen brengen wat voor next generation metadata initiatieven er in nederland en belgië worden ontplooid. de kaart die daarmee werd gevuld laat zien dat in deze regio een sterke vertegenwoordiging is van bibliografische en erfgoed projecten (zie de linker helft van de matrix). verschillende next-generation metadata projecten van de koninklijke bibliotheek nederland werden omschreven, zoals:

automatische metadata creatie, waarbij tools voor het taggen en catalogiseren van naam authority records worden geïdentificeerd en getest.
de entity finder, een tool die wordt ontwikkeld om rda entities (personen, werken en expressies) te helpen ontlenen vanuit authorities en bibliografische records.

de digitale erfgoed referentie architectuur (dera) is ontwikkeld als onderdeel van een nationale strategie voor digitaal erfgoed in nederland. het is een framework voor het beheren en publiceren van erfgoed informatie als linked open data (lod), op basis van overeengekomen conventies en afspraken. het van gogh worldwide platform is een voorbeeld van de applicatie van dera, waar metadata gerelateerd aan de kunstwerken van van gogh, die in bezit zijn van nederlandse erfgoed instellingen en in privé bezit worden geaggregeerd.

een noemenswaardig in kaart gebracht initiatief op het gebied van research informatie management (rim) en scholarly communications was de nederlandse open knowledge base. een in het afgelopen jaar opgestart initiatief binnen de context van de deal tussen elsevier en vsnu, nfu en nwo om gezamenlijk open science services te ontwikkelen op basis van rim systemen, elsevier databases, analytics oplossingen en de databases van de nederlandse onderzoeksinstellingen. de open knowledge base zal nieuwe applicaties kunnen voeden met informatie, zoals een dashboard voor het monitoren van de sustainable development goals van de universiteiten. het uitgangspunt van de knowledge base is het significant kunnen verbeteren van de analyse van de impact van research.

wat houdt ons tegen?

ondanks dat er tijdens de sessie innovatieve projecten in kaart werden gebracht, werd er net als in sommige andere sessies, onduidelijkheid gevoeld over hoe we nu verder door kunnen ontwikkelen. ook was er sprake van enig ongeduld met de snelheid van de transitie naar next generation metadata. sommige bibliotheken waren gefrustreerd over het gebrek aan tools binnen de huidige generatie systemen om deze transitie te versnellen. zoals de integratie van persistant identifiers (pid), lokale authorities of links met externe bronnen. meerdere tools moeten gebruiken voor een workflow voelt als een stap terug in plaats van vooruit.

buiten praktische belemmeringen werd de discussie vooral gedomineerd door de vraag wat ons tegenhoudt in deze ontwikkeling. met zoveel bibliografische data die al als lod gepubliceerd wordt, wat is er dan verder nodig om deze data te linken? zouden we niet op zoek moeten naar partners om samen een kennis-ecosysteem te ontwikkelen?

vertrouwen op externe data

een deelnemer gaf aan dat bibliotheken voorzichtig of terughoudend zijn met de databronnen waarmee ze willen linken. authority files zijn betrouwbare bronnen, waarvoor er nog geen gelijkwaardige alternatieven bestaan in het zich nog ontwikkelende linked data ecosysteem. het gebrek aan conventies voor de betrouwbaarheid is misschien een reden waarom bibliotheken misschien wat terughoudend zijn in het aangaan van linked data partnerschappen of terug deinzen voor het vertrouwen op externe data, zelfs van gevestigde bronnen als wikidata. want, het linken naar een databron is een indicatie van vertrouwen en een erkenning van de datakwaliteit.

het gesprek ging vervolgens verder over linked datamodellen. welke data creëer je zelf? hoe geef je je data vorm en link je met andere data? sommige deelnemers gaven aan dat er nog steeds een gebrek aan afspraken en duidelijkheid is over concepten zoals een “werk”. anderen gaven aan dat het vormgeven van concepten precies is waar linked data om draait en dat meerdere onthologieën naast elkaar kunnen bestaan. in andere woorden, het is misschien niet nodig om de naamgeving in harde standaarden te vatten.

“er is geen uniek semantisch model. wanneer je verwijst naar gegevens die al door anderen zijn gedefinieerd, geef je de controle over dat stukje informatie op, en dat kan een mentale barrière zijn tegen het op de juiste manier werken met linked data. het is veel veiliger om alle data in je eigen silo op te slaan en te beheren. maar op het moment dat je dat los kunt laten, kan de wereld natuurlijk veel rijker worden dan je in je eentje ooit kunt bereiken.”

oefenen met denken in linked data

het gesprek ging verder met een discussie over wat we kunnen doen om bibliotheekmedewerkers die catalogiseren te trainen. een van de deelnemers vond dat het handig zou zijn om te beginnen met ze te leren te denken in linked dataconcepten en om te oefenen met het opbouwen van een knowledge graph en het experimenteren met het bouwen van verschillende structuren. net als dat een kind dat doet door met lego te spelen. de deelnemers waren het erover eens dat we op dit moment nog te weinig kennis hebben van de mogelijkheden en de consequenties van het gebruik van linked data.

“we moeten leren onszelf te zien als uitgevers van metadata, zodat anderen het kunnen vinden – maar we hebben geen idee wie de anderen zijn, we moeten zelfs groter denken dan de naco van de library of congress of worldcat. we hebben het niet langer over de records die we maken, maar over stukjes records die uniek zijn, want veel komt al van elders. we moeten ons dit realiseren en onszelf afvragen: wat is onze rol in het grotere geheel? dit is erg moeilijk om te doen!”

de deelnemers gaven aan dat het erg belangrijk was om deze discussie binnen hun bibliotheek op gang te brengen. maar hoe doe je dat precies? het is een groot onderwerp en het zou mooi zijn als daar vanuit het management ook aandacht voor is.

niet relevant voor mijn bibliotheek

een leidinggevende binnen de deelnemersgroep reageerde hierop en gaf aan:

“het valt me op dat de hoeveelheid bibliotheken die hier nog echt mee te maken hebben kleiner wordt. (…) [in mijn bibliotheek] produceren we nauwelijks zelf nog metadata. (…) als we kijken naar wat we zelf nog produceren is dat bijvoorbeeld nog het beschrijven van foto’s van een studentenvereniging, eigenlijk niets dus. metadata is eigenlijk alleen nog een onderwerp voor een kleine groep specialisten.”

hoe provocerend deze observatie ook was, dit weerspiegelt wel een realiteit die we moeten erkennen en tegelijkertijd in perspectief moeten plaatsen. daar was helaas geen tijd voor, want de sessie liep ten einde. het was zeker een gesprek waar we nog een tijd hadden kunnen doorpraten!

over de oclc research discussie serie over next generation metadata

in maart hield oclc research een discussiereeks gericht op twee rapporten:

“transitioning to the next generation of metadata”

“transforming metadata into linked data to improve digital collection discoverability: a contentdm pilot project”.

de rondetafelgesprekken werden gehouden in verschillende europese talen en de deelnemers konden hun eigen ervaringen delen, een beter begrip krijgen van het onderwerp en kregen handvatten om vol vertrouwen plannen te maken voor de toekomst. .

de plenaire openingssessie opende de vloer voor discussie en verkenning en introduceerde het thema en de bijbehorende onderwerpen. samenvattingen van alle rondetafelgesprekken worden gepubliceerd op de oclc research-blog hanging together.

op de afsluitende plenaire vergadering op april werden de verschillende rondetafelgesprekken samengevat.

the post nederlandse ronde tafel sessie over next generation metadata: denk groter dan naco en worldcat appeared first on hanging together.

- - t : : + : titia van der werf open knowledge foundation: open data day – it’s a wrap https://blog.okfn.org/ / / /open-data-day- -its-a-wrap/

on saturday th march , the eleventh open data day took place with people around the world organising over events to celebrate, promote and spread the use of open data.

thanks to the generous support of this year’s mini-grant funders –microsoft, uk foreign, commonwealth and development office, mapbox, global facility for disaster reduction and recovery, latin american open data initiative, open contracting partnership and datopian – the open knowledge foundation offered more than mini-grants to help organisations run online or in-person events for open data day.

we captured some of the great conversations across asia/the pacific, europe/middle east/africa and the americas using twitter moments.

below you can discover all the organisations supported by this year’s scheme as well as seeing photos/videos and reading their reports to help you find out how the events went, what lessons they learned and why they love open data day:

environmental data

code for pakistan
- a hack day to open and publish the block coordinates of the plantation conducted during the billion tree tsunami in pakistan
- read event report
drm africa (democratic republic of the congo)
- preventing vulnerable communities from river floods through risk data collection, analysis and communication
- read event report
escuela de fiscales (argentina)
- our goal is to show the community and other civil society organizations the importance of open data in preserving and caring for the environment, and the urgency of taking action against climate change and pollution, and how open data can improve public politics with the participation of citizens
- read event report
government degree college bemina,j and k higher education (india)
- make the community aware about the availability and benefits of environmental data for addressing environmental concerns in kashmir valley
- read event report
future lab (mexico)
- engage with the local community and enable citizen participation through the use of open data for the proposal of cleaner and more sustainable public policies
- read event report
mijas multimedia (democratic republic of the congo)
- strengthen the community resilience to the rapid rise of lake tanganyika through the use of open data
- read event report
niger delta snapshots (nigeria)
- use open data to uncover hidden threats damaging nigerian mangrove and demonstrate the necessity for urgent action to save nigerian mangrove
- read event report
open knowledge nepal
- organise a datathon that will bring open data enthusiasts to work on the real-time air quality data and twitter bot enhancement, so that people can use the service and get informed with the recent situations of air quality in their surroundings
- read event report
permapeople (germany)
- present and discuss the importance and challenges of collecting and sharing open source data on plants and growing to assist in the growth of the regenerative movement
- read event report
zanzibar volunteers for environmental conservation (tanzania)
- the main goal is to contribute to open data initiatives by helping the students understand more about open data and environmental issues
- read event report

tracking public money flows

afonte jornalismo de dados
- brazilian are tired of corruption, and open data day porto alegre will provide relevant and open-access information to show the path to investigate public expenses and how they are connected to politicians and even companies
- read event report
dataphyte
- train participants on how to track covid- spending using open government data to unearth malpractices and corruption in the management of the pandemic
- read event report
datos concepción (argentina)
- show companies and organizations that received contracts related to covid-
- read event report
equity watch initiative (nigeria)
- using data to ensure that various gender equality and women empowerment projects in nsukka local government area deliver on promises
- read event report
hackbo / grafoscopio
- to intertwine mini wikis, chatbots and public oversight of public expenses, starting with a particular project in the neighborhood, to showcase how grassroots developed civic tech and open government could be bridged, as an empowering alternative to the opaque extractivist social media where such interaction is happening (facebook) beyond the reach and real interest of civic communities
- read event report
ojoconmipisto (guatemala)
- teaching local journalists data visualisation techniques
- read event report
open knowledge estonia
- estonian procurement registry doesn’t use ocds, but the common european standard (ted). our goal is to cross-match the datasets concerning donations and business registries, in order to automatically detect potential conflict of interests
- read event report
universidad latinoamericana de ciencia y tecnología (ulacit)
- data challenge: take advantage of the first dataset published under the ocds in the country and improve the data literacy of university students
- read event report
water with development initiative (nigeria)
- increase transparency and accountability discussing the use of existing wash data
- read event report

open mapping

dih slovenia
- disseminating existing open mapping solutions, sharing best practices and discussion of possibilities for improving life in communities through open mapping
- read event report
federal university of bahia (brazil)
- strengthen a global network of community data collectors from communities, organisations, as well as academic institutions by ) focusing on sharing experiences from specific cases where particular mapping tools were used as part of strategies of community empowerment and ) using the insights to subsequently co-design a platform to empower data collectors globally
- read event report
geoladies ph (philippines)
- since march is international women’s month and st march is international transgender day of visibility, we would like to hold an event that empowers and engages women (cisgender and transgender) to map out features and amenities (women support desks, breastfeeding stations, gender-neutral comfort rooms, and lgbt safe spaces) and feature lightning talks to highlight women in mapping
- read event report
geosm (cameroon)
- host a “geo-evangelisation”, workshop in the use of josm (java openstreetmap ) and geosm (the first % african open source geolocation platform)
- read event report
ilabs@mak project (uganda)
- to understand and value the need of farmers’ live geo map across food value chain in africa to better food traceability and security
- read event report
labiks – latin american bike knowledge sharing
- to promote and stimulate the sharing of open data about the bike-sharing systems in latin america and to promote and discuss our online open map, aiming to improve it
- read event report
monitor de femicidios de utopix (venezuela)
- monitoring of femicide cases in venezuela
- read event report
periféria policy and research center
- learn about the relevance of open data in collective/critical mapping of gentrification in hungary
- read event report
polimappers (italy)
- host an introductory mapping event on openstreetmap so that students and people interested in collaborating gain the basic skills needed to tackle more advanced tools later in the year
- read event report
smartct (philippines)
- launch the mapatanda initiative (a portmanteau of mapa — which means a map — and tanda — which can mean an older adult but can also mean remember); which is an initiative that seeks to improve the number and quality of data in openstreetmap that are important and relevant to older adults (senior citizens) and the ageing population ( + years old) in the philippines
- read event report
suza youthmappers
- create awareness on open data data use, and how the students can use the data in developing innovative web and mobile applications to solve existing challenges in the society
- read event report
tutela learning network in collaboration with local activists and researchers
- start a debate on alternative, community-managed forms of housing in the city of lisbon based on the model of grant of use and raising awareness on the importance of accessible data on available real estate resources owned by the city
- read event report
unificar ações e informações geoespaciais – uaigeo – universidade federal de são joão del-rei (ufsj)
- disseminate the use and importance of open data to support the solution of territorial tension points, the use of water and the preservation of cultural heritage, as well as providing participants with contacts with collaborative mapping applications
- read event report

data for equal development

youth policy cafe (kenya)
- undertake a webinar via the zoom platform themed “leveraging open data as an asset for inclusive & sustainable development in kenya”
- read event report
accesa (costa rica)
- explore, map, visualize and disseminate key data about the projects being implemented by the territorial councils of rural development, the main participatory bodies for fostering rural development in costa rica, and assess their progress, the money being spent on them, the results obtained, and their impact in narrowing the many social gaps that currently affect the different rural regions of the country
- read event report
afroimpacto
- discuss the importance to the black community of the open data discussion
- read event report
cost honduras
- present how we can promote sustainable infrastructure by using data disclosed under the open contracting for infrastructure data standard and engage citizens and civil society organisations to demand government accountability by using a tool called infras
- read event report
dados abertos de feira (brazil)
- promote and discuss the open data knowledge to our local community (city of feira de santana, countryside of brazil), bringing together the academy, government agents and the society itself
- read event report
datafest tbilisi (georgia)
- highlight and promote the use of data and data-driven products as an effective way to tackle pressing social issues and inequality
- read event report
demokrasya (democratic republic of the congo)
- raise awareness of the congolese community especially the women’s rights community on the use of open data in defending the women’s accessibility to employment
- read event report
fundación eduna (colombia)
- develop activities to address the issue of strengthening the capacity for creative thinking of children and young people in the central region of colombia making use and taking advantage of open data
- read event report
gênero e número (brazil)
- explore open data to get a comprehensive landscape on the labour market for women in brazil during the pandemic
- read event report
girls’ tech-changer community (cameroon)
- show the benefits of open data (such as an increase in efficiency, transparency, innovation, and economic growth) and to encourage the adoption of open data policies in various government bodies, businesses, and civil societies
- read event report
hawa feminist coalition (somalia)
- advance the production, dissemination and openness of sex-disaggregated data in somalia in support of evidence-based planning and policy-making as well as tracking of progress by the government and other stakeholders to achieve the sustainable development goals (sdgs)
- read event report
hope for girls and women tanzania
- teaching community about the benefit of using data for development
- read event report
international youth alliance for family planning- togo (iyafp-togo)
- develop an open map of contraceptive methods and service availability in agbalepedo area
- read event report
ipandetec (panama)
- train panamanian women on their current position, role and future in the world of open data
- read event report
iwatch africa
- demonstrate how equal development within the digital ecosystem in africa can be improved by leveraging data on online abuse and harassment of female journalists
- read event report
kiyita foundation
- encourage local women to get access to data about economic development
- read event report
madagascar initiatives for digital innovation
- make participants understand the value of data for development
- read event report
nepal open source klub
- we will create a glossary of technical terms and words that are commonly used on websites/in software and translate those into nepali
- read event report
nukta africa (tanzania)
- maximizing the use of open data to increase accountability through data journalism
- read event report
programming historian (chile)
- walk participants through the process of visualising qualitative and quantitative development open data for equal development in latin america, using open access tools
- read event report
punch up (thailand)
- emphasise what would be lost if we don’t have open data in our country
- read event report
rausing zimbabwe
- create a platform and outlet for information distribution, updates and discussion with communities on the issues surrounding peace and security in the age of the pandemic
- read event report
vilnius legal hackers (lithuania)
- implement more transparency into funeral business of lithuania
- read event report

thanks to everyone who organised or took part in these celebrations and see you next year for open data day !

need more information?

if you have any questions, you can reach out to the open knowledge foundation’s open data day team by emailing opendataday@okfn.org or on twitter via @okfn.

- - t : : + : stephen abbott pugh digital library federation: the #dlfteach toolkit: participatory mapping in a pandemic https://www.diglib.org/the-dlfteach-toolkit-participatory-mapping-in-a-pandemic/

this post was written by jeanine finn (claremont colleges library), as part of practitioner perspectives: developing, adapting, and contextualizing the #dlfteach toolkit, a blog series from dlf’s digital library pedagogy group highlighting the experiences of digital librarians and archivists who utilize the #dlfteach toolkit and are new to teaching and/or digital tools.

see the original lesson plan in the #dlfteach toolkit.

our original activity was designed around using a live googlesheet in coordination with arcgis online to collaboratively map historic locations for an in-class lesson to introduce students to geospatial analysis concepts. in our example, a history instructor had identified a list of cholera outbreaks with place names from th-century colonial reports.

in the original activity, students were co-located in a library classroom, reviewing the historic cholera data in groups. a google sheet was created and shared with everyone in the class for students to enter “tidied” data from the historic texts collaboratively. the students then worked with a live link from google sheets, allowing the outbreak locations to be served directly to the arcgis online map. it was successful and a useful tool for encouraging engagement and for getting familiar with gis.

then covid- in arrived. instead of a centuries-distant disease outbreak, students learning digital mapping this past year were thrust into socially-distant instructional settings driven by a contemporary pandemic that radically altered their modes of learning. the collaborative affordances of tools like arcgis online were pressed into service to help students collaborate effectively and meaningfully in real-time while learning from home.

as an example, one geology professor at pomona college encouraged her students to explore the geology of their local environment. building on shared readings and lectures on geologic history and rock formations, students were encouraged to research the history of the land around them, and include photographs, observations, and other details to enrich the arcgis storymap. the final map included photographs and geology facts from students’ home locations around the world.

geology of places we live: group projects for module "geology of the solid earth" in geol e. , pomona college, september , header for geology class group storymap at pomona college, fall

a key feature of the arcgis storymap platform that appealed to the instructor was the ability for the students to work collaboratively on the platform itself — not across shared files on folders on box, gsuite, the lms, etc. while this functioned reasonably well, there were several roadblocks to effective collaboration that we encountered along the way. most of the challenges related to permissions settings related to arcgis online administration, as the “shared update” features are not set as default permissions. other challenges included file size limitations for images the students wished to upload, the inability of more than one user to edit the same file simultaneously, and potential security issues (including firewalls) in nations with more restrictive internet laws.

reflecting on these uses of storymaps over this past semester, we encourage instructors and library staff interested in to:

review user license permissions and best practices for arcgis storymap collaboration from esri (some links below).
plan ahead to help students with collecting appropriate images, including discussions of file size and copyright.
encourage the instructor to coordinate student groups with defined roles and responsibilities to lessen the likelihood of multiple editors working on the same storymap at once (which can cause corruption of the files.
get clarity from it and other support staff as needed to determine if students are working remotely from countries that may have restrictions on internet use.

resources:

participatory mapping with google forms, google sheets, and arcgis online (esri community education blog): https://community.esri.com/t /education-blog/participatory-mapping-with-google-forms-google-sheets-and-arcgis/ba-p/

optimize group settings to share stories like never before (esri arcgis blog): https://www.esri.com/arcgis-blog/products/story-maps/constituent-engagement/optimize-group-settings-to-share-stories-like-never-before/

teach with story maps: announcing the story maps curriculum portal (university of minnesota, u-spatial: https://research.umn.edu/units/uspatial/news/teach-story-maps-announcing-story-maps-curriculum-portal

getting started with arcgis storymaps (esri): https://storymaps.arcgis.com/stories/cea a a d cccb d c b bc

vi conclusion recommendations

gather materials ahead of time. photographs from digital archives, maps
there may be data cleaning issues.

the post the #dlfteach toolkit: participatory mapping in a pandemic appeared first on dlf.

- - t : : + : gayle david rosenthal: dogecoin disrupts bitcoin! https://blog.dshr.org/ / /dogecoin-disrupts-bitcoin.html

two topics i've posted about recently, elon musk's cult and the illusory "prices" of cryptocurrencies, just intersected in spectacular fashion. on april the bitcoin "price" peaked at $ . k. early on april , the musk cult saw this tweet from their prophet. immediately, the dogecoin "price" took off like a falcon .

a day later, jemima kelley reported that if you believe, they put a dogecoin on the moon. that was to say that:

dogecoin — the crypto token that was started as a joke and that is the favourite of elon musk — is having a bit of a moment. and when we say a bit of a moment, we mean that it is on a lunar trajectory (in crypto talk: it is going to da moon).

at the time of writing this, it is up over per cent in the past hours — more than tripling in value (for those of you who need help on percentages, it is friday afternoon after all). over the past week it’s up more than per cent (almost seven times higher!).

the headlines tell the story — timothy b. lee's dogecoin has risen percent in the last week because why not and joanna ossinger's dogecoin rips in meme-fueled frenzy on pot-smoking holiday.

the dogecoin "price" graph kelly posted was almost vertical. the same day, peter schiff, the notorious gold-bug, tweeted:

so far in #bitcoin has lost % of its value verses #dogecoin. the market has spoken. dogecoin is eating bitcoin. all the bitcoin pumpers who claim bitcoin is better than gold because its price has risen more than gold's must now concede that dogecoin is better than bitcoin.

below the fold i look back at this revolution in crypto-land.

i'm writing on april , and the bitcoin "price" is around $ k, about % of its peak on april . in the same period dogecoin's "price" peaked at $ . , and is now around $ . , or % of its $ . "price" on april . there are some reasons for bitcoin's slump apart from people rotating out of btc into doge in response to musk's tweet. nivesh rustgi reports:

bitcoin’s hashrate dropped % from all-time highs after an accident in the xinjiang region’s mining industry caused flooding and a gas explosion, leading to deaths with workers trapped since.
...
the leading bitcoin mining data centers in the region have closed operations to comply with the fire and safety inspections.

the chinese central authority is conducting site inspections “on individual mining operations and related local government agencies,” tweeted dovey wan, partner at primitive crypto.
...
the accident has reignited the centralization problems arising from china’s dominance of the bitcoin mining sector, despite global expansion efforts.

the drop in the hash rate had the obvious effects. david gerard reports:

the bitcoin hash rate dropped from exahashes per second to eh/s. the rate of new blocks slowed. the bitcoin mempool — the backlog of transactions waiting to be processed — has filled. transaction fees peaked at just over $ average on april.

the average btc transaction fee is now just short of $ , with a median fee over $ ! the btc blockchain did around k transactions on april , but on april it could only manage k.

it is also true that doge had upward momentum before musk's tweet. after being nearly flat for almost a month, it had already doubled since april .

kelly quotes david kimberley at freetrade:

dogecoin’s rise is a classic example of greater fool theory at play, dogecoin investors are basically betting they’ll be able to cash out by selling to the next person wanting to invest. people are buying the cryptocurrency, not because they think it has any meaningful value, but because they hope others will pile in, push the price up and then they can sell off and make a quick buck.

but when everyone is doing this, the bubble eventually has to burst and you’re going to be left short-changed if you don’t get out in time. and it’s almost impossible to say when that’s going to happen.

kelly also quotes khadim shubber explaining that this is all just entertainment:

bitcoin, and cryptocurrencies in general, are not directly analogous to the fairly mundane practice of buying a lottery ticket, but this part of its appeal is often ignored in favour of more intellectual or high-brow explanations.

it has all the hallmarks of a fun game, played out across the planet with few barriers to entry and all the joy and pain that usually accompanies gambling.

there’s a single, addictive reward system: the price. the volatility of cryptocurrencies is often highlighted as a failing, but in fact it’s a key part of its appeal. where’s the fun in an asset whose price snoozes along a predictable path?

the rollercoaster rise and fall and rise again of the crypto world means that it’s never boring. if it’s down one day (and boy was it down yesterday) well, maybe the next day it’ll be up again.

note the importance of volatility. in a must-read interview that new york magazine entitled bidenbucks is beeple is bitcoin prof. george galloway also stressed the importance of volatility:

young people want volatility. if you have assets and you’re already rich, you want to take volatility down. you want things to stay the way they are. but young people are willing to take risks because they can afford to lose everything. for the opportunity to double their money, they will risk losing everything. imagine a person who has the least to lose: he’s in solitary confinement in a supermax-security prison. that person wants maximum volatility. he prays for such volatility, that there’s a revolution and they open the prison.

people under the age of are fed up. they have less than half of the economic security, as measured by the ratio of wealth to income, that their parents did at their age. their share of overall wealth has crashed. a lot of them are bored. a lot of them have some stimulus money in their pocket. and in the case of gamestop, they did what’s kind of a mob short squeeze.
...
i see crypto as a mini-revolution, just like gamestop. the central banks and governments are all conspiring to create more money to keep the shareholder class wealthy. young people think, that’s not good for me, so i’m going to exit the ecosystem and i’m going to create my own currency.

this all reinforces my skepticism about the "price" and "market cap" of cryptocurrencies. - - t : : + : david. (noreply@blogger.com) david rosenthal: what is the point? https://blog.dshr.org/ / /what-is-point.html during a discussion of nfts, larry masinter pointed me to his proposal the 'tdb' and 'duri' uri schemes, based on dated uris. the proposal's abstract reads:

this document defines two uri schemes. the first, 'duri' (standing
for "dated uri"), identifies a resource as of a particular time.
this allows explicit reference to the "time of retrieval", similar to
the way in which bibliographic references containing uris are often
written.

the second scheme, 'tdb' ( standing for "thing described by"),
provides a way of minting uris for anything that can be described, by
the means of identifying a description as of a particular time.
these schemes were posited as "thought experiments", and therefore
this document is designated as experimental.

as far as i can tell, this proposal went nowhere, but it raises a question that is also raised by nfts. what is the point of a link that is unlikely to continue to resolve to the expected content? below the fold i explore this question.

i think there are two main reasons why duri: went nowhere:

the duri: concept implies that web content in general is not static, but it is actually much more dynamic than that. even the duri: specification admits this:
```
there are many uris which are, unfortunately, not particularly
"uniform", in the sense that two clients can observe completely
different content for the same resource, at exactly the same time.
```
personalization, advertisements, geolocation, watermarks, all make it very unlikely that either several clients accessing the same uri at the same time, or a single client accessing the same uri at different times, would see the same content.
when this proposal was put forward in , it was competing with a less elegant but much more useful competitor that had been in use for years. the duri: specificartion admits that:
```
there are no direct resolution servers or processes for 'duri' or
'tdb' uris. however, a 'duri' uri might be "resolvable" in the sense
that a resource that was accessed at a point in time might have the
result of that access cached or archived in an internet archive
service. see, for example, the "internet archive" project
```
but the duri: uri doesn't provide the information needed to resolve to the "cached or archived" content. the internet archive's wayback machine uses uris which, instead of the prefix duri:[datetime]: have the prefix https://web.archive.org/web/[datetime]/. this is more useful, both because browsers will actually resolve these uris, and because they resolve to a service devoted to delivering the content of the uri at the specified time.

the competition for duri: was not merely long established, but also actually did what users presumably wanted, which was to resolve to the content of the specified url at the specified time.

it is true that a user creating a wayback machine url, perhaps using the "save page now" button, would preserve the content accessed by the wayback machine's crawler. which might be different from that accessed by the user themselves. but the user could compare the two versions at the time of creation, and avoid using the created wayback machine url if the differences were significant. publishing a wayback machine url carries an implicit warranty that the creator regarded any differences as insignificant.

the history of duri: suggests that there isn't a lot of point in "durable" uris lacking an expectation that they will continue to resolve to the original content. nfts have the expectation, but lack the mechanism necessary to satisfy the expectation.

- - t : : + : david. (noreply@blogger.com) hangingtogether: recognizing bias in research data – and research data management http://feedproxy.google.com/~r/hangingtogetherorg/~ / moyeyges/

as the covid pandemic grinds on, vaccinations are top of mind. a recent article published in jama network open examined whether vaccination clinical trials over the last decade adequately represented various demographic groups in their studies. according to the authors, the results suggested they did not: “among us-based vaccine clinical trials, members of racial/ethnic minority groups and older adults were underrepresented, whereas female adults were overrepresented.” the authors concluded that “diversity enrollment targets should be included for all vaccine trials targeting epidemiologically important infections.”

my colleague rebecca bryant and i recently enjoyed an interesting and thought-provoking conversation with dr. tiffany grant, assistant director for research and informatics with the university of cincinnati libraries (an oclc research library partnership member) on the topic of bias in research data. dr. grant neatly summed up the issue by observing that data collected should be inclusive of all the groups who are impacted by outcomes. as the jama article illustrates, that is clearly not always the case – and the consequences can be significant for decision- and policy-making in critical areas like health care.

the issue of bias in research data has been acknowledged for some time; for example, the launch of the human genome project in the late s/early s helped raise awareness of the problem, as did observed differences in health care outcomes across demographic groups. and efforts are underway to help remedy some of the gaps. one initiative, the us national institutes of health’s all of us research program, aims to build a database of health data collected from a diverse cohort of at least one million participants. the rationale for the project is clearly laid out: “to develop individualized plans for disease prevention and treatment, researchers need more data about the differences that make each of us unique. having a diverse group of participants can lead to important breakthroughs. these discoveries may help make health care better for everyone.”

extrapolation of findings observed in one group to all other groups often leads to poor inferences, and researchers should take this into account when designing data collection strategies. the peer review process should act as a filter for identifying research studies that overlook this point in their design – but how well is it working? as in many other aspects of our work and social lives, unconscious bias may play a role here: lack of awareness of the problem on the part of reviewers means that studies with flawed research designs may slip through.

and that leads us to what dr. grant believes is the principal remedy for the problem of bias in research data: education. researchers need training that helps them recognize potential sources of bias in data collection, as well as understand the implications of bias for interpretation and generalization of their findings. the first step in solving a problem is to recognize that there is a problem. some disciplines are further along than others in addressing bias in research data, but in dr. grant’s view, there is still ample scope for raising awareness across campus about this topic.

academic libraries can help with this, by providing workshops and training programs, and gathering relevant information resources. at the university of cincinnati, librarians are often embedded in research teams, providing an excellent opportunity to share their expertise on this issue. raising awareness about bias in research data is also an opportunity to partner with other campus units, such as the office of research, colleges/schools, and research institutes (for more information on how to develop and sustain cross-campus partnerships around research support services see our recent oclc research report on social interoperability).

many institutions are currently implementing equality, diversity, and inclusion (edi) training, and modules addressing bias in research data might be introduced as part of edi curricula for researchers. this could also be an area of focus for professional development programs supporting doctoral, postdoctoral, and other early-career researchers. it seems that many edi initiatives focus on issues related to personal interactions or recruiting more members of underrepresented groups into the field. for researchers, it may be useful to supplement this training with additional programs that focus on edi issues as they specifically relate to the responsible conduct of research. in other words, how do edi-related issues manifest in the research process, and how can researchers effectively address them? a great example is the training offered by we all count, a project aimed at increasing equity in data science.

funders can also contribute toward mitigating bias in research data, by issuing research design guidelines on inclusion of underrepresented groups, and by establishing criteria for scoring grant proposals on the basis of how well these guidelines are addressed. the big “carrots and sticks” wielded by funders are a powerful tool for both raising awareness and shifting behaviors.

bias in research data extends to bias in research data management (rdm). situations where access to and ability to use archived data sets is not equitable is another form of bias. while it is good to mandate that data sets be archived under “open” conditions, as many funders already do, the spirit of the mandate is compromised if the data sets are put into systems that are not accessible and usable to everyone. it is important to recognize that the risk of introducing bias into research data exists throughout the research lifecycle, including curation activities such as data storage, description, and preservation.

our conversation focused on bias in research data in stem fields – particularly medicine – but the issue also deserves attention in the context of the social sciences, as well as the arts and humanities. our summary here highlights just a sample of the topics worthy of discussion in this area, with much to unpack in each one. we are grateful to dr. grant for starting a conversation with us on this important issue and look forward to continuing it in the future as part of our ongoing work on rdm and other forms of research support services.

like so many other organizations, oclc is reflecting on equity, diversity, and inclusion, as well as taking action. check out an overview of that work, and explore efforts being undertaken in oclc’s membership and research division. thanks to tiffany grant, rebecca bryant, and merrilee proffitt for providing helpful suggestions that improved this post!

the post recognizing bias in research data – and research data management appeared first on hanging together.

- - t : : + : brian lavoie lucidworks: enhance product discovery with ai-powered recommenders https://lucidworks.com/post/ai-powered-recommenders-for-product-discovery/

learn how ai-powered recommenders put the right products and content in front of your customers, with just the right amount of human touch.

the post enhance product discovery with ai-powered recommenders appeared first on lucidworks.

- - t : : + : andy wibbels tara robertson: distributing dei work across the organization https://tararobertson.ca/ /distributing-dei-work-across-the-organization/

i enjoyed being a guest on seed&spark‘s first monthly office hours session where stefanie monge, lara mcleod and i talked about distributing diversity, equity and inclusion work across organizations.

here’s some of the work that i mentioned:

megan carpenter’s get it wrong for me: what i need from allies
amy edmondson on psychological safety
roxane gay and tressie mcmillan cottom’s podcast hear to slay, which really is the black feminist podcast of my dreams.
mozilla’s community participation guidelines

the post distributing dei work across the organization appeared first on tara robertson consulting.

- - t : : + : tara robertson terry reese: thoughts on nacos proposed process on updating cjk records https://blog.reeset.net/archives/

i would like to take a few minutes and share my thoughts about an updated best practice recently posted by the pcc and naco related to an update on cjk records. the update is found here: https://www.loc.gov/aba/pcc/naco/cjk/cjk-best-practice-ncr.docx. i’m not certain if this is active or a simply a proposal, but i’ve been having a number of private discussions with members at the library of congress and the pcc as i’ve been trying to understand the genesis for this policy change. i personally believe that formally adopting a policy like this would be exceptionally problematic, and i wanted to flesh out my thoughts on why and some potential better options that could fix the issue that this problem is attempting to solve.

but first, i owe some folks an apology. in chatting with some folks at lc (because, let’s be clear, this proposal was created specifically because there are local, limiting practices at lc that artificially are complicating this work) – it came to my attention that the individuals that spent a good deal of time considering and creating this proposal have received some unfair criticism – and i think i bare a lot of responsibility for that. i have done work creating best practices and standards and its thankless, difficult work. because of that, in cases where i disagree with a particular best practice, my preference has been to address those privately and attempt to understand and share my issues with a set of practices. this is what i have been doing related to this work. however, on the marcedit list (a private list), when a request was made related to a feature request in marcedit to support this work – i was less thoughtful in my response as the proposed change could fundamentally undo almost a decade of work as i have dealt with thousands of libraries stymied by these kinds of best practices that have significant unintended consequences. my regret is that i’ve been told that my thoughts shared on the marcedit list, have been used by others in more public spaces to take this committee’s work to task. this is unfortunate and disappointing, and something i should have been more thoughtful of in my responses on the marcedit list. especially, given that every member of that committee is doing this work as a service to the community. i know i forget that sometimes. so, to the folks that did this work – i’ve not followed (or seen) any feedback you may have received, but in as much that i’m sure i played a part in any push back you may have received, i’m sorry.

what does this problem seek to solve?

if you look at the proposal, i think that the writers do a good job identifying the issue. essentially, this issue is unique to authority records. at present, naco still requires that records created within the program only utilize utf characters that fall within the marc- repertoire. oclc, the pipeline for creating these records, enforces this rule by invalidating records with utf characters outside the marc range. the proposal seeks to address this by encouraging the use of nrc (numeric character reference) data in utf records, to work around these normalization issues.

so, in a nutshell, that is the problem, and that is the proposed solution. but before we move on, let’s talk a little bit about how we got here. this problem currently exists because of, what i believe to be, an extremely narrow and unproductive read of what marc repertoire actually means. for those not in libraries, marc is essentially a made-up character encoding, used only in libraries, that has so outlived its usefulness. modern systems have largely stopped supporting it outside of legacy ingest workflows. the issue is that for every academic library or national library that has transitioned to utf , hundreds of small libraries or organizations around the world have not. marc continues to exist because the infrastructure that supports these smaller libraries is built around it.

but again, i think it is worth thinking about today, what actually is the marc repertoire. previously, this had been a hard set of defined values. but really, that changed in ish when lc updated guidance and introduced the concept of nrcs to preserve lossless data transfer between systems that were fully utf compliant and older marc systems. nrcs in marc were workable, because it left local systems the ability to handle (or not handle) the data as it seen fit and finally provided an avenue for the library community as a whole to move on from the limitations marc was imposing on systems. it allowed for the facilitation of data into non-marc formats that were utf compliant and provided a pathway to allow data from other metadata formats, the ability to reuse that data in marc records. i would argue that today, the marc repertoire includes nrc notation – and to assume or pretend otherwise, is shortsighted and revisionist.

but why is all of this important. well, it is at the heart of the problem that we find ourselves in. for authority data, the library of congress appears to have adopted this very narrow view of what marc means (against their own stated recommendations) and as a result, naco and oclc place artificial limits on the pipeline. there are lots of reasons why lc does this, i recognize they are moving slowly because any changes that they make are often met with some level of resistance from members of our community – but in this case, this paralysis is causing more harm to the community than good.

why this proposal is problematic?

so, this is the environment that we are working in and the issue this proposal sought to solve. the issue, however, is that the proposal attempts to solve this problem by adopting a marc solution and applying it within utf data – essentially making the case that nrc values can be embedded in utf records to ensure lossless data entry. and while i can see why someone might think that – that assumption is fundamentally incorrect. when lc developed its guidance on nrc notation, this was guidance that was specifically directed in the lossless translation of data to marc . utf data has no need for nrc notation. this does not mean that it does not sometimes show up – and as a practical purpose, i’ve spent thousands of hours working with libraries dealing with the issues this creates in local systems. aside from the issues this creates in marc systems around indexing and discovery, it makes data almost impossible to be used outside of that system and in times of migration. in thinking about the implications of this change in the context of marcedit, i had the following, specific concerns:

nrc data in utf records would break existing workflows for users with current generation systems that would have no reason to expect this data as being present in utf marc records
it would make normalization functionally virtually impossible and potentially re-introduce a problem i spent months solving for organizations related to how utf data is normalized and introduced into local systems.
it would break many of the transformation options. marcedit allows for the flow of data to many different metadata formats – all are built on the concept that the first thing marcedit does is clean up character encodings to ensure the output data is in utf .
marcedit is used by ~ k active users and ~ k annual users. over / of those users do not use marc and do not use marc- . allowing the mixing of nrcs and utf data potentially breaks functionality for broad groups of international users.

while i very much appreciate the issue that this is attempting to solve, i’ve spent years working with libraries where this kind of practice would introduce a long-term data issue that is very difficult to identify and fix and often shows up unexpectedly when it comes time to migration or share this information with other services, communities, or organizations.

so what is the solution?

i think that we can address this issue on two fronts. first, i would advise naco and oclc to essentially stop limiting data entry to this very limited notion of marc repertoire. in all other contexts, oclc provides the ability to enter any valid utf data. this current limit within the authority process is artificial and unnecessary. oclc could easily remove it, and naco could amend their process to allow record entry to utilize any valid utf character. this would address the problem that this group was attempting to solve for catalogers creating these records.

the second step could take two forms. if lc continues to ignore their own guidance and cleave to an outdated concept of the marc repertoire – oclc could provide to lc via their pipeline a version of the records where data includes nrc notation for use in lcs own systems. it would mean that i would not recommend using lc as a trusted system for downloading authorities if this was the practice unless i had an internal local process to remove any nrc data found in valid utf records. essentially, we essentially treat lc’s requirements as a disease and quarantine them and their influence in this process. of course, what would be more ideal, is lc making the decision to accept utf data without restrictions and rely on applicable guidance and marc best practice by supporting utf data fully, and for those still needing marc data – providing that data using the lossless process of nrcs (per their own recommendations).

conclusion

ultimately, this proposal is a recognition that the current naco rules and process is broken and broken in a way that it is actively undermining other work in the pcc around linked data development. and while i very much appreciate the thoughtful work that went into the consideration of a different approach, i think the unintended side affects would cause more long-term damage that any short-term gains. ultimately, what we need is for the principles to rethink why these limitations are in place, and, honestly, really consider ways that we start to deemphasize the role lc plays as a standard holder if in that role, lc’s presence continues to be an impediment for moving libraries forward.

- - t : : + : reeset lucidworks: how to deliver impactful digital commerce experiences https://lucidworks.com/post/deliver-relevant-digital-commerce-experiences/

acquia and lucidworks share tips for how to deliver meaningful and relevant digital commerce experiences that create customer connections.

the post how to deliver impactful digital commerce experiences appeared first on lucidworks.

- - t : : + : jenny gomez hangingtogether: accomplishments and priorities for the oclc research library partnership http://feedproxy.google.com/~r/hangingtogetherorg/~ /sv osw ybai/

with well underway, the oclc research library partnership is as active as ever. we are heartened by the positive feedback and engagement our partners have provided in response to our programming and research directions. thank you to those who have shared your stories of success and challenge; listening to your voices is what guides us and drives us forward. we warmly welcome the university of notre dame, university of waterloo, and ocad university into the partnership and are pleased to see how they have jumped right into engagement with shares and other activities.

the shares resource sharing community

the shares community has been a source of support and encouragement as resource sharing professionals around the world strive to meet their communities’ information needs during covid- . during the last year, dennis massie has convened more than shares town halls to date to learn how shares members are changing practice to adapt to quickly evolving circumstances. dennis has documented how resource sharing practices have changed.

inspired by the shares community, we are also excited to have launched the oclc interlibrary loan cost calculator. for library administrators and funders to evaluate collection sharing services properly, they need access to current cost information, as well as benchmarks against which to measure their own library’s data. the cost calculator is a free online tool that has the potential to act as a virtual real-time ill cost study. designed in collaboration with resource sharing experts and built by oclc research staff, the calculator has been in the hands of beta testers and early adopters since october . a recorded webinar gives a guided tour of what the tool does (and does not do), what information users need to gather, how developers addressed privacy issues, and how individual institutions and the library community can benefit.

total cost of stewardship: responsible collection building in archives and special collections

a big thanks to our partners who contributed to the total cost of stewardship: responsible collection building in archives and special collections. this publication addresses the ongoing challenge of descriptive backlogs in archives and special collections by connecting collection development decisions with stewardship responsibilities. the report proposes a total cost of stewardship framework for bringing together these important, interconnected functions. developed by the rlp’s collection building and operational impacts working group, the total cost of stewardship framework is a model that considers the value of a potential acquisition and its alignment with institutional mission and goals alongside the cost to acquire, care for, and manage it, the labor and specialized skills required to do that work, and institutional capacity to care for and store collections.

this publication includes a suite of communication and cost estimation tools to help decision makers assess available resources, budgets, and timelines to plan with confidence and set realistic expectations to meet important goals. the report and accompanying resources provide special collections and archives with tools to support their efforts to meet the challenges of contemporary collecting and to ensure they are equitably serving and broadly documenting their communities.

transitioning to the next generation of metadata

in december, we had a bittersweet moment celebrating senior program officer karen smith-yoshimura’s retirement. as mercy procaccini and others take over the role of coordinating the stalwart metadata managers focus group, we are taking time to refine how this dynamic group works and plans future discussions together to better support their efforts. a synthesis of this group’s discussions from the past six years traces how metadata services are transitioning to the “next generation of metadata.”

transforming metadata into linked data

the rlp’s commitment to advancing learning and operational support for linked data continues with the january publication of transforming metadata into linked data to improve digital collection discoverability: a contentdm pilot project. the report details a pilot project that investigated methods for—and the feasibility of—transforming metadata into linked data to improve the discoverability and management of digitized cultural materials and their descriptions. five institutions partnered with oclc to collaborate on this linked data project, representing a diverse cross-section of different types of institutions: the cleveland public library the huntington library, art museum, and botanical gardens the minnesota digital library temple university libraries university of miami libraries.

oclc has invested in pathbreaking linked data work for over a decade, and it is wonderful to add the publication to this knowledge base.

social interoperability in research support

in the area of research support, rebecca bryant developed a robust series of webinars as a follow-on to the – oclc research project, social interoperability in research support. the resulting report, social interoperability in research support: cross-campus partnerships and the university research enterprise, synthesizes information about the highly decentralized, complex research support ecosystem at us research institutions. the report additionally offers a conceptual model of campus research support stakeholders and provides recommendations for establishing and stewarding successful cross-campus relationships. the social interoperability webinar series complements this work by offering in-depth case studies and “stakeholder spotlights” from rlp institutions, demonstrating how other campus are eager to collaborate with the library. this is a great example of the type of programming you can find in our works in progress webinar series.

equity, diversity, and inclusion

our team has been digging into issues of equity, diversity, and inclusion: we’ve developed a “practice group” to help our team be better situated to engaging in difficult conversations around race, and we also have been learning and engaging in conversations about the difficulty of cataloging topics relating to indigenous peoples in respectful ways.

this work has helped to prepare the way for important new work that i’m pleased to share with you today. oclc will be working in consultation with shift collective on the andrew w. mellon-funded convening, reimagine descriptive workflows. the project will bring together a wide range of community stakeholders to interrogate the existing descriptive workflow infrastructure to imagine new workflows that are inclusive, equitable, scalable, and sustainable. we are following an approach developed in other work we have carried out, such as the research and learning agenda for archives, special, and distinctive collections in research libraries, and more recently, in responsible operations: data science, machine learning, and ai in libraries. in that vein, we will host a virtual convening later this year to inform a community agenda publication.

reimagine descriptive workflows is the next stage of a journey that we’ve been on for some time, informed by numerous webinars, surveys, and individual conversations. i am very grateful to team members and the rlp community for their contributions and guidance. we are truly “learning together.”

looking forward

if you are at an oclc rlp affiliated institution and would like to learn more about how to get the most out of your rlp affiliation, please contact your staff liaison (or anyone on our energetic team) and we be happy to set up a virtual orientation or refresher on our programs and opportunities for active learning.

it is with deep gratitude that i offer my thanks to to our partners for their investment in the research library partnership. we are committed to offering our very best to serve your research and learning needs.

the post accomplishments and priorities for the oclc research library partnership appeared first on hanging together.

- - t : : + : rachel frick open knowledge foundation: watch the net zero challenge pitch contest https://blog.okfn.org/ / / /watch-the-net-zero-challenge-pitch-contest/

this week, five shortlisted teams took part in the final stage of the net zero challenge – a global competition to identify, promote and support innovative, practical and scalable uses of open data that advance climate action.

the five teams presented their three-minute project pitches to the net zero challenge panel of experts, and a live audience. each pitch was followed by a live q&a.

the winner of the pitch contest will be announced in the next few days.

if you didn’t have the chance to attend the event in person, watch the event here ( . .min) or see below for links to individual pitches.

a full unedited video of the event is at the bottom of this page.

introduction – by james hamilton, director of the net zero challenge

watch video here ( . min) // introduction slide deck

pitch – by matt sullivan from snapshot climate tool which provides greenhouse gas emission profiles for every local government region (municipality) in australia.

watch pitch video here ( . min) // snapshot slide deck

pitch – by saif shabou from carbongeoscales which is a framework for standardising open data for green house gas emissions at multiple geographical scales (built by a team from france).

watch pitch video here ( . min) // carbongeoscales slide deck

pitch – by jeremy dickens. he presents citizen science avian index for sustainable forests a new bio monitoring tool that uses open data on bird observations to provide crucial information on forest ecological conditions (from south africa).

watch pitch video here ( . min) // avian index – slide deck

pitch – by cristian gregorini from project yarquen which is a new api tool and website to organise climate relevant open data for use by civil society organisations, environmental activists, data journalists and people interested in environmental issues (built by a team from argentina).

watch pitch video here ( . min)

pitch – by beatriz pagy from clima de eleição which analyses recognition of climate change issues by prospective election candidates in brazil, enabling voters to make informed decisions about who to vote in to office.

watch pitch video here ( . min) // clima de eleição – slide deck

concluding remarks – by james hamilton, director of the net zero challenge

watch video here ( . min)

a full unedited video of the net zero challenge is here ( . min)

there are many people who collaborated to make this event possible.

we wish to thank both microsoft and the uk foreign, commonwealth & development office for their support for the net zero challenge. thanks also to open data charter and the open data & innovation team at transport for new south wales for their strategic advice during the development of this project. the event would not have been possible without the enthusiastic hard work of the panel of experts who will judge the winning entry, and the audience who asked such great questions. finally – to all the pitch teams. your projects inspire us and we hope your participation in the net zero challenge has been – and will continue to be – supportive for your work as you use open data to advance climate action.

- - t : : + : james hamilton hugh rundle: a barbaric yawp https://www.hughrundle.net/a-barbaric-yawp/

over the easter break i made a little rust tool for sending toots and/or tweets from a command line. of course there are dozens of existing tools that enable either of these, but i had a specific use in mind, and also wanted a reasonably small and achievable project to keep learning rust.

for various reasons i've recently been thinking about the power of "the unix philosophy", generally summarised as:

write programs that do one thing and do it well.

write programs to work together.

write programs to handle text streams, because that is a universal interface.

my little program takes a text string as input, and sends the same string to the output, the intention being not so much that it would normally be used manually on its own (though it can be) but more that it can "work together" with other programs or scripts. the "one thing" it does (i will leave the question of "well" to other people to judge) is post a tweet and/or toot to social media. it's very much a unidirectional, broadcast tool, not one for having a conversation. in that sense, it's like whitman's "barbaric yawp", subject of my favourite scene in dead poets society and a pretty nice description of what social media has become in a decade or so. calling the program yawp therefore seemed fitting.

yawp takes text from standard input (stdin), publishes that text as a tweet and/or a toot, and then prints it to standard output (stdout). like i said, it's not particularly complex, and not even all that useful for your daily social media posting needs, but the point is for it to be part of a tool chain. for this reason yawp takes the configuration it needs to interact with the mastodon and twitter apis from environment (env) variables, because these are quite easy to set programatically and a fairly "universal interface" for setting and getting values to be used in programs.

here's a simple example of sending a tweet:

yawp 'hello, world!' -t

we could also send a toot by piping from the echo program (the - tells yawp to use stdin instead of looking for an argument like it uses above):

echo 'hello again, world!' | yawp - -m

in bash, you can send the contents of a file to stdin, so we could do this too:

yawp - -mt <message.txt

but really the point is to use yawp to do something like this:

app_that_creates_message | yawp - -mt | do_something_else.sh >> yawping.log

anyway, enjoy firing your barbaric yawps into the cacophony.

- - t : : + : hugh rundle andromeda yelton: i haven’t failed, i’ve just tried a lot of ml approaches that don’t work https://andromedayelton.com/ / / /i-havent-failed-ive-just-tried-a-lot-of-ml-approaches-that-dont-work/

“let’s blog every friday,” i thought. “it’ll be great. people can see what i’m doing with ml, and it will be a useful practice for me!” and then i went through weeks on end of feeling like i had nothing to report because i was trying approach after approach to this one problem that simply didn’t work, hence not blogging. and finally realized: oh, the process is the thing to talk about…

hi. i’m andromeda! i am trying to make a neural net better at recognizing people in archival photos. after running a series of experiments — enough for me to have written , words of notes — i now have a neural net that is ten times worse at its task.

and now i have , words of notes to turn into a blog post (a situation which gets harder every week). so let me catch you up on the outline of the problem:

download a whole bunch of archival photos and their metadata (thanks, dpla!)
use a face detection ml library to locate faces, crop them out, and save them in a standardized way
benchmark an off-the-shelf face recognition system to see how good it is at identifying these faces
retrain it
benchmark my new system

step : profit, right? well. let me also catch you up on some problems along the way:

alas, metadata

archival photos are great because they have metadata, and metadata is like labels, and labels mean you can do supervised learning, right?

well….

is he “du bois, w. e. b. (william edward burghardt), - ” or “du bois, w. e. b. (william edward burghardt) - ” or “du bois, w. e. b. (william edward burghardt)” or “w.e.b. du bois”? i mean, these are all options. people have used a lot of different metadata practices at different institutions and in different times. but i’m going to confuse the poor computer if i imply to it that all these photos of the same person are photos of different people. (i have gone through several attempts to resolve this computationally without needing to do everything by hand, with only modest success.)

what about “photographs”? that appears in the list of subject labels for lots of things in my data set. “photographs” is a person, right? i ended up pulling in an entire other ml component here — spacy, to do some natural language processing to at least guess which lines are probably names, so i can clear the rest of them out of my way. but spacy only has ~ % accuracy on personal names anyway and, guess what, because everything is terrible, in predictable ways, it has no idea “kweisi mfume” is a person.

is a person who appears in the photo guaranteed to be a person who appears in the photo? nope.

is a person who appears in the metadata guaranteed to be a person who appears in the photo? also nope! often they’re a photographer or other creator. sometimes they are the subject of the depicted event, but not themselves in the photo. (spacy will happily tell you that there’s personal name content in something like “martin luther king day”, but mlk is unlikely to appear in a photo of an mlk day event.)

oh dear, linear algebra

ok but let’s imagine for the sake of argument that we live in a perfect world where the metadata is exactly what we need — no more, no less — and its formatting is perfectly consistent.

here you are, in this perfect world, confronted with a photo that contains two people and has two names. how do you like them apples?

i spent more time than i care to admit trying to figure this out. can i bootstrap from photos that have one person and one name — identify those, subtract them out of photos of two people, go from there? (not reliably — there’s a lot of data i never reach that way — and it’s horribly inefficient.)

can i do something extremely clever with matrix multiplication? like…once i generate vector space embeddings of all the photos, can i do some sort of like dot-product thing across all of my photos, or big batches of them, and correlate the closest-match photos with overlaps in metadata? not only is this a process which begs the question — i’d have to do that with the ml system i have not yet optimized for archival photo recognition, thus possibly just baking bad data in — but have i mentioned i have taken exactly one linear algebra class, which i didn’t really grasp, in ?

what if i train yet another ml system to do some kind of k-means clustering on the embeddings? this is both a promising approach and some really first-rate yak-shaving, combining all the question-begging concerns of the previous paragraph with all the crystalline clarity of black box ml.

possibly at this point it would have been faster to tag them all by hand, but that would be admitting defeat. also i don’t have a research assistant, which, let’s be honest, is the person who would usually be doing this actual work. i do have a -year-old and i am strongly considering paying her to do it for me, but to facilitate that i’d have to actually build a web interface and probably learn more about aws, and the prospect of reading aws documentation has a bracing way of reminding me of all of the more delightful and engaging elements of my todo list, like calling some people on the actual telephone to sort out however they’ve screwed up some health insurance billing.

nowhere to go but up

despite all of that, i did actually get all the way through the steps above. i have a truly, spectacularly terrible neural net. go me! but at a thousand-plus words, perhaps i should leave that story for next week….

- - t : : + : andromeda lucidworks: tips for mixed reality in retail https://lucidworks.com/post/tips-for-mixed-reality-in-retail/

how retailers are turning to virtual reality, augmented reality, and mixed reality applications to recreate the in-store experience from anywhere.

the post tips for mixed reality in retail appeared first on lucidworks.

- - t : : + : andy wibbels erin white: talk: using light from the dumpster fire to illuminate a more just digital world https://erinrwhite.com/talk-using-light-from-the-dumpster-fire-to-illuminate-a-more-just-digital-world/

this february i gave a lightning talk for the richmond design group. my question: what if we use the light from the dumpster fire of to see an equitable, just digital world? how can we change our thinking to build the future web we need?

presentation is embedded here; text of talk is below.

hi everybody, i’m erin. before i get started i want to say thank you to the rva design group organizers. this is hard work and some folks have been doing it for years. thank you to the organizers of this group for doing this work and for inviting me to speak.

this talk isn’t about . this talk is about the future. but to understand the future, we gotta look back.

the web in

travel with me to . twenty-five years ago!

i want to transport us back to the mindset of the early web. the fundamental idea of hyperlinks, which we now take for granted, really twisted everyone’s noodles. so much of the promise of the early web was that with broad access to publish in hypertext, the opportunities were limitless. technologists saw the web as an equalizing space where systems of oppression that exist in the real world wouldn’t matter, and that we’d all be equal and free from prejudice. nice idea, right?

you don’t need to’ve been around since to know that’s just not the way things have gone down.

pictured before you are some of the early web pioneers. notice a pattern here?

these early visions of the web, including barlow’s declaration of independence of cyberspace, while inspiring and exciting, were crafted by the same types of folks who wrote the actual declaration of independence: the landed gentry, white men with privilege. their vision for the web echoed the declaration of independence’s authors’ attempts to describe the world they envisioned. and what followed was the inevitable conflict with reality.

we all now hold these truths to be self-evident:

the systems humans build reflect humans’ biases and prejudices.
we continue to struggle to diversify the technology industry.
knowledge is interest-driven.
inequality exists, online and off.
celebrating, rather than diminishing, folks’ intersecting identities is vital to human flourishing.

the web we have known

profit first: monetization, ads, the funnel, dark patterns
can we?: innovation for innovation’s sake
solutionism: code will save us
visual design: aesthetics over usability
lone genius: “hard” skills and rock star coders
short term thinking: move fast, break stuff
shipping: new features, forsaking infrastructure

let’s move forward quickly through the past years or so of the web, of digital design.

all of the web we know today has been shaped in some way by intersecting matrices of domination: colonialism, capitalism, white supremacy, patriarchy. (thank you, bell hooks.)

the digital worlds where we spend our time – and that we build!! – exist in this way.

this is not an indictment of anyone’s individual work, so please don’t take it personally. what i’m talking about here is the digital milieu where we live our lives.

the funnel drives everything. folks who work in nonprofits and public entities often tie ourselves in knots to retrofit our use cases in order to use common web tools (google analytics, anyone?)

in chasing innovation we often overlook important infrastructure work, and devalue work — like web accessibility, truly user-centered design, care work, documentation, customer support and even care for ourselves and our teams — that doesn’t drive the bottom line. we frequently write checks for our future selves to cash, knowing damn well that we’ll keep burying ourselves in technical debt. that’s some tough stuff for us to carry with us every day.

the “move fast” mentality has resulted in explosive growth, but at what cost? and in creating urgency where it doesn’t need to exist, focusing on new things rather than repair, the end result is that we’re building a house of cards. and we’re exhausted.

to zoom way out, this is another manifestation of late capitalism. emphasis on late. because… happened.

what taught us

hard times amplify existing inequalities
cutting corners mortgages our future
infrastructure is essential
“colorblind”/color-evasive policy doesn’t cut it
inclusive design is vital
we have a duty to each other
technology is only one piece
together, we rise

the past year has been awful for pretty much everybody.

but what the light from this dumpster fire has illuminated is that things have actually been awful for a lot of people, for a long time. this year has shown us how perilous it is to avoid important infrastructure work and to pursue innovation over access. it’s also shown us that what is sometimes referred to as colorblindness — i use the term color-evasiveness because it is not ableist and it is more accurate — a color-evasive approach that assumes everyone’s needs are the same in fact leaves people out, especially folks who need the most support.

we’ve learned that technology is a crucial tool and that it’s just one thing that keeps us connected to each other as humans.

finally, we’ve learned that if we work together we can actually make shit happen, despite a world that tells us individual action is meaningless. like biscuits in a pan, when we connect, we rise together.

marginalized folks have been saying this shit for years.
more of us than ever see these things now.
and now we can’t, and shouldn’t, unsee it.

the web we can build together

current state:
– profit first
– can we?
– solutionism
– aesthetics
– “hard” skills
– rockstar coders
– short term thinking
– shipping

future state:
– people first: security, privacy, inclusion
– should we?
– holistic design
– accessibility
– soft skills
– teams
– long term thinking
– sustaining

so let’s talk about the future. i told you this would be a talk about the future.

like many of y’all i have had a very hard time this year thinking about the future at all. it’s hard to make plans. it’s hard to know what the next few weeks, months, years will look like. and who will be there to see it with us.

but sometimes, when i can think clearly about something besides just making it through every day, i wonder.

what does a people-first digital world look like? who’s been missing this whole time?

just because we can do something, does it mean we should?

will technology actually solve this problem? are we even defining the problem correctly?

what does it mean to design knowing that even “able-bodied” folks are only temporarily so? and that our products need to be used, by humans, in various contexts and emotional states?

(there are also false binaries here: aesthetics vs. accessibility; abled and disabled; binaries are dangerous!)

how can we nourish our collaborations with each other, with our teams, with our users? and focus on the wisdom of the folks in the room rather than assigning individuals as heroes?

how can we build for maintenance and repair? how do we stop writing checks our future selves to cash – with interest?

some of this here, i am speaking of as a web user and a web creator. i’ve only ever worked in the public sector. when i talk with folks working in the private sector i always do some amount of translating. at the end of the day, we’re solving many of the same problems.

but what can private-sector workers learn from folks who come from a public-sector organization?

and, as we think about what we build online, how can we also apply that thinking to our real-life communities? what is our role in shaping the public conversation around the use of technologies? i offer a few ideas here, but don’t want them to limit your thinking.

consider the public sector

here’s a thread about public service.

— dana chisnell (she / her) (@danachis) february ,

i don’t have a ton of time left today. i wanted to talk about public service like the very excellent dana chisnell here.

like i said, i’ve worked in the public sector, in higher ed, for a long time. it’s my bread and butter. it’s weird, it’s hard, it’s great.

there’s a lot of work to be done, and it ain’t happening at civic hackathons or from external contractors. the call needs to come from inside the house.

working in the public sector

government should be
– inclusive of all people
– responsive to needs of the people
– effective in its duties & purpose

— dana chisnell (she / her) (@danachis) february ,

i want you to consider for a minute how many folks are working in the public sector right now, and how technical expertise — especially in-house expertise — is something that is desperately needed.

pictured here are the old website and new website for the city of richmond. i have a whole ‘nother talk about that new richmond website. i foia’d the contracts for this website. there are accessibility errors on the homepage alone. it’s been in development for years and still isn’t in full production.

bottom line, good government work matters, and it’s hard to find. important work is put out for the lowest bidder and often external agencies don’t get it right. what would it look like to have that expertise in-house?

influencing technology policy

we also desperately need lawmakers and citizens who understand technology and ask important questions about ethics and human impact of systems decisions.

pictured here are some headlines as well as a contract from the city of richmond. y’all know we spent $ . million on a predictive policing system that will disproportionately harm citizens of color? and that earlier this month, city council voted to allow richmond and vcu pd’s to start sharing their data in that system?

the surveillance state abides. technology facilitates.

i dare say these technologies are designed to bank on the fact that lawmakers don’t know what they’re looking at.

my theory is, in addition to holding deep prejudices, lawmakers are also deeply baffled by technology. the hard questions aren’t being asked, or they’re coming too late, and they’re coming from citizens who have to put themselves in harm’s way to do so.

technophobia is another harmful element that’s emerged in the past decades. what would a world look like where technology is not a thing to shrug off as un-understandable, but is instead deftly co-designed to meet our needs, rather than licensed to our city for . million dollars? what if everyone knew that technology is not neutral?

closing

this is some of the future i can see. i hope that it’s sparked new thoughts for you.

let’s envision a future together. what has the light illuminated for you?

thank you!

- - t : : + : erinrwhite david rosenthal: nfts and web archiving https://blog.dshr.org/ / /nfts-and-web-archiving.html one of the earliest observations of the behavior of the web at scale was "link rot". there were a lot of s, broken links. research showed that the half-life of web pages was alarmingly short. even in this problem was obvious enough for brewster kahle to found the internet archive to address it. from the wikipedia entry for link rot:

a study found that on the web, about one link out of every broke each week,^{[ ]} suggesting a half-life of weeks. this rate was largely confirmed by a – study of links in yahoo! directory (which had stopped updating in after years of development) that found the half-life of the directory's links to be two years.^{[ ]}

one might have thought that academic journals were a relatively stable part of the web, but research showed that their references decayed too, just somewhat less rapidly. a study found a half-life of . years. see my post the evanescent web.

i expect you have noticed the latest outbreak of blockchain-enabled insanity, non-fungible tokens (nfts). someone "paying $ m for a jpeg" or $ k for a new york times column attracted a lot of attention. follow me below the fold for the connection between nfts, "link rot" and web archiving.

kahle's idea for addressing "link rot", which became the wayback machine, was to make a copy of the content at some url, say:

http://www.example.com/page.html

keep the copy for posterity, and re-publish it at a url like:

https://web.archive.org/web/ /http://www.example.com/page.html

what is the difference between the two urls? the original is controlled by example.com, inc.; they can change or delete it on a whim. the copy is controlled by the internet archive, whose mission is to preserve it unchanged "for ever". the original is subject to "link rot", the second is, one hopes, not subject to "link rot". the wayback machine's urls have three components:

https://web.archive.org/web/ locates the archival copy at the internet archive.
indicates that the copy was made on ^th june, at : : .
http://www.example.com/page.html is the url from which the copy was made.

the fact that the archival copy is at a different url from the original causes a set of problems that have bedevilled web archiving. one is that, if the original goes away, all the links that pointed to it break, even though there may be an archival copy to which they could point to fulfill the intent of the link creator. another is that, if the content at the original url changes, the link will continue to resolve but the content it returns may no longer reflect the intent of the link creator, although there may be an archival copy that does. even in the early days of the web it was evident that web pages changed and vanished at an alarming rate.

the point is that the meaning of a generic web url is "whatever content, or lack of content, you find at this location". that is why url stands for universal resource locator. note the difference with uri, which stands for universal resource identifier. anyone can create a url or uri linking to whatever content they choose, but doing so provides no rights in or control over the linked-to content.

in people's expensive nfts keep vanishing. this is why, ben munster reports that:

over the past few months, numerous individuals have complained about their nfts going “missing,” “disappearing,” or becoming otherwise unavailable on social media. this despite the oft-repeated nft sales pitch: that nft artworks are logged immutably, and irreversibly, onto the ethereum blockchain.

so ntfs have the same problem that web pages do. isn't the blockchain supposed to make things immortal and immutable?

kyle orland's ars technica’s non-fungible guide to nfts provides an over-simplified explanation:

when nft’s are used to represent digital files (like gifs or videos), however, those files usually aren’t stored directly “on-chain” in the token itself. doing so for any decently sized file could get prohibitively expensive, given the cost of replicating those files across every user on the chain. instead, most nfts store the actual content as a simple uri string in their metadata, pointing to an internet address where the digital thing actually resides.

nfts are just links to the content they represent, not the content itself. the bitcoin blockchain actually does contain some images, such as this ascii portrait of len sassaman and some pornographic images. but the blocks of the bitcoin blockchain were originally limited to mb and are now effectively limited to around mb, enough space for small image files. what’s the maximum ethereum block size? explains:

instead of a fixed limit, ethereum block size is bound by how many units of gas can be spent per block. this limit is known as the block gas limit ... at the time of writing this, miners are currently accepting blocks with an average block gas limit of around , , gas. currently, the average ethereum block size is anywhere between to kb in size.

that's a little out-of-date. currently the block gas limit is around . m gas per block and the average block is about kb. nowhere near enough space for a $ m jpeg. the nft for an artwork can only be a link. most nfts are erc- tokens, providing the optional metadata extension:

/// @title erc- non-fungible token standard, optional metadata extension
/// @dev see https://eips.ethereum.org/eips/eip- 
/// note: the erc- identifier for this interface is x b e f.
interface erc metadata /* is erc */ {
 /// @notice a descriptive name for a collection of nfts in this contract
 function name() external view returns (string _name);

 /// @notice an abbreviated name for nfts in this contract
 function symbol() external view returns (string _symbol);

 /// @notice a distinct uniform resource identifier (uri) for a given asset.
 /// @dev throws if `_tokenid` is not a valid nft. uris are defined in rfc
 /// . the uri may point to a json file that conforms to the "erc 
 /// metadata json schema".
 function tokenuri(uint _tokenid) external view returns (string);
}

the metadata json schema specifies an object with three string properties:

name: "identifies the asset to which this nft represents"
description: "describes the asset to which this nft represents"
image: "a uri pointing to a resource with mime type image/* representing the asset to which this nft represents. consider making any images at a width between and pixels and aspect ratio between . : and : inclusive."

note that the json metadata is not in the ethereum blockchain, it is only pointed to by the token on the chain. if the art-work is the "image", it is two links away from the blockchain. so, given the evanescent nature of web links, the standard provides no guarantee that the metadata exists, or is unchanged from when the token was created. even if it is, the standard provides no guarantee that the art-work exists or is unchanged from when the token is created.

caveat emptor — absent unspecified actions, the purchaser of an nft is buying a supposedly immutable, non-fungible object that points to a uri pointing to another uri. in practice both are typically urls. the token provides no assurance that either of these links resolves to content, or that the content they resolve to at any later time is what the purchaser believed at the time of purchase. there is no guarantee that the creator of the nft had any copyright in, or other rights to, the content to which either of the links resolves at any particular time.

there are thus two issues to be resolved about the content of each of the nft's links:

does it exist? i.e. does it resolve to any content?
is it valid? i.e. is the content to which it resolves unchanged from the time of purchase?

these are the same questions posed by the holy grail of web archiving, persistent urls.

assuming existence for now, how can validity be assured? there have been a number of systems that address this problem by switching from naming files by their location, as urls do, to naming files by their content by using the hash of the content as its name. the idea was the basis for bram cohen's highly successful bittorrent — it doesn't matter where the data comes from provided its integrity is assured because the hash in the name matches the hash of the content.

the content-addressable file system most used for nfts is the interplanetary file system (ipfs). from its wikipedia page:

as opposed to a centrally located server, ipfs is built around a decentralized system^{[ ]} of user-operators who hold a portion of the overall data, creating a resilient system of file storage and sharing. any user in the network can serve a file by its content address, and other peers in the network can find and request that content from any node who has it using a distributed hash table (dht). in contrast to bittorrent, ipfs aims to create a single global network. this means that if alice and bob publish a block of data with the same hash, the peers downloading the content from alice will exchange data with the ones downloading it from bob.^{[ ]} ipfs aims to replace protocols used for static webpage delivery by using gateways which are accessible with http.^{[ ]} users may choose not to install an ipfs client on their device and instead use a public gateway.

if the purchaser gets both the nft's metadata and the content to which it refers via ipfs uris, they can be assured that the data is valid. what do these ipfs uris look like? the (excellent) ipfs documentation explains:

https://ipfs.io/ipfs/<cid>
# e.g
https://ipfs.io/ipfs/qme ss arvgxv rxqvpiikmj u nlgmgszg pyrdkeoiu
browsers that support ipfs can redirect these requests to your local ipfs node, while those that don't can fetch the resource from the ipfs.io gateway.

you can swap out ipfs.io for your own http-to-ipfs gateway, but you are then obliged to keep that gateway running forever. if your gateway goes down, users with ipfs aware tools will still be able to fetch the content from the ipfs network as long as any node still hosts it, but for those without, the link will be broken. don't do that.

note the assumption here that the ipfs.io gateway will be running forever. note also that only some browsers are capable of accessing ipfs content without using a gateway. thus the ipfs.io gateway is a single point of failure, although the failure is not complete. in practice nfts using ipfs uris are dependent upon the continued existence of protocol labs, the organization behind ipfs. the ipfs.io uris in the nft metadata are actually urls; they don't point to ipfs, but to a web server that accesses ipfs.

pointing to the nft's metadata and content using ipfs uris assures their validity but does it assure their existence? the ipfs documentation's section persistence, permanence, and pinning explains:

nodes on the ipfs network can automatically cache resources they download, and keep those resources available for other nodes. this system depends on nodes being willing and able to cache and share resources with the network. storage is finite, so nodes need to clear out some of their previously cached resources to make room for new resources. this process is called garbage collection.

to ensure that data persists on ipfs, and is not deleted during garbage collection, data can be pinned to one or more ipfs nodes. pinning gives you control over disk space and data retention. as such, you should use that control to pin any content you wish to keep on ipfs indefinitely.

to assure the existence of the nft's metadata and content they must both be not just written to ipfs but also pinned to at least one ipfs node.

to ensure that your important data is retained, you may want to use a pinning service. these services run lots of ipfs nodes and allow users to pin data on those nodes for a fee. some services offer free storage-allowance for new users. pinning services are handy when:
you don't have a lot of disk space, but you want to ensure your data sticks around.
your computer is a laptop, phone, or tablet that will have intermittent connectivity to the network. still, you want to be able to access your data on ipfs from anywhere at any time, even when the device you added it from is offline.
you want a backup that ensures your data is always available from another computer on the network if you accidentally delete or garbage-collect your data on your own computer.

thus to assure the existence of the nft's metadata and content pinning must be rented from a pinning service, another single point of failure.

in summary, it is possible to take enough precautions and pay enough ongoing fees to be reasonably assured that your $ m nft and its metadata and the jpeg it refers to will remain accessible. whether in practice these precautions are taken is definitely not always the case. david gerard reports :

but functionally, ipfs works the same way as bittorrent with magnet links — if nobody bothers seeding your file, there’s no file there. nifty gateway turn out not to bother to seed literally the files they sold, a few weeks later. [twitter; twitter]

anil dash claims to have invented, with kevin mccoy, the concept of nfts referencing web urls in . he writes in his must-read nfts weren’t supposed to end like this:

seven years later, all of today’s popular nft platforms still use the same shortcut. this means that when someone buys an nft, they’re not buying the actual digital artwork; they’re buying a link to it. and worse, they’re buying a link that, in many cases, lives on the website of a new start-up that’s likely to fail within a few years. decades from now, how will anyone verify whether the linked artwork is the original?

all common nft platforms today share some of these weaknesses. they still depend on one company staying in business to verify your art. they still depend on the old-fashioned pre-blockchain internet, where an artwork would suddenly vanish if someone forgot to renew a domain name. “right now nfts are built on an absolute house of cards constructed by the people selling them,” the software engineer jonty wareing recently wrote on twitter.

my only disagreement with dash is that, as someone who worked on archiving the "old-fashioned pre-blockchain internet" for two decades, i don't believe that there is a new-fangled post-blockchain internet that makes the problems go away. and neither does david gerard:

the pictures for nfts are often stored on the interplanetary file system, or ipfs. blockchain promoters talk like ipfs is some sort of bulletproof cloud storage that works by magic and unicorns.

- - t : : + : david. (noreply@blogger.com) journal of web librarianship: the impact of the covid- pandemic on digital library usage: a public library case study https://www.tandfonline.com/doi/full/ . / . . ?ai= dl&mi=co bk&af=r .
- - t : : + : jelena Ćirić evergreen ils: evergreen . . released https://evergreen-ils.org/evergreen- - - -released/

the evergreen community is pleased to announce the release of evergreen . . . evergreen is highly-scalable software for libraries that helps library patrons find library materials and helps libraries manage, catalog, and circulate those materials, no matter how large or complex the libraries.

evergreen . . is a major release that includes the following new features of note:

support for saml-based single sign on
hold groups, a feature that allows staff to add multiple users to a named hold group bucket and place title-level holds for a record for that entire set of users
the bootstrap public catalog skin is now the default
“did you mean?” functionality for catalog search focused on making suggestions for single search terms
holdings on the public catalog record details page can now be sorted by geographic proximity
library groups, a feature that allows defining groups of organizational units outside of the hierarchy that can be used to limit catalog search results
expired staff accounts can now be blocked from logging in
publisher data in the public catalog display is now drawn from both the and field
the staff catalog can now save all search results (up to , ) to a bucket in a single operation
new opt-in settings for overdue and predue email notifications
a new setting to allow expired patrons to renew loans
porting of additional interfaces to angular, including scan item as missing pieces and shelving location groups

evergreen admins installing or upgrading to . . should be aware of the following:

the minimum version of postgresql required to run evergreen . is postgresql . .
the minimum version of opensrf is . .
this release adds anew opensrf service, open-ils.geo.
the release also adds several new perl module dependencies, geo::coder::google, geo::coder::osm, string::keyboarddistance, and text::levenshtein::damerau::xs.
the database update procedure has more steps than usual; please consult the upgrade section of the release notes.

the release is available on the evergreen downloads page. additional information, including a full list of new features, can be found in the release notes.

- - t : : + : galen charlton lucidworks: build semantic search at speed https://lucidworks.com/post/how-to-build-fast-semantic-search/

learn more about using semantic machine learning methodologies to power more relevant search results across your organization.

the post build semantic search at speed appeared first on lucidworks.

- - t : : + : elizabeth edmiston open knowledge foundation: unveiling the new frictionless data documentation portal https://blog.okfn.org/ / / /unveiling-the-new-frictionless-data-documentation-portal/

have you used frictionless data documentation in the past and been confused or wanted more examples? are you a brand new frictionless data user looking to get started learning?

we invite you all to visit our new and improved documentation portal.

thanks to a fund that the open knowledge foundation was awarded from the open data institute, we have completely reworked the guides of our frictionless data framework website according to the suggestions from a cohort of users gathered during several feedback sessions throughout the months of february and march.

we cannot stress enough how precious those feedback sessions have been to us. they were an excellent opportunity to connect with our users and reflect together with them on how to make all our guides more useful for current and future users. the enthusiasm and engagement that the community showed for the process was great to see and reminded us that the link with the community should be at the core of open source projects.

we were amazed by the amount of extremely useful inputs that we got. while we are still digesting some of the suggestions and working out how to best implement them, we have made many changes to make the documentation a smoother, frictionless experience.

so what’s new?

a common theme from the feedback sessions was that it was sometimes difficult for novice users to understand the whole potential of the frictionless specifications. to help make this clearer, we added a more detailed explanation, user examples and user stories to our introduction. we also added some extra installation tips and a troubleshooting section to our quick start guide.

the users also suggested several code changes, like more realistic code examples, better explanations of functions, and the ability to run code examples in both the command line and python. this last suggestion was prompted because most of the guides use a mix of command line and python syntax, which was confusing to our users. we have clarified that by adding a switch in the code snippets that allows user to work with a pure python syntax or pure command line (when possible), as you can see here. we also put together an faq section based on questions that were often asked on our discord chat. if you have suggestions for other common questions to add, let us know!

the documentation revamping process also included the publication of new tutorials. we worked on two new frictionless tutorials, which are published under the notebooks link in the navigation menu. while working on those, we got inspired by the feedback sessions and realised that it made sense to give our community the possibility to contribute to the project with some real life examples of frictionless data use. the user selection process has started and we hope to get the new tutorials online by the end of the month, so stay tuned!

what’s next?

our commitment to continually improving our documentation is not over with this project coming to an end! do you have suggestions for changes you would like to see in our documentation? please reach out to us or open a pull request to contribute. everyone is welcome to contribute! learn how to do it here.

thanks, thanks, thanks!

once again, we are very grateful to the open data institute for giving us the chance to focus on this documentation in order to improve it. we cannot thank enough all our users who took part in the feedback sessions. your contributions were precious.

more about frictionless data

frictionless data is a set of specifications for data and metadata interoperability, accompanied by a collection of software libraries that implement these specifications, and a range of best practices for data management. the project is funded by the sloan foundation.

- - t : : + : sara petti david rosenthal: cryptocurrency's carbon footprint https://blog.dshr.org/ / /cryptocurrencys-carbon-footprint.html china’s bitcoin mines could derail carbon neutrality goals, study says and bitcoin mining emissions in china will hit million tonnes by , the headlines say it all. excusing this climate-destroying externality of proof-of-work blockchains requires a continuous flow of new misleading arguments. below the fold i discuss one of the more recent novelties.

in bitcoin and ethereum carbon footprints – part , moritz seibert claims the reason for mining is to get the mining reward:

bitcoin transactions themselves don’t cause a lot of power usage. getting the network to accept a transaction consumes almost no power, but having asic miners grind through the mathematical ether to solve valid blocks does. miners are incentivized to do this because they are compensated for it. presently, that compensation includes a block reward which is paid in bitcoin ( . btc per block) as well as a miner fee (transaction fee). transaction fees are denominated in fractional bitcoins and paid by the initiator of the transaction. today, about % of total miners’ rewards are transactions fees, and about % are block rewards.

so, he argues, bitcoin's current catastrophic carbon footprint doesn't matter because, as the reward decreases, so will the carbon footprint:

this also means that the power usage of the bitcoin network won’t scale linearly with the number of transactions as the network becomes predominantly fee-based and less rewards-based (which causes a lot of power to the thrown at it in light of increasing btc prices), and especially if those transactions take place on secondary layers. in other words, taking the ratio of “bitcoin’s total power usage” to “number of transactions” to calculate the “power cost per transaction” falsely implies that all transactions hit the final settlement layer (they don’t) and disregards the fact that the final state of the bitcoin base layer is a fee-based state which requires a very small fraction of bitcoin’s overall power usage today (no more block rewards).

seibert has some vague idea that there are implications of this not just for the carbon footprint but also for the security of the bitcoin blockchain:

going forward however, miners’ primary revenue source will change from block rewards to the fees paid for the processing of transactions, which don’t per se cause high carbon emissions. bitcoin is set to become be a purely fee-based system (which may pose a risk to the security of the system itself if the overall hash rate declines, but that’s a topic for another article because a blockchain that is fully reliant on fees requires that btcs are transacted with rather than held in michael saylor-style as hodling leads to low btc velocity, which does not contribute to security in a setup where fees are the only rewards for miners.)

lets leave aside the stunning irresponsibility of arguing that it is acceptable to dump huge amounts of long-lasting greenhouse gas into the atmosphere now because you believe that in the future you will dump less. how realistic is the idea that decreasing the mining reward will decrease the carbon footprint?

the graph shows the history of the hash rate, which is a proxy for the carbon footprint. you can see the effect of the "halvening", when on may ^th the mining reward halved. there was a temporary drop, but the hash rate resumed its inexorable rise. this experiment shows that reducing the mining reward doesn't reduce the carbon footprint. so why does seibert think that eliminating it will reduce the carbon footprint?

the answer appears to be that seibert thinks the purpose of mining is to create new bitcoins, that the reason for the vast expenditure of energy is to make the process of creating new coins secure, and that it has nothing to do with the security of transactions. this completely misunderstands the technology.

in the economic limits of bitcoin and the blockchain, eric budish examines the return on investment in two kinds of attacks on a blockchain like bitcoin's. the simpler one is a % attack, in which an attacker controls the majority of the mining power. budish explains what this allows the attacker to do:

an attacker could (i) spend bitcoins, i.e., engage in a transaction in which he sends his bitcoins to some merchant in exchange for goods or assets; then (ii) allow that transaction to be added to the public blockchain (i.e., the longest chain); and then subsequently (iii) remove that transaction from the public blockchain, by building an alternative longest chain, which he can do with certainty given his majority of computing power. the merchant, upon seeing the transaction added to the public blockchain in (ii), gives the attacker goods or assets in exchange for the bitcoins, perhaps after an escrow period. but, when the attacker removes the transaction from the public blockchain in (iii), the merchant effectively loses his bitcoins, allowing the attacker to “double spend” the coins elsewhere.

such attacks are endemic among the smaller alt-coins; for example there were three successful attacks on ethereum classic in a single month last year. clearly, seibert's future "transaction only" bitcoin must defend against them.

there are two ways to mount a % attack, from the outside or from the inside. an outside attack requires more mining power than the insiders are using, whereas an insider attack only needs a majority of the mining power to conspire. bitcoin miners collaborate in "mining pools" to reduce volatility of their income, and for many years it would have taken only three or so pools to conspire for a successful attack. but assuming insiders are honest, outsiders must acquire more mining power than the insiders are using. clearly, bitcoin insiders are using so much mining power that this isn't feasible.

the point of mining isn't to create new bitcoins. mining is needed to make the process of adding a block to the chain, and thus adding a set of transactions to the chain, so expensive that it isn't worth it for an attacker to subvert the process. the cost, and thus in the case of proof of work the carbon footprint, is the whole point. as budish wrote:

from a computer security perspective, the key thing to note ... is that the security of the blockchain is linear in the amount of expenditure on mining power, ... in contrast, in many other contexts investments in computer security yield convex returns (e.g., traditional uses of cryptography) — analogously to how a lock on a door increases the security of a house by more than the cost of the lock.

lets consider the possible futures of a fee-based bitcoin blockchain. it turns out that currently fee revenue is a smaller proportion of total miner revenue than seibert claims. here is the chart of total revenue (~$ m/day):

and here is the chart of fee revenue (~$ m/day):

thus the split is about % fee, % reward:

if security stays the same, blocksize stays the same, fees must increase to keep the cost of a % attack high enough.

the chart shows the average fee hovering around $ , so the average cost of a single transaction would be over $ . this might be a problem for seibert's requirement that "btcs are transacted with rather than held".
if blocksize stays the same, fees stay the same, security must decrease because the fees cannot cover the cost of enough hash power to deter a % attack. similarly, in this case it would be times cheaper to mount a % attack, which would greatly increase the risk of delivering anything in return for bitcoin. it is already the case that users are advised to wait blocks (about an hour) before treating a transaction as final. waiting nearly half a day before finality would probably be a disincentive.
if fees stay the same, security stays the same, blocksize must increase to allow for enough transactions so that their fees cover the cost of enough hash power to deter a % attack. since bitcoin blocks have been effectively limited to around mb, and the blockchain is now over one-third of a terabyte growing at over %/yr. increasing the size limit to say mb would solve the long-term problem of a fee-based system at the cost of reducing miners income in the short term by reducing the scarcity value of a slot in a block. doubling the effective size of the block caused a huge controversy in the bitcoin community for precisely this short vs. long conflict, so a much larger increase would be even more controversial. not to mention that the size of the blockchain a year from now would be times bigger imposing additional storage costs on miners.

that is just the supply side. on the demand side it is an open question as to whether there would be times the current demand for transactions costing $ and taking an hour which, at least in the us, must each be reported to the tax authorities.

short vs. long

none of these alternatives look attractive. but there's also a second type of attack in budish's analysis, which he calls "sabotage". he quotes rosenfeld:

in this section we will assume q < p [i.e., that the attacker does not have a majority]. otherwise, all bets are off with the current bitcoin protocol ... the honest miners, who no longer receive any rewards, would quit due to lack of incentive; this will make it even easier for the attacker to maintain his dominance. this will cause either the collapse of bitcoin or a move to a modified protocol. as such, this attack is best seen as an attempt to destroy bitcoin, motivated not by the desire to obtain bitcoin value, but rather wishing to maintain entrenched economical systems or obtain speculative profits from holding a short position.

short interest in bitcoin is currently small relative to the total stock, but much larger relative to the circulating supply. budish analyzes various sabotage attack cases, with a parameter ∆_attack representing the proportion of the bitcoin value destroyed by the attack:

for example, if ∆_attack = , i.e., if the attack causes a total collapse of the value of bitcoin, the attacker loses exactly as much in bitcoin value as he gains from double spending; in effect, there is no chance to “double” spend after all. ... however, ∆_attack is something of a “pick your poison” parameter. if ∆_attack is small, then the system is vulnerable to the double-spending attack ... and the implicit transactions tax on economic activity using the blockchain has to be high. if ∆_attack is large, then a short time period of access to a large amount of computing power can sabotage the blockchain.

the current cryptocurrency bubble ensures that everyone is making enough paper profits from the golden eggs to deter them from killing the goose that lays them. but it is easy to create scenarios in which a rush for the exits might make killing the goose seem like the best way out.

seibert's misunderstanding illustrates the fundamental problem with permissionless blockchains. as i wrote in a note on blockchains:

if joining the replica set of a permissionless blockchain is free, it will be vulnerable to sybil attacks, in which an attacker creates many apparently independent replicas which are actually under his sole control. if creating and maintaining a replica is free, anyone can authorize any change they choose simply by creating enough sybil replicas.

defending against sybil attacks requires that membership in a replica set be expensive.

there are many attempts to provide less environmentally damaging ways to make adding a block to a blockchain expensive, but attempts to make adding a block cheaper are self-defeating because they make the blockchain less secure.

there are two reasons why the primary use of a permissionless blockchain cannot be transactions as opposed to hodl-ing:

the lack of synchronization between the peers means that transactions must necessarily be slow.
the need to defend against sybil attacks means either that transactions must necessarily be expensive, or that blocks must be impractically large.

- - t : : + : david. (noreply@blogger.com) islandora: islandora open meeting: april , https://islandora.ca/content/islandora-open-meeting-april- - islandora open meeting: april , agriffith tue, / / - :

body

we are happy to announce the date of our next open meeting! join us on april , any time between : - : pm edt. the open meetings are drop-in style sessions where users of all levels and abilities gather to ask questions, share use cases and get updates on islandora. there will be experienced islandora users on hand to answer questions or give demos. we would love for your to join us any time during the -hour window, so feel free to pop by any time!

more details about the open meeting, and the zoom link to join, are in this google doc.

registration is not required. if you would like a calendar invite as a reminder, please let us know at community@islandora.ca.

- - t : : + : agriffith digital library federation: call for proposals open for ndsa digital preservation ! https://www.diglib.org/call-for-proposals-open-for-ndsa-digital-preservation- /

the ndsa is very pleased to announce the call for proposals is open for digital preservation : embracing digitality (#digipres ) to be held online this year on november th, during world digital preservation day.

submissions from members and nonmembers alike are welcome, and you can learn more about session format options through the cfp. the deadline to submit proposals is monday, may , at : pm eastern time.

digital preservation (#digipres ) is held in partnership with our host organization, the council on library and information resources’ (clir) digital library federation. separate calls are being issued for clir+dlf’s events, the dlf forum (november - ) and associated workshop series learn@dlf (november - ). ndsa strives to create a safe, accessible, welcoming, and inclusive event, and adheres to dlf’s code of conduct.

we look forward to seeing you online on november th,

~ digipres planning committee

the post call for proposals open for ndsa digital preservation ! appeared first on dlf.

- - t : : + : kussmann hangingtogether: dutch round table on next generation metadata: think bigger than naco and worldcat http://feedproxy.google.com/~r/hangingtogetherorg/~ /jk gsfc ez /

as part of the oclc research discussion series on next generation metadata, this blog post reports back from the dutch language round table discussion held on march , . (a dutch translation is available here).

librarians – with backgrounds in metadata, library systems, reference work, national bibliography, and back-office processes – joined the session, representing a nice mix of academic and heritage institutions from the netherlands and belgium. the participants were engaged, candid, and thoughtful and this stimulated constructive knowledge exchange in a pleasant atmosphere.

mapping exercise

as in all the other round table discussions, participants started with taking stock of next generation metadata projects in their region or initiatives they were aware of elsewhere. the resulting map shows a strong representation of bibliographic and cultural heritage data-projects (see upper- and lower-left quadrants of the matrix). several next-generation metadata research projects of the national library of the netherlands were listed and described, such as:

automatic metadata generation, which identifies and tests tools to support subject tagging and cataloging of name authority records;
the entity finder, a tool being developed to help extract rda entities (persons, works, expressions) from both authority and bibliographic records.

the digital heritage reference architecture (dera) was developed as part of the national strategy for digital heritage in the netherlands. it is a framework for managing and publishing heritage information as linked open data (lod), according to agreed practices and conventions. the van gogh worldwide platform is an exemplar of the application of dera – where metadata, relating to the painter’s art works residing at different dutch heritage institutions and private collectors, have been pulled from source systems by api.

a noteworthy initiative listed in the rim/scholarly communications quadrant of the matrix is the nl-open knowledge base, an initiative in the context of last year’s deal between elsevier and the dutch research institutions, to jointly develop open science services based on their rim systems, elsevier’s databases and analytics solutions and the dutch funding organizations’ databases. the envisaged open knowledge base could potentially feed new applications – for example, a dashboard to monitor the achievement of the universities’ sustainable development goals – and allow to significantly improve the analysis of research impact.

what is keeping us from moving forward?

notwithstanding the state-of-the-art projects mentioned during the mapping exercise, the participants were impatient about the pace of the transition to the next generation of metadata. one participant experienced frustration with having to use multiple tools for a workflow that supports the transition, namely: integration of pids, local authorities, or links to and from external sources. another participant noted that there is still a lot of efficiency to be gained in the value chain:

“when we look at the supply chain, it is absurd to start from scratch because there is already so much data. when a book comes out on the market, it must already have been described. there should not be a need to start from scratch in the library.”

the group also wondered – with so many bibliographic datasets already published as linked open data – what else needs to be done to interconnect them in meaningful ways?

the question of what is keeping us from moving forward dominated the discussion.

trusting external data

one participant suggested that libraries are cautious about the data sources they link up with. authority files are persistent and reliable data sources, which have yet to find their counterparts in the newly emerging linked data ecosystem. the lack of conventions around reliability and persistence might be a reason why libraries are hesitant entering into linked data partnerships or holding back from relying on external data – even from established sources, such as wikidata. after all, linking to a data source is an indication of trust and recognition of data quality.

the conversation moved to data models: which linked data do you create yourself? how will you design it and link it up to other data? some participants found there was still a lack of agreement and clarity about the meaning of key concepts such as a “work”. others pointed out that defining the meaning of concepts used is exactly what linked data is about and this feature allows the co-existence of multiple ontologies – in other words, there is no need any longer to fix semantics in hard standards.

“there is no unique semantic model. when you refer to data that has already been defined by others, you relinquish control over that piece of information, and that can be a mental barrier against doing linked data the proper way. it is much safer to store and manage all the data in your own silo. but the moment you can let go of that, the world can become much richer than you can ever achieve on your own.”

thinking in terms of linked data

the conversation turned to the need to train cataloging staff. one participant thought it would be helpful to get started by learning to think in terms of linked data, to mentally practice building linked data graphs and play with different possible structures, as one does with lego bricks. the group agreed there is still too little understanding of the possibilities and of the consequences of practicing linked data.

“we have to learn to see ourselves as publishers of metadata, so that others can find it – but we have no idea who the others are, we have to think even bigger than the library of congress’s naco or worldcat. we are no longer talking about the records we create, but about pieces of records that are unique, because a lot already comes from elsewhere. we have to wrap our minds around this and ask ourselves: what is our role in the bigger picture? this is very hard to do!”

the group thought it was very important to start having that discussion within the library. but how exactly do you do that? it’s a big topic and it must be initiated by the library’s leadership team.

not relevant for my library

one university library leader in the group reacted to this and said:

“what strikes me is that the number of libraries faced with this challenge is shrinking. (…) [in my library] we hardly produce any metadata anymore. (…) if we look at what we still produce ourselves, it is about describing photos of student fraternities (…). it’s almost nothing anymore. metadata has really become a topic for a small group of specialists.”

the group objected that this observation was overlooking the importance of the discovery needs of the communities libraries serve. however provocative this observation was, it reflects a reality that we need to acknowledge and at the same time put in perspective. alas, there was no time for that, as the session was wrapping up. it had certainly been a conversation to be continued!

about the oclc research discussion series on next generation metadata

in march , oclc research conducted a discussion series focused on two reports:

the round table discussions were held in different european languages and participants were able share their own experiences, get a better understanding of the topic area, and gain confidence in planning ahead.

the opening plenary session opened the forum for discussion and exploration and introduced the theme and its topics. summaries of all eight round table discussions are published on the oclc research blog, hanging together. this is the last post and it is preceded by the posts reporting on the first english session, the italian session, the second english session, the french session, the german session, the spanish session and the third english session.

the closing plenary session on april will synthesize the different round table discussions. registration is still open for this webinar: please join us!

the post dutch round table on next generation metadata: think bigger than naco and worldcat appeared first on hanging together.

- - t : : + : titia van der werf digital library federation: amia cross-pollinator: justine thomas https://www.diglib.org/ -amia-cross-pollinator-justine-thomas/

the association of moving image archivists (amia) and dlf will be sending justine thomas to attend the virtual dlf/amia hack day and amia spring conference! as this year’s “cross-pollinator,” justine will enrich both the hack day event and the amia conference, sharing a vision of the library world from her perspective.

about the awardee

justine thomas (@justinethomasm) is currently a digital programs contractor at the national museum of american history (nmah) focusing on digital asset management and collections information support. prior to graduating in with a master’s in museum studies from the george washington university, justine worked at nmah as a collections processing intern in the archives center and as a public programs facilitator encouraging visitors to discuss american democracy and social justice issues.

about hack day and the award

the seventh amia+dlf hack day (online april - ) will be a unique opportunity for practitioners and managers of digital audiovisual collections to join with developers and engineers to remotely collaborate to develop solutions for digital audiovisual preservation and access.

the goal of the amia + dlf award is to bring “cross-pollinators”–developers and software engineers who can provide unique perspectives to moving image and sound archivists’ work with digital materials, share a vision of the library world from their perspective, and enrich the hack day event–to the conference.

find out more about this year’s hack day activities here.

the post amia cross-pollinator: justine thomas appeared first on dlf.

- - t : : + : gayle evergreen ils: evergreen . -rc available https://evergreen-ils.org/evergreen- - -rc-available/

the evergreen community is pleased to announce the availability of the release candidate for evergreen . . this release follows up on the recent beta release. the general release of . . is planned for wednesday, april . between now and then, please download the release candidate and try it out.

additional information, including a full list of new features, can be found in the release notes.

- - t : : + : galen charlton jez cope: intro to the fediverse https://erambler.co.uk/blog/intro-to-the-fediverse/

wow, it turns out to be years since i wrote this beginners guide to twitter. things have moved on a loooooong way since then.

far from being the interesting, disruptive technology it was back then, twitter has become part of the mainstream, the establishment. almost everyone and everything is on twitter now, which has both pros and cons.

so what’s the problem?

it’s now possible to follow all sorts of useful information feeds, from live updates on transport delays to your favourite sports team’s play-by-play performance to an almost infinite number of cat pictures. in my professional life it’s almost guaranteed that anyone i meet will be on twitter, meaning that i can contact them to follow up at a later date without having to exchange contact details (and they have options to block me if they don’t like that).

on the other hand, a medium where everyone’s opinion is equally valid regardless of knowledge or life experience has turned some parts of the internet into a toxic swamp of hatred and vitriol. it’s easier than ever to forget that we have more common ground with any random stranger than we have similarities, and that’s led to some truly awful acts and a poisonous political arena.

part of the problem here is that each of the social media platforms is controlled by a single entity with almost no accountability to anyone other than shareholders. technological change has been so rapid that the regulatory regime has no idea how to handle them, leaving them largely free to operate how they want. this has led to a whole heap of nasty consequences that many other people have done a much better job of documenting than i could (shoshana zuboff’s book the age of surveillance capitalism is a good example). what i’m going to focus on instead are some possible alternatives.

if you accept the above argument, one obvious solution is to break up the effective monopoly enjoyed by facebook, twitter et al. we need to be able to retain the wonderful affordances of social media but democratise control of it, so that it can never be dominated by a small number of overly powerful players.

what’s the solution?

there’s actually a thing that already exists, that almost everyone is familiar with and that already works like this.

it’s email.

there are a hundred thousand email servers, but my email can always find your inbox if i know your address because that address identifies both you and the email service you use, and they communicate using the same protocol, simple mail transfer protocol (smtp). i can’t send a message to your twitter from my facebook though, because they’re completely incompatible, like oil and water. facebook has no idea how to talk to twitter and vice versa (and the companies that control them have zero interest in such interoperability anyway).

just like email, a federated social media service like mastodon allows you to use any compatible server, or even run your own, and follow accounts on your home server or anywhere else, even servers running different software as long as they use the same activitypub protocol.

there’s no lock-in because you can move to another server any time you like, and interact with all the same people from your new home, just like changing your email address. smaller servers mean that no one server ends up with enough power to take over and control everything, as the social media giants do with their own platforms. but at the same time, a small server with a small moderator team can enforce local policy much more easily and block accounts or whole servers that host trolls, nazis or other poisonous people.

how do i try it?

i have no problem with anyone for choosing to continue to use what we’re already calling “traditional” social media; frankly, facebook and twitter are still useful for me to keep in touch with a lot of my friends. however, i do think it’s useful to know some of the alternatives if only to make a more informed decision to stick with your current choices. most of these services only ask for an email address when you sign up and use of your real name vs a pseudonym is entirely optional so there’s not really any risk in signing up and giving one a try. that said, make sure you take sensible precautions like not reusing a password from another account.

instead of…	try…
twitter, facebook	mastodon, pleroma, misskey
slack, discord, irc	matrix
whatsapp, fb messenger, telegram	also matrix
instagram, flickr	pixelfed
youtube	peertube
the web	interplanetary file system (ipfs)

which, if you can believe it, was formalised nearly years ago in and has only had fairly minor changes since then! ↩︎

- - t : : + : hangingtogether: third english round table on next generation metadata: investing in the utility of authorities and identifiers http://feedproxy.google.com/~r/hangingtogetherorg/~ /angp bt hu/

thanks to george bingham, uk account manager at oclc, for contributing this post as part of the metadata series blog posts.

as part of the oclc research discussion series on next generation metadata, this blog post reports back from the third english language round table discussion held on march , .  the session was scheduled to facilitate a uk-centric discussion with a panel of library representatives from the uk with backgrounds in bibliographic control, special collections, collections management, metadata standards and computer science – a diverse and engaged discussion group.

mapping exercise

as with other round table sessions, the group started with mapping next generation metadata projects that participants were aware of, on a × matrix characterizing the application area: bibliographic data, cultural heritage data, research information management (rim) data, and for anything else, the category, “other”. the resulting map gave a nice overview of some of the building blocks of the emerging next generation metadata infrastructure, focussing in this session on the various national and international identifier initiatives – isni, viaf, fast, lc/naco authority file and lc/saco subject lists, and orcid – and metadata and linked data infrastructure projects such as plan-m (an initiative, facilitated by jisc, to rethink the way that metadata for academic and specialist libraries is created, sold, licensed, shared, and re-used in the uk), bibframe and oclc’s shared entity management infrastructure.

the map also raises interesting questions about some of the potential or actual obstacles to the spread of next generation metadata:

what to do about missing identifiers? how to incorporate extant regional databases and union catalogs into the national and international landscape? how “open” are institutions’ local archive management systems? who is willing to pay for linked data?

contributing to library of congress authorities

the discussion panel agreed that there is a pressing need for metadata to be less hierarchical, which linked data delivers, and that a collaborative approach is the best way forward. one example is the development of the uk funnel for naco and saco, which has reinforced the need for a more national approach in the uk. the funnel allows the uk higher education institutions to contribute to the lc name and subject authorities using a single channel – rather than each library setting up its own channel. because they work together as a group to make their contributions to the authority files, the quality and the “authority” of their contributions is significantly increased.

registering and seeding isnis

one panelist reported on a one-year trial with isni for the institution’s legal deposit library, as a first step into working with linked data. it is hoped that it will prove to be a sustainable way forward. there is considerable enthusiasm and interest for this project amongst the institution’s practitioners, a vital ingredient for a successful next generation metadata initiative.

another panelist expanded on several ongoing projects with the aim of embedding isni identifiers within the value chain and getting them out to where cataloguers can pick them up. for example, publishers are starting to use them in their onix feeds to enable them to create clusters of records. also, cataloging agencies in the uk are being supplied with isni identifiers so that they can embed them in the metadata at source, in the cataloging-in-publication (cip) metadata, that they supply to libraries in the uk.

efforts are also under way to systematically match isni entries against viaf entries, and to provide a reconciliation file to enable oclc to update the viaf with the most recent isni. these could then be fed through to the library of congress, who can then use these to update naco files.

with million files to update, this is a perfect example of a leading edge dynamic next generation metadata initiative that will have to overcome the considerable challenge of scalability for it to succeed at a global level.

challenges faced by identifiers

the discussion moved on to the other challenges faced by identifier schemes. it was noted that encouraging a more widespread collaborative approach would rely on honesty amongst the contributors. there would need to be built in assurances that the tags/data come from a trusted source. would the more collaborative approach introduce too much scope for duplicate identifiers being created, and too many variations on preferred names? cultural expectations would have to be clearly defined and adhered to. and last but by no means least is the challenge of providing the resources needed to upscale to a national and international scope.

obstacles in moving towards next generation metadata

participants raised concerns that library management systems are not keeping pace with current discussions on next generation metadata or with real world implementations, to the extent that they may be the biggest obstacle in the move towards next generation metadata. it was recognized that moving to linked data involves a big conceptual and technical leap from the current string-based metadata creation, sharing and management practices, tools and methodologies.

progress can only be made in small steps, and there is still much work to be done to demonstrate the benefits of next generation metadata, a prerequisite if we are to complete the essential step of gaining the support of senior management and buy-in from system suppliers.

if we don’t lead, will someone else take over?

towards the end of the session, a brief discussion arose around the possibility (and danger) of organizations outside the library sector “taking over” if we can’t manage the transition ourselves. amazon was cited as already becoming regarded as a good model to follow for metadata standards, despite what we know to be its shortcomings: it does not promote high quality data, and there are numerous problems concealed within the data, that are not evident to non-professionals. these quality issues would become very problematic if they are allowed to become pervasive in the global metadata landscape.

“our insistence on ‘perfect data’ is a good thing, but are people just giving up on it because it’s too difficult to attain?”

about the oclc research discussion series on next generation metadata

in march , oclc research conducted a discussion series focused on two reports:

the opening plenary session opened the forum for discussion and exploration and introduced the theme and its topics. summaries of all eight round table discussions are published on the oclc research blog, hanging together. this post is preceded by the posts reporting on the first english session, the italian session, the second english session, the french session, the german session, and the spanish session.

the closing plenary session on april will synthesize the different round table discussions. registration is still open for this webinar: please join us!

the post third english round table on next generation metadata: investing in the utility of authorities and identifiers appeared first on hanging together.

- - t : : + : titia van der werf peter murray: more thoughts on pre-recording conference talks https://dltj.org/article/pre-recording-conference-talks-redux/

over the weekend, i posted an article here about pre-recording conference talks and sent a tweet about the idea on monday. i hoped to generate discussion about recording talks to fill in gaps—positive and negative—about the concept, and i was not disappointed. i’m particularly thankful to lisa janicke hinchliffe and andromeda yelton along with jason griffey, junior tidal, and edward lim junhao for generously sharing their thoughts. daniel s and kate deibel also commented on the code lib slack team. i added to the previous article’s bullet points and am expanding on some of the issues here. i’m inviting everyone mentioned to let me know if i’m mischaracterizing their thoughts, and i will correct this post if i hear from them. (i haven’t found a good comments system to hook into this static site blog.)

pre-recorded talks limit presentation format

lisa janicke hinchliffe made this point early in the feedback:

@datag for me downside is it forces every session into being a lecture. for two decades cfps have emphasized how will this season be engaging/not just a talking head? i was required to turn workshops into talks this year. even tho tech can do more. not at all best pedagogy for learning
— lisa janicke hinchliffe (@lisalibrarian) april ,

jason described the “flipped classroom” model that he had in mind as the nisoplus program was being developed. the flipped classroom model is one where students do the work of reading material and watching lectures, then come to the interactive time with the instructors ready with questions and comments about the material. rather than the instructor lecturing during class time, the class time becomes a discussion about the material. for nisoplus, “the recording is the material the speaker and attendees are discussing” during the live zoom meetings.

in the previous post, i described how the speaker could respond in text chat while the recording replay is beneficial. lisa went on to say:

@datag q+a is useful but isn't an interactive session. to me, interactive = participants are co-creating the session, not watching then commenting on it.
— lisa janicke hinchliffe (@lisalibrarian) april ,

she described an example: the ssp preconference she ran at chs. i’m paraphrasing her tweets in this paragraph. the preconference had a short keynote and an “oprah-style” panel discussion (not pre-prepared talks). this was done live; nothing was recorded. after the panel, people worked in small groups using zoom and a set of google slides to guide the group work. the small groups reported their discussions back to all participants.

andromeda points out (paraphrasing twitter-speak): “presenters will need much more— and more specialized—skills to pull it off, and it takes a lot more work.” and lisa adds: “just so there is no confusion … i don’t think being online makes it harder to do interactive. it’s the pre-recording. interactive means participants co-create the session. a pause to chat isn’t going to shape what comes next on the recording.”

increased technical burden on speakers and organizers

@thatandromeda @datag totally agree on this. i had to pre-record a conference presentation recently and it was a terrible experience, logistically. i feel like it forces presenters to become video/sound editors, which is obviously another thing to worry about on top of content and accessibility.
— junior tidal (@juniortidal) april ,

andromeda also agreed with this: “i will say one of the things i appreciated about niso is that @griffey did all the video editing, so i was not forced to learn how that works.” she continued, “everyone has different requirements for prerecording, and in [code lib’s] case they were extensive and kept changing.” and later added: “part of the challenge is that every conference has its own tech stack/requirements. if as a presenter i have to learn that for every conference, it’s not reducing my workload.”

it is hard not to agree with this; a high-quality (stylistically and technically) recording is not easy to do with today’s tools. this is also a technical burden for meeting organizers. the presenters will put a lot of work into talks—including making sure the recordings look good; whatever playback mechanism is used has to honor the fidelity of that recording. for instance, presenters who have gone through the effort to ensure the accessibility of the presentation color scheme want the conference platform to display the talk “as i created it.”

the previous post noted that recorded talks also allow for the creation of better, non-real-time transcriptions. lisa points out that presenters will want to review that transcription for accuracy, which jason noted adds to the length of time needed before the start of a conference to complete the preparations.

increased logistical burden on presenters

@thatandromeda @datag @griffey even if prep is no more than the time it would take to deliver live (which has yet to be case for me and i'm good at this stuff), it is still double the time if you are expected to also show up live to watch along with everyone else.
— lisa janicke hinchliffe (@lisalibrarian) april ,

this is a consideration i hadn’t thought through—that presenters have to devote more clock time to the presentation because first they have to record it and then they have to watch it. (or, as andromeda added, “significantly more than twice the time for some people, if they are recording a bunch in order to get it right and/or doing editing.”)

no. audience. reaction.

@datag @griffey ) no. audience. reaction. i give a joke and no one laughs. was it funny? was it not funny? talks are a *performance* and a *relationship*; i'm getting energy off the audience, i'm switching stuff on the fly to meet their vibe. prerecorded/webinar is dead. feels like i'm bombing.
— andromeda yelton (@thatandromeda) april ,

wow, yes. i imagine it would take a bit of imagination to get in the right mindset to give a talk to a small camera instead of an audience. i wonder how stand-up comedians are dealing with this as they try to put on virtual shows. andromeda summed this up:

@datag @griffey oh and i mean ) i don't get tenure or anything for speaking at conferences and goodness knows i don't get paid. so the entire benefit to me is that i enjoy doing the talk and connect to people around it. prerecorded talk + f f conf removes one of these; online removes both.
— andromeda yelton (@thatandromeda) april ,

also in this heading could be “no speaker reaction”—or the inability for subsequent speakers at a conference to build on something that someone said earlier. in the code lib slack team, daniel s noted: “one thing comes to mind on the pre-recording [is] the issue that prerecorded talks lose the ‘conversation’ aspect where some later talks at a conference will address or comment on earlier talks.” kate deibel added: “exactly. talks don’t get to spontaneously build off of each other or from other conversations that happen at the conference.”

currency of information

lisa points out that pre-recording talks before en event means there is a delay between the recording and the playback. in the example she pointed out, there was a talk at rluk that pre-recorded would have been about the university of california working on an open access deal with elsevier; live, it was able to be “the deal we announced earlier this week”.

conclusions?

near the end of the discussion, lisa added:

@datag @griffey @thatandromeda i also recommend going forward that the details re what is required of presenters be in the cfp. it was one thing for conferences that pivoted (huge effort!) but if you write the cfp since the pivot it should say if pre-record, platform used, etc.
— lisa janicke hinchliffe (@lisalibrarian) april ,

…and andromeda added: “strong agree here. i understand that this year everyone was making it up as they went along, but going forward it’d be great to know that in advance.”

that means conferences will need to take these needs into account well before the call for proposals (cfp) is published. a conference that is thinking now about pre-recording their talks must work through these issues and set expectations with presenters early.

as i hoped, the twiter replies tempered my eagerness for the all-recorded style with some real-world experience. there could be possibilities here, but adapting face-to-face meetings to a world with less travel won’t be simple and will take significant thought beyond the issues of technology platforms.

edward lim junhao summarized this nicely: “i favor unpacking what makes up our prof conferences. i’m interested in recreating that shared experience, the networking, & the serendipity of learning sth you didn’t know. i feel in-person conferences now have to offer more in order to justify people traveling to attend them.”

related, andromeda said: “also, for a conf that ultimately puts its talks online, it’s critical that it have something beyond content delivery during the actual conference to make it worth registering rather than just waiting for youtube. realtime interaction with the speaker is a pretty solid option.”

if you have something to add, reach out to me on twitter. given enough responses, i’ll create another summary. let’s keep talking about what that looks like and sharing discoveries with each other.

the tree of tweets

it was a great discussion, and i think i pulled in the major ideas in the summary above. with some guidance from ed summers, i’m going to embed the twitter threads below using treeverse by paul butler. we might be stretching the boundaries of what is possible, so no guarantees that this will be viewable for the long term.

- - t : : + : peter murray (jester@dltj.org) peter murray: should all conference talks be pre-recorded? https://dltj.org/article/pre-recording-conference-talks/

the code lib conference was last week. that meeting used all pre-recorded talks, and we saw the benefits of pre-recording for attendees, presenters, and conference organizers. should all talks be pre-recorded, even when we are back face-to-face?

note! after i posted a link to this article on twitter, there was a great response of thoughtful comments. i've included new bullet points below and summarized the responses in another blog post.

as an entirely virtual conference, i think we can call code lib a success. success ≠ perfect, of course, and last week the conference coordinating team got together on a zoom call for a debriefing session. we had a lengthy discussion about what we learned and what we wanted to take forward to the conference, which we’re anticipating will be something with a face-to-face component.

that last sentence was tough to compose: “…will be face-to-face”? “…will be both face-to-face and virtual”? (or another fully virtual event?) truth be told, i don’t think we know yet. i think we know with some certainty that the covid pandemic will become much more manageable by this time next year—at least in north america and europe. (code lib draws from primarily north american library technologists with a few guests from other parts of the world.) i’m hearing from higher education institutions, though, that travel is going to be severely curtailed…if not for health risk reasons, then because budgets have been slashed. so one has to wonder what a conference will look like next year.

i’ve been to two online conferences this year: nisoplus and code lib. both meetings recorded talks in advance and started playback of the recordings at a fixed point in time. this was beneficial for a couple of reasons. for organizers and presenters, pre-recording allowed technical glitches to be worked through without the pressure of a live event happening. technology is not nearly perfect enough or ubiquitously spread to count on it working in real-time. nisoplus also used the recordings to get transcribed text for the videos. (code lib used live transcriptions on the synchronous playback.) attendees and presenters benefited from pre-recording because the presenters could be in the text chat channel to answer questions and provide insights. having the presenter free during the playback offers new possibilities for making talks more engaging: responding in real-time to polls, getting forehand knowledge of topics for subsequent real-time question/answer sessions, and so forth. the synchronous playback time meant that there was a point when (almost) everyone was together watching the same talk—just as in face-to-face sessions.

during the code lib conference coordinating debrief call, i asked the question: “if we saw so many benefits to pre-recording talks, do we want to pre-record them all next year?” in addition to the reasons above, pre-recorded talks benefit those who are not comfortable speaking english or are first-time presenters. (they have a chance to re-do their talk as many times as they need in a much less stressful environment.) “live” demos are much smoother because a recording can be restarted if something goes wrong. each year, at least one presenter needs to use their own machine (custom software, local development environment, etc.), and swapping out presenter computers in real-time is risky. and it is undoubtedly easier to impose time requirements with recorded sessions. so why not pre-record all of the talks?

i get it—it would be different to sit in a ballroom watching a recording play on big screens at the front of the room while the podium is empty. but is it so different as to dramatically change the experience of watching a speaker at a podium? in many respects, we had a dry-run of this during code lib . it was at the early stages of the coming lockdowns when institutions started barring employee travel, and we had to bring in many presenters remotely. i wrote a blog post describing the setup we used for remote presenters, and at the end, i said:

i had a few people comment that they were taken aback when they realized that there was no one standing at the podium during the presentation.

some attendees, at least, quickly adjusted to this format.

for those with the means and privilege of traveling, there can still be face-to-face discussions in the hall, over meals, and social activities. for those that can’t travel (due to risks of traveling, family/personal responsibilities, or budget cuts), the attendee experience is a little more level—everyone is watching the same playback and in the same text backchannels during the talk. i can imagine a conference tool capable of segmenting chat sessions during the talk playback to “tables” where you and close colleagues can exchange ideas and then promote the best ones to a conference-wide chat room. something like that would be beneficial as attendance grows for events with an online component, and it would be a new form of engagement that isn’t practical now.

there are undoubtedly reasons not to pre-record all session talks (beyond the feels-weird-to-stare-at-an-unoccupied-ballroom-podium reasons). during the debriefing session, one person brought up that having all pre-recorded talks erodes the justification for in-person attendance. i can see a manager saying, “all of the talks are online…just watch it from your desk. even your own presentation is pre-recorded, so there is no need for you to fly to the meeting.” that’s legitimate.

so if you like bullet points, here’s how it lays out. pre-recording all talks is better for:

accessibility: better transcriptions for recorded audio versus real-time transcription (and probably at a lower cost, too)
engagement: the speaker can be in the text chat during playback, and there could be new options for backchannel discussions
better quality: speakers can re-record their talk as many times as needed
closer equality: in-person attendees are having much the same experience during the talk as remote attendees

downsides for pre-recording all talks:

feels weird: yeah, it would be different
erodes justification: indeed a problem, especially for those for whom giving a speech is the only path to getting the networking benefits of face-to-face interaction
limits presentation format: it forces every session into being a lecture. for two decades cfps have emphasized how will this season be engaging/not just a talking head? (lisa janicke hinchliffe)
increased technical burden on speaker and organizers: conference organizers asking presenters to do their own pre-recording is a barrier (junior tidal), and organizers have added new requirements for themselves
no audience feedback: pre-recording forces the presenter into an unnatural state relative to the audience (andromeda yelton)
currency of information: pre-recording talks before en event naturally introduces a delay between the recording and the playback. (lisa janicke hinchliffe)

i’m curious to hear of other reasons, for and against. reach out to me on twitter if you have some. the covid- pandemic has changed our society and will undoubtedly transform it in ways that we can’t even anticipate. is the way that we hold professional conferences one of them?

can we just pause for a moment and consider the decades of work and layers of technology that make a modern teleconference call happen? for you younger folks, there was a time when one couldn’t assume the network to be there. as in: the operating system on your computer couldn’t be counted on to have a network stack built into it. in the earliest years of my career, we were tickled pink to have macintoshes at the forefront of connectivity through gatorboxes. go read the first paragraph of that wikipedia article on gatorboxes…tcp/ip was tunneled through localtalk running over phonenet on unshielded twisted pairs no faster than about kbit/second. (and we loved it!) now the network is expected; needing to know about tcp/ip is pushed so far down the stack as to be forgotten…assumed. sure, the software on top now is buggy and bloated—is my zoom client working? has zoom’s service gone down?—but the network…we take that for granted. ↩

- - t : : + : peter murray (jester@dltj.org) islandora: upcoming dig sprint https://islandora.ca/content/upcoming-dig-sprint upcoming dig sprint agriffith thu, / / - :

body

the islandora documentation interest group is holding a sprint!

to support the upcoming release of islandora, the dig has planned a -week documentation, writing-and-updating sprint to occur as part of the release process. to prepare for that effort, we’re going to spend april – th on an auditing sprint, where volunteers will review existing documentation and complete this spreadsheet, providing a solid overview of the current status of our docs so we know where to best deploy our efforts during the release. this sprint will run alongside the upcoming pre-release code sprint, so if you’re not up for coding, auditing docs is a great way to contribute during sprint season!

we are looking for volunteers to sign up to take on two sprint roles:

auditor: review a page of documentation and fill out a row in the spreadsheet indicating things like the current status (‘good enough’ or ‘needs work’) , the goal for that particular page (e.g., “explain how to create an object,” or “compare islandora concepts to islandora concepts”), and the intended audience (beginners, developers, etc.).

reviewer: read through a page that has been audited and indicate if you agree with the auditor’s assessment, add additional notes or suggestions as needed; basically, give a second set of eyes on each page.

you can sign up for the sprint here, and sign up for individual pages here.

- - t : : + : agriffith samvera: registration now open for samvera virtual connect, april – https://samvera.org/ / / /registration-now-open-for-samvera-virtual-connect/

registration is now open for samvera virtual connect ! samvera virtual connect will take place april th - st from am – pm edt. registration is free and open to anyone with an interest in samvera.

this year’s program is packed with presentations and lightning talks of interest to developers, managers, librarians, and other current or potential samvera community participants and technology users.

the post registration now open for samvera virtual connect, april – appeared first on samvera.

- - t : : + : heather greer klein lucidworks: chatbots for self-resolution and happier customers https://lucidworks.com/post/chatbots-self-resolution/

how chatbots and conversational applications with deep learning are helping customers resolve issues faster than ever.

the post chatbots for self-resolution and happier customers appeared first on lucidworks.

- - t : : + : sommer antrim digital library federation: dlf forum, digipres, and learn@dlf calls for proposals https://www.diglib.org/ -dlf-forum-digipres-and-learndlf-calls-for-proposals/

we’re delighted to share that it’s cfp season for clir’s annual events.

based on community feedback, we’ve made the decision to take our events online again in . we look forward to new and better ways to come together—as always, with community at the center.

our events will take place on the following dates:

the dlf forum (#dlfforum, november - ), our signature event, includes digital library practitioners and others from member institutions and the broader community, for whom it serves as a meeting place, marketplace, and congress. learn more and check out the cfp here: https://forum .diglib.org/call-for-proposals/
ndsa’s digital preservation : embracing digitality (#digipres , november ), ndsa’s major meeting and conference, will help to chart future directions for both the ndsa and digital stewardship, and is expected to be a crucial venue for intellectual exchange, community-building, development of best practices, and national-level agenda-setting in the field. learn more and check out the cfp for this year’s event here: https://ndsa.org/conference/digital-preservation- /cfp/
learn@dlf (#learnatdlf, november - ) is our dedicated workshop series for digging into tools, techniques, workflows, and concepts. through engaging, hands-on sessions, attendees will gain experience with new tools and resources, exchange ideas, and develop and share expertise with fellow community members. learn more and check out the cfp here: https://forum .diglib.org/call-for-proposals/

for all events, we encourage proposals from members and non-members; regulars and newcomers; digital library practitioners and those in adjacent fields such as institutional research and educational technology; and students, early-career professionals and senior staff alike. proposals to more than one event are permitted, though please submit different proposals for each.

the dlf forum and learn@dlf cfp is here: https://forum .diglib.org/call-for-proposals/

ndsa’s digital preservation : embracing digitality cfp is here: https://ndsa.org/conference/digital-preservation- /cfp/

session options range from -minute lighting talks at the forum to half-day workshops at learn@dlf, with many options in between.

the deadline for all opportunities is monday, may , at : pm eastern time.

if you have any questions, please write to us at forum@diglib.org, and be sure to subscribe to our forum newsletter to stay up on all forum-related news. we’re looking forward to seeing you this fall.

-team dlf

the post dlf forum, digipres, and learn@dlf calls for proposals appeared first on dlf.

- - t : : + : gayle peter sefton: what did you do in the lockdowns pt? part - music videos http://ptsefton.com/ / / /lockdowns /index.html

post looks too long? don't want to read? here's the summary. last year gail mcglinn* and i did the lockdown home-recording thing. we put out at least one song video per week for a year (and counting - we're up to over weeks). searchable, sortable website here. we learned some things, got better at performing for the phone camera and our microphones and better at mixing and publishing the result.

* disclosure gail's my wife. we got married; she proposed, i accepted.

i may i might - is this the world's best marriage proposal acceptance song? (it did win a prize at a ukulele festival for best song)

(this post is littered with links to our songs, sorry but there are of them and someone has to link to them.)

in the second quarter of gail mcglinn and i went from playing and singing in community music events (jams, gigs, get togethers) at least once a week to being at home every evening, like everyone else. like lots of people we decided to put our efforts into home recording, not streaming cos that would be pointless for people with basically no audience, but we started making videos and releasing them under our band name team happy.

by release i mean "put on facebook" and "sometimes remember to upload to youtube".

this post is about that experience and what we learned.

team happy is the name we use to perform as a duo at open mic events and the odd community or ukulele festival. we were originally called "the narrownecks" in honour of where we live, for one gig, but then we found out there's another group with that name. actually they're much better than us, just go watch them.

coming in to we already had a youtube channel and it had a grand total of two videos on it with a handful of views - as in you could count them on your fingers. it's still a sad thing to behold, how many views we have - but it's not about views it's about getting discovered and having our songs performed by, oh i dunno, casey chambers? keith urban? (oh yeah, that would mean we'd need views. bugger.) either that or it's about our personal journey and growth as people. or continuing to contribute to our local music communities in lockdown (which is what gail says it's about.). seriously though, we think i called your name and dry pebbles would go well on someone else's album.

dry pebbles, by gail mcglinn - a song written tramping through the bush.

i called your name by peter sefton

anyway, in late march we got out our recording gear and started. while phone cameras are fine for the quality of video we need, we wanted to do better than phone-camera sound. (here's an example of that sound from one of our first recordings on my song seventeen - it's pretty muddy, like the lighting.)

seventeen by peter sefton

initial attempts to get good audio involved feeding usb-audio from a sound mixer with a built in audio interface (a yamaha mx ) into the phone itself and recording an audio track with the video - but this is clunky and you only get two tracks even though the mixer has multiple inputs. we soon graduated to using a daw - a digital audio workstation with our mixer, still only two tracks but much less mucking around with the phone.

so this is more or less what we ended up with for the first few weeks - we'd record or "track" everything on the computer and then use it again to mix.

our first-generation recording rig with annoying recording via a laptop

there's a thing you have to do to audio files called mastering which means getting them to a suitable volume level and dynamic range for distribution. without it loud stuff is too quiet and quiet stuff is too quiet, and the music has no punch. this was a complete mystery to me to start with so i paid for online services that use ai to master tracks - kind of but not really making everything louder. at some point i started doing it myself, beginning the long process of learning the mysteries of compression and limiting and saving money. haven't mastered it yet, though. mastering is an actual profession, by the way and i'm not going to reach those heights.

in may, we got a new bit of gear, the tascam model an all in one mixer-recorder-interface that lets you track (that is record tracks) without a computer - much easier to deal with. a bit later we got a zoom h portable recorder with built in mics and a couple of extra tracks for instruments so we can do stuff away from home - this got used on our month-long holiday in march . well it was almost a month, but there was a rain event and we came home a bit early. these machines let you capture tracks, including adding new ones without touching the computer which is a big win as far as i am concerned.

gail singing closer to fine on the strand in townsville, in north queensland, recorded on the h and (partly) mixed in the car on holidays.

after a bit, and depending on the level of lockdown we'd have guests around to visit and when that was happening, we kept our distance at either end of our long lounge room and used a phone camera and microphone at each end.

our second-generation recording rig with stand-alone laptop-free tracking

this new setup made it much easier to do overdubs - capture more stuff into the model and make videos each time, like on this song of mine they say dancing where i overdubbed guitar and bass over a live track.

they say dancing by peter sefton

so what did we learn?

perfect is the enemy of done. well, we knew that, but if you've decided to release a song every week, even if you're away on a holiday, or there are other things going on then there's no time to obsess over details - you have to get better at getting a useable take quickly or you won't be able to keep going for a year or more.
practice may not make perfect, but it's a better investment than new gear, or doing endless takes with the cameras rolling. we got better at picking a song (or deciding to write one or finish one off), playing it for a week or two and then getting the take.
simplify! we learned that to get a good performance sometimes it was better for only one of us to play or sing, that fancy parts increased the chance of major errors, meaning yet another take. if in doubt (like my harmony singing that's always in doubt) we're learning to leave it out.
nobody likes us! actually we know that's not true, some of the songs get hundreds of plays on facebook but not many people actually click the like button, maybe twenty or so. but then you run into people in the supermarket; they say "love the songs keep it up"! and there are quite a few people who listen every week on fb we just can't tell they're enjoying it. there are complex reasons for this lack of engagement - some people don't like to like things so that (they think) the evil fb can't track them. i think the default auto-play for video might be a factor too - the video starts playing, and that might not be a good time, so people skip forward to something else.

it's kind of demoralizing that it is much easier to get likes with pictures of the dog.

our spoiled covid-hound, floki - about months old. much more likeable on the socials than our music.
youtube definitely doesn't like us. i figured that some of the songs we sang would attract some kind of youtube audience - we often search to see what kinds of covers of songs are out there and thought others might find us the same way, but we get almost no views on that platform. i also thought that adding some text about the gear we used might bring in some views. for example we were pretty early adopters of the tascam model . i had tried to find out what one sounded like in real life before i bought, with no success - and i thought people might drop by to hear us, but i don't think google/youtube is giving us any search-juice at all.

our personal favourites

our favourite cover we did (and we actually agreee on this - team happy is not an ironic name) was colour my world. we'd just got the tascam and gail was able to double track herself - no mucking around with computers. we had fun that night.

colour my world - one of our fave covers to perform

and my favourite original? well i'm very proud of all l'amour for you with lots of words and a bi-lingual pun - i wanted to do that on the local community radio just last weekend when we were asked in, but the host richard 'duck' keegan kind of mentioned the aforementioned i called your name so we did that instead along with dry pebbles and seventeen.

all l'amour for you the last word on love and metaphors for love? by peter sefton.

gail's fave original? i may i might, the song that snagged her the best husband in south katoomba over . m tall. and she likes the tear jerker goodbye mongrel dog i wrote, on which she pays some pumpin' banjo.

goodbye mongrel dog - a song that says goodbye to a (deceased) mongrel dog who went by the name of spensa.

music-tech stuff and mixing tips

for those of you who care, here's a roundup of the main bits of kit that work well. we've reached the point where there's actually nothing on the shopping list - we can do everything for the foreseeable future with what we have.

i have mentioned that we track using the tascam model and the zoom h - these are both great. the only drawback of the zoom is that you can't see the screen (and thus the levels) from performance position. it also needed a better wind shield - i bought a dead-cat, shaggy thing to go over the mics that works if the wind is moderate.

when i bought the tascam i thought it was going to be all analogue through the mixer stage like their model and model , but no, it's all digital. i don't think this is an issue having used it but it was not something they made all that explicit at launch. there's a digital zoom equivalent (the l ) which is a bit smaller, and has more headphone outputs but at the expense of having to do mode-switching to to access all the functions. i think the tascam will be easier to use for live shows when those start happening again.

for video we just use our phones - for a while we had matching pixel xls then a pixel which drowned in a tropical stream. yes they're waterproof, those models, but not when they have tiny cracks in the screen. no more $ phones for me.

reaper is bloody marvelous software. it's cheap for non-commercial use, incredibly powerful and extensible. i have not used any other digital audio workstation other than garage band, that comes for free on the apple platform but as far as i can see there's no reason for non-technophobic home producers to pay any more than the reaper fee for something else.

our mainstay mics are a slightly battered pair of audio technica at s - we had these for performing live with gail's band u ria - everyone gathered around a condenser mic, bluegrass style. for recording we either put one at either end of the room or mount them vertically in an x/y configuration - ° to get stereo. they're fairly airy and have come to be a big part of our sound. we tried some other cheap things that didn't work very well, and i got a pair of australian rode m pencil condenser mics, not expensive, that i hoped might be easier to mount x/y but we didn't like them for vocals at all, though they're great on stringed instruments. we do have an sm and sm -- gotta love a microphone with a wikipedia page -- which see occasional use as vocal mics if we want a more rock 'n roll sound, or the guest singer is more used to a close-mic. and the sm for guitar amps sometimes.

we tend to play our favourite acoustic instruments but when we have bass we use the trace elliot elf amp which has a great compressor and a di output (it can send a signal to the mixer/interface without going via the speaker). sometimes we run the speaker and try not to let it bleed too much into the at s, very occasionally we wear headphones for the first track and go direct so there's no bass bleed. i have done a bit of electric guitar with the boss katana - to me it sounds good in the room that amp, but has not recorded well either via the headphone out or via an sm . i get better results thru the bass amp. i don't have any kind of actual electric guitar tone sorted though i have seen lot of videos about how to achieve the elusive tone. maybe one day.

one thing that i wasn't expecting to happen - i dropped the top e of my little made in mexico martin ooo jr guitar to d (you know, like keef) some time early in and it ended up staying there. gives some nice new chord voicings ( ths mostly) and it's the same top strings as a string banjo with some very easy-to-grab chords. have started doing it to ukuleles too, putting them in open c.

a note on the bass: playing bass is fun (we knew that before we started) but mixing it so it can be heard on a phone speaker is a real challenge. one approach that helps is using an acoustic bass which out of a lot more high frequency than a solid body electric this also helps because you don't have to have an amp on while you're tracking it live, but you can take a direct input from a pickup (or two) and mic the bass giving you lots of signals with different eq to play with. i gaffa-taped a guitar humbucker into my artist guitars string acoustic and it sounds huge.

the basic (ha!) trick i try to use for getting more high frequency for tiny speakers is to create a second track, saturate the signal with distortion and/or saturation effects to boost the upper harmonic content and then cut all the low frequency out and mix that so it can just be heard and imply the fundamental bass frequency in addition to the real bassy bass. helps if you have some bridge pickup or under-saddle pickup in the signal if those are available and if you remember.

i also like to add some phaser effect that gives some motion in the upper frequencies - for example my perfect country pop song - too much phaser? probably, but i can hear the bass on my phone and it bounces :). phaser is team happy's favourite effect, nothing says perfect country pop (which is what we are, right?) like a phaser.

perfect country pop song - is it perfect or merely sublime? (this one has a cute puppy in it).

everything i know about music production is from youtube. everything i know about song writing is from deep in my soul. thank you for reading all the way to the bottom. normal service will resume next week.

- - t : : + : ptsefton lucidworks: let fusion handle search to get the most out of sharepoint https://lucidworks.com/post/lucidworks-fusion-augments-sharepoint-capabilities-for-best-knowledge-management-experience/

augment sharepoint with a flexible search platform to deliver the best knowledge management experience in the market.

the post let fusion handle search to get the most out of sharepoint appeared first on lucidworks.

- - t : : + : jenny gomez jez cope: collaborations workshop : collaborative ideas & hackday https://erambler.co.uk/blog/collabw -part- /

my last post covered the more “traditional” lectures-and-panel-sessions approach of the first half of the ssi collaborations workshop. the rest of the workshop was much more interactive, consisting of a discussion session, a collaborative ideas session, and a whole-day hackathon!

the discussion session on day one had us choose a topic (from a list of topics proposed leading up to the workshop) and join a breakout room for that topic with the aim of producing a “speed blog” by then end of minutes. those speed blogs will be published on the ssi blog over the coming weeks, so i won’t go into that in more detail.

the collaborative ideas session is a way of generating hackday ideas, by putting people together at random into small groups to each raise a topic of interest to them before discussing and coming up with a combined idea for a hackday project. because of the serendipitous nature of the groupings, it’s a really good way of generating new ideas from unexpected combinations of individual interests.

after that, all the ideas from the session, along with a few others proposed by various participants, were pitched as ideas for the hackday and people started to form teams. not every idea pitched gets worked on during the hackday, but in the end teams of roughly equal size formed to spend the third day working together.

my team’s project: “aha! an arts & humanities adventure”

there’s a lot of fomo around choosing which team to join for an event like this: there were so many good ideas and i wanted to work on several of them! in the end i settled on a team developing an escape room concept to help arts & humanities scholars understand the benefits of working with research software engineers for their research.

five of us rapidly mapped out an example storyline for an escape room, got a website set up with github and populated it with the first few stages of the game. we decided to focus on a story that would help the reader get to grips with what an api is and i’m amazed how much we managed to get done in less than a day’s work!

you can try playing through the escape room (so far) yourself on the web, or take a look at the github repository, which contains the source of the website along with a list of outstanding tasks to work on if you’re interested in contributing.

i’m not sure yet whether this project has enough momentum to keep going, but it was a really valuable way both of getting to know and building trust with some new people and demonstrating the concept is worth more work.

other projects

here’s a brief rundown of the other projects worked on by teams on the day.

coding confessions: everyone starts somewhere and everyone cuts corners from time to time. real developers copy and paste! fight imposter syndrome by looking through some of these confessions or contributing your own. https://coding-confessions.github.io/
carpenpi: a template to set up a raspberry pi with everything you need to run a carpentries (https://carpentries.org/) data science/software engineering workshop in a remote location without internet access. https://github.com/carpenpi/docs/wiki
research dugnads: a guide to running an event that is a coming together of a research group or team to share knowledge, pass on skills, tidy and review code, among other software and working best practices (based on the norwegian concept of a dugnad, a form of “voluntary work done together with other people”) https://research-dugnads.github.io/dugnads-hq/
collaborations workshop ideas: a meta-project to collect together pitches and ideas from previous collaborations workshop conferences and hackdays, to analyse patterns and revisit ideas whose time might now have come. https://github.com/robintw/cw-ideas
howdescribedis: integrate existing tools to improve the machine-readable metadata attached to open research projects by integrating projects like somef, codemeta.json and howfairis (https://howfairis.readthedocs.io/en/latest/index.html). complete with ci and badges! https://github.com/knowledgecaptureanddiscovery/somef-github-action
software end-of-project plans: develop a template to plan and communicate what will happen when the fixed-term project funding for your research software ends. will maintenance continue? when will the project sunset? who owns the ip? https://github.com/elichad/software-twilight
habeas corpus: a corpus of machine readable data about software used in covid- related research, based on the cord dataset. https://github.com/softwaresaved/habeas-corpus
credit-all: extend the all-contributors github bot (https://allcontributors.org/) to include rich information about research project contributions such as the casrai contributor roles taxonomy (https://casrai.org/credit/) https://github.com/dokempf/credit-all

i’m excited to see so many metadata-related projects! i plan to take a closer look at what the habeas corpus, credit-all and howdescribedis teams did when i get time. i also really want to try running a dugnad with my team or for the glam data science network.

- - t : : + : journal of web librarianship: meeting a higher standard: a case study of accessibility compliance in libguides upon the adoption of wcag . guidelines https://www.tandfonline.com/doi/full/ . / . . ?ai= dl&mi=co bk&af=r .
- - t : : + : michael chee ed summers: twarc https://inkdroid.org/ / / /twarc /

this post was originally published on medium but i spent time writing it so i wanted to have it here too.

tl;dr twarc has been redesigned from the ground up to work with the new twitter v api and their academic research track. many thanks for the code and design contributions of betsy alpert, igor brigadir, sam hames, jeff sauer, and daniel verdeer that have made twarc possible, as well as early feedback from dan kerchner, shane lin, miles mccain, 李荣蓬, david thiel, melanie walsh and laura wrubel. extra special thanks to the institute for future environments at queensland university of technology for supporting betsy and sam in their work, and for the continued support of the mellon foundation.

back in august of last year twitter announced early access to their new v api, and their plans to sunset the v . api that has been active for almost the last years. over the lifetime of their v . api twitter has become deeply embedded in the media landscape. as magazines, newspapers and television have moved onto the web they have increasingly adopted tweets as a mechanism for citing politicians, celebrities and organizations, while also using them to document current events, generate leads and gather feedback for evolving stories. as a result twitter has also become a popular object of study for humanities and social science researchers looking to understand the world as reflected, refracted and distorted by/in social media.

on the surface the v api update seems pretty insignificant since the shape of a tweet, its parts, properties and affordances, aren’t changing at all. tweets with characters of text, images and video will continue to be posted, retweeted and quoted. however behind the scenes the representation of a tweet as data, and the quotas that control the rates at which this data can flow between apps and other third party services will be greatly transformed.

needless to say, v represents a big change for the documenting the now project. along with community members we’ve developed and maintained open source tools like twarc that talk directly to the twitter api to help users to search for and collect live tweets that match criteria like hashtags, names and geographic locations. today we’re excited to announce the release of twarc v which has been designed from the ground up to work with the v api and twitter’s new academic research track.

clearly it’s extremely problematic having a multi-national corporation act as a gatekeeper for who counts as an academic researcher, and what constitutes academic research. we need look no further than the recent experiences of timnit gebru and margaret mitchell at google for an example of what happens when research questions run up against the business objectives of capital. we only know their stories because gebru and mitchell’s bravely took a principled approach, where many researchers would have knowingly or unknowingly shaped their research to better fit the needs of the company.

so it is important for us that twarc still be usable by people with and without access to the academic research track. but we have heard from many users that the academic research track presents new opportunities for twitter data collection that are essential for researchers interested in the observability of social media platforms. twitter is making a good faith effort to work with the academic research community, and we thought twarc should support it, even if big challenges lie ahead.

so why are people interested in the academic research track? once your application has been approved you are able to collect data from the full history of tweets, at no cost. this is a massive improvement over the v . access which was limited to a one week window and researchers had to pay for access. access to the full archive means it’s now possible to study events that have happened in the past back to the beginning of twitter in . if you do create any historical datasets we’d love for you to share the tweet identifier datasets in the catalog.

however this opening up of access on the one hand comes with a simultaneous contraction in terms of how much data can be collected at one time. the remainder of this post describes some of the details and the design decisions we have made with twarc to address them. if you would prefer to watch a quick introduction to using twarc v please check out this short video:

installation

if you are familiar with installing twarc nothing is changed. you still install (or upgrade) with pip as you did before:

$ pip install --upgrade twarc

in fact you will still have full access to the v . api just as you did before. so the old commands will continue to work as they did

$ twarc search blacklivesmatter > tweets.jsonl

twarc was designed to let you to continue to use twitter’s v . api undisturbed until it is finally turned off by twitter, at which point the functionality will be removed from twarc. all the support for the v api is mediated by a new command line utility twarc . for example to search for blacklivesmatter tweets and write them to a file tweets.jsonl:

$ twarc search blacklivesmatter > tweets.jsonl

all the usual twarc functionality such as searching for tweets, collecting live tweets from the streaming api endpoint, requesting user timelines and user metadata are all still there, twarc --help gives you the details. but while the interface looks the same there’s quite a bit different going on behind the scenes.

representation

truth be told, there is no shortage of open source libraries and tools for interacting with the twitter api. in the past twarc has made a bit of a name for itself by catering to a niche group of users who want a reliable, programmable way to collect the canonical json representation of a tweet. javascript object notation (json) is the language of web apis, and twitter has kept its json representation of a tweet relatively stable over the years. rather than making lots of decisions about the many ways you might want to collect, model and analyze tweets twarc has tried to do one thing and do it well (data collection) and get out of the way so that you can use (or create) the tools for putting this data to use.

but the json representation of a tweet in the twitter v api is completely burst apart. the v base representation of a tweet is extremely lean and minimal, and just includes the text of the tweet its identifier and a handful of other things. all the details about the user who created the tweet, embedded media, and more are not included. fortunately this information is still available, but the user needs to craft their api request to request tweets using a set of expansions that tell the twitter api what additional entities to include. in addition for each expansion there are a set of field options to include that control what of these expansions is returned.

so rather than there being a single json representation of a tweet api users now have the ability to shape the data based on what they need, much like how graphql apis work. this kind of makes you wonder why twitter didn’t make their graphql api available. for specific use cases this customizability is very useful, but the mutability of the representation of a tweet presents challenges when collecting data for future use. if you didn’t request the right expansions or fields when collecting the data then you won’t be able to analyze that data later when doing your research.

to solve for this twarc has been designed to collect the richest possible representation for a tweet, by requesting all possible expansions and field combinations for tweets. see the expansions module for the details if you are interested. this takes a significant burden off of users to digest the api documentation, and craft the correct api requests themselves. in addition the twarc community will be monitoring the twitter api documentation going forward to incorporate new expansions and fields as they will inevitably be added in the future.

flattening

this is diving into the weeds a little bit, but it’s worth noting here that twitter’s introduction of expansions allows data that was once duplicated across multiple tweets (such as user information, media, retweets, etc) to be included once per response from the api. this means that instead of seeing information about the user who created a tweet in the context of their tweet the user will be referenced using an identifier, and this identifier will map to user metadata in the outer envelope of the response.

it makes sense why twitter have introduced expansions since it means in a set of tweets from a given user the user information will just be included once rather than repeated times, which means less data, less network traffic and less money. it’s even more significant when consider the large number of possible expansions. however this pass by-reference rather than by-value presents some challenges for stream based processing which expects each tweet to be self-contained.

for this reason we’ve introduce the idea of flattening the response data when persisting the json to disk. this means that tools and data pipelines that expect to operate on a stream of tweets can continue to do so. since the representation of a tweet is so dependent on how data is requested we’ve taken the opportunity to introduce a small stanza of twarc specific metadata using the __twarc prefix.

this metadata records what api endpoint the data was requested from, and when. this information is critically important when interpreting the data, because some information about a tweet like its retweet and quote counts are constantly changing.

data flows

as mentioned above you can still collect tweets from the search and streaming api endpoints in a way that seems quite similar to the v api. the big changes however are the quotas associated with these endpoints which govern how much can be collected. these quotas control how many requests can be sent to twitter in minute intervals.

in fact these quotas are not much changed, but what’s new are app wide quotas that constrain how many tweets a given application (app) can collect every month. an app in this context is a piece of software (e.g. your twarc software) identified by unique api keys set up in the twitter developer portal. the standard api access sets a , tweet per month limit. this is a huge change considering there were no monthly app limits before. if you get approved for the academic research track your app quota is increased to million per month. this is markedly better but the achievable data volume is still nothing like the v . api, as these graphs attempt to illustrate:

twarc will still observe the same rate limits, but once you’ve collected your portion for the month there’s not much that can be done, for that app at least.

apart from the quotas twitter’s streaming endpoint in v is substantially changed which impacts how users interact with twarc. previously twarc users would be able to create up to to two connections to the filter stream api. this could be done by simply:

twarc filter obama > obama.jsonl

however in the twitter v api only apps can connect to the filter stream, and they can only connect once. at first this seems like a major limitation but rather than creating a connection per query the v api allows you to build a set of rules for tweets to match, which in turns controls what tweets are included in the stream. this means you can collect for multiple types of queries at the same time, and the tweets will come back with a piece of metadata indicating what rule caused its inclusion.

this translates into a markedly different set of interactions at the command line for collecting from the stream where you first need to set your stream rules and then open a connection to fetch it.

twarc stream-rules add blacklivesmatter twarc stream > tweets.jsonl

one useful side effect of this is that you can update the stream (add and remove rules) while the stream is in motion:

twarc stream-rules add blm

while you are limited by the api quota in terms of how many tweets you can collect, tweets are not “dropped on the floor” when the volume gets too high. once upon a time the v . filter stream was rumored to be rate limited when your stream exceeds % of the total volume of new tweets.

plugins

in addition to twarc helping you collect tweets the github repository has also been a place to collect a set of utilities for working with the data. for example there are scripts for extracting and unshortening urls, identifying suspended/deleted content, extracting videos, buiding wordclouds, putting tweets on maps, displaying network graph visualizations, counting hashtags, and more. these utilities all work like unix filters where the input is a stream of tweets and the output varies depending on what the utility is doing, e.g. a gephi file for a network visualization, or a folder of mp files for video extraction.

while this has worked well in general the kitchen sink approach has been difficult to manage from a configuration management perspective. users have to download these scripts manually from github or by cloning the repository. for some users this is fine, but it’s a bit of a barrier to entry for users who have just installed twarc with pip.

furthermore these plugins often have their own dependencies which twarc itself does not. this lets twarc can stay pretty lean, and things like youtube_dl, networkx or pandas can be installed by people that want to use utilities that need them. but since there is no way to install the utilities there isn’t a way to ensure that the dependencies are installed, which can lead to users needing to diagnose missing libraries themselves.

finally the plugins have typically lacked their own tests. twarc’s test suite has really helped us track changes to the twitter api and to make sure that it continues to operate properly as new functionality has been added. but nothing like this has existed for the utilities. we’ve noticed that over time some of them need updating. also their command line arguments have drifted over time which can lead to some inconsistencies in how they are used.

so with twarc we’ve introduced the idea of plugins which extend the functionality of the twarc command, are distributed on pypi separately from twarc, and exist in their own github repositories where they can be developed and tested independently of twarc itself. this is all achieved through twarc ’s use of the click library and specifically click-plugins. so now if you would like to convert your collected tweets to csv you can install the twarc-csv:

$ pip install twarc-csv $ twarc search covid > covid .jsonl $ twarc csv covid .jsonl > covid .csv

or if you want to extract embedded and referenced videos from tweets you can install twarc-videos which will write all the videos to a directory:

$ pip install twarc-videos $ twarc videos covid .jsonl --download-dir covid -videos

you can write these plugins yourself and release them as needed. check out the plugin reference implementation tweet-ids for a simple example to adapt. we’re still in the process of porting some of the most useful utilities over and would love to see ideas for new plugins. check out the current list of twarc plugins and use the twarc issue tracker on github to join the discussion.

you may notice from the list of plugins that twarc now (finally) has documentation on readthedocs external from the documentation that was previously only available on github. we got by with github’s rendering of markdown documents for a while, but github’s boilerplate designed for developers can prove to be quite confusing for users who aren’t used to selectively ignoring it. readthedocs allows us to manage the command line and api documentation for twarc, and to showcase the work that has gone into the spanish, japanese, portuguese, swedish, swahili and chinese translations.

feedback

thanks for reading this far! we hope you will give twarc a try. let us know what you think either in comments here, in the docnow slack or over on github.

✨ ✨ happy twarcing! ✨ ✨ ✨

windows users will want to indicate the output file using a second argument rather than redirecting output with >. see this page for details.↩

- - t : : + : peter sefton: fair data management; it's a lifestyle not a lifecycle http://ptsefton.com/ / / /rdmpic/index.html

i have been working with my colleague marco la rosa on summary diagrams that capture some important aspects of research data management, and include the fair data principles; that data should be findable, accessible, interoperable and reusable.

but first, here's a rant about some modeling and diagramming styles and trends that i do not like.

i took part in a fun twitter thread recently kicked off by fiona tweedie.

fiona tweedie @fctweedie so my current bugbear is university processes that seem to forget that the actual work of higher ed is doing research and/ or teaching. this "research lifecycle" diagram from @uw is a stunning example:

the uw myresearch lifecycle with the four stages: plan/propose, setup, manage, and closeout

in this tweet dr tweedie has called out yet another research lifecycle diagram that leaves out the process of you know, actually doing research. this process-elision happened more than once when i was working as an eresearch manager - management would get in the consultants to look at research systems, talk to the research office and graduate school and come up with a "journey map" of administrative processes that either didn't mention the actual doing research or represented it as a tiny segment, never mind that it's, you know, the main thing researchers do when they're being researchers rather than teachers or administrators.

at least the consultants would usually produce a 'journey map' that got you from point a to point b using chevrons to >> indicate progress and didn't insist that everything was a 'lifecycle'.

something like:

plan / propose >> setup >> manage / do research >> closeout

but all too commonly processes are represented using the tired old metaphor of a lifecycle.

reminder: a lifecycle is a biological process; how organisms come into existence, reproduce and die via various means including producing seeds, splitting themselves in two, um, making love, laying eggs and so on.

it's really stretching the metaphor to talk about research in this way - maybe the research outputs in the uw "closeout" phase are eggs that hatch into new bouncing baby proposals?

regrettably, arranging things in circles and using the "lifecycle" metaphor is very common - see this google image search for "research lifecycle":

i wonder if the diagramming tools that are available to people are part of the issue - microsoft word, for example can build cycles and other diagrams out of a bullet list.

(i thought it would be amusing to draw the uw diagram from above as a set cogs but this happened - you can only have cogs in a word diagram.)

attempt to use microsoft word to make a diagram cogs for plan/propose, setup, manage, and closeout but it will only draw three of them

research data management as a cycle

now that i've got that off my chest let's look at research data management. here's a diagram which is in fairly wide use, from the university of california.

(this image has a cc-by logo which means i can use it if i attribute it - but i'm not % clear on the original source of the diagram - it seems to be from uc somewhere.)

marco used this one in some presentations we gave. i thought we could do better.

the good part of this diagram is that it shows research data management as a cyclical, recurring activity - which for fair data it needs to be.

what i don't like:

i think it is trying to show a project (ie grant) level view of research with data management happening in one spot on the journey. typically researchers do research all the time (or in between teaching or when they can get time on equipment) not at a particular point in some administrative "journey map". we often hear feedback that their research is a lifetime activity and does not happen the way administrators and it think it does.
"archive" is shown as a single-step pre-publication. this is a terrible message; if we are to start really doing fair data then data need to be described and made findable and accessible asap.
the big so-called lifecycle is (to me) very contrived and looks like a librarian view of the world with data searching as a stand-alone process before research data management planning. not clear whether publication means articles or data.
"data search / reuse" is a type of "collection", and why is it happening before data management planning? "re-collection" is also a kind of collection, so we can probably collapse all those together (the findable and accessible in fair).
it’s not clear whether publication means articles or data or both.
most research uses some kind of data storage but very often not directly; people might be interacting with a lab notebook system or a data repository - at uts we arrived at the concept of "workspaces" to capture this.

the "minimum viable fair diagram"

marco and i have a sketch of a new diagram that attempts to address these issues and addresses what needs to be in place for broad-scale fair data practice.

two of the fair principles suggest services that need to be in place; ways to find and access data. the i and r in fair are not something that can be encapsulated in a service, as such, rather they imply that data are well described for re-use and interoperation of systems and in reusable formats.

as it happens, there is a common infrastructure component which encapsulates finding data and accessing; the repository. repositories are services which hold data and make it discoverable and accessible, with governance that ensures that data does not change without notice and is available for access over agreed time frames - sometimes with detailed access control. repositories may be general purpose or specialized around a particular type of data: gene sequences, maps, code, microscope images etc. they may also be ad-hoc - at a lab level they could be a well laid out, well managed file system.

some well-funded disciplines have established global or national repositories and workflows for some or all of their data, notably physics and astronomy, bioinformatics, geophysical sciences, climate and marine science. some of these may not be thought of by their community as repositories - but according to our functional definition they are repositories, even if they are "just" vast shared file systems or databases where everyone knows what's what and data managers keep stuff organized. also, some institutions have institutional data repositories but it is by no means common practice across the whole of the research sector that data find their way into any of these repositories.

remember: data storage is not all files-on-disks. researchers use a very wide range of tools which may make data inaccessible outside of the tool. examples include: cloud-based research (lab) notebook systems in which data is deposited alongside narrative activity logs; large shared virtual laboratories where data are uploaded; secure eresearch platforms (serps) which allow access only via virtualized desktops with severely constrained data ingress and egress; survey tools; content management systems; digital asset management systems; email (yes, it's true some folks use email as project archives!); to custom-made code for a single experiment.

our general term for all of the infrastructures that researchers use for rdm day to day including general purpose storage is “workspaces”.

many, if not most workspaces do not have high levels of governance, and data may be technically or legally inaccessible over the long term. they should not be considered as suitable archives or repositories - hence our emphasis on making sure that data can be described and deposited into general purpose, standards-driven repository services.

the following is a snapshot of the core parts of an idealised fair data service. it shows the activities that researchers undertake, acquiring data from observations, instruments and by reuse, conducting analysis and data description in a working environment, and depositing results into one or more repositories.

we wanted it to show:

that infrastructure services are required for research data management - researchers don't just "archive" their data without support - they and those who will reuse data need repository services in some form.
that research is conducted using workspace environments - more infrastructure.

a work-in-progress sketch of fair research data management.

we (by which i mean marco) will make this prettier soon.

and yes, there is a legitimate cycle in this diagram it's the find -> access -> reuse -> describe -> deposit cycle that's inherent in the fair lifestyle.

things that might still be missing:

some kind of rubbish bin - to show that workspaces are ephemeral and working data that doesn't make the cut may be culled, and that some data is held only for a time.
what do you think's missing?

thoughts anyone? comments below or take it up on twitter with @ptsefton.

(i have reworked parts of a document that marco and i have been working on with guido aben for this document, and thanks to recent graduate florence sefton for picking up typos and sense-checking).

- - t : : + : ptsefton david rosenthal: elon musk: threat or menace? https://blog.dshr.org/ / /elon-musk-threat-or-menace.html although both tesla and spacex are major engineering achievements, elon musk seems completely unable to understand the concept of externalities, unaccounted-for costs that society bears as a result of these achievements.

first, in tesla: carbon offsetting, but in reverse, jaime powell reacted to tesla taking $ . b in carbon offsets which provided the only profit tesla ever made and putting them into bitcoin:

looked at differently, a single bitcoin purchase at a price of ~$ , has a carbon footprint of tons, the equivalent of ice cars.

tesla’s average selling price in the fourth quarter of ? $ , .

we’re not sure about you, but ft alphaville is struggling to square the circle of “buy a tesla with a bitcoin and create the carbon output of internal combustion engine cars” with its legendary environmental ambitions.

unless, of course, that was never the point in the first place.

below the fold, more externalities musk is ignoring.

second, there is musk's obsession with establishing a colony on mars. even assuming spacex can stop their starship second stage exploding on landing, and do the same with the much bigger first stage, the mars colony scheme would have massive environmental impacts. musk envisages a huge fleet of starships ferrying people and supplies to mars for between and years. the climate effects of dumping this much rocket exhaust into the upper atmosphere over such a long period would be significant. the idea that a world suffering the catastrophic effects of climate change could sustain such an expensive program over many decades simply for the benfit of a miniscule fraction of the population is laughable.

these externalities are in the future. but there are a more immediate set of externalities.

back in i expressed my skepticism about "level " self-driving cars in techno-hype part , stressing that the problem was that to get to level , or as musk calls it "full self-driving", you need to pass through the levels where the software has to hand-off to the human. and the closer you get to level , the harder this problem becomes:

suppose, for the sake of argument, that self-driving cars three times as good as waymo's are in wide use by normal people. a normal person would encounter a hand-off once in , miles of driving, or less than once a year. driving would be something they'd be asked to do maybe times in their life.

even if, when the hand-off happened, the human was not "climbing into the back seat, climbing out of an open car window, and even smooching" and had full "situational awareness", they would be faced with a situation too complex for the car's software. how likely is it that they would have the skills needed to cope, when the last time they did any driving was over a year ago, and on average they've only driven times in their life? current testing of self-driving cars hands-off to drivers with more than a decade of driving experience, well over , miles of it. it bears no relationship to the hand-off problem with a mass deployment of self-driving technology.

mack hogan's tesla's "full self driving" beta is just laughably bad and potentially dangerous starts:

a beta version of tesla's "full self driving" autopilot update has begun rolling out to certain users. and man, if you thought "full self driving" was even close to a reality, this video of the system in action will certainly relieve you of that notion. it is perhaps the best comprehensive video at illustrating just how morally dubious, technologically limited, and potentially dangerous autopilot's "full self driving" beta program is.

hogan sums up the lesson of the video:

tesla's software clearly does a decent job of identifying cars, stop signs, pedestrians, bikes, traffic lights, and other basic obstacles. yet to think this constitutes anything close to "full self-driving" is ludicrous. there's nothing wrong with having limited capabilities, but tesla stands alone in its inability to acknowledge its own shortcomings.

hogan goes on to point out the externalities:

when technology is immature, the natural reaction is to continue working on it until it's ironed out. tesla has opted against that strategy here, instead choosing to sell software it knows is incomplete, charging a substantial premium, and hoping that those who buy it have the nuanced, advanced understanding of its limitations—and the ability and responsibility to jump in and save it when it inevitably gets baffled. in short, every tesla owner who purchases "full self-driving" is serving as an unpaid safety supervisor, conducting research on tesla's behalf. perhaps more damning, the company takes no responsibility for its actions and leaves it up to driver discretion to decide when and where to test it out.

that leads to videos like this, where early adopters carry out uncontrolled tests on city streets, with pedestrians, cyclists, and other drivers unaware that they're part of the experiment. if even one of those tesla drivers slips up, the consequences can be deadly.

of course, the drivers are only human so they do slip up:

the tesla arrives at an intersection where it has a stop sign and cross traffic doesn't. it proceeds with two cars incoming, the first car narrowly passing the car's front bumper and the trailing car braking to avoid t-boning the model . it is absolutely unbelievable and indefensible that the driver, who is supposed to be monitoring the car to ensure safe operation, did not intervene there.

an example of the kinds of problems that can be caused by autonomous vehicles behaving in ways that humans don't expect is reported by timothy b. lee in fender bender in arizona illustrates waymo’s commercialization challenge:

a white waymo minivan was traveling westbound in the middle of three westbound lanes on chandler boulevard, in autonomous mode, when it unexpectedly braked for no reason. a waymo backup driver behind the wheel at the time told chandler police that "all of a sudden the vehicle began to stop and gave a code to the effect of 'stop recommended' and came to a sudden stop without warning."

a red chevrolet silverado pickup behind the vehicle swerved to the right but clipped its back panel, causing minor damage. nobody was hurt.

the tesla in the video made a similar unexpected stop. lee stresses that, unlike tesla's, waymo's responsible test program has resulted in a generally safe product, but not one that is safe enough:

waymo has racked up more than million testing miles in arizona, california, and other states. this is far more than any human being will drive in a lifetime. waymo's vehicles have been involved in a relatively small number of crashes. these crashes have been overwhelmingly minor with no fatalities and few if any serious injuries. waymo says that a large majority of those crashes have been the fault of the other driver. so it's very possible that waymo's self-driving software is significantly safer than a human driver.
...
the more serious problem for waymo is that the company can't be sure that the idiosyncrasies of its self-driving software won't contribute to a more serious crash in the future. human drivers cause a fatality about once every million miles of driving—far more miles than waymo has tested so far. if waymo scaled up rapidly, it would be taking a risk that an unnoticed flaw in waymo's programming could lead to someone getting killed.

i'm a pedestrian, cyclist and driver in an area infested with teslas owned, but potentially not actually being driven, by fanatical early adopters and members of the cult of musk. i'm personally at risk from these people believing that what they paid good money for was "full self driving". when spacex tests starship at their boca chica site they take precautions, including road closures, to ensure innocent bystanders aren't at risk from the rain of debris when things go wrong. tesla, not so much.

of course, tesla doesn't tell the regulators that what the cult members paid for was "full self driving"; that might cause legal problems. as timothy b. lee reports, tesla: “full self-driving beta” isn’t designed for full self-driving:

"despite the "full self-driving" name, tesla admitted it doesn't consider the current beta software suitable for fully driverless operation. the company said it wouldn't start testing "true autonomous features" until some unspecified point in the future.
...
tesla added that "we do not expect significant enhancements" that would "shift the responsibility for the entire dynamic driving task to the system." the system "will continue to be an sae level , advanced driver-assistance feature."

sae level is industry jargon for a driver-assistance systems that perform functions like lane-keeping and adaptive cruise control. by definition, level systems require continual human oversight. fully driverless systems—like the taxi service waymo is operating in the phoenix area—are considered level systems."

there is an urgent need for regulators to step up and stop this dangerous madness:

the nhtsa should force tesla to disable "full self driving" in all its vehicles until the technology has passed an approved test program
any vehicles taking part in such a test program on public roads should be clearly distinguishable from teslas being driven by actual humans, for example with orange flashing lights. self-driving test vehicles from less irresponsible companies such as waymo are distinguishable in this way, teslas in which some cult member has turned on "full self driving beta" are not.
the ftc should force tesla to refund, with interest, every dollar paid by their customers under the false pretense that they were paying for "full self driving".

- - t : : + : david. (noreply@blogger.com) jez cope: collaborations workshop : talks & panel session https://erambler.co.uk/blog/collabw -part- /

i’ve just finished attending (online) the three days of this year’s ssi collaborations workshop (cw for short), and once again it’s been a brilliant experience, as well as mentally exhausting, so i thought i’d better get a summary down while it’s still fresh it my mind.

collaborations workshop is, as the name suggests, much more focused on facilitating collaborations than a typical conference, and has settled into a structure that starts off with with longer keynotes and lectures, and progressively gets more interactive culminating with a hack day on the third day.

that’s a lot to write about, so for this post i’ll focus on the talks and panel session, and follow up with another post about the collaborative bits. i’ll also probably need to come back and add in more links to bits and pieces once slides and the “official” summary of the event become available.

updates

- - added links to recordings of keynotes and panel sessions

provocations

the first day began with two keynotes on this year’s main themes: fair research software and diversity & inclusion, and day had a great panel session focused on disability. all three were streamed live and the recordings remain available on youtube:

fair research software

dr michelle barker, director of the research software alliance, spoke on the challenges to recognition of software as part of the scholarly record: software is not often cited. the fair rs working group has been set up to investigate and create guidance on how the fair principles for data can be adapted to research software as well; as they stand, the principles are not ideally suited to software. this work will only be the beginning though, as we will also need metrics, training, career paths and much more. resa itself has focus areas: people, policy and infrastructure. if you’re interested in getting more involved in this, you can join the resa email list.

equality, diversity & inclusion: how to go about it

dr chonnettia jones, vice president of research, michael smith foundation for health research spoke extensively and persuasively on the need for equality, diversity & inclusion (edi) initiatives within research, as there is abundant robust evidence that all research outcomes are improved.

she highlighted the difficulties current approaches to edi have effecting structural change, and changing not just individual behaviours but the cultures & practices that perpetuate iniquity. what initiatives are often constructed around making up for individual deficits, a bitter framing is to start from an understanding of individuals having equal stature but having different tired experiences. commenting on the current focus on “research excellent” she pointed out that the hyper-competition this promotes is deeply unhealthy. suggesting instead that true excellence requires diversity, and we should focus on an inclusive excellence driven by inclusive leadership.

equality, diversity & inclusion: disability issues

day ’s edi panel session brought together five disabled academics to discuss the problems of disability in research.

dr becca wilson, ukri innovation fellow, institute of population health science, university of liverpool (chair)
phoenix c s andrews (phd student, information studies, university of sheffield and freelance writer)
dr ella gale (research associate and machine learning subject specialist, school of chemistry, university of bristol)
prof robert stevens (professor and head of department of computer science, university of manchester)
dr robin wilson (freelance data scientist and ssi fellow)

nb. the discussion flowed quite freely so the following summary, so the following summary mixes up input from all the panel members.

researchers are often assumed to be single-minded in following their research calling, and aptness for jobs is often partly judged on “time send”, which disadvantages any disabled person who has been forced to take a career break. on top of this disabled people are often time-poor because of the extra time needed to manage their condition, leaving them with less “output” to show for their time served on many common metrics. this can partially affect early-career researchers, since resources for these are often restricted on a “years-since-phd” criterion. time poverty also makes funding with short deadlines that much harder to apply for. employers add more demands right from the start: new starters are typically expected to complete a health and safety form, generally a brief affair that will suddenly become an -page bureaucratic nightmare if you tick the box declaring a disability.

many employers claim to be inclusive yet utterly fail to understand the needs of their disabled staff. wheelchairs are liberating for those who use them (despite the awful but common phrase “wheelchair-bound”) and yet employers will refuse to insure a wheelchair while travelling for work, classifying it as a “high value personal item” that the owner would take the same responsibility for as an expensive camera. computers open up the world for blind people in a way that was never possible without them, but it’s not unusual for mandatory training to be inaccessible to screen readers. some of these barriers can be overcome, but doing so takes yet more time that could and should be spent on more important work.

what can we do about it? academia works on patronage whether we like it or not, so be the person who supports people who are different to you rather than focusing on the one you “recognise yourself in” to mentor. as a manager, it’s important to ask each individual what they need and believe them: they are the expert in their own condition and their lived experience of it. don’t assume that because someone else in your organisation with the same disability needs one set of accommodations, it’s invalid for your staff member to require something totally different. and remember: disability is unusual as a protected characteristic in that anyone can acquire it at any time without warning!

lightning talks

lightning talk sessions are always tricky to summarise, and while this doesn’t do them justice, here are a few highlights from my notes.

data & metadata

malin sandstrom talked about a much-needed refinement of contributor role taxonomies for scientific computing
stephan druskat showcased a project to crowdsource a corpus of research software for further analysis

learning & teaching/community

matthew bluteau introduced the concept of the “coding dojo” as a way to enhance community of practice. a group of coders got together to practice & learn by working together to solve a problem and explaining their work as they go
- he described models: a code jam, where people work in small groups, and the randori method, where people do pair programming while the rest observe. i’m excited to try this out!
steve crouch talked about intermediate skills and helping people take the next step, which i’m also very interested in with the glam data science network
esther plomp recounted experience of running multiple carpentry workshops online, while diego alonso alvarez discussed planned workshops on making research software more usable with guis
shoaib sufi showcased the ssi’s new event organising guide
caroline jay reported on a diary study into autonomy & agency in rse during covid
- lopez, t., jay, c., wermelinger, m., & sharp, h. ( ). how has the covid- pandemic affected working conditions for research software engineers? unpublished manuscript.

wrapping up

that’s not everything! but this post is getting pretty long so i’ll wrap up for now. i’ll try to follow up soon with a summary of the “collaborative” part of collaborations workshop: the idea-generating sessions and hackday!

- - t : : + : journal of web librarianship: examination of academic library websites regarding covid- responsiveness https://www.tandfonline.com/doi/full/ . / . . ?ai= dl&mi=co bk&af=r .
- - t : : + : kristine condic terry reese: marcedit . update https://blog.reeset.net/archives/

changelog: https://marcedit.reeset.net/software/update .txt

highlights preview changes

one of the most requested features over the years has been the ability to preview changes prior to running them. as of . . – a new preview option has been added to many of the global editing tools in the marceditor. currently, you will find the preview option attached to the following functions:

replace all
add new field
delete field
edit subfield
edit field
edit indicator
copy field
swap field

functions that include a preview option will be denoted with the following button:

when this button is pressed, the following option is made available

when preview results is selected, the program will execute the defined action, and display the potential results in a display screen. for example:

to protect performance, only results at a time will be loaded into the preview grid, though users can keep adding results to the grid and continue to review items. additionally, users have the ability to search for items within the grid as well as jump to a specific record number (not row number).

these new options will show up first in the windows version of marcedit, but will be added to the marcedit mac . .x branch in the coming weeks.

new json => xml translation

to better support the translation of data from json to marc, i’ve included a json => marc algorithm in the marcengine. this will allow json data to serialized into xml. the benefit of including this option, is that i’ve been able to update the xml functions options to allow json to be a starting format. this will specifically useful for users that want to make use of linked data vocabularies to generate marc authority records. users can direct marcedit to facilitate the translation from json to xml, and then create xslt translations that can then be used to complete the process to marcxml and marc. i’ve demonstrated how this process works using a vocabulary of interest to the #critcat community, the homosaurus vocabulary (how do i generate marc authority records from the homosaurus vocabulary? – terry’s worklog (reeset.net)).

oclc api interactions

working with the oclc api is sometimes tricky. marcedit utilizes a specific authentication process that requires oclc keys be setup and configured to work a certain way. when issues come up, it is sometimes very difficult to debug them. i’ve updated the process and error handling to surface more information – so when problems occur and xml debugging information isn’t available, the actual exception and inner exception data will be surfaced instead. this often can provide information to help understand why the process isn’t able to complete.

wrap up

as noted, there have been a number of updates. while many fall under the category of house-keeping (updating icons, ux improvements, actions, default values, etc.) – this update does include a number of often asked for, significant updates, that i hope will improve user workflows.

–tr

- - t : : + : reeset terry reese: how do i generate marc authority records from the homosaurus vocabulary? https://blog.reeset.net/archives/

step by step instructions here: https://youtu.be/fjsdqi pzpq

ok, so last week, i got an interesting question on the listserv where a user asked specifically about generating marc records for use in one’s ils system from a jsonld vocabulary. in this case, the vocabulary in question as homosaurus (homosaurus vocabulary site) – and the questioner was specifically looking for a way to pull individual terms for generation into marc authority records to add to one’s ils to improve search and discovery.

when the question was first asked, my immediate thought was that this could likely be accommodated using the xml/json profiling wizard in marcedit. this tool can review a sample xml or json file and allow a user to create a portable processing file based on the content in the file. however, there were two issues with this approach:

the profile wizard assumes that data format is static – i.e., the sample file is representative of other files. unfortunately, for this vocabulary, that isn’t the case.
the profile wizard was designed to work with json – json ld is actually a different animal due to the inclusion of the @ symbol.

while i updated the profiler to recognize and work better with json-ld – the first challenge is one that doesn’t make this a good fit to create a generic process. so, i looked at how this could be built into the normal processing options.

to do this, i added a new default serialization, json=>xml == which marcedit now supports. this allows the tool to take a json file, and deserialize the data so that is output reliably as xml. so, for example, here is a sample json-ld file (homosaurus.org/v /adoptiveparents.jsonld):

{ "@context": { "dc": "http://purl.org/dc/terms/", "skos": "http://www.w .org/ / /skos/core#", "xsd": "http://www.w .org/ /xmlschema#" }, "@id": "http://homosaurus.org/v /adoptiveparents", "@type": "skos:concept", "dc:identifier": "adoptiveparents", "dc:issued": { "@value": " - - ", "@type": "xsd:date" }, "dc:modified": { "@value": " - - ", "@type": "xsd:date" }, "skos:broader": { "@id": "http://homosaurus.org/v /parentslgbtq" }, "skos:hastopconcept": [ { "@id": "http://homosaurus.org/v /familymembers" }, { "@id": "http://homosaurus.org/v /familieslgbtq" } ], "skos:inscheme": { "@id": "http://homosaurus.org/terms" }, "skos:preflabel": "adoptive parents", "skos:related": [ { "@id": "http://homosaurus.org/v /socialparenthood" }, { "@id": "http://homosaurus.org/v /lgbtqadoption" }, { "@id": "http://homosaurus.org/v /lgbtqadoptiveparents" }, { "@id": "http://homosaurus.org/v /birthparents" } ] }

in marcedit, the new json=>xml process can take this file and output it in xml like this:

<?xml version=" . "?> <records> <record> <context> <dc>http://purl.org/dc/terms/</dc> <skos>http://www.w .org/ / /skos/core#</skos> <xsd>http://www.w .org/ /xmlschema#</xsd> </context> <id>http://homosaurus.org/v /adoptiveparents</id> <type>skos:concept</type> <identifier>adoptiveparents</identifier> <issued> <value> - - </value> <type>xsd:date</type> </issued> <modified> <value> - - </value> <type>xsd:date</type> </modified> <broader> <id>http://homosaurus.org/v /parentslgbtq</id> </broader> <hastopconcept> <id>http://homosaurus.org/v /familymembers</id> </hastopconcept> <hastopconcept> <id>http://homosaurus.org/v /familieslgbtq</id> </hastopconcept> <inscheme> <id>http://homosaurus.org/terms</id> </inscheme> <preflabel>adoptive parents</preflabel> <related> <id>http://homosaurus.org/v /socialparenthood</id> </related> <related> <id>http://homosaurus.org/v /lgbtqadoption</id> </related> <related> <id>http://homosaurus.org/v /lgbtqadoptiveparents</id> </related> <related> <id>http://homosaurus.org/v /birthparents</id> </related> </record> </records>

the ability to reliably convert json/jsonld to xml means that i can now allow users to utilize the same xslt/xquery process marcedit utilizes for other library metadata format transformation. all that was left to make this happen was to add a new origin data format to the xml function template – and we are off and running.

the end result is users could utilize this process with any json-ld vocabulary (assuming they created the xslt) to facilitate the automation of marc authority data. in this case of this vocabulary, i’ve created an xslt and added it to my github space: https://github.com/reeset/marcedit_xslt_files/blob/master/homosaurus_xml.xsl

but have included the xslt in the marcedit xslt directory in current downloads.

in order to use this xslt and allow your version of marcedit to generate marc authority records from this vocabulary – you would use the following steps:

be using marcedit . . + or marcedit mac . . + (mac version will be available around / ). i have not decided if i will backport to . -
open the xml functions editor in marcedit
add a new transformation – using json as the original format, and marc as the final. make sure the xslt path is pointed to the location where you saved the downloaded xslt file.
save

that should be pretty much it. i’ve recorded the steps and placed them here: https://youtu.be/fjsdqi pzpq, including some information on values you may wish to edit should you want to localize the xslt.

- - t : : + : reeset peter murray: publishers going-it-alone (for now?) with getftr https://dltj.org/article/publishers-alone-with-getftr/

in early december , a group of publishers announced get-full-text-research, or getftr for short. i read about this first in roger schonfeld’s “publishers announce a major new service to plug leakage” piece in the scholarly kitchen via jeff pooley’s twitter thread and blog post. details about how this works are thin, so i’m leaning heavily on roger’s description. i’m not as negative about this as jeff, and i’m probably a little more opinionated than roger. this is an interesting move by publishers, and—as the title of this post suggests—i am critical of the publisher’s “go-it-alone” approach.

first, some disclosure might be in order. my background has me thinking of this in the context of how it impacts libraries and library consortia. for the past four years, i’ve been co-chair of the niso information discovery and interchange topic committee (and its predecessor, the “discovery to delivery” topic committee), so this is squarely in what i’ve been thinking about in the broader library-publisher professional space. i also traced the early development of ra and more recently am volunteering on the seamlessaccess entity category and attribute bundles working group; that’ll become more important a little further down this post.

i was nodding along with roger’s narrative until i stopped short here:

the five major publishing houses that are the driving forces behind getftr are not pursuing this initiative through one of the major industry collaborative bodies. all five are leading members of the stm association, niso, orcid, crossref, and chorus, to name several major industry groups. but rather than working through one of these existing groups, the houses plan instead to launch a new legal entity.

while [vice president of product strategy & partnerships for wiley todd] toler and [senior director, technology strategy & partnerships for the american chemical society ralph] youngen were too politic to go deeply into the details of why this might be, it is clear that the leadership of the large houses have felt a major sense of mismatch between their business priorities on the one hand and the capabilities of these existing industry bodies. at recent industry events, publishing house ceos have voiced extensive concerns about the lack of cooperation-driven innovation in the sector. for example, judy verses from wiley spoke to this issue in spring , and several executives did so at frankfurt this fall. in both cases, long standing members of the scholarly publishing sector questioned if these executives perhaps did not realize the extensive collaborations driven through crossref and orcid, among others. it is now clear to me that the issue is not a lack of knowledge but rather a concern at the executive level about the perceived inability of existing collaborative vehicles to enable the new strategic directions that publishers feel they must pursue.

this is the publishers going-it-alone. to see roger describe it, they are going to create this web service that allows publishers to determine the appropriate copy for a patron and do it without input from the libraries. librarians will just be expected to put this web service widget into their discovery services to get “colored buttons indicating that the link will take [patrons] to the version of record, an alternative pathway, or (presumably in rare cases) no access at all.” (let’s set aside for the moment the privacy implications of having a fourth-party web service recording all of the individual articles that come up in a patron’s search results.) librarians will not get to decide the “alternative pathway” that is appropriate for the patron: “some publishers might choose to provide access to a preprint or a read-only version, perhaps in some cases on some kind of metered basis.” (roger goes on to say that he “expect[s] publishers will typically enable some alternative version for their content, in which case the vast majority of scholarly content will be freely available through publishers even if it is not open access in terms of licensing.” i’m not so confident.)

no, thank you. if publishers want to engage in technical work to enable libraries and others to build web services that determine the direct link to an article based on a doi, then great. libraries can build a tool that consumes that information as well as takes into account information about preprint services, open access versions, interlibrary loan and other methods of access. but to ask libraries to accept this publisher-controlled access button in their discovery layers, their learning management systems, their scholarly profile services, and their other tools? that sounds destined for disappointment.

i am only somewhat encouraged by the fact that ra started out as a small, isolated collaboration of publishers before they brought in niso and invited libraries to join the discussion. did it mean that it slowed down deployment of ra ? undoubtedly yes. did persnickety librarians demand transparent discussions and decisions about privacy-related concerns like what attributes the publisher would get about the patron in the shibboleth-powered backchannel? yes, but because the patrons weren’t there to advocate for themselves. will it likely mean wider adoption? i’d like to think so.

have publishers learned that forcing these kinds of technologies onto users without consultation is a bad idea? at the moment it would appear not. some of what publishers are seeking with getftr can be implemented with straight-up openurl or—at the very least—limited-scope additions to openurl (the z . open standard!). so that they didn’t start with openurl, a robust existing standard, is both concerning and annoying. i’ll be watching and listening for points of engagement, so i remain hopeful.

a few words about jeff pooley’s five-step “laughably creaky and friction-filled effort” that is seamlessaccess. many of the steps jeff describes are invisible and well-established technical protocols. what jeff fails to take into account is the very visible and friction-filled effect of patrons accessing content beyond the boundaries of campus-recognized internet network addresses. those patrons get stopped at step two with a “pay $ please” message. i’m all for removing that barrier entirely by making all published content “open access”. it is folly to think, though, that researchers and readers can enforce an open access business model on all publishers, so solutions like seamlessaccess will have a place. (which is to say nothing of the benefit of inter-institutional resource collaboration opened up by a more widely deployed shibboleth infrastructure powered by seamlessaccess.)

- - t : : + : peter murray (jester@dltj.org) peter murray: what is known about getftr at the end of https://dltj.org/article/getftr-update/

in early december , a group of publishers announced get-full-text-research, or getftr for short. there was a heck of a response on social media, and the response was—on the whole—not positive from my librarian-dominated corner of twitter. for my early take on getftr, see my december rd blog post “publishers going-it-alone (for now?) with getftr.” as that post title suggests, i took the five founding getftr publishers to task on their take-it-or-leave-it approach. i think that is still a problem. to get you caught up, here is a list of other commentary.

roger schonfeld’s december rd “publishers announce a major new service to plug leakage” piece in the scholarly kitchen
tweet from herbert van de sompel, the lead author of the openurl spec, on solving the appropriate copy problem
december th post “get to fulltext ourselves, not getftr.” on the open access button blog
twitter thread on december th between @cshillum and @lisalibrarian on the positioning of getftr in relation to link resolvers and an unanswered question about how getftr aligns with library interests
twitter thread started by @tac_niso on december th looking for more information with a link to an stm association presentation added by @aarontay
a tree of tweets starting from @mrgunn’s [i don’t trust publishers to decide] is the crux of the whole thing. in particular, threads of that tweet that include jason griffey of niso saying he knew nothing about getftr and bernhard mittermaier’s point about hidden motivations behind getftr
twitter thread started by @aarontay on december th saying “getftr is bad for researchers/readers and librarians. it only benefits publishers, change my mind.”
lisa janicke hinchliffe’s december th “why are librarians concerned about getftr?” in the scholarly kitchen and take note of the follow-up discussion in the comments
twitter thread between @alison_mudditt and @lisalibrarian clarifying plos is not on the advisory board with some @tac_niso as well.
ian mulvany’s december th “thoughts on getftr” on scholcommsprod
getftr’s december th “updating the community” post on their website
the spanish federation of associations of archivists, librarians, archaeologists, museologists and documentalists (anabad)’s december th “getftr: new publishers service to speed up access to research articles” (original in spanish, google translate to english)
december th news entry from econtent pro with the title “what getftr means for journal article access” which i’ll only quarrel with this sentence: “thus, getftr is a service where academic articles are found and provided to you at absolutely no cost.” no—if you are in academia the cost is born by your library even if you don’t see it. but this seems like a third party service that isn’t directly related to publishers or libraries, so perhaps they can be forgiven for not getting that nuance.
wiley’s chemistry views news post on december th titled simply “get full text research (getftr)” is perhaps only notable for the sentence “growing leakage has steadily eroded the ability of the publishers to monetize the value they create.”

if you are looking for a short list of what to look at, i recommend these posts.

getftr’s community update

on december —after the two posts i list below—an “updating the community” web page was posted to the getftr website. from a public relations perspective, it was…interesting.

we are committed to being open and transparent

this section goes on to say, “if the community feels we need to add librarians to our advisory group we will certainly do so and we will explore ways to ensure we engage with as many of our librarian stakeholders as possible.” if the getftr leadership didn’t get the indication between december and december that librarians feel strongly about being at the table, then i don’t know what will. and it isn’t about being on the advisory group; it is about being seen and appreciated as important stakeholders in the research discovery process. i’m not sure who the “community” is in this section, but it is clear that librarians are—at best—an afterthought. that is not the kind of “open and transparent” that is welcoming.

later on in the questions about library link resolvers section is this sentence:

we have, or are planning to, consult with existing library advisory boards that participating publishers have, as this enables us to gather views from a significant number of librarians from all over the globe, at a range of different institutions.

as i said in my previous post, i don’t know why getftr is not engaging in existing cross-community (publisher/technology-supplier/library) organizations to have this discussion. it feels intentional, which colors the perception of what the publishers are trying to accomplish. to be honest, i don’t think the publishers are using getftr to drive a wedge between library technology service providers (who are needed to make getftr a reality for libraries) and libraries themselves. but i can see how that interpretation could be made.

understandably, we have been asked about privacy.

i punted on privacy in my previous post, so let’s talk about it here. it remains to be seen what is included in the getftr api request between the browser and the publisher site. sure, it needs to include the doi and a token that identifies the patron’s institution. we can inspect that api request to ensure nothing else is included. but the fact that the design of getftr has the browser making the call to the publisher site means that the publisher site knows the ip address of the patron’s browser, and the ip address can be considered personally identifiable information. this issue could be fixed by having the link resolver or the discovery layer software make the api request, and according to the questions about library link resolvers section of the community update, this may be under consideration.

so, yes, an auditable privacy policy and implementation is key for for getftr.

getftr is fully committed to supporting third-party aggregators

this is good to hear. i would love to see more information published about this, including how discipline-specific repositories and institutional repositories can have their holdings represented in getftr responses.

my take-a-ways

in the second to last paragraph: “researchers should have easy, seamless pathways to research, on whatever platform they are using, wherever they are.” that is a statement that i think every library could sign onto. this updating the community is a good start, but the project has dug a deep hole of trust and it hasn’t reached level ground yet.

lisa janicke hinchliffe’s “why are librarians concerned about getftr?”

posted on december th in the scholarly kitchen, lisa outlines a series of concerns from a librarian perspective. i agree with some of these; others are not an issue in my opinion.

librarian concern: the connection to seamless access

many librarians have expressed a concern about how patron information can leak to the publisher through ill-considered settings at an institution’s identity provider. seamless access can ease access control because it leverages a campus’ single sign-on solution—something that a library patron is likely to be familiar with. if the institution’s identity provider is overly permissive in the attributes about a patron that get transmitted to the publisher, then there is a serious risk of tying a user’s research activity to their identity and the bad things that come from that (patrons self-censoring their research paths, commoditization of patron activity, etc.). i’m serving on a seamless access task force that is addressing this issue, and i think there are technical, policy, and education solutions to this concern. in particular, i think some sort of intermediate display of the attributes being transmitted to the publisher is most appropriate.

librarian concern: the limited user base enabled

as lisa points out, the population of institutions that can take advantage of seamless access, a prerequisite for getftr, is very small and weighted heavily towards well-resourced institutions. to the extent that projects like seamless access (spurred on by a desire to have getftr-like functionality) helps with the adoption of saml-based infrastructure like shibboleth, then the whole academic community benefits from a shared authentication/identity layer that can be assumed to exist.

librarian concern: the insertion of new stumbling blocks

of the issues lisa mentioned here, i’m not concerned about users being redirected to their campus single sign-on system in multiple browsers on multiple machines. this is something we should be training users about—there is a single website to put your username/password into for whatever you are accessing at the institution. that a user might already be logged into the institution single sign-on system in the course of doing other school work and never see a logon screen is an attractive benefit to this system.

that said, it would be useful for an api call from a library’s discovery layer to a publisher’s getftr endpoint to be able to say, “this is my user. trust me when i say that they are from this institution.” if that were possible, then the seamless access where-are-you-from service could be bypassed for the getftr purpose of determining whether a user’s institution has access to an article on the publisher’s site. it would sure be nice if librarians were involved in the specification of the underlying protocols early on so these use cases could be offered.

update

lisa reached out on twitter to say (in part): “issue is getftr doesn’t redirect and sa doesnt when you are ipauthenticated. hence user ends up w mishmash of experience.” i went back to read her scholarly kitchen post and realized i did not fully understand her point. if getftr is relying on a seamless access token to know which institution a user is coming from, then that token must get into the user’s browser. the details we have seen about getftr don’t address how that seamless access institution token is put in the user’s browser if the user has not been to the seamless access select-your-institution portal. one such case is when the user is coming from an ip-address-authenticated computer on a campus network. do the getftr indicators appear even when the seamless access institution token is not stored in the browser? if at the publisher site the getftr response also uses the institution ip address table to determine entitlements, what does a user see when they have neither the seamless access institution token nor the institution ip address? and, to lisa’s point, how does one explain this disparity to users? is the situation better if the getftr determination is made in the link resolver rather than in the user browser?

librarian concern: exclusion from advisory committee

see previous paragraph. that librarians are not at the table offering use cases and technical advice means that the developers are likely closing off options that meet library needs. addressing those needs would ease the acceptance of the getftr project as mutually beneficial. so an emphatic “agree!” with lisa on her points in this section. publishers—what were you thinking?

librarian concern: getftr replacing the library link resolver

libraries and library technology companies are making significant investments in tools that ease the path from discovery to delivery. would the library’s link resolver benefit from a real-time api call to a publisher’s service that determines the direct url to a specific doi? oh, yes—that would be mighty beneficial. the library could put that link right at the top of a series of options that include a link to a version of the article in a green open access repository, redirection to a content aggregator, one-click access to an interlibrary-loan form, or even an option where the library purchases a copy of the article on behalf of the patron. (more likely, the link resolver would take the patron right to the article url supplied by getftr, but the library link resolver needs to be in the loop to be able to offer the other options.)

my take-a-ways

the patron is affiliated with the institution, and the institution (through the library) is subscribing to services from the publisher. the institution’s library knows best what options are available to the patron (see above section). want to know why librarians are concerned? because they are inserting themselves as the arbiter of access to content, whether it is in the patron’s best interest or not. it is also useful to reinforce lisa’s closing paragraph:

whether getftr will act to remediate these concerns remains to be seen. in some cases, i would expect that they will. in others, they may not. publishers’ interests are not always aligned with library interests and they may accept a fraying relationship with the library community as the price to pay to pursue their strategic goals.

ian mulvany’s “thoughts on getftr”

ian’s entire post from december th in scholcommsprod is worth reading. i think it is an insightful look at the technology and its implications. here are some specific comments:

clarifying the relation between seamlessaccess and getftr

there are a couple of things that i disagree with:

ok, so what is the difference, for the user, between seamlessaccess and getftr? i think that the difference is the following - with seamless access you the user have to log in to the publisher site. with getftr if you are providing pages that contain dois (like on a discovery service) to your researchers, you can give them links they can click on that have been setup to get those users direct access to the content. that means as a researcher, so long as the discovery service has you as an authenticated user, you don’t need to even think about logins, or publisher access credentials.

to the best of my understanding, this is incorrect. with seamlessaccess, the user is not “logging into the publisher site.” if the publisher site doesn’t know who a user is, the user is bounced back to their institution’s single sign-on service to authenticate. if the publisher site doesn’t know where a user is from, it invokes the seamlessaccess where-are-you-from service to learn which institution’s single sign-on service is appropriate for the user. if a user follows a getftr-supplied link to a publisher site but the user doesn’t have the necessary authentication token from the institution’s single sign-on service, then they will be bounced back for the username/password and redirected to the publisher’s site. getftr signaling that an institution is entitled to view an article does not mean the user can get it without proving that they are a member of the institution.

what does this mean for green open access

a key point that ian raises is this:

one example of how this could suck, lets imagine that there is a very usable green oa version of an article, but the publisher wants to push me to using some “e-reader limited functionality version” that requires an account registration, or god forbid a browser exertion, or desktop app. if the publisher shows only this limited utility version, and not the green version, well that sucks.

oh, yeah…that does suck, and it is because the library—not the publisher of record—is better positioned to know what is best for a particular user.

will getftr be adopted?

ian asks, “will google scholar implement this, will other discovery services do so?” i do wonder if getftr is big enough to attract the attention of google scholar and microsoft research. my gut tells me “no”: i don’t think google and microsoft are going to add getftr buttons to their search results screens unless they are paid a lot. as for google scholar, it is more likely that google would build something like getftr to get the analytics rather than rely on a publisher’s version.

i’m even more doubtful that the companies pushing getftr can convince discovery layers makers to embed getftr into their software. since the two widely adopted discovery layers (in north america, at least) are also aggregators of journal content, i don’t see the discovery-layer/aggregator companies devaluing their product by actively pushing users off their site.

my take-a-ways

it is also useful to reinforce ian’s closing paragraph:

i have two other recommendations for the getftr team. both relate to building trust. first up, don’t list orgs as being on an advisory board, when they are not. secondly it would be great to learn about the team behind the creation of the service. at the moment its all very anonymous.

where do we stand?

wow, i didn’t set out to write , words on this topic. at the start i was just taking some time to review everything that happened since this was announced at the start of december and see what sense i could make of it. it turned into a literature review of sort.

while getftr has some powerful backers, it also has some pretty big blockers:

can getftr help spur adoption of seamless access enough to convince big and small institutions to invest in identity provider infrastructure and single sign-on systems?
will getftr grab the interest of google, google scholar, and microsoft research (where admittedly a lot of article discovery is already happening)?
will developers of discovery layers and link resolvers prioritize getftr implementation in their services?
will libraries find enough value in getftr to enable it in their discovery layers and link resolvers?
would libraries argue against getftr in learning management systems, faculty profile systems, and other campus systems if its own services cannot be included in getftr displays?

i don’t know, but i think it is up to the principles behind getftr to make more inclusive decisions. the next steps is theirs.

- - t : : + : peter murray (jester@dltj.org) pleroma — a lightweight fediverse server documentation api blog news documentation api blog news pleroma free and open communication for everyone. pleroma is social networking software compatible with other fediverse software such as mastodon, misskey, pixelfed and many others. for a friendly introduction to pleroma and the fediverse, check the big pleroma and fediverse faq and read what is pleroma? getting started start using pleroma by joining an existing pleroma instance or check the installation guide to setting up your own server. join an instance installation guide about pleroma our latest release is v . . . pleroma is free software, all development and issue tracking happens over at the project's gitlab instance. there are multiple frontends to use with pleroma to suit all kinds user preferences: pleroma fe, our 'official' highly customizable frontend. soapbox, a simple easy to learn and use alternative. masto fe, a pleroma focused fork of mastodon's multi column frontend. featured instances want to try pleroma out but don't know which one of the many instances to join? here's a short list of public community ran instances with open registration. outerheaven.club stereophonic.space cawfee.club shitposter.club blob.cat fedi.absturztau.be cdrom.tokyo udongein.xyz other helpful resources statistics and configuration of pleroma instances only statistics of pleroma instances #pleroma and #pleroma-dev irc channels on freenode contact you can contact us via email at contact@pleroma.social. software freedom day | nosk sfd , nosk speakers schedule makers venue sponsors register at ncit balkumari, lalitpur september september · at ncit · balkumari, lalitpur let's celebrate software freedom day (sfd). sfd is a public education effort with the aim of increasing awareness of free software and its virtues, and encouraging its use. see the schedule -> register while it lasts hurry! book seat for workshops and tech talks register but wait. what is sfd? celebration of free and open source software software freedom day (sfd) is an annual worldwide celebration of free software. sfd is a public education effort with the aim of increasing awareness of free software and its virtues, and encouraging its use. speakers learn directly from the leaders that have powered one of the most transformational periods in information technology here in nepal. tech talk speakers er. dipesh das manager, gdg birgunj er. kumar pudasaini network engineer sushil kumar sah kantipur media group opening speakers mohan khadka er. nipesh shrestha mentors ashish tiwari independent developer ashmina kattel mobile app developer, makura creations nischal lal shrestha independent standard-based developer ramesh giri mobile app developer, makura creations sagar devkota game developer, time and update saroj maharjan nepal television suman gautam mobile app developer, chaitanya designs umesh basnet mobile app developer, young innovations experience attend talks and workshops on various foss, play games, code, eat and celebrate at the software freedom day. take a look at this small taste of the edition: allowfullscreen venue where is it? ncit balkumari, lalitpur experience the future at one of the best it colleges in nepal. ncit, a pioneer private institution providing engineering education in nepal, is renowned for excellence in teaching & research, while maintaining close and mutually beneficial links with various sectors get directions for ncit · balkumari, lalitpur -> sponsors your generous contribution provides an unique opportunity for over a thousand young men and woman to learn about the benefits of using free and open source software plus celebrate the worldwide event. event organizers supported by bronze sponsor other sponsor register while we welcome every one of you to our event, we are only able to accomodate a few of you to our tech talks and workcamps. so, hurry up now, and register for the activities you're interested in. register at ncit balkumari, lalitpur september september · at ncit · balkumari, lalitpur let's celebrate software freedom day (sfd). sfd is a public education effort with the aim of increasing awareness of free software and its virtues, and encouraging its use. see the schedule -> register while it's still available register speakers venue sponsors schedule register makers of sfd code of conduct facebook github teach with story maps: announcing the story maps curriculum portal | office of the vice president for research skip to content university of minnesota go to the u of m home page one stop myu search search submit search query office of the vice president for research u-spatial news events the spatial university contact us covid- guidance for the research community about us overview staff list contact us training u-spatial training geospatial skills badge - storytelling help desk software resources spatial data teach with story maps gis courses at umn esri innovation program (eip) esri conference session videos services hazard mitigation planning acknowledging u-spatial mapping prize overview best maps best maps best maps best maps best maps best maps best maps menu close about us overview staff list contact us training u-spatial training geospatial skills badge - storytelling help desk software resources spatial data teach with story maps gis courses at umn esri innovation program (eip) esri conference session videos services hazard mitigation planning acknowledging u-spatial mapping prize overview best maps best maps best maps best maps best maps best maps best maps news events the spatial university contact us one stop myu you are here u-spatial home » news » teach with story maps: announcing the story maps curriculum portal teach with story maps: announcing the story maps curriculum portal feb , u-spatial is excited to announce the recent launch of the story maps curriculum portal, a site that provides pedagogical materials for university classes working with esri story maps. this portal is designed to provide tools for both instructors and students who want to work with story maps, but who do not have an extensive background in gis or digital projects more generally. its resources range from assignment templates, to short how to’s, to exemplary student work. story_maps_image.jpg over the last few years, there has been an increased interest in story maps from instructors and students who would normally never encounter gis in their disciplines. since the platform offers a simple and compelling way to engage with spatial thinking, story maps has been particularly popular among those seeking to engage in the digital humanities. yet, many instructors and students alike have remained intimidated by gis, which has prompted the need for accessible resources to assist with the implementation of story maps in the classroom. this need was addressed by the efforts of a team of researchers and educators from across the university of minnesota, whose work has culminated in this site. with the assistance of an academic innovation grant from the college of liberal arts, this team worked with a variety of instructors to develop resources to enable them to teach with story maps. this work has already paid off, with at least courses across cla running story map assignments this spring semester. if you or your colleagues are interested in using story maps in the classroom, check out http://storymaps.umn.edu/ or contact u-spatial to get connected with the story maps team. the umn story maps curriculum team sarah chambers, phd - cla innovation grant pi, department of history faculty chris saladin - history phd student, graduate research assistant shana crosson - academic technologist, liberal arts technologies and innovation services (latis) kate carlson - spatial technology consultant and training coordinator, u-spatial melinda kernik - spatial data analyst and curator, umn libraries len kne - associate director, u-spatial ben wiggins - program director, digital arts, sciences, & humanities (dash) subscribe to ovpr's inquiry newsletter email address * leave this field blank u-spatial blegen hall th ave. s minneapolis, mn email: uspatial@umn.edu phone: ( ) - office of the vice president for research johnston hall pleasant st. se minneapolis, mn email: research@umn.edu phone: ( ) - ovpr home ovpr department directory website feedback/questions maps & directions parking & transportation last modified: february , - : am. back to top © regents of the university of minnesota. all rights reserved. the university of minnesota is an equal opportunity educator and employer. privacy statement report accessibility concerns archival connections archival connections project site platform monopolies and archives i am at the interpares trust north american team meeting in vancouver, and the issue of platform monopolies has risen to the top of my mind. here is a quick list of readings i&# ;ve thrown together while listening to and engaging in the discussion: for now, i don&# ;t have much to say, other than this: as a &# ; continue reading platform monopolies and archives sia workshop links just sharing a few links for use during the sia workshop i&# ;ll be teaching later today: google form for exercises sia workshop slides scaling machine-assisted description of historical records one of the questions i&# ;ve been grappling with as part of the archival connections research project is simple: is there a future for the finding aid? i&# ;m inclined to think not, at least not in the form we are used to. looking to the future, i recently had the chance to propose something slightly different, and &# ; continue reading scaling machine-assisted description of historical records social feed manager takeaways later this week, i&# ;ll be introducing the archival connections project at the society of indiana archivists meeting. during the first year of this project, one focus of my work was evaluating and developing some recommendations for using social feed manager, a tool developed by george washington university libraries. my full report is here, for those interested: https://gwu-libraries.github.io/sfm-ui/resources/sfmreportprom .pdf. without &# ; continue reading social feed manager takeaways arrangement and description in the cloud: a preliminary analysis i&# ;m posting a preprint of some early work related to the archival connections project. this work will be published as a book chapter/proceedings by the archiveschule in marburg. in the meantime, here is the preprint: archival arrangement and description in the cloud a preliminary analysis installing social feed manager locally the easiest way to get started with social feed manager is to install docker on a local machine, such as a laptop or (preferably) desktop computer with a persistent internet connection. running sfm locally for anything other than testing purposes is not recommended. it will not be sufficient for a long-term documentation project and would &# ; continue reading installing social feed manager locally preserving email report summary earlier today, i provided a summary of preserving email, a technology watch report i wrote back in . i'll leave it to others to judge how well that report holds up, but i had the following takeaways when re-reading it: introducing archival connections welcome! this shares information from a five-year research project that i am coordinating at the university of illinois at urbana-champaign. the project aims to make it easier for people to find and use the materials managed by archival repositories like the university of illinois archives, where i work. you can read more about the project on the &# ; continue reading introducing archival connections the founding fathers of the web we're using cookies to improve your experience. find out more. hidden main menu item mashable video entertainment movies gaming television culture web culture sex & relationships celebrities memes parenting social media tech business apps gadgets reviews mobile smart home how to mashable choice science climate space social good lgbtq feminism gender equality activism non-profits amplify shop tech vpn headphones speakers laptops web hosting antivirus lifestyle black friday home kitchen gift guides gaming culture dating pets subscription boxes carry on best of tech best vpn best cheap vpn best streaming services best cheap laptops best running headphones best bluetooth speakers best of culture best dating sites best free dating sites best dating sites for introverts best dna tests best dog dna tests best subscription boxes best of lifestyle best airfryer best cordless vacuum best instant pot best gifts under $ best robot vacuums best vacuum for pet hair black friday search more channels video entertainment culture tech science social good amplify company masthead licensing & reprints archive mashable careers contact contact us submit news mashable shop advertise advertise adchoices legal privacy policy terms of use cookie policy accessibility statement do not sell my personal information resources travel security how to mashable deals gift guides sites job board social good summit international mashable australia mashable benelux mashable india mashable italia mashable me mashable pakistan mashable se asia mashable uk entertainment like follow the founding fathers of the web by christina warren - - : : utc while the phrase "founding fathers" is often used in conjunction with men like benjamin franklin, thomas jefferson and george washington, we wanted the think about the phrase on the global level. and what is more global than the world wide web? thus, this holiday, we're taking a look at individuals who have been instrumental in helping to shape the world wide web and the culture of the internet as we know it today. check out our round up below to learn about some of the most influential people in the creation and development of the ideas and technologies that have led to today's web experience. let us know in the comments if you think we've missed anyone! . tim berners-lee why he matters: tim berners-lee is credited as the inventor of the world wide web. a physicist, berners-lee and his team built the world's very first web browser, worldwideweb, the first web server and the hypertext-based markup language html. berners-lee founded and is the current director of the world wide web consortium (w c), a standards body that oversees the development of the web as a whole. while the internet itself dates back , it was berners-lee who was able to bring together the concept of the internet and hypertext, which set the foundation for the internet as we know it today. because cern (the european organization for nuclear research) didn't make the world wide web proprietary and never charged for dues, its protocols were widely adopted. . marc andreessen why he matters: marc andreessen co-authored mosaic, the first widely-used web browser and he founded netscape communications. while mosaic wasn't the first graphical web browser, it was the first to garner significant attention. it was also the first browser to display images inline with text. after designing and programing mosaic, andreessen went on to co-found netscape communications. netscape's flagship product, netscape navigator, had an enormous impact, by helping to bring the web to mainstream users. in , netscape released the code base for netscape communicator under an open source license. that project, known as mozilla, became the basis of what we now know as firefox. . brian behlendorf why he matters: brian behlendorf was the primary developer of the apache web server and one of the founding members of the apache group. while working as the webmaster for wired magazines's hotwired web site, behlendorf found himself making changes and patches to the http server first developed at ncsa at the university of illinois at urbana-champaign. after realizing that others were also adding their own patches, he put together an electronic mailing list to help coordinate the work. by february , the project had been given a name - apache - and the entire codebase from the original ncsa server was rewritten and re-optimized. the real genius with apache, other than its free and open source nature, was that it was built to be extensible. that meant that isps could easily add their own extensions or plugins to better optimize the server, allowing hundreds of sites to be hosted from just one computer server. apache remains the most popular web server on the internet. , , . rasmus lerdorf, andi gutmans and zeev suraski why they matter: lerdorf, gutmans and suraski are all responsible for what we know as php, the scripting language that remains one of the most used web languages for creating dynamic web pages. rasmus lerdorf first created php in and he was the main developer of the project for its first two versions. in , gutmans and suraski decided to extend php, rewriting the parser and creating what became known as php . the two then went on to rewrite the core of php, naming it the zend engine, and using that to power php . gutmans and suraski further went on to found zend technologies, which continues to do much of the development of php. while larry wall's perl was one of the first general-purpose scripting languages to really take off on the web, the ease of use and embedability of php is what has made it take over as the defacto "p" in the lamp stack (lamp being a default set of components on which many web applications are based). . brad fitzpatrick why he matters: creator of livejournal, in many ways the proto-social network, the original author of memcached and the original authentication protocol for openid. fitzpatrick created livejournal in college, as a way for he and his friends to keep one another up to date with what they were doing. it evolved into a larger blogging community and implemented many features, like friends lists, the ability to create user polls, support for blog clients, the ability to send text messages to users, the ability to post by phone, post by e-mail, create group blogs and more that have become a standard part of communities like facebook, tumblr, myspace, wordpress.com and posterous today. as livejournal grew and started to use more and more resources, fitzpatrick started the memcached project as a way to speed up dynamic web applications and alleviate database load. it does this by pooling together the free memory from across your web servers and then allocate it out as needed. this makes it easy for large projects to scale. memcached is in use by wikipedia, flickr, facebook, wordpress, twitter, craigslist and more. . brendan eich why he matters: he created javascript and now serves as the cto of the mozilla corporation. eich created javascript while at netscape, first under the name mocha, then under the name livescript, and finally as javascript. javascript made its official debut in december of . javascript quickly became one of the most popular web programming languages, even if its use cases in the early days were often visual abominations. however, as time has progressed, the advent of javascript libraries and frameworks, coupled with the power of ajax has made javascript an integral part of the standards-based web. . john resig why he matters: john resig is the creator and lead developer of jquery, the most popular javascript library on the web. while other javascript libraries, such as sam stephenson's protoype, preceded jquery, jquery's goal of being compatible across web browsers is what really sets it apart. in the last two years especially, the momentum around jquery has exploded and it is now reportedly in use by % of the top , most visited websites. it's extensibility and the jquery ui toolkit has also made it a popular adoption target in enterprise application development. any javascript library that can make the leap from web developers to enterprise app builders is the real deal. javascript continues to be one of the big forces within the standards-based web and jquery is helping to lead the charge. . jonathan gay why he matters: he co-founded futurewave software and for more than a decade was the main programmer and visionary behind flash. while not everyone is a fan of adobe flash, it's important to remember how influential and instrumental the technology has been over the course of the last years. gay wrote a vector drawing program called smartsketch back in for the penpoint operating system, and after penpoint was discontinued, the technology in smartsketch was repurposed as a tool that could create animation that could be played back on web pages. this product, futuresplash animator, was acquired by macromedia in and renamed flash. after the acquisition, gay became vice president of engineering at macromedia and he led the flash engineering team. over the years, his team implemented new elements to flash, like actionscript. however, perhaps gay's pinnacle achievement with flash was in the team he spearheaded to create what was then known as the flash communication server (it's now the flash media server) which let flash player use the rtmp protocol to stream audio and video over the web. in essence, this technology is what allowed youtube to be, well, youtube. more development and design resources from mashable: - top resources for design inspiration - how to: get up-to-date on wordpress . - hackathons around the world and the web - web design bloggers you should follow - top beautiful minimalist icon sets [img credits: european parliament, marc andreessen, ilya schurov, chrys/sebastian bergmann, crucially, jsconf, badubadu] topics: brendan eich, dev & design, founding fathers, john resig, marc andreessen, rasmus lerdorf, social media, web development, world wide web masthead jobs advertise mashable shop contact privacy terms facebook mashable twitter mashable feeds mashable pinterest mashable youtube mashable stumbleupon mashable linkedin mashable better business bureau accredited business is a global, multi-platform media and entertainment company. powered by its own proprietary technology, mashable is the go-to source for tech, digital culture and entertainment content for its dedicated and influential audience around the globe. © mashable, inc. all rights reserved. mashable, mashbash and mashable house are among the federally registered trademarks of ziff davis, llc and may not be used by third parties without explicit permission. twarc toggle navigation inkdroid about bookmarks photos music software social talks twarc april , python twitter this post was originally published on medium but i spent time writing it so i wanted to have it here too. tl;dr twarc has been redesigned from the ground up to work with the new twitter v api and their academic research track. many thanks for the code and design contributions of betsy alpert, igor brigadir, sam hames, jeff sauer, and daniel verdeer that have made twarc possible, as well as early feedback from dan kerchner, shane lin, miles mccain, 李荣蓬, david thiel, melanie walsh and laura wrubel. extra special thanks to the institute for future environments at queensland university of technology for supporting betsy and sam in their work, and for the continued support of the mellon foundation. back in august of last year twitter announced early access to their new v api, and their plans to sunset the v . api that has been active for almost the last years. over the lifetime of their v . api twitter has become deeply embedded in the media landscape. as magazines, newspapers and television have moved onto the web they have increasingly adopted tweets as a mechanism for citing politicians, celebrities and organizations, while also using them to document current events, generate leads and gather feedback for evolving stories. as a result twitter has also become a popular object of study for humanities and social science researchers looking to understand the world as reflected, refracted and distorted by/in social media. on the surface the v api update seems pretty insignificant since the shape of a tweet, its parts, properties and affordances, aren’t changing at all. tweets with characters of text, images and video will continue to be posted, retweeted and quoted. however behind the scenes the representation of a tweet as data, and the quotas that control the rates at which this data can flow between apps and other third party services will be greatly transformed. needless to say, v represents a big change for the documenting the now project. along with community members we’ve developed and maintained open source tools like twarc that talk directly to the twitter api to help users to search for and collect live tweets that match criteria like hashtags, names and geographic locations. today we’re excited to announce the release of twarc v which has been designed from the ground up to work with the v api and twitter’s new academic research track. clearly it’s extremely problematic having a multi-national corporation act as a gatekeeper for who counts as an academic researcher, and what constitutes academic research. we need look no further than the recent experiences of timnit gebru and margaret mitchell at google for an example of what happens when research questions run up against the business objectives of capital. we only know their stories because gebru and mitchell’s bravely took a principled approach, where many researchers would have knowingly or unknowingly shaped their research to better fit the needs of the company. so it is important for us that twarc still be usable by people with and without access to the academic research track. but we have heard from many users that the academic research track presents new opportunities for twitter data collection that are essential for researchers interested in the observability of social media platforms. twitter is making a good faith effort to work with the academic research community, and we thought twarc should support it, even if big challenges lie ahead. so why are people interested in the academic research track? once your application has been approved you are able to collect data from the full history of tweets, at no cost. this is a massive improvement over the v . access which was limited to a one week window and researchers had to pay for access. access to the full archive means it’s now possible to study events that have happened in the past back to the beginning of twitter in . if you do create any historical datasets we’d love for you to share the tweet identifier datasets in the catalog. however this opening up of access on the one hand comes with a simultaneous contraction in terms of how much data can be collected at one time. the remainder of this post describes some of the details and the design decisions we have made with twarc to address them. if you would prefer to watch a quick introduction to using twarc v please check out this short video: installation if you are familiar with installing twarc nothing is changed. you still install (or upgrade) with pip as you did before: $ pip install --upgrade twarc in fact you will still have full access to the v . api just as you did before. so the old commands will continue to work as they did $ twarc search blacklivesmatter > tweets.jsonl twarc was designed to let you to continue to use twitter’s v . api undisturbed until it is finally turned off by twitter, at which point the functionality will be removed from twarc. all the support for the v api is mediated by a new command line utility twarc . for example to search for blacklivesmatter tweets and write them to a file tweets.jsonl: $ twarc search blacklivesmatter > tweets.jsonl all the usual twarc functionality such as searching for tweets, collecting live tweets from the streaming api endpoint, requesting user timelines and user metadata are all still there, twarc --help gives you the details. but while the interface looks the same there’s quite a bit different going on behind the scenes. representation truth be told, there is no shortage of open source libraries and tools for interacting with the twitter api. in the past twarc has made a bit of a name for itself by catering to a niche group of users who want a reliable, programmable way to collect the canonical json representation of a tweet. javascript object notation (json) is the language of web apis, and twitter has kept its json representation of a tweet relatively stable over the years. rather than making lots of decisions about the many ways you might want to collect, model and analyze tweets twarc has tried to do one thing and do it well (data collection) and get out of the way so that you can use (or create) the tools for putting this data to use. but the json representation of a tweet in the twitter v api is completely burst apart. the v base representation of a tweet is extremely lean and minimal, and just includes the text of the tweet its identifier and a handful of other things. all the details about the user who created the tweet, embedded media, and more are not included. fortunately this information is still available, but the user needs to craft their api request to request tweets using a set of expansions that tell the twitter api what additional entities to include. in addition for each expansion there are a set of field options to include that control what of these expansions is returned. so rather than there being a single json representation of a tweet api users now have the ability to shape the data based on what they need, much like how graphql apis work. this kind of makes you wonder why twitter didn’t make their graphql api available. for specific use cases this customizability is very useful, but the mutability of the representation of a tweet presents challenges when collecting data for future use. if you didn’t request the right expansions or fields when collecting the data then you won’t be able to analyze that data later when doing your research. to solve for this twarc has been designed to collect the richest possible representation for a tweet, by requesting all possible expansions and field combinations for tweets. see the expansions module for the details if you are interested. this takes a significant burden off of users to digest the api documentation, and craft the correct api requests themselves. in addition the twarc community will be monitoring the twitter api documentation going forward to incorporate new expansions and fields as they will inevitably be added in the future. flattening this is diving into the weeds a little bit, but it’s worth noting here that twitter’s introduction of expansions allows data that was once duplicated across multiple tweets (such as user information, media, retweets, etc) to be included once per response from the api. this means that instead of seeing information about the user who created a tweet in the context of their tweet the user will be referenced using an identifier, and this identifier will map to user metadata in the outer envelope of the response. it makes sense why twitter have introduced expansions since it means in a set of tweets from a given user the user information will just be included once rather than repeated times, which means less data, less network traffic and less money. it’s even more significant when consider the large number of possible expansions. however this pass by-reference rather than by-value presents some challenges for stream based processing which expects each tweet to be self-contained. for this reason we’ve introduce the idea of flattening the response data when persisting the json to disk. this means that tools and data pipelines that expect to operate on a stream of tweets can continue to do so. since the representation of a tweet is so dependent on how data is requested we’ve taken the opportunity to introduce a small stanza of twarc specific metadata using the __twarc prefix. this metadata records what api endpoint the data was requested from, and when. this information is critically important when interpreting the data, because some information about a tweet like its retweet and quote counts are constantly changing. data flows as mentioned above you can still collect tweets from the search and streaming api endpoints in a way that seems quite similar to the v api. the big changes however are the quotas associated with these endpoints which govern how much can be collected. these quotas control how many requests can be sent to twitter in minute intervals. in fact these quotas are not much changed, but what’s new are app wide quotas that constrain how many tweets a given application (app) can collect every month. an app in this context is a piece of software (e.g. your twarc software) identified by unique api keys set up in the twitter developer portal. the standard api access sets a , tweet per month limit. this is a huge change considering there were no monthly app limits before. if you get approved for the academic research track your app quota is increased to million per month. this is markedly better but the achievable data volume is still nothing like the v . api, as these graphs attempt to illustrate: twarc will still observe the same rate limits, but once you’ve collected your portion for the month there’s not much that can be done, for that app at least. apart from the quotas twitter’s streaming endpoint in v is substantially changed which impacts how users interact with twarc. previously twarc users would be able to create up to to two connections to the filter stream api. this could be done by simply: twarc filter obama > obama.jsonl however in the twitter v api only apps can connect to the filter stream, and they can only connect once. at first this seems like a major limitation but rather than creating a connection per query the v api allows you to build a set of rules for tweets to match, which in turns controls what tweets are included in the stream. this means you can collect for multiple types of queries at the same time, and the tweets will come back with a piece of metadata indicating what rule caused its inclusion. this translates into a markedly different set of interactions at the command line for collecting from the stream where you first need to set your stream rules and then open a connection to fetch it. twarc stream-rules add blacklivesmatter twarc stream > tweets.jsonl one useful side effect of this is that you can update the stream (add and remove rules) while the stream is in motion: twarc stream-rules add blm while you are limited by the api quota in terms of how many tweets you can collect, tweets are not “dropped on the floor” when the volume gets too high. once upon a time the v . filter stream was rumored to be rate limited when your stream exceeds % of the total volume of new tweets. plugins in addition to twarc helping you collect tweets the github repository has also been a place to collect a set of utilities for working with the data. for example there are scripts for extracting and unshortening urls, identifying suspended/deleted content, extracting videos, buiding wordclouds, putting tweets on maps, displaying network graph visualizations, counting hashtags, and more. these utilities all work like unix filters where the input is a stream of tweets and the output varies depending on what the utility is doing, e.g. a gephi file for a network visualization, or a folder of mp files for video extraction. while this has worked well in general the kitchen sink approach has been difficult to manage from a configuration management perspective. users have to download these scripts manually from github or by cloning the repository. for some users this is fine, but it’s a bit of a barrier to entry for users who have just installed twarc with pip. furthermore these plugins often have their own dependencies which twarc itself does not. this lets twarc can stay pretty lean, and things like youtube_dl, networkx or pandas can be installed by people that want to use utilities that need them. but since there is no way to install the utilities there isn’t a way to ensure that the dependencies are installed, which can lead to users needing to diagnose missing libraries themselves. finally the plugins have typically lacked their own tests. twarc’s test suite has really helped us track changes to the twitter api and to make sure that it continues to operate properly as new functionality has been added. but nothing like this has existed for the utilities. we’ve noticed that over time some of them need updating. also their command line arguments have drifted over time which can lead to some inconsistencies in how they are used. so with twarc we’ve introduced the idea of plugins which extend the functionality of the twarc command, are distributed on pypi separately from twarc, and exist in their own github repositories where they can be developed and tested independently of twarc itself. this is all achieved through twarc ’s use of the click library and specifically click-plugins. so now if you would like to convert your collected tweets to csv you can install the twarc-csv: $ pip install twarc-csv $ twarc search covid > covid .jsonl $ twarc csv covid .jsonl > covid .csv or if you want to extract embedded and referenced videos from tweets you can install twarc-videos which will write all the videos to a directory: $ pip install twarc-videos $ twarc videos covid .jsonl --download-dir covid -videos you can write these plugins yourself and release them as needed. check out the plugin reference implementation tweet-ids for a simple example to adapt. we’re still in the process of porting some of the most useful utilities over and would love to see ideas for new plugins. check out the current list of twarc plugins and use the twarc issue tracker on github to join the discussion. you may notice from the list of plugins that twarc now (finally) has documentation on readthedocs external from the documentation that was previously only available on github. we got by with github’s rendering of markdown documents for a while, but github’s boilerplate designed for developers can prove to be quite confusing for users who aren’t used to selectively ignoring it. readthedocs allows us to manage the command line and api documentation for twarc, and to showcase the work that has gone into the spanish, japanese, portuguese, swedish, swahili and chinese translations. feedback thanks for reading this far! we hope you will give twarc a try. let us know what you think either in comments here, in the docnow slack or over on github. ✨ ✨ happy twarcing! ✨ ✨ ✨ windows users will want to indicate the output file using a second argument rather than redirecting output with >. see this page for details.↩ unless otherwise noted all the content here is licensed cc-by posts on mark a. matienzo posts on mark a. matienzo recent content in posts on mark a. matienzo iah forecast - disquiet junto project an experiment with recording a new single using vcv rack and reaper based on a compositional prompt. i ended up recording two tracks. perfecting a favorite: oatmeal chocolate chip cookies i have a horrible sweet tooth, and i absolutely love oatmeal chocolate chip cookies. i tend to bake as a means to cope with stress, and of course, more often then that means making these cookies. after making many iterations, i’ve settled upon this recipe as the ultimate version to which all compare. in memoriam and appreciation of rob casson ( - ) the world lost one of its brightest and most charming lights earlier this week, rob casson. many of us knew rob through the code lib community and conferences and his work at miami university libraries. we miss his generosity, patience, sense of humor, and genuine kindness. those of us who got the chance to socialize with him also remember his passion for music, and some of us were even lucky to see live shows in the evenings between conference sessions and other social activities. on sunday, october at : pm pacific/ : pm eastern, those of us who knew him through code lib and the world of libraries are encouraged to gather to share our memories of him and to appreciate his life and work. please join me and my co-organizers, mike giarlo and declan fleming on zoom (registration required). robert casson (robcasson), jan - sep . photo: declan fleming. first sota activation about a month ago, i got my ham radio license, and soon after i got pretty curious about summits on the air (sota), an award scheme focused on safe and low impact portable operation from mountaintops. while i like to hike, i’m arguably a pretty casual hiker, and living in california provides a surprising number of options within minutes driving time for sota newbies. optimizing friction over and in response to the last few months, i’ve been reflecting about intentionality, and how i spend my time creating things. i have tried to improve the indiewebbiness of my site, and understanding what it means to “scratch my own itch”. this resonates particularly lately because it’s leading me to mull over which parts should be hard and easy. unsurprisingly, much of that is personal preference, and figuring out how i want to optimize from the perspective of user experience. friction in ux can be a powerful tool, part of what i’m trying to find is where i want to retain friction as it helps me remain intentional. a hugo shortcode for embedding mirador i spent a little time over the last day or so trying to bodge together a shortcode for hugo to embed an instance of mirador. while it’s not quite as simple (or full-featured) as i’d like, it’s nonetheless a starting point. the shortcode generates a snippet of html that gets loaded into hugo pages, but (unfortunately) most of the heavy lifting is done by a separate static page that gets included as an <iframe/> within the page. that page parses url parameters to pass some of the parameters when mirador gets instantiated. getting a consistent way to load multiple iiif manifests, either into comparison view or for populating a resource list also needs some work, which also led me to grapple with thinking through the iiif content state api spec, which will require some more attention, too. besieged i have spent the last four and a half months feeling like everything is slipping from my grasp – personally, professionally, and in between. the torpor of life under a pandemic and a world wracked with pain has led me to feel like i am stuck in slowly-drying glue. planning too far ahead seems nearly pointless. and yet, every day, we are asked to undertake haruspicy, to speculate about how our organizations and ourselves should respond to the remaining uncertainty, ideally with precision. the world keeps turning and we are asked to keep up, while taking care of family members, grieving our losses, or dealing with other challenges amplified by the present circumstances. at the same time, i feel myself slowing down, or at least to continue trying to slow down. i have not read anything more substantial than an article since february, despite getting a stack of books out of the library in preparation for more time at home. the cognitive load of mailing packages can sometimes be too much. comments on revisions to saa statement on diversity and inclusion the saa council has issued a call for comments on the saa statement on diversity and inclusion. as noted in the announcement, the revision includes changes to expand the statement to cover equity as well. comments are open on the revisions until march , , and what follows are the comments that i’ve submitted. books read, january-february i’m trying to do a better job tracking what i’ve been reading. here’s a start. solidarity, logistics, and infrastructure on prime day july and th are “prime day,” amazon’s attempt to drive up sales and artificial demand around things we don’t need at prices they’ve convinced us that we can afford. thanks to mar hicks, many of us heard that workers at a shakopee, minnesota fulfillment center are holding a six-hour work stoppage on one of the busiest days of the year. alongside, many have called for a boycott on amazon and its subsidiaries (whole foods, goodreads, twitch, etc.), and others have called for a general strike to protest amazon’s collaboration with palantir in aiding ice. with all of this in mind, i’ve been reflecting on what larger scale industrial actions could look like when we look at amazon’s simultaneous leveraging of centralization and unreliability of single resources to provide critical infrastructure for the it sector and its own operations. : a year in gratitude this year was largely complicated and often felt like a massive garbage fire to myself and my crew. i didn’t accomplish a number of my goals and was inconsistent about others, so recapping awesome things i did doesn’t feel appropriate and also happens to be a soft reminder of either failure or things not going as planned. i also tend to hate “best of the year” lists but i find them helpful to remember about where i found joy or the ability to connect to something outside of myself. i suppose this is an attempt to reconcile those things, or perhaps more in line with the end of year spirit, a way to articulate gratitude to the people and things around me that impacted me. when basil has gone to seed: contemplative pesto we are growing three kinds of basil in our garden: “regular” basil, purple basil, and magic mountain basil. the regular basil and magic mountain basil have been thriving quite a bit; the purple basil, less so, as it is growing at the base of the regular basil plant. but the other two, my goodness. the regular old basil was going to seed, though, much to the chagrin of my partner. i’d promised for weeks on end to do something with all that basil, as the stems grew woodier, and as the flowers turned from brilliant white to the brown of kraft paper. meanwhile, the magic mountain basil also grew tall and bushy, went to flower, but only because that’s what it’s supposed to do. evidence of them: digitization, preservation, and labor this is a lightly edited version of the presentation i gave as part of as a part of session : digitization is/not preservation at the society of american archivists annual meeting. the session was overall pure fire, with thoughtful, funny, provocative, and challenging presentations by julia kim, frances harrell, tre berney, andrew robb, snowden becker, fletcher durant, siobhan hagan, and sarah werner. my heart goes out to all of them. all of the images used in the presentation were adapted from the art of google books. what one says and does not say: vulnerability, leadership, and professional trajectories an extended reflection on professional trajectories, leadership, vulnerability, community, and finding my voice, written as part of my participation in the it leadership program. beyond hearing (one another): radical empathy in archives-as-workplace i am writing this amidst being crammed into a seat flying back from new york city, after a few days of intensive meetings. between a number of good and less ideal things, my mind has felt really unsettled lately, and i’m working through some professional malaise, and feeling a bit rudderless. in an attempt to give myself something be myself optimistic about and to set some direction, i reread michelle caswell and marika cifor’s archivaria article “from human rights to feminist ethics: radical empathy in archives”. part of their analysis outlines four affective shifts in archival relationships based on radical empathy - those between ) archivist and records creator, ) archivist and records subject, ) archivist and user, and ) archivist and larger community. given a long list of topics on my mind (precarity, developing inclusive workplaces and cultures, my own uncertain pathway), it felt like there was plenty of space to identify other shifts. sending websub notifications from static sites using netlify functions as part of my iterative intentions for , i started a project to rebuild and simplify my website. i’ve used jekyll for quite some time (either by itself or with octopress), and as part of the latest iteration of the site, i’ve been working to align the site more with indieweb principles, and to smooth the deployment path for my site by hosting it on netlify. one challenge with jekyll and other static site generators is that “dynamic-ish” functionality, including sending notifications through protocols like websub. the trouble is knowing where these actions fit into the build process for your site: you don’t want to send the notifications before your site gets built, or pushed to the cdn hosting your site. recently, netlify announced a private beta for its new netlify functions service, which provides lambda-style functions deployed as part of your site deployment. one of the neat features that exists as of the beta is the ability to trigger the functions via netlify events, like when your site successfully deploys. notes on itlp workshop readings i completed my reading and viewing assignments for my cohort’s it leadership program workshop (january -january at uc berkeley.) this is a brief set of notes for my own use about how all of them tie together. iterative intentions for while i enjoy seeing what my friends are setting their intentions towards in the new year, i don’t really believe in new year’s resolutions for myself. they tend to wear on me heavily whenever i’ve proclaimed a long list of things i’m hoping to get better at. instead, this year, i’m starting with a very short list. my hope is that i can commit to a small number of good habits at a time, which i can then build on iteratively. i want to have the windows of reinforcement stay small at first (maybe a week or two), and once i feel satisfied about whichever habits i’ve committed to, i can add more. i’m starting with three items: rebuilding this website: simplified tooling; new layout/style; using and publishing more structured data, and a partial implementation of a stack following indieweb and solid principles. the last part is intentionally slippery, but i mostly really care about sending and receiving notifications at this point. a push-to-talk conference call foot pedal my current position at dpla, especially since we are remote-first organization, requires me to be on lots of conference calls, both video and audio. while i’ve learned the value of staying muted while i’m not talking, there are a couple of things that make this challenging. first, i usually need the window for the call to have focus to unmute myself by the platform’s designated keystroke. forget that working well if you need to bring something up in another window, or switch to another application. secondly, while we have our own preferred platform internally (google hangouts), i have to use countless others, too; each of those platforms has its own separate keystroke to mute. this all leads to a less than ideal situation, and naturally, i figured there must be a better way. how we work: the dpla technology team core values one of the most important aspects of the work of the dpla technology team is ensuring that we maintain a common frame of reference for all of our efforts. this is situated in multiple aspects - in terms of our shared technical knowledge, the overall dpla strategic plan, and more. overall, however, the guiding principles for our work are best understood through the core values that inform how we work together within our team, as well as with our colleagues at dpla and across the network of our stakeholders and collaborators. these values are not only designed to be aspirational; instead, they also inform practical aspects of our day to day work, allowing us to work together effectively through their articulation of cultural norms and expectations. in addition, our values encourage us to be intentional about our work, even when faced with challenges from deadlines, staff capacity, and other external pressures. open, free, and secure to all: dpla launches full support for https dpla is pleased to announce that the entirety of our website, including our portal, exhibitions, primary source sets, and our api, are now accessible using https by default. dpla takes user privacy seriously, and the infrastructural changes that we have made to support https allows us to extend this dedication further and become signatories of the library digital privacy pledge of - , developed by our colleagues at the library freedom project. dpla and the international image interoperability framework dpla, along with representatives of a number of institutions including stanford university, the yale center for british art, the bibliothèque nationale de france, and more, is presenting at access to the world’s images, a series of events related to the international image interoperability framework (iiif) in new york city, hosted by the museum of modern art and the new york academy of medicine. the events will showcase how institutions are leveraging iiif to reduce total cost and time to deploy image delivery solutions, while simultaneously improving end user experience with a new host of rich and dynamic features, and promote collaboration within the iiif community through facilitated conversations and working group meetings. ever to excel: towards an apologetics of the spreadsheet this is the written version of my presentation from code lib in philadelphia, on march , . my presentation was part of a panel with my friends christina harlow, ted lawless, and matt zumwalt, after which we had some discussion moderated by matt miller. my slides are available, as are the video of all talks from the panel. my jekyll todo list a running list of things i want to do or have done. a lot of this relates to adopting the indieweb ethos indiewebcamp nyc i’m at indiewebcamp nyc and i just added some microformats data to my site. hurrah! edit: and i’ve successfully sent a webmention by hand from the command line. time to add that to the jekyll build process… developing and implementing a technical framework for interoperable rights statements within the technical working group of the interoperability-working-on-rights/">international rights statements working group, we have been focusing our efforts on identifying a set of requirements and a technically sound and sustainable plan to implement the rights statements under development. now that two of the working group’s white papers have been released, we realized it was a good time to build on the introductory blog post by our co-chairs, emily gore and paul keller. accordingly, we hope this post provides a good introduction to our technical white paper, recommendations for the technical infrastructure for standardized international rights statements, and more generally, how our thinking has changed throughout the activities of the working group. dplafest attendees: support lgbtq youth in indiana! this is a joint blog post by dplafest attendees benjamin armintor and christina harlow, and dpla staff members mark matienzo and tom johnson. after the passage of sea (the indiana religious freedom restoration act), many scheduled attendees of dplafest were conflicted about its location in indianapolis. emily gore, dpla director for content, captured both this conflict and the opportunity the location provides when she wrote: we should want to support our hosts and the businesses in indianapolis who are standing up against this law… at dplafest, we will also have visible ways to show that we are against this kind of discrimination, including enshrining our values in our code of conduct. we encourage you to use this as an opportunity to let your voice and your dollars speak. as dplafest attendees, patronizing businesses identifying themselves with open for service is an important start, but some of us wanted to do more. during our visit to indianapolis, we are donating money to local charities supporting the communities and values that sea threatens. profit & pleasure in goat keeping two weeks ago, we officially announced the initial release of krikri, our new metadata aggregation, mapping, and enrichment toolkit. in light of its importance, we would like to take a moment for a more informal introduction to the newest members of dpla’s herd. krikri and heiðrún (a.k.a. heidrun; pronounced like hey-droon) are key to many of dpla’s plans and serve as a critical piece of infrastructure for dpla. they are also names for, or types, of goats. what dpla and dlf can learn from code lib this post has been crossposted to the digital library federation blog. code lib was held last week from february - , in portland, oregon. the code lib conferences have grown in the last ten years, both in terms of size and scope of topics. this growth is particularly impressive when you consider that much of the work of organizing the conference falls upon a circulating group of volunteers, with additional organizational support from organizations like the digital library federation. it has become clear to me that the code lib community is interested in ensuring that it can develop and support compelling and useful conferences for everyone who chooses to participate. a helping hand: free software and the dpla as you probably know, dpla is committed to making cultural heritage materials held in america's libraries, archives, and museums freely available to all, and we provide maximally open data to encourage transformative uses of those materials by developers. in addition, dpla is also proud to distribute the software we produce to support our mission to the wider community. the greatest adventure with apologies to rankin/bass and glenn yarbrough, the greatest adventure is what lies ahead. after almost four great years working for manuscripts and archives at the yale university library and two and a half rewarding years as the technical architect on archivesspace, i am excited to announce that i&# ;ve accepted a position as the director of technology for the digital public library of america, a small but well-supported non-profit dedicated to free and open access to cultural heritage materials. more information about my new position can be found in the press release. while i am sad to be leaving a great institution and a great project, both with fantastic colleagues, i look forward to contributing my time, energy and expertise to the addressing the huge challenges and encouraging the exciting possibilities of dpla. if you&# ;d like to join me in this adventure, i&# ;m also happy to announce that dpla will be hiring two technology specialists very soon, so if you&# ;re interested or have any questions, please don&# ;t hesitate to contact me! computer anonymous new york in my previous post, i wrote about wanting to address issues of privilege in the space between archives and technology. as a first step, i mentioned organizing a new york group of computer anonymous. i’m pleased to announce that we’ve scheduled our first meeting: tuesday, october , , : pm - ?, at pacific standard, fourth avenue, brooklyn, ny we have about seven people who have indicated that they’re planning on attending. if you’re interested, please comment here, contact me via twitter or email, or leave a comment on this github issue. i believe that a computer anonymous group in new york is a great chance to start having both tough and positive conversations. i realize that it won’t solve everything, and that our initial location may not be ideal, but i’m certainly amenable to other ideas and doing better outreach. i want to see both the technology and archives professions become more diverse, more equitable, and healther communities that in which i can encourage others to join. cha(lle)nging the dynamics of privilege in archives and technology like others, i found the presidential address of jackie dooley last august’s society of american archivists annual meeting to be problematic. at the time, i had little more to add than what was articulated by others, such as sam winn’s post on professional privilege. as the dust settles, though i’ve gotten a lot more clarity. the society of american archivists is not really an easy place to examine our privilege or our struggle. there are many ways in which we desperately need to examine privilege within the context of our profession as well as the overall organization, but for now, i’m going to limit this post to addressing an issue that has been racing through my head since the saa annual meeting, which concern privilege and the intersection of archives and technology, the area in which i work. i am nothing if not enthusiastic about open culture and open source software and their transformative potential. i release my own work (meaning software, presentations, writing, etc. collaboration before preservation: recovering born digital records in the stephen gendin papers for some, the phrase “born digital resources” may be unfamiliar, but ricky erway, senior program officer at oclc research wrote a brief essay entitled defining “born digital”, which provides a handy, working definition: “items created and managed in digital form.” manuscripts and archives, the beinecke rare book and manuscript library, and yale university library overall have had a notable history of working with born digital resources over the past ten years. emotion, archives, interactive fiction, and linked data [edit (feb , ): thanks to the fantastic work of tara robertson, the video of my lightning talk is now available!] i gave a lightning talk entitled [“wielding the whip: affect, archives, and ontological fusion”]({{ root_url }}/storage/ / feb-code lib-lightning-talk) at the code lib conference in chicago, illinois. this lightning talk was one of the most difficult presentations i’ve ever given for a number of reasons, including the emotional aspect of the content itself, as well as the fact that several of the ideas i was trying to articulate weren’t fully baked. i’ve been thinking about this for the four to six months in various capacities and with different focuses, especially as i read more interactive fiction and learn more about it (as well as about hypertext in general). this post serves as an expansion of some of the ideas in my lightning talk and as a way to further the discussion around the following question: can we write interactive fiction and (semi-/para-)fictional hypertext that leverages linked data to create an emotional connection to the “real world”? hours: the day of digital archives thursday, october was the day of digital archives, organized by friend and colleague gretchen gueguen at the university of virginia. i missed the post deadline yesterday, but it's been a busy week, so i might as well walk through some of the highlights of my work related to digital archives that occurred during that hours from am thursday to am friday. am: it's late, but i'm finishing the last bit of work of writing up lecture notes. this fall, i am teaching a class on digital preservation as an adjunct in the ischool at drexel university. the ischool is on the quarter system, so we have only ten weeks to cover a wide variety of material. last week the students got an introduction to the reference model for an open archival information system, and this week's topics (on which i am writing the lecture notes) are selection and appraisal, assessment, provenance, and authenticity. some of the sources of the week's material include a forthcoming case study from the city of vancouver archives, the dcc curation manual's chapter on appraisal and selection, sections of the clir publication authenticity in a digital environment, and the final report of the w c provenance incubator group. how to hack saa inspired by my friend declan fleming's "how to hack code lib," i have been motivated to put together a guide to surviving and enjoying the annual meeting. it can be a seemingly scary (and potentially lonely) experience if it's your first conference, and we archivists are not always known for our extrovertedness. so, without further ado, here is my brief list of suggestions - again, some of which have been shamelessly stolen adapted from declan's guide. tweeting up at saa thanks to the great work of lance (@newmsi), rachel donahue (@sheepeeh), and angelique richardson (@randomarchivist) last year, the first saa tweetup was pulled off successfully in washington, dc. given that this year's saa annual meeting is just a few weeks away, hillel arnold (@helrond) and i have elected to organize one in chicago, as well. we're holding this year's tweetup on thursday, august , starting at pm, at the clark street ale house, which is about a mile from the conference hotel and easily walkable and accessible by public transportation. feel free to join us after the alumni mixers - and please join us even if you don't use twitter. please rsvp at http://twtvite.com/saa tweetup; while rsvps are not required, they will help us and the bar plan ahead. supporting hyatt workers and unite here local at the annual meeting of saa some of us archivists have growing concerns regarding the long-standing labor dispute between unite here local and the management of the hyatt regency chicago, the location of the annual meeting of the society of american archivists. most recently, this labor dispute has led to a one-day strike of housekeepers, dishwashers, bellmen and other hotel workers on june , . saa has not given its membership any guidance to its membership about how to support unite here local and the hyatt's hotel workers. accordingly, my colleague hillel arnold and i have put together an website for archivists to find and share ideas. this website, support hyatt workers at saa : an unofficial resource, is now live, and provides ideas for actions that anyone can perform, plus lists of those specifically for individuals who have either chosen not to attend and for those that are attending. this site allows anyone to contribute and comment either generally on a given page or in response to particular ideas. sumer is icumen in i have spent the last several months in a fog. emotions tend to get the better of me whenever faced with a barrier in my work life. it's gotten increasingly difficult for me to see the forest for the trees, no matter how much i tell myself that my work is for the greater good of my unit, my institution, and archivy. self-doubt creeps in, as does stress, frustration, depression. positivity begins to wane, with optimism replaced by apathy and sarcasm. you stop seeing the good in things and other people, and you stop being inspired. you desperately want to get away, pull the plug, clean the slate, or otherwise just put everything to a grinding halt. you stop asking "why can't i do that?" and start asking "why should i care?" instead. i don't think this is the first time i've faced burnout, and while it certainly won't be the last, the extent to which it's affected me this time around is astounding. in memoriam: robert frost, - i am sad to announce the passing of robert l. "bob" frost ( - ). bob was an associate professor at the university of michigan school of information, my alma mater, where he had taught since . bob had been battling cancer for over two years. ed vielmetti has written an obituary of bob on his blog, including the announcement from si dean jeffrey mackie-mason. bob was an inspiration to many of us si alums, and his magnetic personality, sharp wit, and joie de vivre ensured he had a bevy of his students and colleagues buzzing around him at any given time. i had the opportunity to take his class material culture and the interpretation of objects in the spring of , my final semester at si. the class was intense in a way that few of my other classes at michigan were, and it provoked my continuing curiosity in identifying theoretical frameworks to analyze the everyday world. bob reinforced my fascination with wilhelm reich and the fugs by introducing me to dušan makavejev's w. wikileaks & the archives & records profession: a panel discussion update: the text of my remarks can be now found online at https://matienzo.org/presentations/ /wikileaks/. i am honored to be one of the speakers at "wikileaks & the archives & records profession," a panel discussion organized by the archivists roundtable of metropolitan new york and the metropolitan new york city chapter of arma international. the panel will be on january , at the center for jewish history. from the announcement: do wikileaks and its complex, attendant issues shift our conceptualization of our roles as information professionals? how might wikileaks change the public's views on usage of and access to archives and records? to what extent is the most recent release of diplomatic cables a product of information mismanagement? addressing these and many more questions, our confirmed speakers include trudy peterson, former acting archivist of the united states ( - ) and current representative for the society of american archivists on the department of state's historical advisory committee; fred pulzello, solutions architect in the information governance practice at microlink llc; jim fortmuller, manager of systems security at kelley drye & warren llp in washington, dc; mark matienzo, digital archivist in manuscripts and archives at yale university library; and derek bambauer, associate professor of law at brooklyn law school. what's your delicious story? update: i've added a question on quora about this too - feel free to contribute your story there. in my last post, i talked a bit about the notion of delicious being a platform with a myriad of uses, and i've been actively wondering about this since then. upon further reflection, i've realized that the best way to figure this out is actually to engage and ask people directly. accordingly, i'm asking for your help. of course it's upsetting that delicious is being sunsetted, but other than individual users and archive team, people seem to be doing very little about it. delicious is clearly more than the bookmarks. i want to gather information about how people like you and me actually used it beyond it's obvious functionality. did you use it to manage resources for your dissertation? did you use it to communicate with family about a serious event or illness? how did you go beyond the boundaries of it being just " delicious and the preservation of "platforms" just as plenty of others have, i recoiled in horror when i heard that delicious (née del.icio.us) was being "sunsetted". regardless of the red flags that have been raised about its potentially imminent demise, i've still been using it on a daily basis. i've been an active user for over . years, which is longer than i can say for just about any other web platform or service. i deleted my friendster and myspace accounts quite a while ago; i've been on flickr almost as long as delicious, but the bookmarking wins out by a good four months or so. i started using delicious in my final semester of library school, and it shows. i used it for procrastinating as well as a way to organize research materials before i had zotero. the bulk of the bookmarks from that first day of use (february , ) were likely imports from my browser, but i quickly showed a facility for adding stuff that i saw as interesting, useful, etc. update: aus-archivists not dead? earlier today i'd posted about the australian society of archivists' announcement about the aus-archivists listserv being "lost." tim sherratt, an australian colleague and friend of this blog, announced this post on archiveslive, the ning group created by the asa seemingly to replace the listserv. pat jackson, asa president, has already responded with an update: the asa national office has not lost the aus-archivists list-serv. we have moved from an outsourced service provider to managing our new server at the national office. the aus-archivists list-serv was a bit too ancient for our spanking new server to manage. in terms of the posterity of the contents of the list-serv, the wonderful discussions and debate it fostered and engendered, they are not lost. it is our intention to post them to the asa website where they can be perused. further to that, it is my understanding that the aus-archivists list-serv is also deemed to be permanent under the asa retention schedule. the asa will be investigating other methods of storing the list-serv for permanent retention. goodbye, aus-archivists: listservs and the commitment to digital preservation [update: aus-archivists might not be gone for good, as asa intends to share the entire run of postings on its website. see this post for details.] despite my relative distaste for the a&a list, i have previously found it useful and argued for its retention when it was threatened in . i still agree with most of what i wrote . years ago, although i might have toned things down in retrospect. in an effort to find other e-mail discussion lists on archives that engaged my interest, i joined arcan-l (the canadian archivists' listserv) and aus-archivists (the australian archivists' listserv, maintained by the australian society of archivists). surprisingly, aus-archivists had been idle since around the end of october. i noticed this tweet from the australian society of archivists only in passing at the beginning of november: the asa office would just like everyone to know that our list serv is still currently unavailable, we apologize for any inconvenience... i didn't hear anything else between then and earlier today. i should note that i'm not a member of asa, and so i can't speak to any communication they had with their membership. however, today a message was sent out by pat jackson, the asa president, to all aus-archivists subscribers, announcing that the listserv was lost entirely. disco-powered pymarc i'd been long interested in starting to develop code using some sort of mapreduce implementation for distributed computing. i have never been able to get my head around hadoop, so i gave up with that pretty quickly. i recently discovered disco, a mapreduce framework with an erlang-based core. disco also allows you to to write your worker code in python, which was a huge plus to me. after stumbling through the tutorial, i took the word count demo and put together some basic code using pymarc that gathered tag count statistics for a bunch of marc files. the code's still in a very early form, and arguably should carve up large files into smaller chunks to pass off to the worker processes; i've gotten around this for the time being by splitting up the files using yaz-marcdump. once i split the files, i pushed them into a tag of ddfs, the disco distributed file system. this was a useful way for me to write some demo code both for using pymarc and disco. the future of archivesblogs every project has it's day. i've administered archivesblogs for four years now. originally, i created it to fill a void when blogging was new to the archival profession, and archivists were having to make the case for dedicating staff time to shepherding early social media projects. four years later, things are much different; i'm less interested in web . (professionally speaking), more archivists are blogging, and more repositories are maintaining their own blogs. despite the changes in the archival blogosphere and repository administration, archivists still contact me occasionally and remind me of the value of archivesblogs. it's also lead to some interesting debates in the past. i still think it has its place, but i don't want to be the only person shaping its future. i've also been thinking for a while that i want to get out of the aggregation business, and i believe time to put together a succession plan. the reality is that i don't have the time to rethink what archivesblogs could be, or even give it the care and feeding it needs to keep running. why i have given up on the archives and archivists list i am certainly not the first person to chime in on this topic, and i certainly hope not to be the last. inspired by two fantastic posts by ben bromley and maureen callahan, i have chosen to discuss the reasons why i have given up on the archives and archivists list. unlike ben and maureen, who discuss why they choose not to post to the list, i'm also including reasons why i choose not to read or subscribe to the list anymore. for what it's worth, until yesterday, i had been on the a&a list for almost nine long years. i don't think the majority of the traffic is terribly useful. this can be incredibly frustrating, especially there's a question on topic you happen to know something about. telling someone how to perform a google search is not an adequate response.given the signal-to-noise ratio of the list, useful or timely messages can be easily buried. off-topic messages seem to be the rule rather than the exception. with little fanfare, dlist goes down i've been meaning to blog about this for a while. dlist, the digital library of information science and technology, maintained by the university of arizona school of information resources and library science, has been down for at least three months. any url formerly part of dlist gets automatically redirected to an announcement page that reads as follows: aging hardware and conversion issues following a system crash have taken their toll on dlist, the university of arizona's digitial library of information science and technology. we are currently exploring choices and alternatives both to short term recovery and long term sustainability. the resources and metadata are fully recovered, and we hope to put them back online in a new repository soon. if you or your institution would like to assist with the dlist project, please contact us at sirls@email.arizona.edu. thanks for your support! while i feel for the difficulties they've had in maintaining it, i have to admit that it's a bit frustrating for me from the standpoint of someone who submitted material to dlist. code lib : southern hospitality i recently returned from a trip to asheville, north carolina for this year's code lib conference. despite the unavoidable hiccups that some attendees experienced as they tried to head home from the conference, i believe that this year's conference was the most successful one that i happened to attend. if i'm right, i think this year had a record number of attendees, a record number of new attendees, and much tighter organization to make the new folks feel welcome. the social activities were certainly more planned and organized than last year, which was a welcome change. while i certainly didn't mind hollering out to the crowd that i would be going to see some bands or to a particular restaurant like i had in previous years, it was nice to see other folks take the lead. the newcomer dinners seemed to go pretty well; the brews cruise and barbecue excursions went smoothly; and even the game(s) of werewolf seemed to take a life of their own. description peddlers and data.gov: two peas in a pod as you may have heard, the national archives issued a press release today announcing the release of three data sets on data.gov: the first milestone of the open government directive was met on january with the release of new datasets on data.gov. each major government agency has uploaded at least three datasets in this initial action. the national archives released the — code of federal regulations and two datasets from its archival research catalog. this is the first time this material is available as raw data in xml format. the archival research catalog, or arc, is nara's primary access system for archival description, representing % of nara's entire holdings. this breaks down to the following: , , cubic feet record groups , collections , series , , file units , items in addition, there are , , , logical data records and , artifacts described in arc. nara's decision to share this data is a breakthrough for archives and people who love data. onward and upward... it's fitting that this the hundredth (gosh, only the hundredth?) post, because i have rather important news. first, my fellow developers/producers/ux designers at the new york public library and i have been dealing with every minute detail on the upcoming, drupal-based replacement to the nypl website. you can see a live preview at http://new.nypl.org/. i can proudly say that this project has helped both me personally and nypl overall play nice in the open source world - we've been actively contributing code, reporting bugs, and sending patches to the drupal project. also, our site search is based on solr, which always bears mention. in addition, after a working tirelessly as a developer at nypl for the last year and a half, i have decided to move onward and upward. i am leaving the cozy environs of the still-recently renovated office space i share with my spectacular coworkers. it was not an easy decision by far, but it feels like the best one overall. clifford lynch clarifies position on open source ilses clifford lynch, executive director of the coalition for networked information, has responded to the leaked sirsidynix report that spreads horrific untruths about open source. marshall breeding posted lynch's response on guideposts. in particular, lynch notes the following: i don't think that i ever wrote those words down in an article; i suppose i may have said something to that effect in an interview or q&a in some conference program like ala top tech, though perhaps no quite as strongly as it's expressed here. i have without question spoken out about my concerns regarding investment in open source ils development in the last few years. if i did say this, it feels like it's used a little out of context -- or maybe the better characterization is over-simplistically -- in the report. ... i think there are still major problems -- many of which we really don't know how to solve effectively, and which call for sustained and extensive research and development -- in various areas where ils get involved in information discovery and the support of research and teaching. sirsidynix report leaked, spreading fear, uncertainty and doubt about open source thanks to twitter, i discovered that wikileaks has posted a report written by sirsidynix vice president for innovation stephenabram which spreads a fantastic amount of fear, uncertainty and doubt about both open source software in general and, more specifically, the suitability of open source integrated library systems. as the summary provided by wikileaks states, this document was released only to a select number of existing customers of the company sirsidynix, a proprietary library automation software vendor. it has not been released more broadly specifically because of the misinformation about open source software and possible libel per se against certain competitors contained therein ... the source states that the document should be leaked so that everyone can see to what extent sirsidynix will attempt to spread falsehoods and smear open source and the proponents of open source. in addition, as you may have heard, the queens library is suing sirsidynix for breach of contract; for what it's worth, the initial conference is scheduled for next monday, november , . pybhl: accessing the biodiversity heritage library's data using openurl and python via twitter, i heard about the biodiversity heritage library's relatively new openurl resolver, announced in their blog about a month ago. more specifically, i head about matt yoder's new ruby library, rubybhl, which exploits the bhl openurl resolver to provide metadata about items in their holdings and does some additional screenscraping to return things like links to the ocred version of the text. in typical fashion, i've ported matt's library to python, and have released my code. pybhl is available from my site, pypi, and github. use should be fairly straightforward, as seen below: import pybhl import pprint b = pybhl.bhlopenurlrequest(genre='book', aulast='smith', aufirst='john', date=' ', spage=' ', volume=' ') r = b.get_response() len(r.data['citations']) pprint.pprint(r.data['citations'][ ]) {u'atitle': u'', u'authors': [u'smith, john donnell,'], u'date': u' ', u'epage': u'', u'edition': u'', u'genre': u'journal', u'isbn': u'', u'issn': u'', u'itemurl': u'http://www.biodiversitylibrary.org/item/ ', u'language': u'latin', u'lccn': u'', u'oclc': u' ', u'pages': u'', u'publicationfrequency': u'', u'publishername': u'h.n. patterson,', u'publisherplace': u'oquawkae [ill.] :', u'spage': u'page ', u'stitle': u'', u'subjects': [u'central america', u'guatemala', u'plants', u''], u'title': u'enumeratio plantarum guatemalensium imprimis a h. access and description reconsidered what exactly is archival access, and how does archival description make it possible? i feel like that in some form or another i've been struggling with this question throughout my career. recently, this blog post from the top shelf, the blog of the university of texas at san antonio archives and special collections department, came across my radar, wherein they write (emphasis in original): utsa archives and special collections is among the growing number of archives to create an online presence for every one of its collections. ... we were able to utilize inventories generated by former and current collection assistants to create guides to the collection with folder-level and box-level descriptions. the project resulted in access to more than collections and linear feet of materials. what defines that accessibility? i certainly don't intend to be a negative nancy about this - adding finding aids and other descriptive metadata about collections is obviously useful. but how has it necessarily increased access to the materials themselves? aip receives nhprc funding to digitize samuel goudsmit papers i'm happy to pass on the news that my former employer, the niels bohr library & archives of the american institute of physics, has received funding from the national historical publications and records commission to digitize the entirety of the samuel goudsmit papers. from the announcement on the center for history of physics/niels bohr library & archives facebook page: goudsmit ( — ) was a dutch-educated physicist who spent his career in the us and was involved at the cutting edge of physics for over years. he was an important player in the development of quantum mechanics in the s and s; he then served as scientific head of the alsos mission during world war ii, which assessed the progress of the german atomic bomb project. goudsmit became a senior scientist at brookhaven national laboratory and editor-in-chief of the american physical society. the papers consist of an estimated , documents, which include correspondence, research notebooks, lectures, reports, and captured german war documents; the collection is the most used in the library. a gentle reminder on the eve of teaching my first class of my course (lis - , or, building digital libraries: infrastructural and social aspects) at liu's palmer school of information and library science, i'd like to remind you of the following. the syllabus is available on online, if you're curious. privacy, censorship, and good records management: brooklyn public library in the crosshairs over at librarian.net, jessamyn west has a brief write up about a post on the new york times' city room blog about placing access restrictions on offensive material (in this case, one of hergé's early tintin books at the brooklyn public library). more interestingly, she notes, is that the times was given access and accordingly republished challenges from bpl patrons and other community members. quite astutely, jessamyn recognizes that the patrons' addresses are removed but their names and city/state information are published. if your name is, for example, [name redacted], redacting your address doesn't really protect your anonymity. i'm curious what the balance is between patron privacy and making municipal records available. it's a good question that doesn't have an incredibly straightforward answer. my first concern was about whether bpl had kept the challenge correspondence beyond the mandated dates in the new york state records schedules. after doing some digging, on the new york state archives' website, i came across schedule mi- (" everything is bigger in texas, including my talks on the semantic web i'll be at the society of american archivists annual meeting next week in austin, texas. it looks to be a jam-packed week for me, with a full-day standards committee/tsds meeting on tuesday, followed by thatcamp austin in the evening, an (expanded version of my) presentation on linked data and archival description during the ead roundtable on wednesday, and thursday's session (number ): "building, managing, and participating in online communities: avoiding culture shock online" (with jeanne kramer-smyth, deborah wythe, and camille cloutier). and to think i haven't even considered which other sessions i'm going to! anyhow, i hope to see you there, and please make either or both of my presentations if you can. must contextual description be bound to records description? i've been struggling with the fact that (american) archival practice seems to bind contextual description (i.e., description of records creators) to records description. much of these thoughts have been stirring in my head as a result of my class at rare book school. if we take a relatively hardline approach, e.g. the kind suggested by chris hurley ("contextual data should be developed independently of the perceived uses to which it will be put", , see also ), it makes total sense to separate them entirely. in fact, it starts making me mad that the <bioghist> tag exists at all in ead. contextual description requires that it be written from a standpoint relative to that of the creator it describes. i guess what i keep getting hung up on is if there could be a relevant case that really merits this direct intellectual binding. i therefore appeal to you, humble readers, to provide me with your counsel. do you think there are any such cases, and if so, why? seeking nominations for co-chair, rlg programs roundtable apologies for any duplication - we're just trying to get the word out! as co-chairs of the rlg programs roundtable of the society of american archivists, we're seeking nominees to co-chair of the roundtable for - . if you'd like to nominate yourself or someone else, please email mark matienzo, co-chair, at mark at matienzo.org. please submit all nominations no later than pm eastern time on friday, august . serving in a leadership position for a section or roundtable is a great way to learn about saa and its governance, contribute to new directions for the society, and work with other archivists on interesting projects. it is also a great way to serve the society! your rlg roundtable co-chairs, thomas g. knoles marcus a. mccorison librarian american antiquarian society mark matienzo applications developer, digital experience group the new york public library the archival, the irreconcilable, and the unwebbable: three horsemen and/or stooges this week in charlottesville has been a whirlwind exploration of standards and implementation strategies thus far during my class, designing archival description systems, at rare book school. my classmates and i have been under the esteemed tutelage of daniel pitti, who has served as the technical architect for both ead and eac. interestingly, there's been a whole lot of talk about linking data, linked data, and linked data, date normalization, and print versus online presentation, among other things. in addition, a few things have floated past on my radar screen this week that have seemed particularly pertinent to the class. the first of these was a post by stefano mazzocchi of metaweb, "on data reconciliation strategies and their impact on the web of data". in stefano's post, he wrote about the problem of a priori data reconciliation vs. a posteriori; in other words, whether you iron out the kinks, apply properties like owl:sameas, etc., on the way in or on the way out. "summer camp for archivists" sounds so much better crossposted to nypl labs. i'm staying with colleagues and good friends during my week-long stint in charlottesville, virginia for rare book school. if you're here - particularly if you're in my class (daniel pitti's designing archival description systems) - let me know. i'm looking forward to a heady week dealing with descriptive standards, knowledge representation, and as always, doing my best to sell the archives world on linked data. notes and thoughts will follow, as always, on here. "using the oclc worldcat apis" now available in python magazine as of last thursday, i have been inducted into the pantheon of published python programmers (aye, abuse of alliteration is always acceptable). my article, "using the oclc worldcat apis," appears in the latest issue (june ) of python magazine. i'd like to thank my editor, brandon craig rhodes, for helping me along in the process, not the least of which includes catching bugs that i'd overlooked. the article includes a brief history lesson about oclc, worldcat, and the worldcat affiliate apis, a detailed introduction to worldcat, my python module to interact with oclc's apis, and a brief introduction to simile exhibit, which helps generate the holdings mashup referenced earlier on my blog. subscribers to python magazine have access to a copy of the code containing a functional oclc web services key ("wskey") to explore the application. nyart presentation: archives & the semantic web this last tuesday, i spoke at the annual meeting of the archivists' roundtable of metropolitan new york, where i gave a talk on archives and the semantic web. the presentation went over very well, and colleagues from both the archives field and the semantic technology field were in attendance. i did my best to keep the presentation not overtly technical and cover just enough to get archivists to think about how things could be in the future. i also have to give a big hat tip to dan chudnov, whose recent keynote at the texas conference on digital libraries helped me organize my thoughts. enjoy the slides, and as always, i relish any feedback from the rest of you. drupal for archivists: documenting the asian/pacific american community with drupal over the course of the last academic year, i have been part of a team working on survey project aimed at identifying and describing archival collections relating to the asian and pacific american community in the new york city metropolitan area. the results of the fifty-plus collections we surveyed have been posted on our drupal-powered website, which has been an excellent fit for the needs of this project and has also enabled us to engage many of the challenges the project has presented. by way of introduction, this survey project seeks to address the underrepresentation of east coast asian/pacific americans in historical scholarship and archival repositories by working with community-based organizations and individuals to survey their records and raise awareness within the community about the importance of documenting and preserving their histories. funded by a documentary heritage project grant from metro: metropolitan new york library council, the project is a collaborative effort between the asian/pacific/american institute and the tamiment library/robert f. worldcat in the wild at oclc's worldcat mashathon in amsterdam it's good to see other people using your code. thanks to the oclc devnet blog, i found out that etienne posthumus used worldcat for a demo application he built during the worldcat mashathon in amsterdam last week. even more interesting is that etienne's application was deployed on google app engine. courtesy of oclc's alice sneary, there is a brief video of etienne presenting his application to the other mashathon attendees: batch reindexing for drupal + solr crossposted to nypl labs. sorry for any duplication! hey, do you use drupal on a site with several thousand nodes? do you also use the apache solr integration module? if you're like me, you've probably needed to reindex your site but couldn't be bothered to wait for those pesky cron runs to finish — in fact, that's what led me to file a feature request on the module to begin with. well, fret no more, because thanks to me and greg kallenberg, my illustrious fellow applications developer at nypl dgtl, you can finally use drupal's batch api to reindex your site. the module is available as an attachment from that same issue node on drupal.org. nota bene: this is a really rough module, with code swiped pretty shamelessly from the example use of the batch api page on drupal.org. it works, though, and it works well enough as we tear stuff down and build it back up over and over again. digitalnz and brooklyn museum api modules for python i've been busy the last few weeks, so i didn't even really announce this to begin with! i've been playing around with some of the cultural heritage apis that are available, some of which i learned about while i was at museums and the web . while i was away i released code for a python module for interacting with the brooklyn museum collections api. after chatting with virginia gow from digitalnz, i also got motivated to write a python module to interact with the digitalnz api. the code for both is fairly unpolished, but i'm always ready for feedback! both modules are available as mercurial repositories linked from my bitbucket account. there's also a small cluster of us working on a museum api wiki to begin sorting out some of these issues. comparably speaking, the library and archives world has it somewhat easy... the medium is not the message "electronic records" is a particularly awful phrase and does not even actually capture anything about the underlying records at all. as far as the term goes, it's not too far off from "machine readable records." as a profession, can we start actually thinking critically about the underlying technical issues and push for using terms that more accurately describe what it is we're dealing with? i understand it's a convenient catch-all term, but there is a large range of issues that differ with the kinds of data and systems. drupal for archivists: a drupal-built archives reference blog when mark asked me to write about our use of drupal at the dickinson college archives and special collections, the first thing i thought about was when our archives reference blog was initially launched in april . i couldn't believe that it has been two years already. i am pleased to report that my colleagues at dickinson and i are enormously happy with the results of those two years. i hope others may find this brief explanation of how and why we are using drupal as a reference management tool to be helpful and instructive. the concept for our implementation of drupal was a simple one. i was thinking about the fact that we help researchers everyday to locate information that they want, but that what they discover among our collections or learn from them seldom gets shared, except by those who write for publication. so, what if we shared via the web, through a simple blog format, the basic questions posed by our researchers along with a simple summary of the results? why you should support linked data if you don't, i'll make your data linkable. coming soon: drupal for archivists i've been fairly quiet lately as i've been busy with this and that, but i thought i'd let everyone know that i've been beginning to put together a series of posts entitled "drupal for archivists." drupal, as you may or may not know, is a flexible and extensible open source content management system. there will be a general overview of some of the important concepts, but it'll focus less on the basics of getting people up and running — there are plenty of resources out there, such as the wonderful tutorials and articles available from lullabot. instead, i've drafted a handful of guest bloggers to discuss how and why they're using drupal. keep your eyes peeled! brooklyn museum releases api the always groundbreaking brooklyn museum has now released an api to allow the public to interact with their collections data. i can't even tell you how happy i am about this in terms of an open data perspective. also, this is the direction that makes the whole "detailed curation by passionate amateurs" thing possible. there are only three simple methods for accessing the data. ideally, it would be nice to see them put their collections metadata up as linked data, but now i'm daring to dream a little. hey, wait a minute! i think that's the perfect way to start playing around with the api. doing some digging through the documentation, i'm seeing that all the objects and creators seem to have uris. take a crack at it - the registration form is ready for you. moving worldcat to mercurial and bitbucket it's official - i've moved the codebase for worldcat, my python module for working with the oclc worldcat apis, to be hosted on bitbucket, which uses the mercurial distributed version control system. you can find the new codebase at http://bitbucket.org/anarchivist/worldcat/. make me a structured vocabulary or i'll make one for you the society of american archivists released the thesaurus for use in college and university archives as an electronic publication this week. specifically, it was issued as a series of pdf files. is this data stored in some sort of structured format somewhere? if so, it's not available directly from the saa site. there's no good reason why tucua shouldn't be converted to structured, linkable data, expressed using skos, the simple knowledge organization system. it's not like i need another project, but i'm sure i could write some scraper to harvest the terms out of the pdf, and while i'm at it, i could write one to also harvest the glossary of archival terminology. someone, please stop me. i really don't need another project. go foaf yourself i'm really looking forward to next week's code lib conference in providence, despite my utter failure to complete or implement the project on which i am presenting. in particular, i'm really looking forward to the linked data preconference. like some of my other fellow attendees, i've hammered out a foaf file for the preconference already so that ed summers' combo foaf crawler and attendee info web app. this is what the sample output looks using my foaf data. it's good to see we're well on our way to have an easily creatable sample type of rdf data for people to play with. at a bare minimum, you can create your foaf data using foaf-a-matic and then edit it to add the assertions you need to get it to play nice with ed's application. see you in providence, but go foaf yourself first. developing metrics for experimental forms of outreach archivesnext recently inquired about how archivists measure success of . initiatives. it's hard to determine some . -ish initiatives will really impact statistics when you don't really define what the results you're trying to see. i'd like to open the question further — how do we begin developing metrics for things that sit on the cusp between forms of outreach? furthermore, i'm curious to see where this information is captured — do archivists wait until the end to gather survey data, or if they working towards something like we at nypl labs are doing with infomaki, our new usability tool developed by michael lascarides, our user analyst. dead reckoning # : mixing/matching with namespaces and application profiles so, it's time for another rant about my issues with ead. this one is a pretty straightforward and short one, and comes down to the issue that i should essentially be able to mix and match metadata schemas. this is not a new idea, and i'm tired of the archives community treating it like it is one. application profiles, as they are called, allow us to define a structured way to combine elements from different schemas, prevent addition of new and arbitrary elements, and tighten existing standards for particular use cases. however, to a certain extent, the ead community has accepted the concept of combining xml namespaces but on a very limited level. the creation of the ead schema allows ead data to be embedded into other xml documents, such as mets. however, i can't do it the other way around; for example, i can't work a mods or marcxml record into a finding aid. why not? you're all sheep made by twittersheep, a new project made (in part) by my acquaintance ted roden, a creative technologist for new york times research & development. a bird's eye view of archival collections mitchell whitelaw is a senior lecturer in the faculty of design and creative practice at the university of canberra and the winner of the national archives of australia's ian maclean award. according to the naa's site, the ian maclean award commemorates archivist ian maclean, and is awarded to individuals interested in conducting research that will benefit the archival and historical profession in australia and promote the important contribution that archives make to society. dr. whitelaw has been keeping the world up to date on his work using his blog, the visible archive. his work fits well with my colleague jeanne kramer-smyth's archival data visualization project, archivesz, as well as the multidimensional visualization projects underway at the humanities advanced technology & information institute at the university of glasgow. however, his project fascinates me for a few specific reasons. first of all, the scale of the datasets he's working with are astronomically larger than those that any other archival visualization project has tried to tackle so far. api fun: visualizing holdings locations in my previous post, i included a screenshot of a prototype, but glossed over what it actually does. given an oclc record number and a zip code, it plots the locations of the nearest holdings of that item on a google map. pulled off in python (as all good mashups should be), along with simile exhibit, it uses the following modules: geopy simplejson web.py and, of course, worldcat. if you want to try it out, head on over here. the curent of the code will soon be able as part of the examples directory in the distribution for worldcat, which can be found in my subversion repository. this is all i'm going to say on this here blogsite concerning the brouhaha about the policy for use and transfer of worldcat records because i have other, more interesting and more complex problems to solve (and so do you) the moderated discussion hosted and sponsored by nylink went pretty well. also, i don't need the records to have fun with the data "” i just need robust apis. (in fact, as i said today, i'd prefer not to have to deal with the marc records directly.) robust apis would help making prototypes like this one i hacked together in a few hours into a real, usable service. lightening the load: drupal and python man, if this isn't a "you got your peanut butter in my chocolate thing" or what! as i wrote over on the nypl labs blog, we've been up to our necks in drupal at mpow, and i've found that one of the great advantages of using it is rapid prototyping without having to write a whole lot of code. again, that's how i feel about python, too, but you knew that already. once you've got a prototype built, how do you start piping stuff into it? in drupal , a lot of the contrib modules to do this need work - most notably, i'm thinking about node_import, which as of yet still has no (official) cck support for drupal and cck . in addition, you could be stuck with having to write php code for the heavy lifting, but where's the joy in that? well, it so happens that the glue becomes the solvent in this slow, slow dance. dead reckoning # : a fatheaded failure for faceted terms and headings in ead a while back, i wrote a bad marc rant, and i considered titling this a bad metadata rant. however, as the kids say, i got mad beef with a little metadata standard called encoded archival description. accordingly, i figured i should begin a new series of posts discussing some of these issues that i have with something that is, for better or for worse, a technological fixture of our profession. this is in part prompted by thoughts that i've had as a result of participating in ead@ and attending the something new for something old conference sponsored by the pacscl consortial survey initiative. anyhow, onto my first bone to pick with ead. i'm incredibly unsatisfied with the controlled access heading tag <controlaccess/>. first of all, it can occur within itself, and because of this, i fear that there will be some sort of weird instance where i have to end up parsing a series of these tags levels deep. also, it can contain a <chronlist/>, which also seems pretty strange given that i've never seen any example of events being used as controlled access terms in this way. going off the rails: really rapid prototyping with drupal previously posted on http://labs.nypl.org/. the other labs denizens and i are going off the rails on a crazy train deeper down the rabbit hole of reimplementing the nypl site in drupal. as i pile my work on the fire, i've found that building things in drupal is easier than i'd ever thought it to be. it's a scary thought, in part because i'm no fan of php (the language of drupal's codebase). really, though, doing some things can be dead simple. it's a bit of a truism in the drupal world at this point that you can build a heck of a lot just by using the cck and views modules. the important part is that you can build a heck of a lot without really having to know a whole lot of code. this is what threw me off for so long - i didn't realize that i was putting too much thought into building a model like i normally would with another application framework. does saa need to support who i am? there's been a whole lot of discussion in the archivoblogosphere about the perceived need for quasi-informal interest groups that are fundamentally driven by identity. while i agree with this in theory, i must register my opposition to having saa promote, support, or provide any sort of infrastructure for such groups. fundamentally, i am against this because i believe it poses a strong threat to the privacy of archivists. deliciouscopy: a dumb solution for a dumb problem you'd think there was some sort of tried and true script for delicious users to repost bookmarks from their inboxes into their accounts, especially given that there are often shared accounts where multiple people will tag things as "for:foo" to have them show up on foo's delicious account. well, there wasn't, until now (at least as far as i could tell). enter deliciouscopy. it uses pydelicious, as well as the universal feed parser and simplejson. it reads a user's inbox, checks to see if poster of the for:whomever tag was added to your network, and reposts accordingly, adding a via: tag for attribution. it even does some dead simple logging if you need that sort of thing. the code's all there, and gpl license blah blah blah. i hacked this together in about an hour for something at mpow - namely to repost things to our shared account. it's based on michael noll's deliciousmonitor.py but diverges from it fairly quickly. enjoy, and give any feedback if you must. idle hands are the devil's plaything i've had my hands full lately. two weeks ago i was at the mcn conference (wherein, among other things, i have continued my dominion as archduke of archival description by taking over the mcn standards sig chair position from the bancroft library's mary elings), and next week i'm off to philadelphia for the pacscl something new for something old conference. i hammered out the coherent, written version of my paper i gave at ead@ . i prepared a proposal for next february's code lib conference in providence (ahem, vote for mine, if you're so inclined): building on galen charlton's investigations into distributed version control systems for metadata management, i offer a prototype system for managing archival finding aids in ead (encoded archival description). my prototype relies on distributed version control to help archivists maintain transparency in their work and uses post-commit hooks to initiate indexing and publishing processes. in addition, this prototype can be generalized for any xml-based metadata schema. on top of that, i'm working with a fine group of folks on the rlg programs project to analyze ead editing and creation tools, doing hardcore schema mapping at work, and somehow finding enough time to play a little doukutsu monogatari to unwind. developing automated repository deposit modules for archivists' toolkit? i'd like to gauge interest for people to help add code to archivists' toolkit to automate the deposit of digital objects into digital repositories. at first glance, the biggest issue is having to deal with differing deposit apis for each repository, but using something like sword would make sense to bridge this gap. any and all feedback is welcome! python worldcat module v . . now available in preparation for the upcoming worldcat hackathon starting this friday, i've made a few changes to worldcat, my python module for interacting with oclc's apis. most notably, i've added iterators for sru and opensearch requests, which (like the rest of the module) painfully need documentation. it's available either via download from my site or via pypi; please submit bug reports to the issue tracker as they arise. edit: i've bumped up the version number another micro number to . . as i've just added the improvements mentioned by xiaoming liu on the worldcat devnet blog (lccn query support, support for tab-delimited and csv responses for xissnrequests, and support for php object responses for all xidrequests). edit: thanks to thomas dukleth, i was told that code for the hackathon was to be licensed under the bsd license. accordingly, i've now dual licensed the module under both gpl and bsd. v -powered libraries and the happiness engines that run them previously posted on http://labs.nypl.org/. a week ago today, a few of my deg colleagues and i went to see liz lawley from rit's lab for social computing give a talk entitled "libraries as happiness engines." it was a modified version of a talk she gave at this year's cil conference. the gist of the talk was that gaming in libraries means not just using established games to draw the public into the library, but also to begin implementing game mechanics into libraries that allow them to flourish as social spaces. in particular, these game mechanics include things like collecting, points, feedback, exchanges, and customization. i've been ruminating on this for the last week or so in a couple different ways. first of all, i've been trying to figure out how we could implement game mechanics within nypl. an open letter to saa council and the program committee i apologize for using my blog to soapbox, but i felt like this was a significant concern that i should share with my readers. if you wish to support my position, please consider sending an e-mail to saa council and the program committee chairs. dear program committee members and saa council members, i understand that we are nearing the deadlines for submission of proposals for sessions at the annual meeting of the society of american archivists. i also understand the reasons behind having an earlier deadline than past years. however, i am deeply concerned with the decision to have the deadline set to be october , , which is yom kippur and the day which the jewish high holidays end. as is often the case, conference proposals often coalesce at the last minute, and this is further complicated by the fact that the beginning of rosh hashana fell on september , . i recognize that the deadline is most likely immutable at this point, but i am asking that saa council and future program committees pay attention to when the high holidays fall in future years. the apex of hipster xml geekdom: tei-encoded dylan via language log: the electronic textual cultures lab (etcl) at the university of victoria has, in an effort to draw more attention to tei, chosen to prepare an encoded version of the lyrics to bob dylan's "subterranean homesick blues" and overlaid the resulting xml over the song's video. the resulting video is available, naturally, on youtube. etcl's ray siemens writes about the reasoning behind this on the tei video widgets blog: at the last gathering of the text encoding initiative consortium, in maryland, a few of us were discussing the ways in which tei has eluded some specific types of social-cultural representation that are especially current today . . . things like an avatar, or something that could manifest itself as a youtube posting. a quick search of youtube did reveal a significant and strong presence of sorts, but it was that of tei the korean pop singer (pronounced, we're told, "˜tay'); so, our quest began there, setting out modestly to create a video widget that would balance t-e-i and tei in the youtube world. introducing djabberdjaw djabberdjaw is an alpha-quality jabber bot written in python that uses django as an administrative interface to manage bot and user profiles. i've included a couple of plugins out of the box that will allow you to perform queries against z . targets and oclc's xisbn api (assuming you have the requisite modules). djabberdjaw requires django . or later, jabberbot, and xmpppy. it's available either from pypi (including using easy_install) or via subversion. you can browse the subversion repository, too. archivesblogs . thanks to jeanne from spellbound blog, i was made aware of the fact that archivesblogs hadn't really been doing its job. so, i ripped out its guts and put it back together. it's running the latest, shiniest versions of wordpress, feedwordpress, and auto delete posts, and now it has added feedburner and wp stats goodness. let me know if you discover any peculiarities in the updated set up. slaying the scary monsters previously posted on http://labs.nypl.org/. getting up to speed is hard anywhere, and it's especially difficult in a large, complex institution like nypl. other than just understanding the projects that you're given, you also are thrown headfirst into making sense of the culture, the organization, and all the unspoken and occasionally unseen things that allow you to do your job. there's no clear place to start this, so a good portion of the time you have to keep on top of that while you start thrashing away at your work. the question remains, though, how do you organize this stuff? how do you enable sensemaking in yourself and your peers? everything old is new again goodbye, wordpress - i've been drinking more of the koolaid. i rebuilt my personal/professional site (not this blog) in drupal. migrating the content was pretty easy (about static pages, no posts). the functionality is astounding - i only started working on redoing it yesterday and i've already got a great infrastructure. expect a detailed post before too long, or at least a link to a colophon on said site. matienzo, the san francisco treat i'm packing up and heading out to sfo this evening for saa . right now i'm frantically backing up my zotero repository, making sure i have a bunch of sources to peruse on the plane as i hack away on my slides for ead@ . you might be surprised that my idea of me jumping out of a cake in the shape of an <archdesc> tag wearing a bathing suit was not even considered, so it looks like i'll actually have to put some coherent thoughts together. i've got to make a grand entrance somehow. i'll be chairing the description section meeting as well, so behave yourselves, kids. bad marc rant # : leader positions and i understand why the marc leader position is a good idea in theory. in fact, marbi proposal - suggests: a change in definition to leader/ code "a" for clarification; making code "t" (manuscript language materials) obsolete in leader/ and using code "a" instead; redefinitions of codes "a" and "p" in leader/ ; renaming the for books to "textual (nonserial); and deleting field for mixed material. i can safely say that some pretty funky stuff gets cataloged with the leader position set as "a," and much of it is incorrect, at $mpow and otherwise. what is leader/ actually supposed to be used for? marbi proposal - again states: code a indicates that the material is described according to archival descriptive rules, which focus on the contextual relationships between items and on their provenance rather than on bibliographic detail. the specific set of rules for description may be found in $e. all forms of material can be controlled archivally. python worldcat api module now available i'd like to humbly announce that i've written a pre-pre-alpha python module for working with the worldcat search api and the xid apis. the code needs a fair amount of work, namely unit tests and documentation. i've released the code under the gpl. the module, called "worldcat", is available from the python package index. you can also checkout a copy of the code from my subversion repository. seriously, follow our lead oclc's lorcan dempsey makes a great point as usual in his post "making tracks": in recent presentations, i have been suggesting that libraries will need to adopt more archival skills as they manage digital collections and think about provenance, evidential integrity, and context, and that they will also need to adopt more museum perspectives as they think about how their digital collections work as educational resources, and consider exhibitions and interpretive environments. i doubt that any archivist would disagree with this. even better, i think this offers a great opportunity to reach out and have those in allied fields really understand how and why we've done things slightly different for so long. i'm glad to see that my new employer has picked up on this holistic approach with platforms like the nypl blogs. now, it can be told after a little over two years processing, referencing, and cataloging, and hacking at aip, i'm skipping up to the city that never sleeps to join jay datema, josh greenberg, and company in the nypl labs. i'd be lying if i said i wasn't thrilled about this opportunity, and i'm ready to see where my new job will take me. the next major hurdle will be finding a place to live, so if you're privy to anything in brooklyn, please let me know. ica releases international standard for describing functions the ica's committee of best practices and standards released the first edition of the international standard for describing functions (isdf). like much of ica's other work in descriptive standards for archives, isdf is designed to be used in conjunction with established standards such as isad(g) and isaar(cpf), as well as standards in preparation such as isiah. isdf will assist both archivists and users to understand the contextual aspects of the creation of records of corporate bodies. through isdf and related standards, archivists will be able to develop improved descriptive systems that can be potentially implemented using a linked data model. google message discovery amidst this week of notorious hoaxes, google has launched google message discovery as an enterprise-focused add on for its google apps platform. google message discovery goes well beyond a simple and reliable e-mail backup system and provides three key features of interest to records managers: content-addressable storage for electronic mail stored immediately upon sending or retrieval creating explicit retention policies based upon time compliance with relevant laws and best practices straightforward discovery for any use, regardless if internal or concerning litigation google message discovery, as well as other related offerings such as e-mail security, clearly has its origins in google's acquisition of postini last year. postini isn't some startup with dubious or perpetually beta offerings (e.g. dodgeball or grandcentral); some of their better known clients include basf and merrill lynch. at $ per user per year, the service seems to be an incredible steal. easy peasy: using the flickr api in python since i'm often required to hit the ground running at $mpow on projects, i was a little concerned when i roped myself into assisting our photo archives with a flickr project. the first goal was to get a subset of the photos uploaded, and quickly. googling and poking around the cheeseshop led me to beej's flickrapi for python. little did i know that it would be dead simple to get this project going. to authenticate: def create_session(api_key, api_secret): """creates as session using flickrapi.""" session = flickrapi.flickrapi(api_key, api_secret) (token, frob) = session.get_token_part_one(perms='write') if not token: raw_input("hit return after authorizing this program with flickr") session.get_token_part_two((token, frob)) return session that was less painful than the ppd test for tuberculosis. oh, and uploading? flickr.upload(filename=fn, title=title, description=desc, tags=tags, callback=status) using this little code plus a few other tidbits, i created an uploader that parses csv files of image metadata exported from an access database. and when done, the results look a little something like this. movin' and shakin' in the archives world archivesnext recently discussed library journal's annual list of "movers and shakers," pondering what a comparable list in the archival profession would look like. for those who don't know, the list recognizes "library advocates, community builders, . gurus, innovators, marketers, mentors, and problem solvers transforming libraries." after some rumination, archivesnext is now calling for nominations to generate a similar list. do your civic duty and nominate either a project, an individual, or even a situation worthy of this recognition! behind the times: where i finally speak of code lib ok, ok. a post about code libcon is long overdue. the minor details: the weather was nice, food was decent, good beer was abundant, and live music was enjoyable. onto the real meat... this time around, i felt like i got a whole lot more out of attending; i'm not sure if this is due to the changing nature of my job, increased attention, or some other factor, like neckferrets and dongles. the great majority of the talks, be they keynotes, traditional presentations, or even just lightning talks, were excellent. furthermore, this time around i felt a whole lot more connected to the miasma - so much so, in fact, that i ended up giving two lightning talks (or three, depending on if you consider the one i gave with gabriel farrell on kobold_chiefain fac-back-opac). the most impressive thing overall, though, were lolcats that came out to play: thanks to the work of noel peden and dan scott, the videos should be up soon enough. dataportability.org and the dream of a web . backup system i just discovered dataportability.org through peter van garderen's blog post about it. i was entirely surprised that i'd heard nary a peep about it. some basic examination (running a whois query on the domain) shows that it's still a fairly new project. i have to say, though, that i'm entirely impressed. those involved have given a whole lot of thought to how they're going to be doing things, as evidenced by those who have signed up to be involved and the dataportability charter. to wit, the charter's principles tend to speak for themselves: we want sovereignty over the profiles, relationships, content and media we create and maintain. we want open formats, protocols and policies for identity discovery, data import, export and sync. we want to protect user rights and privacy. and, of course, the thing that made me squeal with delight like a pig in mud: . dataportability will not inventing any new standards. i mean, that's probably the best news that someone like me could get. announcing zgw.py, or, how i stopped worrying and learned to love z . after more than a few late nights and long weekends, i'm proud to announce that i've completed my latest pet programming project. zgw.py is a lightweight z . -web gateway, written, naturally, in python. none of this would be possible without the following python modules: aaron lav's pyz , the beast of burden; ed summers' pymarc, the smooth-talking translator; and web.py, quite possibly the best and most straightforward python web framework available. i initially undertook this project as an excuse to play with pyz and to teach myself the workings of web.py; i'd played with django, but it seemed entirely excessive for what i was working on. first, i should mention that zgw.py isn't designed to be a complete implementation of a z . gateway. there are many areas in which there is much to be desired, and it's probably not as elegant as some would like. however, that wasn't the point of the project. my ultimate goal was to create a simple client that could be used as a starting point from which to develop a complete web application. no excuses to the power of infinity i have no excuses for not updating this blog. i thought about forcing myself to comply some sort of resolution - you know, given the new year and all - but everyone knows how those turn out. regardless, i have a whole backlog of things to post about, most notably being the countless python programming projects i've been working on lately. expect more posts to arise over the next few days as a result of this. also, i have no excuses for botching up archivesblogs temporarily by mucking about and wiping out some of wordpress's databases that make feedwordpress, the plugin that grabs content for archivesblogs, do its thing. the recovery was simpler than i thought it would be, but this is probably the largest amount of unplanned downtime we've had. keep your eyes open, as a replacement for feedwordpress may itself becoming along sooner or later. web . , disaster, and archives many of web . 's detractors argue about it's real value, but given the wildfires in southern california, i was happy to see it really put to good use. kpbs, a san diego radio station, has been using flickr and, even more shocking (at least for some), twitter as ways to disseminate information and news quickly. the use of twitter is particularly interesting as it can send out sms messages. you might recall a few years ago when protesters in the philippines used sms to organize political rallies and warn of police retaliation. the california state library blog also has provided information from the california state archivist about archives affected by the fires. in addition, information about disaster recovery for libraries and archives is available both on a regional level by the san diego/imperial county libraries disaster response network and on the state level by the california preservation program. please hold those affected by the fires in your thoughts, and if you can, contact sildrn or the cpp to help. archivesblogs upgrades & related weirdness i've updated archivesblogs to the latest version of wordpress, as well as the latest versions of the plugins that do the heavy lifting (feedwordpress and auto delete posts). in so doing, i found that the database structure of wordpress . is radically different, causing some of my elegant work to break (namely, the use of the auto delete posts plugin, for which i wrote a patch). you may have seen duplicate posts, no new posts on specific feeds (language and blog type), and possibly other unpredicted outcomes. everything seems to be working properly now, so if you see anything strange or that doesn't work, let me know. dust in the wind(y city) saa came and went. everyone knows that i'm no good at liveblogging or semi-liveblogging, so don't expect an exhaustive report - potentially better sources include archivesnext and spellbound blog. here are my personal highlights, which is just about the best that this here boy archivist can pull off. the pre-conference saa research forum. while i only got to see the second half of the day, this is where the meat was according to those who were there for the whole thing. the description section steering committee meeting. this was probably the most instructive for me as i'm the incoming chair. hacking away on my remarks most of the week and successfully pulling off our session. jennifer schaffner from oclc/rlg programs substituted for merrilee proffitt and did a swell job. she's a great person to discuss all these crazy ideas with for two reasons - she's established in the profession and new to oclc! i eagerly await her posts at hangingtogether. hey chicago i'm in the windy city for saa . i'll be pretty busy the first few days in town, but remember, if you want to find me, just look for the glasses. also, make sure you come to the description section meeting and session on friday! happy birthday archivesblogs! it was one year ago today that i made archivesblogs available to the public. time sure seems to fly by fast! since then there have been a lot of changes - layout, platform, and hosting - but still, i remain involved for the long haul. thanks to all who provided suggestions, submitted blogs to be syndicated, and any other guidance along the way. archivesblogs now syndicates nearly blogs in languages! when life hands you marc, make pymarc it's a bad pun, but what can you expect from someone who neglects his blogs as much as i do? i've been busy, somewhat, and one of my latest forays has been getting a grip on python, an absolutely wonderful programming language. i actually enjoy writing code again, which is more than a bit scary. i was sick of the mangled scripts and workflows i came up with at mpow to handle converting marc data to html and other such nonsense. writing perl made me feel unclean. after playing around with ed summers' pymarc module, i began hacking about and putting my own hooks into the code here and there. i longed for marc to unicode conversion, which is a necessary evil. digging around, i came across aaron lav's pyz module, which had its own little marc code. after bugging ed via #code lib, and hassling aaron in the process, ed began incorporating the code and i started some testing. just a short while later, the conversion code worked. archives camp: talking about archives . archivesnext recently discussed the possibility of having some "archives . "-themed events this summer, and i think it's a great idea. now, we may not be able to throw something together in time for saa, but it seems like the idea of at least meeting up informally is percolating. there's a wealth of opportunities available for archives and archivists to improve access to their holdings through social software and the like. my vision, as i said in a comment on the post, would be to end up with an unconference along the lines of a library camp (or more generally, a barcamp), maybe with lightning talks if enough of us have something to show off or talk about. like library camp, i'd like to see a "bridging the gap" session where we learn and share ways about how to talk to it staff and other stakeholders essential to our ideas taking off. i facilitated a such a session at library camp east, and although trying at times, it was really instructive. nara frees their data, somewhat i'm a bit surprised that this hasn't come across anyone's radar, because it seems awfully damn significant to me. according to this post on the a&a listserv by michael ravnitzky, the national archives and records administration released an exhaustive database of box holdings of all the federal records centers. he doesn't really say how he obtained this database, but my guess is he just asked based upon his background and interest in public access to government information - i've come across his name on material relating to foia before. the file he received from nara is a mb microsoft access database, and soon after he posted about it to the listserv, jordan hayes and phil lapsley took the opportunity to host the database, converted it to mysql, and wrote a few simple query forms for the database in php. hayes also provided some basic documentation on how to use the forms since mysql query syntax is probably not familiar to most people on the a&a list. sticking my neck out it's been some time since i've had a substantive post, and i don't really intend to write one now. i figured i should mention, however, that i've been featured lately in print and in the blogosphere. jessamyn west of librarian.net interviewed me for an article ("saving digital history") in library journal netconnect. in addition, i was tapped by the wonderful folks at booktruck for the latest installment in their "ask a male librarian" series. i swear someday soon i'll write something much more interesting and less self-promotional. upgrading kubuntu to feisty beta breaks privoxy while i fully intend to go over my full experience upgrading to the latest development release of kubuntu, one of the things that i first noticed was that privoxy didn't seem to work or to be speaking with tor, preventing me from that lovely "anonymous" browsing experience. i noticed that in the upgrade the ever important "forward-socks a / localhost: ." line in /etc/privoxy/config wasn't in the upgraded version (actually, it shouldn't be). apparently during the upgrade, i told it to clobber my config file with the one distributed, saving my old version (luckily) to /etc/privoxy/config.dpkg-old. once that i added that line back, i'm now able to surf a bit more safely. protection from human pests a few months ago (while i was at naco training) i got a reader's card at the library of congress. for a while i pretty actively went and requested books on saturday afternoons. in particular, i was interested in archival manuals from outside the united states. one of the most interesting books i found was s. m. jaffar's problems of an archivist, a manual written in pakistan in . i was struck by the following passage ("protection from human pests"), taken from pp. - : "human pests" and "white huns" are the common epithets applied to human species acting as enemies of archives. history has recorded many such instances of vandalism as the wholesale destruction of priceless treasures of art and literature, the burning of big and beautiful libraries, the transport of camel-loads of books to distant countries and the sale of valuable manuscripts at ridiculously low prices. the transfer of artistic and literary treasures of subjugated countries by the conquerors to their homelands to adorn their own museums and libraries has depleted those countries of that wealth. five non-library blogs i read i won't bother waiting to be tagged to do this, because all the cool kids already are. i read too many blogs already, so here we go. mary eats is, as one would easily assume, a blog about food. mary started the blog while she and her husband were living in korea, and thus there's an overwhelming emphasis on korean food and restaurants. she moved to seattle relatively recently and began culinary school, too. my two favorite parts of this blog are when she makes videos and when she makes comics, like this one about konbu. language log is a blog written by linguistics faculty from around the world, wherein they tackle important and not-so-important issues like linguistic prescriptivism, scammers, the pirahà language, and cheese steak rolls served at chinese restaurants in philadelphia, all with a good sense of humor. information aesthetics covers all sorts of stuff related to information visualization. essentially, it's just one massive blog full of data porn, from treemaps to youtube videos using isotype symbols. two work-safe tidbits about archives and erotica first, via my associates at booktruck.org, i came across a review of the comic book demonslayer v. . , by a certain marat mychaels, et al. at comics should be good. while the fact that the reviewers pan the comic book seems only marginally of interest to those of us wading in archivy, i should draw your attention to a specific part of this issue. apparently one of the characters goes to visit the director of archives at the new york museum of natural history, who has chosen to decorate his office in the style of some seemingly life-sized works by (fellow peruvian) boris vallejo. secondly, everyone knows how much of a pain digital preservation is, particularly in terms of born-digital cultural materials. so, who should archivists and curators look to for guidance? kurt bollacker, digital research manager at the long now foundation (and formerly of the internet archive), holds up the pornography industry as a potential leader of the pack. possible archivesblogs downtime: software upgrade i finally noticed that feedwordpress, the plugin i use to maintain archivesblogs, has been updated within the last month to work with wordpress . and higher. i hope to get this working pretty soon, but i apologize in advance if it ends up going down for a few days. throwing out the baby, the bathwater, and the bathtub: the sad state of the archives and archivists listserv today, nancy beaumont, executive director of the society of american archivists, made an announcement on the archives & archivists listserv that saa would no longer retain the first thirteen years of posts from the listserv. during this time the listserv was hosted by miami university of ohio, and last september, the list was moved to an saa server. this stems from a decision made by saa council that they not retain the archives for three reasons: ) an appraisal decision informed by the saa's archives at the university of wisconsin - milwaukee, ) a consideration of administrative issues, and ) a consideration of cost. while the appraisal decision is well-informed by the claim that the list archives do not have evidential value as saa records, the belief that these records have little informational value does not sit well with me. the list archives document the development of archives becoming a stronger profession in the face of technology and the creation of a tight-knit social network. braindump i'm really behind on posting, and i apologize. there are a few action items that i should mention before i clear my brain to allow me to start posting things with actual content. archivesblogs moved, but mail to archivesblogs.com was not working for a while. a few people mentioned this to me, but i didn't get this resolved until just last week. after who knows how many attempts trying to get something posted on boing boing, i finally made it when i had more information about the hottest chili peppers in the world. i now have a food blog, so if you're interested, check it out. it's called feeding the hungry ghost. now that that stuff is out of the way, i can start posting about "important" things again, like my trip to georgia for code lib . tomato "foam"? i know, i know - you're probably thinking "foams are so over," regardless which side of the molecular gastronomy fence you sit on. if you're a fan of the strange powders and physical state changes of food, you might be saying "c'mon, everybody knows that espuma is the new foam!" yeah, right - and aire is the new espuma. they're all pretty much the same thing, and you've got to be bullshitting yourself if you think that adrià and his ilk don't know this already. if you're convinced that all this stuff is mumbo jumbo designed to take away from traditional technique, then fine. i don't particularly care either way. i made a foam that wasn't really a foam ... or was it? i was bored tonight when i was about to make supper for myself. yesterday i got a whole bunch of free samples from national starch, but i haven't really been able to do anything with them since i've left them sitting in my office. eatin' fresh and (mostly) raw i left work early yesterday for a doctor's appointment, which left little time for lunch. on the way there, i snacked on some almonds and raisins to tide me over. by the time i finally got done with the tests and consultation my stomach was making unholy groans that sounded like ghosts were plaguing my gi tract. since there were a few things i wanted to pick up anyhow, i headed to whole foods and stopped by the deli first to get a sandwich. for what it's worth, i got the "tuna niçoise" sandwich, which wasn't all that niçoise (it tasted alright, though). despite warnings of the possibility of olive pits listed on the wrapper, i couldn't find a single piece of olive anywhere close to it except in another area of the deli case. the sandwich was much larger than the amount of food i've become used to eating in one sitting, so i roamed the aisles stuffed to the gills with tuna, bread, green beans, and hardboiled eggs. adventures in fermentation: yogurt for beginners i decided somewhat spontaneously to make my own yogurt after casually reading about the process and realizing how incredibly simple it is. after consulting a wide variety of sources - both print and electronic, like any good information professional would - i set to the task at hand. mise en place a large stainless steel or aluminum saucepan a large stainless steel, aluminum, unglazed ceramic, or heat-resistant glass bowl a large wooden or stainless steel spoon a kitchen thermometer a heating pad towels a ladle containers for storing the finished yogurt ingredients quart of high-quality, organic milk pint of organic heavy cream (if so desired) / to / cup organic yogurt with live cultures (see below) some, but not all, of the directions that i read suggested that you sterilize all equipment before you begin making the yogurt by immersing it in boiling water. if you decide to do so, i would strongly suggest that you avoid using plastic containers to store the yogurt. archivesblogs on the move thanks to the wonderful people at ibiblio, archivesblogs will be changing hosts! if you're not familiar with ibiblio, it's one of the largest and oldest public digital library collections on the internet. in addition to the upcoming hosting of archivesblogs, ibiblio also hosts librarian.net and library web. pardon any interruptions in access given the impending move; everything should be settled within a few days. also, a few changes i've made to the backend should fix most of the continuing issues with certain feeds not aggregating. let me know if there are any problems that still occur. archivesblogs revamped after many late nights toiling away, i'm done with the latest version of archivesblogs. i've changed things quite a bit - most notably, i've switched platforms from plagger to wordpress using the feedwordpress plugin to do the heavy lifting of syndication. i've decided to do away with the old opml structure as well since the taxonomy wasn't as refined as i would have liked. instead, feedwordpress can categorize posts as they come in, which has allowed me to create a brand new taxonomy for archivesblogs based on language. each language can also have its own feed now. the one thing missing that i'm really itching to put back in place are the social bookmarking links; none of the plugins i've come across so far seem to like my theme, so i may just end up writing my own plugin. anyhow, please give me feedback - i'm itching to do more. is open data the point? i've been thinking about the biblioblogosphere's reaction to casey bisson's decision to use the $ , he was awarded by the mellon foundation for his work on wpopac to purchase lc bibliographic data and open it up to anyone who wanted to take a crack at it. yes, this is a "good thing," and valuable to the library community as a whole, but i feel like there are some things we're overlooking. dan chudnov and i seem to agree, but i'm not going to go so far to damn those who herald this as a "new era." it's a little premature to say where it will go, but i have to admit that i'm occasionally confused and often a little bit insulted by some of the talk surrounding this issue. i wonder how interesting all the bibliographic data of lc is to begin with. what's in the dump paid for by the mellon award money? i'd guess monographs and serials, and probably audiovisual materials. the state of open source archival management software it's been a while since i've written here, but other responsibilities at both at home and work have kept me busy. to get back into the swing of writing regularly, i thought i'd take a look at one of the biggest hot-button topics in archives this year: the development and release of open source archival management systems. between this year's and last year's saa conferences, there were three sessions that, at least in part, dealt with the development of open source software for archives. in turn, this reflected the three major projects that archivists have been developing: archivists' toolkit, archon, and ica-atom. archivists' toolkit is the oldest of the three projects; the first meeting and project proposal date from . it may very well be the best funded of the three projects, as it received a $ , grant from the mellon foundation. however, it also seems to be the least mature, in my opinion, as i've not seen a live demo that's publicly accessible. marac friday afternoon report the mid-atlantic archivists are in a brief recess between now and the final session of the day, and it's been thoroughly interesting to say the least. i missed the caucus meetings this morning, unfortunately, but the plenary session was well worth it because it's got the gears turning about archival access systems even though it wasn't directly about them. paul israel of the edison papers project spoke at length about edison's legacy and collaboration with others. the talk emphasized that thomas edison was much more than a great inventor and owed a great deal of his success to his entrepreneurial nature, which i didn't know much about. while we didn't get to see him give us an interactive presentation of the site, i noticed how exhaustive the digital edition was. while the architecture of the site is a little confusing for me, there's so much content i didn't know where to begin or even what to search for! the series notes are a great way to browse through the collection, though. morristown calling: marac fall meeting i'm at the westin governor morris in morristown, new jersey for the marac fall meeting. i just got back from visiting the morris museum with a few folks, and now i'm enjoying the (expensive) wireless connection here. this time around i don't know so many folks here, so shoot me an e-mail or comment if you're in attendance. expect a more detailed post soon; i'm exhausted from being up early to catch amtrak! library camp east post-mortem i know this post is well overdue, but the last few weeks have kept me extremely busy. library camp east was amazing; fun, thought-provoking, and inspiring. john blyberg and alan kirk gray (as well as the rest of the darien library staff) did a heck of a job preparing for all of us descending into the auditorium. they even gave me a cool mug that my co-workers envy. i also finally got to meet dan chudnov and casey bisson, whose blogs i've followed for a while now. jessamyn west and john posted nearly exhaustive lists of posts by lce attendees for reference. (for what it's worth, jessamyn also tips her hat to archivesblogs and apologizes for us not meeting at two conferences so far. i share the blame!) fortunately for my readers, i have precious little to add in terms of comments (although i tagged some library camp-related links on unalog). i actually was called into service to lead a session by accident (i happened to be scratching my nose), but i was happy enough to moderate the discussion on how techies and non-techies can learn to talk to each other. archivesblogs . after doing some frantic hacking this week i'm happy to announce that i've unveiled the second major revision to archivesblogs. other than a change in color, i have added the subscription list in the sidebar using a slightly modified version of dan mctough's optimal browser for opml.the opml file it renders is also the subscription list used by plagger. anyhow, let me know what you think. i'm sure there are some kinks that need to be ironed out. i'm off to library camp east early tomorrow (a : am train out of dc). i hope to write-up a post-mortem soon after. on what "archives blogs" are and what archivesblogs is not i had fully earmarked addressing thomas g. lannon's "archive blogs" post on documenting sources, his blog, for over a week now after discovering it in my requisite vanity search of technorati. other things (even reading) have kept me busy, though, hence the unintentional neglect. i've had plenty of time to reflect upon it at this point, so i might as well respond to some of his points. he first asks the following: what is an archive blog? this should be a crucial question as the growing field of "blogs about archives" offers up posts stretching from the recent saa conference to south carolina gamecocks. perhaps it would it be helpful to make a distinction between official blogs relating to news and services from archival repositories and personal blogs written by people who happen to work in archives? it is an important question indeed. when i came up with the idea for archivesblogs (and when i was still calling it " saa session proposal: the changing nature of description and opacs during the description section meeting at this year's saa conference, i made an informal proposal for a session concerning the changing nature of opacs, changes in the library cataloging world, and the impact of those on descriptive practice in archives and manuscript repositories. i'd like to invite any of you, if you're interested, to let me know if you'd be interested in assisting me with putting together a proposal on this topic. a small group of us met briefly after the description section meeting and discussed the possible formats and areas of discussion. we determined that a seminar-style discussion seemed most appropriate, with perhaps a brief presentation on a specific area presented by the panelists on a given aspect of these issues. possible areas for presentation and discussion include: the changing nature of the opac in the library world: open-source, problems with vendors, adding web . -like features (the "next generation of finding aids" session at this year's conference included good examples of this) the impact of changes at lc and the oclc/rlg merger: lc's decision to end creating series authority records, rumors of abandoning lcsh, decreased importance of cataloging in general to lc administrators, the future of nucmc and archivegrid the impact of meissner and greene's " coming soon: archivesblogs . ? after two weeks of use, plagger has proven itself to be pretty resilient. i've been asking myself how i can make archivesblogs even better, and i've finally got a few ideas. a site redesign. i'd like different colors. categorizing the feeds, e.g. separating blogs by individuals from repository blogs. this will probably end up with me creating a couple of plagger configurations and dumping them into different subdirectories on archivesblogs. better support for tags. it'd be nice to pull them out and have automagically linked technorati tags. scrubbing html from the feeds to create valid xhtml for the syndication page(s). plagger supports the perl module html::scrubber so it seems. this is a big deal to someone like me. adding a directory - most likely in opml - for as many blogs about archives and archivists as possible since it's just not possible to do that for some blogs using plagger. the most straightforward example are archives with blogs that are part of a library-wide blog and therefore don't have their own feeds. archivesblogs news: disappearing blogspot blogs archivesblogs has been going strong for over a week now. if you use blogspot and had a blog previously syndicated by archivesblogs, your content may be temporarily unsyndicated. the specific problem is http error, which seems to indicate a problem with a proxy server at blogspot. in any rate, they should return soon enough -- it would be nice to have the blogs back! archivesblogs update: service links i've upgraded plagger (the software behind archivesblogs) to the latest version and it's allowed me to add service links to del.icio.us, unalog, digg, reddit, and technorati. i suppose i could add more (ma.gnolia, furl, etc.), but i'll hold off doing that for the sake of cluttering the interface for the time being. if you have any service links you'd like to see, let me know and i might be able to hack something together. announcing archivesblogs since my last post about syndicating blogs about archives, i've played around with the idea and different software packages to do it, including planet and plagger. i'm happy to announce that after a few days work i was able to put something together. archivesblogs is an aggregator for blogs about archives. it runs plagger and updates hourly, outputting html, rss, atom, opml (for import into other aggregator), and a foafroll. the site design is simple, but i'm happy with it. i took whatever archives blogs i knew about and added them, so if you know of any others or you want yours removed, let me know. syndicating archives blogs i still haven't had enough time to process everything i took in or ideas i came up with as a result of the saa conference. many were more diligent than i and i'm sorry to say i didn't meet them, but some highlights follow: geof huth took notes on the saa awards ceremony, christie peterson pitted archon against archivist's toolkit, jessamyn west blogged about her session on blogs, peter van garderen discusses his experience at the conference including his session on archives and web . , and merrilee proffitt from rlg mentioned the blog session and rlg roundtable. i'm not even up to speed on the rest of the archival blogs out there. in a stroke of genius and madness i've got an idea that i may put into motion. i'm thinking about setting up an instance of planet, a python-powered web-based news aggregator. it's pretty common in the floss world, and has been picked up by the code lib folks; they're running theirs as planet code lib. report from saa: archival solidarity and international cooperation the archival solidarity session was really great and generated a lot of dialog. it was originally organized by nancy marrelli of concordia university (montréal), but she couldn't make it on account of a family emergency. trudy huskamp peterson led the discussion in her place and did a wonderful job. essentially, archival solidarity is a project involving the ica's section of professional associations that concerns "international archival development" through bilateral projects. there are several major issues at play. first, existing methods of international development are not working for archival projects, either because of bureaucracy in general or archives being of lower priority in comparison to needs such as sanitation, adequate health care, and the like. we identified that one of the most critical aspects is the lack of communication or methods to share information. there is no central "hub," formal or informal, that allows archivists to share information about assistance needed or offered. the international fund for archival development (fida), coordinated by the ica, was supposed to serve as such, but apparently operational issues prevent it from working effectively. report from saa: give me free wifi i'm at the hilton washington, the site of the saa conference. i've registered and picked up my free totebag. i, and others, have bemoaned the lack of connectivity in the conference area. wireless is only available in the lobby, so it seems, and it's rather pricy ($ . for hours or $ . for hours). i know archivists are often thought of as being technologically behind (whether we are is a pandora's box that i won't open in this post), but i feel that some sort of net access is necessary at every conference. i'm just barely able to get it through my cell phone, which is how i'm posting now. unfortunately, i get no reception on the conference floor so i needed to make my way up to the lobby anyhow. i missed the standards committee meeting since i was a little late and i didn't want to barge in since the doors were closed. it's nearly time for the archival solidarity session, which sounds interesting to me since i'd like to get involved in ica. conference time i'm one of several bloggers attending the saa conference the rest of this week. nothing against cosa or nagara, but i'm attending the conference for the organization to which i belong. my schedule is pretty packed, and if you're one of us be sure to attend the description section meeting since i'm running for vice-chair. socketscdr audio zine out soon! i'm going to be on the latest installment of the socketscdr audio zine, curated this time around by rebecca mills of the caution curves. sean, the socketscdr label honcho, just posted the cover artwork for it and it looks like a great line-up, including friends like the plums and stamen & pistils. this will be my first release in a while (other than the collab cd with myself, cotton museum, and actual birds on casanova temptations). more details will follow, naturally. upgrading kubuntu breezy to dapper upon hearing about yesterday's release of kubuntu . , i decided to upgrade from the previous release, kubuntu . . i'd like to say that it went off without a hitch, but it didn't. it did, however, go mostly well, and i realized that my problem was that i continued to use applications while adept installed the new packages. i couldn't install all the packages, and i ended up with a minorly disfunctional kernel that wouldn't allow ndiswrapper to load properly, preventing me from using my internal wireless card. once i rebooted (and used a spare pcmcia wireless card to gain connectivity), i was able to finish installing the rest of the packages that had not finished properly and rebooted again. everything pretty much worked, but i'm having to tweak some lost settings, most notably in kmail. other than that, it's been working out fine! rlg + oclc = clog roc? the technical services world has been in an uproar lately, between lc's decision to stop creating series authority records (particularly since they didn't consult pcc members beforehand) and the fallout after calhoun report. we might as well have another drink, because as librarian.net reports (along with several others), oclc and rlg are about to merge. it's mindblowing to think that rlg employees did not find out any sooner than the rest of us, and that either organization has yet to consult its members. however, rlg plans to do so, but it will be interesting to see how this pans out. in particular, some folks worried about the merging of data and the future of redlightgreen. i know it's not considerably better, but they seem to be overlooking open worldcat. change of platform nearly a year ago i switched from wordpress to drupal. i chose to switch back, partially because it was capable of doing way more than i needed it to! i thought i didn't want to be limited by blog software, but apparently that's not a terribly huge concern anymore. the old site had frightfully little content (three posts in dalliance, a few personal posts, and links to papers). i'm redoing my non-blog site with purls since i don't have access to an e-prints server to which i can upload my varied previous academic work. anyhow, the important stuff is soon to come, with dalliance possibly moving to another host (probably wordpress.com). anything linking to one of the papers or my code snippets will be edited as needed. an updated version of nick gerakines' mail rss.pl a little over a month ago, nick gerakines posted a perl script to be called from a procmail configuration file. it seemed to work pretty well, but the anal-retentive cataloger/standards geek in me decided to pass the results through a feed validator. it failed in a few key areas: missing version attribute in the rss tag, improper guid and link tags, and a pubdate with a non-rfc date. these all seemed pretty easy to fix, so i went ahead and made some changes. my fixes are a bit inelegant, but they create valid rss . . it was pretty trivial to add an rss version number and to fix the guid error; the latter just required adding the ispermalink="false" attribute to that tag. however, nick's original code required parsing the pubdate tags to determine when to kill data that was over hours old. i didn't want to be bothered parsing an rfc date with this, so i moved that information into a category tag. mid-november updates: dalliance off the ground! site changes galore! dc not bad! i've finally gotten around to doing some serious work on the site. i've completed the first post for my defunct blog, and it's about one of my favorite songwriters ever, dr. franklin bruno. i've also figured out some of the odd intricacies of drupal and am finally getting this site to have a look and feel of which i can be proud. i've settled in nicely to washington, dc, and i'm living in a decent area of town within a reasonable interest of a decent watering hole, groceries, and the metro. halloween has come and gone; i dressed up as everyone's favorite st. vitus dancer, ian curtis, complete with requisite noose. my friend corey took similar cues as far as the era and scope of his costume, and chose to dress up as henry rollins. the weather has stayed mostly warm, so i've been spoiled on that front too. more changes are coming soon, so stay alert. off on my way: in transition to washington, dc i'm pleased to announce that i will be joining the staff of the national anthropological archives and human studies film archives of the smithsonian institution's department of anthropology as a project archivist. i will have two initial primary responsibilities: cataloging plains indian ledger art for the non-profit artstor project, and original cataloging and bibliographic enhancement of audio, film and video collections in support of the naa's new endangered languages program. this program also collaborates with the university of utah's center for american indian languages and is also part of the documenting endangered languages project, supported by the national science foundation and the national endowment for the humanities. i will be starting work for the naa/hsfa on september , , and will be working on a month term contract. samvera wiki {"serverduration": , "requestcorrelationid": " ac cfd "} libraries | university of cincinnati skip to main content use the form to search uc's web site for pages, programs, directory profiles and more. libraries online library for faculty for graduate students for undergraduates for staff libraries archives and rare books about the archives and rare books library annual summary research policies staff faqs desiderata collections urban studies rare books university archives german americana local government records search arb collections records management disposal submission form online exhibits special projects services genealogy research image reproduction and use archives and rare books teaching support internship program ccm ccm catalog search ccm research about the ccm library ccm staff directory ccml faqs ccm services ccm special-collections ceas about ceas library history floor plan guide for new faculty course research guides research resources ask a librarian tutorial videos senior design reports special collections the armstrong collection the cooperative engineer the strauss collection contact us ask a librarian reserve a room cech about faculty & staff our collections borrowing guidelines find us services poster printing reserves instruction resources makerlab technology for checkout study spaces info commons chemistry-biology about staff oesper history of chemistry collection services getting around the library ask a librarian classics about the classics library classics library guide snapshot of the classics collections highlights of classics books classics collection development policy why a classics library? classics book of the month classics library's open access link of the month classics library book desiderata virtual tour of the classics library usage statistics staff directory recent book acquisitions classicizing cincinnati classics library policies classics library collections german classics dissertations modern greek journal collection classics map collection greek rare book collection latin rare book collection classics books with author signatures uc department of classics archive classics library services group study room in classics scanners, printers, copiers in classics tours and drop-ins classics library picture gallery daap collections architecture drawings related regional libraries exhibits instruction services study rooms contact us daap library covid- updates geology-mathematics-physics about the library history getting around the library help ask-a-librarian help for faculty help for students help for undergraduate students services new books special collections rare book collection willis g. meyer map collection guidebook collection health sciences services membership room reservations borrow hsl-it research help hsl history hsl directions hsl staff directory winkler center about cecil striker society & lecture resources services langsam law uc blue ash about the ucba library ucba library faculty & staff ucba library policies vision and core values annual reports student employment at ucba library covid- services faqs borrowing materials at ucba library borrowing & returning reserves equipment lending study spaces resources for ucba faculty and staff course reserve guidelines collections library liaison program space reservations teaching support ask the ucba library uc clermont library about uc clermont library student employment support the library collection development contact us - faqs borrow materials technology & equipment textbook reserves study spaces policies and guidelines teaching support information literacy course materials course reserves ask uc clermont winkler center other area libraries ask find, request, borrow search for materials call number locator (langsam library) borrow materials borrow equipment renew materials request materials reserves e-reserves faculty guidelines traditional course reserves reserves - contacts copyright resources textbook affordability help finding and using materials interlibrary loan special collections faq digital collections research & teaching support research data services lab spaces workshops and education meet the team uc data day data & computational science series data tools testimonials data visualization showcase digital scholarship center citing sources copyright repositories subject librarians uc press teaching support workshops & trainings ask a librarian online reference shelf library materials for online teaching spaces & tech room reservations adaptive technologies library media space student technology resources center borrow equipment about covid- click & collect health and safety protocols hours and locations contact us employment ohiolink-luminaries staff directory giving adopt-a-book funding donors strategic plan tenets pillars ten initiatives news and events policies acceptable use gift policy source library faculty resources for library faculty library faculty directory dean's welcome core beliefs login off campus access affiliate and guest access help and troubleshooting tools vpn interlibrary loan interlibrary lending policies my library record pay fines fine appeal form articles books journals databases search summon to find articles, books, and more advanced summon search | find by doi or pmid | more search options |help search the library catalog for books and more advanced catalog search | guest access | more search options | help find e-journals or print journals e-journals | print journals | browzine | more search options | help search the a-z indexes databases list browse databases | top databases | academic search complete | more search options uc online library whether onsite or online, we continue to connect students, faculty, researchers and scholars to dynamic data, information and resources. uc online library service updates off campus access contact us interlibrary loan research guides browse all guides spring return to campus as we step into spring term , our motto “strength in unity” continues to take on added meaning. health and safety remain a top priority in an environment featuring virtual, hybrid, hyflex and in-person classes, testing as a critical component toward a safer community as well as remote work options. visit uc's public health site online library searching for a resource, have a question or simply browsing for fun? we've brought all online resources together in one place. online library digital technologies & innovation uc libraries creates and utilizes learning tools and research platforms that transform the user experience and the creation of new knowledge. special collections uc libraries preserves and provides access to special collections and the scholarly and historical record of the university, including archival as well as born-digital content and datasets. view with . million volumes and access to thousands of electronic resources available / through our online library catalog, uc libraries' virtual and physical locations offer resources for everyone. uc libraries includes the walter c. langsam library, the archives and rare books library, the donald c. harrison health sciences library, and eight college and departmental libraries serving constituents in applied science, architecture, art, biology, chemistry, classics, design, education, engineering, geology, mathematics, music, physics and planning. give to uc libraries library news "off the shelf and into the lab" webinar may april , event: may , : pm join the henry r. winkler center for the history of the health professions and the cecil striker society for the history of medicine at p.m., wednesday, may , for the third lecture in the cecil striker webinar series. faculty awards : arlene johnson april , through her many roles in her years at the university of cincinnati, arlene johnson has served students, faculty and staff in the pursuit of knowledge — fitting for the recipient of the faculty senate exemplary service to the university award. ‘can uc my mask’ canned food sculpture temporarily installed in... march , the masked bearcat is showing school pride while reminding everyone to stay safe by wearing a mask. debug query for this more news library blog news from the library blog ucba library needs you! now hiring for summer semester fri, apr ucba library needs you! now hiring for summer semester are you… friendly and welcoming? eager to help students, staff and faculty? if so, consider joining the ucba library team! apply: https://libraries.uc.edu/libraries/ucba/about/employment.html april service note: access to library resources is currently down tue, apr update: all access has been restored. ________________________________ all access to library resources through the proxy server is currently down. oclc is working on the issue and we expect a resolution shortly. we apologize for the inconvenience. if you know the resource url you are attempting to access, try this page: https://libapps.libraries.uc.edu/proxy/proxygoto.php. the url for the […] the preservation lab celebrates preservation week : preservation in action mon, apr join the preservation lab april - as they celebrate the american library association’s (ala) preservation week, “preservation in action.” more information, including a schedule of the week’s events, is available on the preservation’s blog. “off the shelf and into the lab” may th webinar to highlight medical history, preservation and the uc libraries’ adopt-a-book program wed, apr join the henry r. winkler center for the history of the health professions and the cecil striker society for the history of medicine, thursday, may at : p.m. for the rd lecture in the cecil striker webinar series. off the shelf and into the lab: medical history, preservation and the university of cincinnati libraries’ […] ending the hiv epidemic, a panel discussion april mon, apr join uc libraries online wednesday, april , : p.m. for “ending the hiv epidemic,” a panel discussion. learn from various cincinnati area hiv/aids service providers about how long-standing hiv prevention efforts combined with education on treatment, viral load suppression and concerted efforts by multiple agencies are being utilized to make hiv infection a thing of […] university of cincinnati libraries po box cincinnati, ohio - contact us | staff directory uc tools canopy & canvas one stop email catalyst shuttle tracker it help uc vpn bearcats landing about us maps & directions jobs news diversity governance & policies directory events calendar university of cincinnati | clifton ave. | cincinnati, oh | ph: - - alerts | clery and heoa notice | notice of non-discrimination | eaccessibility concern | privacy statement | copyright information © university of cincinnati university of cincinnati libraries po box cincinnati, ohio - contact us | staff directory © university of cincinnati chat loading... inkdroid toggle navigation inkdroid about bookmarks photos music software social talks - - ~ - - ~ twarc - - ~ $ j - - ~ strengths and weaknesses - - ~ data speculation - - ~ recovering foucault - - ~ teaching oop in the time of covid - - ~ gpt- jam - - ~ mimetypes - - ~ northwest branch cairn - - ~ blow back derelict wind - - ~ outgoing - - ~ trump's tweets - - ~ noarchive - - ~ what's the diff? - - ~ for - - ~ diss music - - ~ years of robots.txt - - ~ curation communities - - ~ mystery file! - - ~ kettle - - ~ static-dynamic - - ~ dark reading - - ~ seeing software - - ~ curating corpora - - ~ fuzzy - - ~ penny - - ~ fuzzy file formats - - ~ pandoc - - ~ fuzzy matching - - ~ less is (sometimes) more - - ~ teaching digital curation - - ~ rss - - ~ organizations on twitter - - ~ bibdesk, zotero and jabref - - ~ disinformation metadata - - ~ equipment - - ~ twitter - - ~ music for hard times - - ~ digital curation - - ~ dependency hell - - ~ keyboard - - ~ tech tree - - ~ appraisal talk in web archives - - ~ talk talk - - ~ original voice - - ~ write it down - - ~ sun and moon - - ~ first thought - - ~ studying the covid- web « prev next » unless otherwise noted all the content here is licensed cc-by brown university library digital technologies brown university library digital technologies bundler . . and homeless accounts this week we upgraded a couple of our applications to ruby . and bundler . . and one of the changes that we noticed was that bundler was complaining about not being able to write to the /opt/local directory. turns out this problem shows up because the account that we use to run our application is &# ; continue reading bundler . . and homeless accounts upgrading from solr to solr a few weeks ago we upgraded the version of solr that we use in our discovery layer, we went from solr . to solr . . although we have been using solr .x in other areas of the library this was a significant upgrade for us because searching is the raison d&# ;être of our discovery layer &# ; continue reading upgrading from solr to solr pypi packages recently, we published two python packages to pypi: bdrxml and bdrcmodels. no one else is using those packages, as far as i know, and it takes some effort to put them up there, but there are benefits from publishing them. putting a package on pypi makes it easier for other code we package up to &# ; continue reading pypi packages new riamco website a few days ago we released a new version of the rhode island archival and manuscript collections online (riamco) website. the new version is a brand new codebase. this post describes a few of the new features that we implemented as part of the rewrite and how we designed the system to support them. the &# ; continue reading new riamco website deploying with shiv i recently watched a talk called &# ;containerless django &# ; deploying without docker&# ;, by peter baumgartner. peter lists some benefits of docker: that it gives you a pipeline for getting code tested and deployed, the container adds some security to the app, state can be isolated in the container, and it lets you run the exact &# ; continue reading deploying with shiv checksums in the bdr, we calculate checksums automatically on ingest (fedora provides that functionality for us), so all new content binaries going into the bdr get a checksum, which we can go back and check later as needed. we can also pass checksums into the bdr api, and then we verify that fedora calculates the &# ; continue reading checksums exporting django data we recently had a couple cases where we wanted to dump the data out of a django database. in the first case (&# ;tracker&# ;), we were shutting down a legacy application, but needed to preserve the data in a different form for users. in the second case (&# ;deposits&# ;), we were backing up some obsolete data before &# ; continue reading exporting django data searching for hierarchical data in solr recently i had to index a dataset into solr in which the original items had a hierarchical relationship among them. in processing this data i took some time to look into the ancestor_path and descendent_path features that solr provides out of the box and see if and how they could help to issue searches based &# ; continue reading searching for hierarchical data in solr monitoring passenger’s requests in queue over time as i mentioned in a previous post we use phusion passenger as the application server to host our ruby applications. a while ago upon the recommendation of my coworker ben cail i created a cron job that calls passenger-status every minutes to log the status of passenger in our servers. below is a sample &# ; continue reading monitoring passenger&# ;s requests in queue over time looking at the oxford common filesystem layout (ocfl) currently, the bdr contains about tb of content. the storage layer is fedora , and the data is stored internally by fedora (instead of being stored externally). however, fedora is end-of-life. this means that we either maintain it ourselves, or migrate to something else. however, we don&# ;t want to migrate tb, and then have &# ; continue reading looking at the oxford common filesystem layout (ocfl) none lita blog – empowering libraries through technology lita blog empowering libraries through technology toggle navigation about regular contributors get involved! join lita lita jobs jobs in information technology: august , august , august , | jenny levine new this week coordinator of digital scholarship and programs, marquette university libraries, milwaukee wi digital scholarship coordinator, unc charlotte, charlotte, nc visit the lita jobs site for additional job openings and information on submitting your own job posting. continue reading lita jobs jobs in information technology: august , august , august , | jenny levine new this week information systems manager (pdf), the community library association, ketchum, id children’s librarian, buhl public library, buhl, id technology integration librarian, drexel university libraries, philadelphia, pa visit the lita jobs site for additional job openings and information on submitting your own job posting. continue reading core update your core community update august , | chrishelle thomas much has been happening behind-the-scenes to prepare for core’s upcoming launch on september st, so we want to update you on the progress we’ve made. at the ala virtual conference council meetings, the ala council approved the creation of core, so we’re official! it’s been a difficult summer for everyone given the global situation, but this was a milestone we’re excited to reach. what we’ve been doing in may, the core transition committee (the division presidents plus senior staff) formed working groups of members from all divisions to make recommendations about how to proceed with our awards/scholarships, budget/finance, committees, communications, conference programming, continuing education, fundraising/sponsorships, interest groups, member engagement, nominations for president-elect, publications, and standards. these groups have done an amazing amount of work in a very short time period, and we’re grateful to these members for their commitment and effort. we’re happy to report… continue reading education free lita webinar ~ library tech response to covid- ~ august th july , | chrishelle thomas sign up for this free lita webinar: library tech response to covid- libraries are taking the necessary precautions to create a safe environment during the pandemic. social distancing isn’t the only solution, but providing access to loanable technologies, including handling and quarantine of equipment, cleaning, and other safety and health concerns are just some of the measures put in place. with the ongoing disruption to library services caused by covid- , what reopening planning policies should be considered for usage? in this free -minute presentation, our presenters will share tips that might be helpful to other librarians before they reopen. the presenters will also talk about the evolution of the phased plan from the establishment of a temporary computer lab in the library as covid- began to spread in march , to the current phased approach for gradual reopening. justin will also offer insight into managed access, technology and services, workflows, messaging,… continue reading lita jobs jobs in information technology: july , july , | jenny levine new this week library director, walpole town library, walpole, nh visit the lita jobs site for additional job openings and information on submitting your own job posting. continue reading program planning core call for ala annual program proposals july , | chrishelle thomas submit an ala annual conference program proposal for ala’s newest division, core: leadership, infrastructure, futures, which will begin on september , . proposals are due september , , and you don’t need to be a core member to submit a proposal. submit your idea using this proposal form. core welcomes topics of interest to a wide range of library professionals in many different areas, including… . access and equity advocacy in areas such as copyright, equity of access, open access, net neutrality, and privacy preservation week equity, diversity, and inclusion, both within the division and the profession, as related to core’s subject areas . assessment emphasizing the role of assessment in demonstrating the impacts of libraries or library services assessment tools, methods, guidelines, standards, and policies and procedures . leadership and management developing leaders at every level best practices for inclusion by using an equity lens to examine leadership… continue reading education core call for webinar proposals july , | chrishelle thomas submit a webinar proposal for ala’s newest division, core: leadership, infrastructure, futures, which will begin on september , . proposals are due september , , and you don’t need to be a core member to submit a proposal. early submissions are encouraged and will be considered for september and october presentations. submit your idea using this proposal form. core webinars reach a wide range of library professionals in many different areas, including… . access and equity advocacy in areas such as copyright, equity of access, open access, net neutrality, and privacy preservation week equity, diversity, and inclusion, both within the division and the profession, as related to core’s subject areas . assessment emphasizing the role of assessment in demonstrating the impacts of libraries or library services assessment tools, methods, guidelines, standards, and policies and procedures . leadership developing leaders at every level best practices for inclusion by using an equity lens to examine… continue reading core virtual forum core virtual forum is excited to announce our keynote speakers! july , july , | chrishelle thomas core virtual forum welcomes our keynote speakers, dr. meredith d. clark and sofia leung! both speakers embody our theme in leading through their ideas and are catalysts for change to empower our community and move the library profession forward. dr. clark is a journalist and assistant professor in media studies at the university of virginia. she is academic lead for documenting the now ii, funded by the andrew w. mellon foundation. dr. clark develops new scholarship on teaching students about digital archiving and community-based archives from a media studies perspective. she will be a - fellow with data & society. she is a faculty affiliate at the center on digital culture and society at the university of pennsylvania. and, she sits on the advisory boards for project information literacy, and for the center for critical race and digital studies at new york university. clark is an in-demand media consultant… continue reading ital catch up on the june issue of information technology and libraries july , | chrishelle thomas the june issue of information technology and libraries (ital) was published on june . editor ken varnum and lita president emily morton-owens reflect on the past three months in their letter from the editor, a blank page, and lita president’s message, a framework for member success, respectively. kevin ford is the author of this issue’s “editorial board thoughts” column, seeing through vocabularies. rounding out our editorial section, the june “public libraries leading the way” section offers two items. chuck mcandrew of the lebanon (new hampshire) public libraries describes his leadership in the imls-funded libraryvpn project. melody friedenthal, of the worcester (massachusetts) public library talks about how she approached and teaches an intro to coding using python course. peer-reviewed content virtual reality as a tool for student orientation in distance education programs: a study of new library and information science students dr. sandra valenti, brady lund, ting wang virtual reality… continue reading lita jobs jobs in information technology: july , july , july , | jenny levine new this week dean of libraries, san jose state university, san jose, ca deputy library director, city of carlsbad, carlsbad, ca visit the lita jobs site for additional job openings and information on submitting your own job posting. continue reading posts navigation older posts upcoming events bibliometrics for librarians presented by phillip doehle and clarke lakovakis on july , – july , virtual reality, augmented reality, mixed reality and the academic library presenters: dr. plamen miltenoff and mark gill offered: august , – august , core virtual forum visit our website for the latest updates on the core virtual forum in fall . recent posts jobs in information technology: august , jobs in information technology: august , your core community update free lita webinar ~ library tech response to covid- ~ august th jobs in information technology: july , archives archives select month august july june may april march february january december november october september august july june may april march february january december november october september august july june may april march february january december november october september august july june may april march february january december november october september august july june may april march february january december november october september august july june may april march february january december november october september august july june may april march february january december november october september august july june may april march february january december november october september august july june may april march february january december november october september august july june may april march february january december november october september august july june may april march february january december november october september august july june may april march february january december november october september august july june may april march february january december november october september august july june may april march february january december november october september august july june may april march february january november october september august july june categories categoriesselect category a new division discussions ala annual conferences ala midwinter meetings awards and scholarships begin transmission bigwig blogging help committees and interest groups core update education emerging technologies general information imagineering institutional repositories instruction and online learning legislation & regulation library experiences lita board of directors lita bylaws lita elections lita forums core virtual forum lita jobs lita officers news & noteworthy original content podcast president’s post program planning publications ital roundup sf notables spotlight series standards watch sunday routines technical services top technology trends topic + reaction twitter chats uncategorized website management and user experience the blog of the library and information technology association privacy policy powered by wordpress | wordpress theme by tidyhive lita blog lita blog empowering libraries through technology jobs in information technology: august , new this week coordinator of digital scholarship and programs, marquette university libraries, milwaukee wi digital scholarship coordinator, unc charlotte, charlotte, nc visit the lita jobs site for additional job openings and information on submitting your own job posting. jobs in information technology: august , new this week information systems manager (pdf), the community library association, ketchum, id children&# ;s librarian, buhl public library, buhl, id technology integration librarian, drexel university libraries, philadelphia, pa visit the lita jobs site for additional job openings and information on submitting your own job posting. your core community update much has been happening behind-the-scenes to prepare for core’s upcoming launch on september st, so we want to update you on the progress we’ve made. at the ala virtual conference council meetings, the ala council approved the creation of core, so we’re official! it’s been a difficult summer for everyone given the global situation, but this was a milestone we’re excited to reach. what we’ve been doing in may, the core transition committee (the division presidents plus senior staff) formed working groups of members from all divisions to make recommendations about how to proceed with our awards/scholarships, budget/finance, committees, communications, conference programming, continuing education, fundraising/sponsorships, interest groups, member engagement, nominations for president-elect, publications, and standards. these groups have done an amazing amount of work in a very short time period, and we’re grateful to these members for their commitment and effort. we’re happy to report... free lita webinar ~ library tech response to covid- ~ august th sign up for this free lita webinar: library tech response to covid- libraries are taking the necessary precautions to create a safe environment during the pandemic. social distancing isn’t the only solution, but providing access to loanable technologies, including handling and quarantine of equipment, cleaning, and other safety and health concerns are just some of the measures put in place. with the ongoing disruption to library services caused by covid- , what reopening planning policies should be considered for usage? in this free -minute presentation, our presenters will share tips that might be helpful to other librarians before they reopen. the presenters will also talk about the&# ;evolution of the phased plan from the establishment of a temporary computer lab in the library as covid- began to spread in march , to the current phased approach for gradual reopening. justin will also offer insight into managed access, technology and services, workflows, messaging,... jobs in information technology: july , new this week library director, walpole town library, walpole, nh visit the lita jobs site for additional job openings and information on submitting your own job posting. core call for ala annual program proposals submit an ala annual conference program proposal for ala’s newest division, core: leadership, infrastructure, futures, which will begin on september , . proposals are due september , , and you don’t need to be a core member to submit a proposal. submit your idea using this proposal form. core welcomes topics of interest to a wide range of library professionals in many different areas, including… . access and equity advocacy in areas such as copyright, equity of access, open access, net neutrality, and privacy preservation week equity, diversity, and inclusion, both within the division and the profession, as related to core’s subject areas . assessment emphasizing the role of assessment in demonstrating the impacts of libraries or library services assessment tools, methods, guidelines, standards, and policies and procedures . leadership and management developing leaders at every level best practices for inclusion by using an equity lens to examine leadership... core call for webinar proposals submit a webinar proposal for ala’s newest division, core: leadership, infrastructure, futures, which will begin on september , . proposals are due september , , and you don’t need to be a core member to submit a proposal. early submissions are encouraged and will be considered for september and october presentations. submit your idea using this proposal form. core webinars reach a wide range of library professionals in many different areas, including… .&# ;access and equity advocacy in areas such as copyright, equity of access, open access, net neutrality, and privacy preservation week equity, diversity, and inclusion, both within the division and the profession, as related to core’s subject areas .&# ;assessment emphasizing the role of assessment in demonstrating the impacts of libraries or library services assessment tools, methods, guidelines, standards, and policies and procedures .&# ;leadership developing leaders at every level best practices for inclusion by using an equity lens to examine... core virtual forum is excited to announce our keynote speakers! core virtual forum welcomes our keynote speakers, dr. meredith d. clark and sofia leung! both speakers embody our theme in leading through their ideas and are catalysts for change to empower our community and move the library profession forward. dr. clark is a journalist and assistant professor in media studies at the university of virginia. she is academic lead for documenting the now ii, funded by the andrew w. mellon foundation. dr. clark develops new scholarship on teaching students about digital archiving and community-based archives from a media studies perspective. she will be a - fellow with data &# ; society. she is a faculty affiliate at the center on digital culture and society at the university of pennsylvania. and, she sits on the advisory boards for project information literacy, and for the center for critical race and digital studies at new york university. clark is an in-demand media consultant... catch up on the june issue of information technology and libraries the june issue of information technology and libraries (ital) was published on june . editor ken varnum and lita president emily morton-owens reflect on the past three months in their letter from the editor, a blank page, and lita president’s message, a framework for member success, respectively. kevin ford is the author of this issue’s “editorial board thoughts” column, seeing through vocabularies. rounding out our editorial section, the june “public libraries leading the way” section offers two items. chuck mcandrew of the lebanon (new hampshire) public libraries describes his leadership in the imls-funded libraryvpn project. melody friedenthal, of the worcester (massachusetts) public library talks about how she approached and teaches an intro to coding using python course. peer-reviewed content virtual reality as a tool for student orientation in distance education programs: a study of new library and information science students dr. sandra valenti, brady lund, ting wang virtual reality... jobs in information technology: july , new this week dean of libraries, san jose state university, san jose, ca deputy library director, city of carlsbad, carlsbad, ca visit the lita jobs site for additional job openings and information on submitting your own job posting. jobs in information technology: july , new this week web services librarian, chester fritz library, university of north dakota, grand forks, nd visit the lita jobs site for additional job openings and information on submitting your own job posting. jobs in information technology: june , new this week metadata librarian, librarian i or ii, university of northern british columbia, prince george, british columbia, canada visit the lita jobs site for additional job openings and information on submitting your own job posting. jobs in information technology: june , new this week information technology librarian,&# ;university of maryland, baltimore county, baltimore, md associate university librarian for research and learning,&# ;columbia university libraries, new york, ny library technology/programmer analyst iii,&# ;virginia beach public library,&# ;virginia beach, va visit the lita jobs site for additional job openings and information on submitting your own job posting. core virtual happy hour social ~ june our joint happy hour social at midwinter was such a success that next week we’re bringing happy hour to you online—and registration is free! we invite members of alcts, lita, and llama to join us on friday, june , : - : pm central time for virtual happy hour networking and/or play with your peers in a game of scattergories.&# ;wear your favorite pop culture t-shirt, bring your best zoom background, grab a beverage, and meet us online for a great time! attendees will automatically be entered to win free registration to attend the core virtual forum.&# ;winner must be present to redeem prize.&# ;registration is required. register now at: bit.ly/ nenprh michael carroll awarded lita/christian larew memorial scholarship michael carroll has been selected to receive the &# ;lita/christian larew memorial scholarship ($ , ) sponsored by the library and information technology association (lita) and baker &# ; taylor. this scholarship is for master’s level study, with an emphasis on library technology and/or automation, at a library school program accredited by the american library association. criteria for the scholarship includes previous academic excellence, evidence of leadership potential, and a commitment to a career in library automation and information technology. the larew scholarship committee was impressed by&# ;what michael has already accomplished and look forward to seeing what he will accomplish after graduation in .&# ;michael&# ;has already shown&# ;a strong interest in digitization projects.&# ;he currently manages&# ;a&# ;team of students working with digitization.&# ;previously, he has scanned and cataloged many collections.&# ;he has also assisted the presbyterian historical society&# ;in creating&# ;sustainable processes for digitization.&# ;michael has also shown his willingness and ability to work&# ;with a wide variety of&# ;projects and technologies that span&# ;both&# ;technical&# ;and non-technical&# ;including... we are back on twitter friday for #litachat the fourth in this series of #litachats will start on friday, june from - central standard time on twitter. we will be asking you to chat with us about self-care. what are you doing to take care of yourselves during this time? how do you unplug without feeling guilty?&# ; we hope you’ll join us for #litachat and chat about self-care techniques and figuring out how to better take care of ourselves during these tough times. we&# ;re looking forward to hearing from you! join lita on twitter catch up on the last #litachat join us for alcts/lita/llama e-forum! please join us for a joint alcts/lita/llama e-forum discussion. it’s free and open to everyone! registration information is at the end of the message, along with subscription management options for existing listserv members. continuing to manage the impact of covid- on libraries june - , moderated by alyse jordan, steven pryor, nicole lewis and rebecca uhl please join us for an e-forum discussion. it’s free and open to everyone! registration information is at the end of the message. each day, discussion begins and ends at: pacific: a.m. – p.m. mountain: a.m. – p.m. central: a.m. – p.m. eastern: a.m. – p.m. over the past several months, covid- has significantly impacted libraries and library technical service units and departments, including requiring staff to work remotely and determining what services they can provide. as states begin to reopen, libraries face challenges as they determine... together against racism ala and core are committed to dismantling racism and white supremacy. along with the ala executive board, we endorse the&# ;black caucus of the american library association (bcala)’s may statement&# ;condemning the brutal murder of george floyd at the hands of minneapolis police department officers. in their statement, bcala cites floyd’s death as “the latest in a long line of recent and historical violence against black people in the united states.” not only does core support the sentiments of bcala, we vow to align our values regarding equity, diversity, and inclusion with those of bcala and other organizations that represent marginalized communities within ala. we also stand strong with the asian/pacific american community, which has been the target of xenophobia and racism in the wake of the outbreak of covid- , and support the&# ;asian/pacific american librarians association (apala) and their statement&# ;that, “there is no excuse for discriminatory sentiments and actions towards asians... we are back on twitter tomorrow for #litachat are you ready for the next twitter #litachat? join the discussion on friday, may , from - pm central time. we will be asking you to tell us about challenges with working from home. are there things you can’t do and wish you could? are there issues with your home setup in general?&# ;anne pepitone will lead the discussion. we invite you to join us tomorrow to share your experiences and chat with your colleagues. follow lita on twitter catch up on the last #litachat we&# ;re looking forward to hearing from you! -the lita membership development committee lita job board analysis report – laura costello (chair, assessment & research) lita assessment & research and diversity & inclusion committees background &# ; data this report comes from a joint analysis conducted by lita&# ;s assessment &# ; research and diversity &# ; inclusion committees in fall . the analysis focused on the new and emerging trends in skills in library technology jobs and the types of positions that are currently in demand. it also touches on trends in diversity and inclusion in job postings and best practices for writing job ads that attract a diverse and talented candidate pool.&# ; the committees were provided with a list of job postings from the lita job board between - . data included the employer information, the position title, the location (city/state) the posting date. some postings also included a short description. the assessment &# ; research committee augmented the dataset with job description, responsibilities, qualifications, and salary information for a % sample of the postings from each year using archival job posting information. committee members also assigned... congratulations to dr. jian qin, winner of the lita/oclc kilgour research award dr. jian qin has been selected as the recipient of the &# ;frederick g. kilgour award for research in library and information technology, sponsored by oclc and the library and information technology association (lita). she&# ;is the professor and director at the ischool, syracuse university.&# ;&# ;the kilgour award honors research relevant to the development of information technologies, especially work which shows promise of having a positive and substantive impact on any aspect(s) of the publication, storage, retrieval and dissemination of information, or the processes by which information and data are manipulated and managed. it recognizes a body of work probably spanning years, if not the majority of a career. the winner receives $ , , and a citation. dr. qin’s recent research projects include metadata modeling for gravitational wave research data management and big metadata analytics using genbank metadata records for dna sequences, both with funding from nsf. she also collaborated with a colleague to develop a capability maturity model... lita/ala survey of library response to covid- the library and information technology association (lita) and its ala partners are seeking a new round of feedback about the work of libraries as they respond to the covid- crisis, releasing a survey and requesting feedback by : p.m. cdt, monday, may , . please complete the survey by clicking on the following link: https://www.surveymonkey.com/r/libraries-respond-to-covid- -may- .&# ; lita and its ala partners know that libraries across the united states are taking unprecedented steps to answer the needs of their communities, and this survey will help build a better understanding of those efforts. lita and its ala partners will use the results to advocate on behalf of libraries at the national level, communicate aggregated results with the public and media, create content and professional development opportunities to address library staff needs, and share some raw, anonymized data elements with state-level staff and library support organizations for their own advocacy needs.&# ; additional information about... #coreforum is now a virtual event! join your ala colleagues from across divisions for the forum, which is now a virtual event!&# ; where: in light of the covid- public health crisis, leadership within lita, alcts, and llama made the decision to move the conference online to create a safe, interactive environment accessible for all. what: call for proposals have been extended to friday june , .&# ; when: forum is scheduled november and , how: share your ideas and experiences with library projects by submitting a talk for the inaugural event for core:&# ; https://forum.lita.org/call-for-proposals for more information about the lita, alcts, llama (core) forum, please visit https://forum.lita.org&# ; jobs in information technology: may , new this week web services librarian, fairfield university, fairfield, ct visit the lita jobs site for additional job openings and information on submitting your own job posting. wfh? boost your skill set with lita ce! reserve your spot and learn new skills to enhance your career with lita online continuing education offerings. buying strategies for information technologywednesday, may&# ; , , : - : pm central timepresenter:&# ;michael rodriguez, collections strategist at the university of connecticut in this -minute webinar, you’ll learn&# ;best practices, terminology, and concepts for effectively negotiating contracts for the purchase of information technology (it) products and services. view details&# ;and&# ;register here. using images from the internet in a webpage: how to find and citewednesday, june&# ; , , : - : pm central timepresenter:&# ;lauren bryant, priority associate librarian of ray w. howard library in this -minute&# ;webinar, you’ll learn&# ;practical ways&# ;to quickly find and filter creative commons licensed images online, learn how to hyperlink a citation for a website, and&# ;how to use creative&# ;commons images for thumbnails in videos and&# ;how to cite the image in unconventional situations like this. view details&# ;and&# ;register here. troublesome technology trends: bridging the learning dividewednesday, june , , : - : pm... may / twitter #litachat last week, anne pepitone kicked off the discussion with zoom virtual backgrounds, shared her favorites, and provided tips on how to use them. the next twitter #litachat will be on friday, may , from - pm central time when we&# ;ll talk about apps that help you work from home. what do you use to help with project management, time management, deadlines, or to just stay focused? we invite you to join us tomorrow to share, learn, and chat about it with your colleagues. follow lita on twitter. we&# ;re looking forward to hearing from you! -the lita membership development committee jobs in information technology: april , new this week two associate dean positions, james madison university libraries, harrisonburg, va visit the lita jobs site for additional job openings and information on submitting your own job posting. data privacy while working from home today&# ;s guest post is brought to you by our recent presenter, becky yoose. special thanks to becky for being willing to answer the questions we didn&# ;t have time for during our webinar! hello everyone from your friendly neighborhood library data privacy consultant! we covered a lot of material earlier this month in &# ;a crash course in protecting library data while working from home,&# ; co-sponsored by lita and oif. we had a number of questions during the webinar, some of which were left unanswered at the end. below are three questions in particular that we didn’t get to in the webinar. enjoy! working from home without a web-based ils we don&# ;t have a web-based version of our ils and our county-based it department says they can&# ;t set up remote desktop (something to do with their firewall)… do you have any recommendations on how to advocate for remote desktop? if i have... strategies for surviving a staffing crisis library staff are no strangers to budget and staffing reductions. most of us have way too much experience doing more with less, covering unfilled positions, and rigging solutions out of the digital equivalent of chewing gum and bailing wire, because we can’t afford to buy all the tools we need. in the last two years, my department at northern arizona university’s cline library operated with roughly half the usual amount of staff. in this post, i’ll share a few strategies that helped us get through this challenging time. first, a quick introduction. my department, content, discovery &# ; delivery services, includes the digital services unit (formerly library technology services) as well as collection management (including electronic resources management), acquisitions, cataloging, physical processing, interlibrary loan and document delivery, and course reserves. we are a technology-intensive department, both as users and implementers/supporters of technology. here are some of the strategies we used to... april / twitter #litachat a lot has changed since we had our last twitter #litachat, core passed and then covid happened. we are all navigating new territory in our jobs and life overall. so we wanted to bring you a weekly set of litachats discussing our shared experiences during these strange times.&# ; the first in this series of litachats will start on friday, april from - pm central standard time. we will be asking you to show us your zoom virtual backgrounds! we know that zoom conferencing has been popular among many workplaces so we thought what would be better than showcasing some of the creative backgrounds everyone has been using. if you don’t have a background no worries, you can share about the best backgrounds you have seen from colleagues. don’t know how to turn on zoom virtual backgrounds? we will cover that too! we hope you’ll join us on twitter for... congratulations to samantha grabus, winner of the lita/ex libris student writing award samantha grabus has been selected as the winner of the student writing award sponsored by ex libris group and the library and information technology association (lita) for her paper titled “evaluating the impact of the long s upon th-century encyclopedia britannica automatic subject metadata generation results.” grabus is a research assistant and phd student at drexel university metadata research center. &# ;this valuable work of original research helps to quantify the scope of a problem that is of interest not only in the field of library and information science, but that also, as grabus notes in her conclusion, could affect research in fields from the digital humanities to the sciences,&# ; said julia bauder, the chair of this year&# ;s selection committee. when notified she had won, grabus remarked, “i am thrilled and honored to receive the lita/ex libris student writing award. i would like to extend my gratitude to the award committee... jobs in information technology: april , new this week web and digital scholarship technologies librarian, marquette university libraries, milwaukee, wi ceo / library director, orange county library system, orlando, fl visit the lita jobs site for additional job openings and information on submitting your own job posting. ala lita emerging leaders: inventing a sustainable division in january , the latest cohort of emerging leaders met at ala midwinter to begin their projects. lita sponsored two emerging leaders this year: kelsey flynn, adult services specialist at white oak library, and paige walker, digital collections &# ; preservation librarian at boston college. kelsey and paige are part of emerging leaders group g, &# ;inventing a sustainable division,&# ; in which they’ve been charged with identifying measures that lita can take to improve its fiscal and environmental sustainability. as a first step in their assessment, the group distributed a survey to lita members that will quantify interest in sustainable measures such as virtual conferences and webinars. want to help? complete the survey to give feedback that may shape the direction of our chapter. group g is fortunate to have several other talented library workers on its team:&# ; kristen cooper, plant sciences librarian at university of minnesota tonya ferrell, oer coordinator at... latest in lita elearning so much has changed since covid- . online learning is in greater demand and we are working hard to provide you with resources and more professional development opportunities that strengthens the library community. we hope you are well and staying safe. there&# ;s a seat waiting for you. register today! digital inception: building a digital scholarship/humanities curriculum as a subject librarian wednesday, april , : &# ; : p.m. central time presenter: marcela isuster, education and humanities librarian, mcgill university this presentation will guide attendees in building a digital scholarship curriculum from a subject librarian position. it will explore how to identify opportunities, reach out to faculty, and advertise your services. it will also showcase activities, lesson plans, and free tools for digital publication, data mining, text analysis, mapping, a section on finding training opportunities and strategies to support colleagues and create capacity in your institutions. in this -minute webinar, you&# ;ll learn:... join us this fall for #coreforum – proposal deadline extended! call for proposals have now been extended to friday, may , . share your&# ;ideas and experiences about library technology, leadership, collections, preservation, assessment, and metadata at the inaugural meeting of core, a joining of lita/alcts/llama. we welcome your session proposal. for more information about the call for proposals and our theme of exploring ideas and making them reality, visit the forum website: https://forum.lita.org&# ; event details november - , baltimore, md renaissance baltimore harborplace hotel covid- planning the lita/alcts/llama forum planning committee is currently evaluating a contingency plan, should the covid- public health crisis impact forum in november. core is approved! we’re thrilled to announce that core: leadership, infrastructure, futures is moving forward, thanks to our members. the three existing divisions’ members all voted to approve the bylaws change that will unite alcts, lita, and llama to form core: alcts: % yes lita: % yes llama: % yes the presidents of the three divisions, jennifer bowen, alcts, emily morton-owens, lita, and anne cooper moore, llama, shared the following statement: “we first want to thank our members for supporting core. their belief in this vision, that we can accomplish more together than we can separately, has inspired us, and we look forward to working with all members to build this new and sustainable ala division. we also want to thank the core steering committee, and all the members who were part of project teams, town halls and focus groups. we would not have reached this moment without their incredible work.” ala executive... free lita webinar: protect library data while working from home a crash course in protecting library data while working from home presenter: becky yoose, founder / library data privacy consultant, ldh consulting services thursday, april , : &# ; : pm central time there’s a seat waiting for you…&# ;register for this free lita webinar today! libraries across the u.s. rapidly closed their doors to both public and staff in the last two weeks, leaving many staff to work from home. several library workers might be working from home for the first time in their current positions, while many others were not fully prepared to switch over to remote work in a matter of days, or even hours, before the library closed. in the rush to migrate library workers to remote work and to migrate physical library programs and services to online, data privacy and security sometimes gets lost in the mix. unfamiliar settings, new routines, and increased reliance on vendor... jobs in information technology: march , new this week head of library technology services, east carolina university, greenville, nc visit the lita jobs site for additional job openings and information on submitting your own job posting. march ital issue now available the march issue of information technology and libraries (ital) is available now. in this issue, ital editor ken varnum shares his support of lita, alcts, and llama merging to form a new ala division, core. our content includes a message from lita president, emily morton-owens. “a framework for member success,“ morton-owens discusses the current challenges of lita as a membership organization and reinvention being the key to survival. also in this edition, laurie willis discusses the pros and cons of handling major projects in-house versus hiring a vendor in &# ;tackling big projects.&# ; sheryl cormicle knox and trenton smiley discuss using digital tactics as a cost-effective way to increase marketing reach in &# ;google us!&# ; featured articles: “user experience methods and maturity in academic libraries,” scott w. h. young, zoe chao, and adam chandler this article presents a mixed-methods study of the methods and maturity of user experience (ux) practice in... learn how to build your own digital scholarship/humanities curriculum with this lita webinar are you a subject librarian interested in building digital scholarships? join us for the upcoming webinar &# ;digital inception: building a digital scholarship/humanities curriculum as a subject librarian,&# ; on wednesday, april , from : &# ; : pm cst. digital scholarship is gaining momentum in academia. what started as a humanities movement is now present in most disciplines. introducing digital scholarship to students can benefit them in multiple ways: it helps them interact with new trends in scholarship, appeals to different kinds of learners, helps them develop new and emerging literacies, and gives them the opportunity to be creative. this -minute&# ;presentation will guide attendees in building a digital scholarship curriculum from a subject librarian position. it will explore how to identify opportunities, reach out to faculty, and advertise your services. it will also showcase activities, lesson plans, and free tools for digital publication, data mining, text analysis, mapping, etc. finally, the presentation will... jobs in information technology: march , new this week project manager for resource sharing initiatives,&# ;harvard university,&# ;cambridge, ma research data services librarian,&# ;university of kentucky libraries,&# ;lexington, ky digital archivist,&# ;rice university, fondren library,&# ;houston, tx associate director, technical services,&# ;yale university,&# ;new haven, ct visit the lita jobs site for additional job listings and information on submitting your own job posting. congratulations to alison macrina, winner of the lita/library hi tech award the lita/library hi tech awards committee is pleased to select alison macrina as the recipient of the lita/library hi-tech award. macrina led the tor relay initiative in new hampshire, is the founder and executive director of the library freedom project, and has written and taught extensively in the areas of digital privacy, surveillance, and user anonymity in the context of libraries and librarianship. in this role, macrina was instrumental in creating the library freedom institute, which trained its first cohort in and will train its third cohort in . macrina has also spoken on digital privacy and the work of the library freedom project across the united states and published&# ;anonymity, the first book in ala&# ;s library futures series, in . the committee was fortunate to receive several outstanding nominations for the award. macrina stood out in this strong pool of candidates for the broad reach and impact... nominate yourself or someone you know for the next lita top tech trends panel of speakers lita is looking for dynamic speakers with knowledge about the top trends in technology and how they intersect with information security and privacy. library technology is quickly evolving with trends such as vr, cloud computing and ai. as library technology continues to impact our profession and those that we serve, security and privacy are quickly becoming top concerns. we hope this panel will provide insight and information about these technology trends for you to discuss within your own organization. if you or someone you know would be a great fit for this exciting panel, please submit your nomination today.&# ;&# ; submit your nominations – the deadline is april , . the session is planned for sunday, june , , : – : pm, at the ala annual conference in chicago, il. a moderator and several panelists will each discuss trends impacting libraries, ideas for use cases, and practical approaches for... jobs in information technology: march , new this week wilson distinguished professorship, university of north carolina at chapel hill, chapel hill, nc coordinator of library technical services, berea college, berea, ky ui/ux designer, university of rochester libraries, rochester, ny technical support and hardware specialist &# ; openings, st. lawrence university, canton, ny software engineer, library systems, stanford health care, palo alto, ca visit the lita jobs site for additional job listings and information on submitting your own job posting. hebah emara is our - lita/oclc spectrum scholar lita and oclc are funding hebah emara&# ;s participation in the ala spectrum scholars program as part of their commitment to help diversify the library technology field. emara&# ;is a second year distance student at the university of missouri – columbia school of information science and learning technologies mlis program. she is interested in the ways libraries and technology intersect. her background in it and love of learning about technology, computers, and programming drew her to working in library technology. libraries’ ability to bridge the digital divide and their use of technology to provide opportunities to their communities and solve problems are also of particular interest to emara. her decision to apply to the spectrum scholarship was fueled by a desire to learn from a community of peers and mentors.&# ; emara&# ;is currently the co-chair of a tech unconference to be held in april and organized by mentornj in collaboration with the... share your ideas and library projects by submitting a session proposal for the forum! forum call for proposals submission deadline: march , november - , baltimore, maryland renaissance baltimore harborplace hotel do you have an idea or project that you would like to share? does your library have a creative or inventive solution to a common problem? submit a proposal for the lita/alcts/llama forum! submission deadline is march th. our library community is rich in ideas and shared experiences. the forum theme embodies our purpose to share knowledge and gain new insights by exploring ideas through an interactive, hands-on experience. we hope that this forum can be an inspiration to share, finish, and be a catalyst to implement ideas… together. we invite those who choose to lead through their ideas to submit proposals for&# ;sessions or preconference workshops, as well as&# ;nominate keynote speakers. this is an opportunity to share your ideas or unfinished work, inciting collaboration and advancing the library profession... early-bird registration for the exchange ends in three days! the march early-bird registration deadline for the exchange is approaching. register today and save! there&# ;s still time to register for the exchange at a discount, with early-bird registration rates at $ for alcts, lita, and llama members; $ for ala individual members; $ for non-members; $ for student/retired members; $ for groups; and $ for institutions. early-bird registration ends march . taking place may , , and , the exchange will engage a wide range of presenters and participants, facilitating enriching conversations and learning opportunities in a three-day, fully online, virtual forum. programming includes keynote presentations from emily drabinski and rebekkah smith aldrich, and sessions focusing on leadership and change management, continuity and sustainability, and collaborations and cooperative endeavors. in addition to these sessions, the exchange will offer lightning rounds and virtual poster sessions. for up-to-date details on sessions, be sure to check the exchange website as new information... jobs in information technology: february , new this week back end drupal web developer,&# ;multnomah county library, portland, or distance education &# ; outreach librarian,&# ;winona state university,&# ;winona, mn senior systems specialist,&# ;prairiecat, library consortium,&# ;coal valley, il training and outreach coordinator,&# ;prairiecat, library consortium,&# ;coal valley, il visit the lita jobs site for additional job listings and information on submitting your own job posting. deadline extended to march – submit a proposal to teach for lita the deadline to submit lita education proposals has been extended to march th. we&# ;re seeking instructors passionate about library technology topics to share their expertise and teach a webinar, webinar series, or online course for lita this year. instructors receive a $ honorarium for an online course or $ for a webinar, split among instructors. check out our list of current and past course offerings to see what topics have been covered recently. be part of another slate of compelling and useful online education programs this year! submit your lita education proposal today! for questions or comments related to teaching for lita, contact us at lita@ala.org or ( ) - . the census starts in two weeks — are your computers ready? post courtesy of gavin baker, ala office of public policy and advocacy, deputy director, public policy and government relations on march , millions of american households will begin receiving mailings inviting them to respond to the census. to get an accurate count, everyone has to respond – if they don’t, our libraries and communities will lose needed funding. as the mailings arrive, patrons may come to your library with questions – and, with a new option to respond online, to complete the questionnaire using the library’s computers or internet. to help you prepare, ala has a new, two-page tip sheet, &# ;libraries and the census: responding to the census,&# ; that provides key dates, options for responding, and advice for libraries preparing for the census. for instance, the tip sheet explains these important facts: ways to respond: households can respond to the census online, by phone, or by mail... news regarding the future of lita after the core vote dear lita members, we&# ;re writing about the implications of lita’s budget for the upcoming - fiscal year, which starts september , . we have reviewed the budget and affirmed that lita will need to disband if the core vote does not succeed. since the great recession, membership in professional organizations has been declining consistently. lita has followed the same pattern and as a result, has been running at a deficit for a number of years. each year, lita spends more on staff, events, equipment, software, and supplies than it takes in through memberships and event registrations. we were previously able to close our budgets through the use of our net asset balance which is, in effect, like a nest egg for the division. of course, that could not continue indefinitely. our path towards sustainability has culminated in the proposal to form core: leadership, infrastructure, futures. the new division would come with... boards of alcts, lita and llama put core on march ballot the boards of the association for library collections &# ; technical services (alcts), library information technology association (lita) and the library leadership &# ; management association (llama) have all voted unanimously to send to members their recommendation that the divisions form a new division, core: leadership, infrastructure, futures.&# ; alcts, lita and llama will vote on the recommendation during the upcoming american library association (ala) election. if approved by all three memberships, and the ala council, the three long-time divisions will end operations on august , , and merge into core on september . members of the three boards emphasized that core will continue to support the groups in which members currently find their professional homes while also creating new opportunities to work across traditional division lines. it is also envisioned that core would strengthen member engagement efforts and provide new career-support services. if one or more of the division memberships do not... jobs in information technology: february , new this week librarian (emphasis in user experience and technology), chabot college, hayward, ca librarian ii (ils admin &# ; tech services), duluth public library, duluth, mn distance education &# ; outreach librarian, winona state university, winona, mn head, digital initiatives &# ; tisch library, tufts university, medford, ma online learning and user experience librarian, ast or asc professor, siu edwardsville, edwardsville, il discovery and systems librarian, hamilton college, clinton, ny visit the lita jobs site for additional job listings and information on submitting your own job posting. early-bird registration ends march st for the exchange with stimulating programming, including discussion forums and virtual poster sessions, the exchange will engage a wide range of presenters and participants, facilitating enriching conversations and learning opportunities in a three-day, fully online, virtual forum. programming includes keynote presentations from emily drabinski and rebekkah smith aldrich, and sessions focusing on leadership and change management, continuity and sustainability, and collaborations and cooperative endeavors. the exchange will take place may , , and . in addition to these sessions, the exchange will offer lightning rounds and virtual poster sessions. for up-to-date details on sessions, be sure to check the exchange website as new information is being added regularly. early-bird registration rates are $ for alcts, lita, and llama members, $ for ala individual members, $ for non-members, $ for student members, $ for groups, and $ for institutions. early-bird registration ends march . want to register your group or institution? groups watching the... jobs in information technology: february , new this week upper school librarian (pdf), st. christopher&# ;s school, richmond, va diversity and engagement librarian, ast or asc professor, siu edwardsville, edwardsville, il repository services manager, washington university, saint louis, mo information technology librarian, albin o. kuhn library &# ; gallery (umbc), baltimore, md visit the lita jobs site for additional job listings and information on submitting your own job posting. lita blog call for contributors we&# ;re looking for new contributors for the lita blog! do you have just a single idea for a post or a series of posts? no problem! we&# ;re always looking for guest contributors with new ideas. do you have thoughts and ideas about technology in libraries that you&# ;d like to share with lita members? apply to be a regular contributor! if you&# ;re a member of lita, consider either becoming a regular contributor for the next year or submitting a post or two as a guest. apply today! learn the latest in library ux with this lita webinar there’s a seat waiting for you… register for this lita webinar today! how to talk about library ux &# ; redux presenter: michael schofield librarian / director of engineering, whereby.us wednesday, march , : – : pm central time the last time we did this webinar was in &# ; and a lot&# ;s changed. the goal then was to help establish some practical benchmarks for how to think about the user experience and ux design in libraries, which suffered from a lack of useful vocabulary and concepts: while we might be able to evangelize the importance of ux, libuxers struggled with translating their championship into the kinds of bureaucratic goals that unlocked real budget for our initiatives. it&# ;s one thing to say, &# ;the patron experience is critical!&# ; it&# ;s another thing to say, &# ;the experience is critical &# ; so pay for optimalworkshop, or hire a ux librarian, or give me a... joint working group on ebooks and digital content in libraries john klima, the lita representative to the working group on ebooks and digital content, recently agreed to an interview about the latest update from ala midwinter . watch the blog for more updates from john about the working group in the coming months! what is the mission and purpose of the working group on ebooks and digital content? quoting from the minutes of the ala executive board fall meeting in october of : [the purpose of this working group is] to address library concerns with publishers and content providers specifically to develop a variety of digital content license models that will allow libraries to provide content more effectively, allowing options to choose between one-at-a-time, metered, and other options to be made at point of sale; to make all content available in print and for which digital variants have been created to make the digital content equally available to libraries without... forum call for proposals lita, alcts and llama are now accepting proposals for the forum, november - at the renaissance baltimore harborplace hotel in baltimore, md. intention and serendipity: exploration of ideas through purposeful and chance connections submission deadline: march , our library community is rich in ideas and shared experiences. the forum theme embodies our purpose to share knowledge and gain new insights by exploring ideas through an interactive, hands-on experience. we hope that this forum can be an inspiration to share, finish, and be a catalyst to implement ideas…together. we invite those who choose to lead through their ideas to submit proposals for&# ;sessions or preconference workshops, as well as&# ;nominate keynote speakers. this is an opportunity to share your ideas or unfinished work, inciting collaboration and advancing the library profession forward through meaningful dialogue. we encourage diversity in presenters from a wide range of background, libraries, and experiences. we deliberately... lita announces the excellence in children’s and young adult science fiction notable lists the lita committee recognizing excellence in children’s and young adult science fiction presents the excellence in children’s and young adult science fiction notable lists. the lists are composed of notable children’s and young adult science fiction published between november and october and organized into three age-appropriate categories. the annotated lists will be posted on the website at&# ;www.sfnotables.org. the golden duck notable picture books list is selected from books intended for pre-school children and very early readers, up to years old. recognition is given to the author and the illustrator: field trip to the moon by john hare. margaret ferguson books hello by aiko ikegami. creston books how to be on the moon by viviane schwarz. candlewick press out there by tom sullivan. balzer + bray the babysitter from another planet by stephen savage. neal porter books the space walk by brian biggs. dial books for young... jobs in information technology: february , new this week (tenure-track) senior assistant librarian, sonoma state universityrohnert park, ca data services librarian for the sciences, harvard universitycambridge, ma visit the lita jobs site for additional job listings and information on submitting your own job posting. teach for lita: submit proposals by february reminder: the deadline to submit lita education proposals is february th. please share our cfp with your colleagues. we are seeking instructors passionate about library technology topics to share their expertise and teach a webinar, webinar series, or online course for lita this year. all topics related to the intersection of technology and libraries are welcomed, including: machine learning it project management data visualization javascript, including: jquery, json, d .js library-related apis change management in technology big data, high performance computing python, r, github, openrefine, and other programming/coding topics in a library context supporting digital scholarship/humanities virtual and augmented reality linked data implementation or participation in open source technologies or communities open educational resources, creating and providing access to open ebooks and other educational materials managing technology training diversity/inclusion and technology accessibility issues and library technology technology in special libraries ethics of library technology (e.g., privacy concerns, social justice implications) library/learning management... jobs in information technology: january , new this week stem, instruction, and assessment librarian,&# ;mcdaniel college, westminster, md data science/analysis research librarian,&# ;hamilton college, clinton, ny electronic resources librarian, brown university, providence, ri systems librarian, brown university, providence, ri head, technical services, brown university, providence, ri network and systems administrator,&# ;st. lawrence university, canton, ny visit the lita jobs site for additional job listings and information on submitting your own job posting. emily drabinski, rebekkah smith aldrich to deliver keynotes at the exchange virtual forum the association for library collections and technical services (alcts), the library information technology association (lita) and the library leadership and management association (llama) have announced that emily drabinski and rebekkah smith aldrich will deliver keynote addresses at the exchange virtual forum. the theme for the exchange is &# ;building the future together,&# ; and it will take place on the afternoons of may , and . each day has a different focus, with day exploring leadership and change management; day examining continuity and sustainability; and day focusing on collaborations and cooperative endeavors. drabinski&# ;s keynote will be on may , and smith aldrich&# ;s will be on may . emily drabinski is the critical pedagogy librarian at mina rees library, graduate center, city university of new york (cuny). she is also the liaison to the school of labor and urban studies and other cuny masters and doctoral programs. drabinski&# ;s research includes... jobs in information technology: january , new this week information technology and web services (itws) department head, auraria library, denver, co visit the lita jobs site for additional job listings and information on submitting your own job posting. advice for the new systems librarian – building relationships . advice for the new systems librarian &# ; building relationships, part previous articles in this series: building relationships, helpful resources, a day in the life i am at the two-year mark of being in my role as systems librarian at jacksonville university, and i continue to love what i do. i am working on larger-scale projects and continuing to learn new things every week. there has not been a challenge or new skill to learn yet that i have been afraid of. my first post in this series highlighted groups and departments that may be helpful in learning your new role. now that i’m a little more seasoned, i have had the opportunity to work with even more departments and individuals at my institution on various projects. some of these departments may be unique to me, but i would imagine you would find counterparts where you work. the academic technology... jobs in information technology: january , new this week performing and visual arts librarian, butler university, indianapolis, in librarian, the college of lake county, grayslake, il user experience (ux) librarian, unc charlotte, j. murrey atkins library, charlotte, nc southeast asia digital librarian, cornell university, ithaca, ny head of digital infrastructure services at uconn library, university of connecticut, storrs, ct visit the lita jobs site for additional job listings and information on submitting your own job posting. lita education call for proposals for what library technology topics are you passionate about? have something you can help others learn? lita invites you to share your expertise with an international audience! our courses and webinars are based on topics of interest to library technology workers and technology managers at all levels in all types of libraries. taught by experts, they reach beyond physical conferences to bring high quality continuing education to the library world. we deliberately seek and strongly encourage submissions from underrepresented groups, such as women, people of color, the lgbtqa+ community, and people with disabilities. submit a proposal by february th to teach a webinar, webinar series, or online course for winter/spring/summer/fall . all topics related to the intersection of technology and libraries are welcomed, including: machine learning it project management data visualization javascript, including: jquery, json, d .js library-related apis change management in technology big data, high performance computing python, r, github, openrefine,... jobs in information technology: january , new this week web services &# ; discovery manager, american university library, washington, dcsenior research librarian, finnegan, washington, dc electronic resources and discovery librarian, auburn university, al discovery &# ; systems librarian, california state university, dominguez hills, carson, ca visit the lita jobs site for additional job listings and information on submitting your own job posting. ux “don’ts” we still need from erika hall the second edition of erika hall’s just enough research dropped october ; although this excellent volume was previously unknown to me i am taking the opportunity now to consume, embody, and evangelize hall’s approach to user research. or, as hall might put it, i’m a willing convert to the gospel of “enoughening”. hall is a seasoned design consultant and co-founder of mule design studio but her commercial approach is tempered by a no-nonsense attitude that makes her solutions and suggestions palatable to a small ux team such as my own at indiana university bloomington libraries. rather than conduct a formulaic book review of just enough research, i want to highlight some specific things hall tells the reader not to do in their ux research. this list of five “don’ts” summarize hall’s tone, style, and approach. it will also highlight the thesis of the second edition’s brand new chapter on surveys.... jobs in information technology: december , new this week vice provost and university librarian, university of oregon,&# ;eugene, or data migration specialist (telecommuting position), bywater solutions,&# ;remote position research librarian, oak ridge national laboratory, oak ridge, tn visit the lita jobs site for additional job listings and information on submitting your own job posting. announcing the new lita elearning coordinator we are proud to announce that kira litvin will be the new lita elearning coordinator. litvin has been the continuing education coordinator at the colorado school for public health for the past six months. she provides distance/online learning library services and instruction and works regularly with other librarians, instructional designers, faculty, and educators to collaborate on instructional delivery projects. &# ;i am passionate about being a librarian and working with people in an online environment! &# ;for the past nine years i have worked with libraries that are exclusively online. my roles include administering and managing electronic library systems, including springshare products, and providing virtual reference and instruction to students, faculty and staff. more recently i have transitioned to working as an elearning instructional designer which means i design and develop instructional content available for asynchronous learning and professional development. as online learning continues to grow, i believe that libraries need to... submit a nomination for awards and scholarships hugh c. atkinson memorial award the award honors the life and accomplishments of hugh c. atkinson by soliciting nominations and recognizing the outstanding accomplishments of an academic librarian who has worked in the areas of library automation or library management and has made contributions (including risk taking) toward the improvement of library services or to library development or research. nomination deadline: january , winner receives a cash award and a plaque. learn more about the requirements for the atkinson memorial award. ex libris student writing award the lita/ex libris student writing award is given for the best unpublished manuscript on a topic in the area of libraries and information technology written by a student or students enrolled in an ala-accredited library and information studies graduate program. application deadline: february , winner receives a $ , cash and a plaque. learn more about the requirements for the ex libris student... submit a nomination for the hugh c. atkinson memorial award lita, acrl, alcts, and llama invite nominations for the hugh c. atkinson memorial award. please submit your nominations by january , . the award honors the life and accomplishments of hugh c. atkinson by recognizing the outstanding accomplishments of an academic librarian who has worked in the areas of library automation or library management and has made contributions (including risk taking) toward the improvement of library services or to library development or research. winners receive a cash award and a plaque. this award is funded by an endowment created by divisional, individual, and vendor contributions given in memory of hugh c. atkinson. the nominee must be a librarian employed in one of the following during the year prior to application for this award: university, college, or community college library non-profit consortium, or a consortium comprised of non-profits that provides resources/services/support to&# ; academic libraries the nominee must have a minimum... core update – / / greetings again from the steering committee of core: leadership, infrastructure, futures, a proposed division of ala. coming up this friday, december is the last of four town halls we are holding this fall to share information and elicit your input. please join us! register for town hall today. alcts, lita, and llama division staff will lead this town hall with a focus on core’s mission, vision, and values; benefits organizationally; benefits to members; and opportunities in the future. our speakers will be jenny levine (lita executive director), julie reese (alcts deputy executive director), and kerry ward (llama executive director and interim alcts executive director). we’re excited to share an updated core proposal document for ala member feedback and review, strengthened by your input. we invite further comments on this updated proposal through sunday, december . meanwhile, division staff will incorporate your comments and finalize this proposal document for... jobs in information technology: december , new this week senior specialist &# ; makerspace,&# ;middle tennessee state university, walker library,&# ;murfreesboro, tn user experience librarian,&# ;auburn university,&# ;auburn university, al visit the lita jobs site for additional job listings and information on submitting your own job posting. announcing the new lita blog editor we are proud to announce that jessica gilbert redman will be the new editor of the&# ;lita&# ;blog.&# ; gilbert redman has been the web services librarian at the university of north dakota for the past three years. she coordinates and writes for the library blog and maintains the library website. she has completed a post-graduate certificate in user experience and always seeks to ensure that end users are able to easily find the information they need to complete their research. additionally, she realizes communication is the key component in any relationship, be it between libraries and their users or between colleagues, and she always strives to make communication easier for all involved. &# ;i am excited to become more involved in lita, and i think the position of lita blog editor is an excellent way to meet more people within lita and ala, and to maintain a finger on the pulse of new... jobs in information technology: december , new this week digital discovery librarian/assistant librarian, miami university, oxford, oh visit the lita jobs site for additional job listings and information on submitting your own job posting. jobs in information technology: november , new this week web and digital scholarship technologies librarian,&# ;marquette university libraries, milwaukee, wi digital access and metadata librarian,&# ;marquette university libraries, milwaukee, wi librarian (san ramon campus),&# ;contra costa community college district,&# ;san ramon, ca visit the lita jobs site for additional job listings and information on submitting your own job posting. support lita scholarships this #givingtuesday it’s almost #givingtuesday, so we’re highlighting the difference that lita scholarships can make, and inviting you to join us in increasing access to lita events by donating to our scholarship fund today. you can help us to provide more scholarships to events like avramcamp and lita forum, as well as sponsor emerging leaders, with your donation today! your donation of $ could open up untold opportunities for other library technology professionals. “the lita scholarship afforded me the opportunity to present at the avramcamp and ala conference. it was an incredible opportunity to network with dozens of information professionals, build connections with people in the field, ask them all of my questions and exchange our technical acumen and job experiences. as a result, i have been offered two interviewing opportunities that were an incredibly valuable experience for my career development. i am very grateful to lita for the opportunity to... jobs in information technology: november , new this week metadata specialist iii, metadata services, the new york public library, new york, ny eresources librarian,&# ;university of maryland, baltimore county, baltimore, md multiple librarian positions,&# ;george washington university, washington dc information technology analyst,&# ;san mateo county libraries, san mateo county, ca visit the lita jobs site for additional job listings and information on submitting your own job posting. call for blog coordinator for the exchange: an alcts/lita/llama collaboration the exchange: an alcts/lita/llama collaboration brings together experiences, ideas, expertise, and individuals from the three ala divisions. broadly organized around the theme of “building the future together,” the exchange will examine the topic in relation to collections, leadership, technology, innovation, sustainability, and collaborations. participants from diverse areas of librarianship will find the three days of presentations, panels, and lightning rounds both thought-provoking and highly relevant to their current and future career paths. the exchange will engage a wide range of presenters and participants, facilitating enriching conversations and learning opportunities. divisional members and non-members alike are encouraged to register and bring their questions, experiences, and perspectives to the events. as part of the conference experience, the exchange plans to host regular blog posts in advance of the conference. blog posts will serve multiple purposes: generate excitement and interest in content, encourage participation outside of simply watching presentations, and provide an avenue... the exchange call for proposals and informational webinar alcts, lita, and llama are now accepting proposals for the exchange: building the future together, a virtual forum scheduled for may , , and , . the twelve hour virtual event will take place over three afternoons, featuring the following themes and topics: day &# ; leadership and change management day &# ; continuity and sustainability day &# ; collaborations and cooperative endeavors session formats the exchange will feature the following session formats: full-session proposals presenters prepare content for a -minute session, with an additional -minute q&# ;a period for all presenters. full-session proposals may include multiple presentations with content that is topically related. lightning round each participant is given five minutes to give a presentation. at the end of the lightning round, there will be a - -minute q&# ;a period for all presenters in the session. topics for lightning rounds related to innovative projects or research are encouraged. proposals will be... registration is now open for the exchange in may , join alcts, lita, and llama for an exciting and engaging virtual forum. registration is now open! &# ; the exchange: an alcts/lita/llama collaboration brings together experiences, ideas, expertise, and individuals from the three ala divisions. broadly organized around the theme of “building the future together,” the exchange will examine the topic in relation to collections, leadership, technology, innovation, sustainability, and collaborations. participants from diverse areas of librarianship will find the three days of presentations, panels, and lightning rounds both thought-provoking and highly relevant to their current and future career paths. the exchange will engage a wide range of presenters and participants, facilitating enriching conversations and learning opportunities. divisional members and non-members alike are encouraged to register and bring their questions, experiences, and perspectives to the events. “building on the rich educational traditions of the three divisions, the exchange provides the opportunity to break down silos and explore synergies... core call for comment greetings again from the steering committee of&# ;core: leadership, infrastructure, futures, a proposed division of ala. the steering committee welcomes comments on the&# ;draft division proposal documentation&# ;through november th. please join the conversation! your perspectives and input are shaping the identity and priorities of the proposed division. we’re asking for you to respond to the documents with key questions in mind, including: does this make sense to someone new to alcts/ lita/ llama? does this piece of the plan reflect how members want the new division to function? are there any points that are cause for concern? if you’re interested in helping us in the review process or other work ahead, please&# ;consider volunteering&# ;for&# ;core.&# ;we’re eager to collaborate with you! we’re working hard to ensure everyone can participate in the&# ;core&# ;conversation, so please&# ;let us know&# ;what could make&# ;core&# ;a compelling and worthy division home for you.&# ;keep the feedback and input coming! full details for all our&# ;upcoming events&# ;are... lis students: apply for the larew scholarship for tuition help the library and information technology association (lita) and baker &# ; taylor are accepting applications for the lita/christian (chris) larew memorial scholarship for those who plan to follow a career in library and information technology, demonstrate potential leadership, and hold a strong commitment to library automation. the winner will receive a $ , check and a citation.&# ;the application form is open through march , . criteria for the scholarship includes previous academic excellence, evidence of leadership potential, and a commitment to a career in library automation and information technology. candidates should illustrate their qualifications for the scholarships with a statement indicating the nature of their library experience, letters of reference and a personal statement of the applicant’s view of what they can bring to the profession.&# ;winners must have been accepted to&# ;a master of library science (mls) program recognized by the american library association. references, transcripts, and other documents must be postmarked no... jobs in information technology: november , new this week full time faculty &# ; non tenure track,&# ;sjsu school of information, san jose, ca digital collections librarian, union college,&# ;schenectady, ny web services librarian,&# ;university of oregon libraries, eugene, or galileo programmer/analyst,&# ;university of georgia libraries, athens, ga visit the lita jobs site for additional job listings and information on submitting your own job posting. lita opens call for innovative lis student writing award for the library and information technology association (lita), a division of the american library association (ala), is pleased to offer an award for the best unpublished manuscript submitted by a student or students enrolled in&# ;an ala-accredited graduate program. sponsored by&# ;lita&# ;and&# ;ex&# ;libris, the award consists of $ , , publication in&# ;lita’s&# ;referred journal,&# ;information technology and libraries (ital), and a certificate. the deadline for submission of the manuscript is february , . the award recognizes superior student writing and is intended to enhance the professional development of students. the manuscript can be written on any aspect of libraries and information technology. examples include, but are not limited to, digital libraries, metadata, authorization and authentication, electronic journals and electronic publishing, open source software, distributed systems and networks, computer security, intellectual property rights, technical standards, desktop applications, online catalogs and bibliographic systems, universal access to technology, and library consortia. to be eligible, applicants must follow&# ;these&# ;guidelines&# ;and fill out&# ;the application form&# ;(pdf).... jobs in information technology: november , new this week open educational resources production manager, oregon state university &# ; ecampus, corvallis, or user experience librarian, northwestern university, evanston, il institute for clinical and translational research (ictr) librarian, university of maryland, baltimore, baltimore, md director of collections &# ; access, wheaton college, norton, ma visit the lita jobs site for additional job listings and information on submitting your own job posting. nominate a colleague doing cutting edge work in tech education for the lita library hi tech award nominations are open&# ;for the &# ;lita/library hi tech award, which is given each year to an individual or institution for outstanding achievement in educating the profession about cutting edge technology within the field of library and information technology. sponsored by the&# ;library and information technology association&# ;(lita) and library hi tech, the award includes a citation of merit and a $ , stipend provided by&# ;emerald publishing, publishers of library hi tech. the deadline for nominations is december , . the award, given to either a living individual or an institution, may recognize a single seminal work or a body of work created during or continuing into the five years immediately preceding the award year. the body of work need not be limited to published texts but can include course plans or actual courses and/or non-print publications such as visual media. awards are intended to recognize living persons rather than to honor the deceased; therefore,... propose a topic for the ital “public libraries leading the way” column information technology and libraries (ital), the quarterly open-access journal published by ala’s library information technology association, is looking for contributors for its regular “public libraries leading the way” column. this column highlights a technology-based innovation or approach to problem solving from a public library perspective. topics we are interested in include the following, but proposals on any other technology topic are welcome. -d printing and makerspaces civic technology drones diversity, equity, and inclusion and technology privacy and cyber-security virtual and augmented reality artificial intelligence big data internet of things robotics geographic information systems and mapping library analytics and data-driven services anything else related to public libraries and innovations in technology to propose a topic, use this brief form, which will ask you for three pieces of information: your name your email address a brief ( - word) summary of your proposed column that describes your library, the technology you wish to... alcts, lita and llama collaborate for virtual forum the association for library collections &# ; technical services (alcts), the library and information technology association (lita) and the library leadership &# ; management association (llama) have collaborated to create the exchange, an interactive, virtual forum designed to bring together experiences, ideas, expertise and individuals from these american library association (ala) divisions. modeled after the alcts exchange, the exchange will be held may , may and may in with the theme “building the future together.” as a fully online interactive forum, the exchange will give participants the opportunity to share the latest research, trends and developments in collections, leadership, technology, innovation, sustainability and collaborations. participants from diverse areas of librarianship will find the three days of presentations, panels and activities both thought-provoking and highly relevant to their current and future career paths. the exchange will engage an array of presenters and participants, facilitating enriching conversations and learning opportunities.... submit your annual meeting request by feb the lita meeting request form is now open for the ala annual conference in chicago, il. all lita committee and interest group chairs should use it to let us know if you plan to meet at annual. we&# ;re looking forward to seeing what you have planned. the deadline to submit your meeting request is friday, february , . we&# ;re going to change how we&# ;ve listed meetings in the past. if you do not submit this form, your group will not be included in the list of lita session on our website, the online scheduler, or the print program. while we&# ;ll still hold the joint chairs meeting on saturday from : - : am and use that same room for committee and ig meetings from : - : am, your group will only be listed if you submit this form. you should also use it if you want to request a meeting on a different day... submit a nomination for the prestigious kilgour technology research award lita and oclc invite nominations for the frederick g. kilgour award for research in library and information technology. submit your nomination no later than december , . the kilgour research award recognizes research relevant to the development of information technologies, in particular research showing promise of having a positive and substantive impact on any aspect of the publication, storage, retrieval, and dissemination of information or how information and data are manipulated and managed. the winner receives $ , cash, an award citation, and an expense-paid trip (airfare and two nights lodging) to the ala annual conference in chicago, il. nominations will be accepted from any member of the american library association. nominating letters must address how the research is relevant to libraries; is creative in its design or methodology; builds on existing research or enhances potential for future exploration; and/or solves an important current problem in the delivery of... core update – october , greetings again from the steering committee of core: leadership, infrastructure, futures, a proposed division of ala. thank you for all of your questions and feedback about the proposed new division!&# ;the steering committee has been revising core documents based on what we’ve heard from you so far in order to share draft bylaws and other information with you soon. we want you to know that we are continuing to listen and incorporate the feedback you’re providing via town halls, twitter chats, the core feedback form, and more.&# ; in our next steering committee meeting, we will be discussing how we can support the operational involvement of interested volunteers. if you have ideas on how members should be involved, please share them with us through&# ;the feedback form.&# ; we’re working hard to ensure everyone can participate in the core conversation, so please&# ;let us know&# ;what could make core a compelling and worthy division home for... jobs in information technology: october , new this week metadata &# ; research support specialist, open society research services, open society foundations, new york, ny head of public services in the daniel library, the citadel, the military college of south carolina, charleston, sc engineering and science liaison, mit, cambridge, ma head of technical services &# ; library, the citadel, the military college of south carolina, charleston, sc analyst programmer , oregon state university libraries and press, corvallis, or collection information specialist, isabella stewart gardner museum, boston, ma visit the lita jobs site for additional job listings and information on submitting your own job posting. jobs in information technology: october , new this week metadata librarian for distinctive collections, mit, cambridge, ma electronic access librarian, university of rochester, rochester, ny dean, university libraries, university of northern colorado, greeley, co administrative/metadata specialist, asr international corp., monterey, ca core systems librarian, university of oregon libraries, eugene, or visit the lita jobs site for additional job listings and information on submitting your own job posting. september ital issue now available the september issue of information technology and libraries (ital) is available now. in this issue, ital editor ken varnum announces six new members of the ital editorial board. our content includes a recap of emily morton-owens&# ; president&# ;s inaugural message, &# ;sustaining lita&# ;, discussing the many ways lita strives to provide a sustainable member organization. in this edition of our &# ;public libraries leading the way&# ; series, thomas lamanna discusses ways libraries can utilize their current resources and provide ideas on how to maximize effectiveness and roll new technologies into operations in &# ;on educating patrons on privacy and maximizing library resources.&# ; featured articles: &# ;library-authored web content and the need for content strategy,&# ; courtney mcdonald and heidi burkhardt increasingly sophisticated content management systems (cms) allow librarians to publish content via the web and within the private domain of institutional learning management systems. “libraries as publishers”may bring to mind roles in scholarly communication and... jobs in information technology: october , new this week information research specialist, harvard business school, boston, ma - library residency program (provost’s postdoctoral fellowship), new york university, division of libraries, new york, ny executive director, library connection, inc, windsor, ct associate university librarian, cornell university, ithaca, ny visit the lita jobs site for additional job listings and information on submitting your own job posting. new vacancy listings are posted on wednesday afternoons. latest lita learnings there&# ;s a seat waiting for you&# ; register today for a lita webinar! guiding students through digital citizenship presenter: casey davis instructional designer (it), arizona state university wednesday, october , : &# ; : pm central time as academic librarians, we help build our students into digital citizens.&# ;it&# ;s our duty to make sure students have the tools and resources to be savvy tech users, become information literate, and understand the permanence of their digital actions. in this -minute webinar,&# ;you&# ;ll learn research-based best practices you can implement using the framework of the hero&# ;s journey&# ;without creating an additional burden on faculty, staff, and students. learning objectives for this program include: • an expanded understanding of digital citizenship within the context of college/university life •&# ;examining areas where increased awareness and practice is needed within the college/university community • creating authentic training for increasing digital citizenship within the college/university community view details&# ;and&# ;register here. in-house vs.... hectic pace – a view on libraries, the library business, and the business of libraries skip to content hectic pace my pre-covid things posted on dec by andrew k. pace authors note: these parodies are always about libraries and always based on christmas songs, stories, or poems. being what it is, this year is an exception to both…that’s right, i’m siding with my family and admitting that my favorite things is not a christmas song. (sung to the tune of “my favorite things”) [click the youtube link to listen while you sing along.] eating in restaurants and movies on big screenspeople who don’t doubt the virtue of vaccines.inspiring leaders who don’t act like kings.these were a few of my pre-covid things. live music venues and in-person classes.no masks or … tagged with: / category: christmas parody / leave a comment sitting in the reading room all day posted on dec by andrew k. pace (sung to the tune of “walking in a winter wonderland”) [click the youtube link to listen while you sing along.] people shhhhhh, are you listening? in the stacks, laptops glistening the reading light’s bright the library’s right for sitting in the reading room all day. gone away are the book stacks here to stay, the only town’s fax. we share all our books without judgy looks. sitting in the reading room all day. in the lobby we could build a book tree. readers guide is green and they stack well. i’ll say ‘do we have ’em?’ you’ll say, ‘yeah man.’ … tagged with: / category: christmas parody / comment it’s the best library time of the year posted on dec by andrew k. pace (sung to the tune of “it’s the most wonderful time of the year”) press play to sing along with the instrumental track! it’s the best library time of the year with no more children yelling and no one is telling you “get it in gear!” it’s the best library time of the year it’s the qui-quietest season at school only smile-filled greetings and no more dull meetings where bosses are cruel it’s the qui-quietest season at school there’ll be books for re-stocking vendor end-of-year-hawking and overdue fine cash for beer send the word out to pre-schools drag queen visit … tagged with: / category: christmas parody / leave a comment posts navigation … next about a blog and its author search the archive search for: archives archives select month december ( ) december ( ) december ( ) december ( ) july ( ) april ( ) march ( ) february ( ) december ( ) november ( ) october ( ) september ( ) august ( ) may ( ) december ( ) october ( ) september ( ) july ( ) april ( ) march ( ) january ( ) december ( ) june ( ) april ( ) december ( ) october ( ) august ( ) july ( ) june ( ) may ( ) april ( ) march ( ) january ( ) december ( ) september ( ) august ( ) july ( ) june ( ) may ( ) april ( ) march ( ) january ( ) december ( ) november ( ) october ( ) september ( ) august ( ) july ( ) june ( ) may ( ) april ( ) march ( ) january ( ) december ( ) november ( ) october ( ) september ( ) august ( ) july ( ) june ( ) may ( ) april ( ) march ( ) february ( ) january ( ) december ( ) november ( ) october ( ) september ( ) august ( ) july ( ) june ( ) categories categoriesselect category . ( ) ala ( ) april fool’s ( ) catalogs ( ) christmas parody ( ) community development ( ) e-books ( ) ebsco ( ) education ( ) equity, diversity, & inclusion ( ) general ( ) google ( ) innovation ( ) lita ( ) mergers & acquisitions ( ) metasearch ( ) niso ( ) oclc ( ) open source ( ) openurl ( ) product management ( ) public libraries ( ) publishers ( ) sacred cows ( ) search ( ) standards ( ) vendors ( ) web-scale ( ) wms ( ) worldshare ( ) oclc next we persevere through challenges when we rely on each other skip prichard why a “library on-demand” vision benefits from pandemic wisdom cathy king the oclc network: collaboration, innovation, and efficiency oclc oclc colleagues . : the dewey blog the digital shift (roy tennant) hanging together lorcan dempsey’s weblog oclc next webjunction library colleagues librarian.net screwy decimal the shifted librarian librarian in black free range librarian the travelin’ librarian david lee king jenny arch justin the librarian mr. library dude thoughts from carl grant search worldcat enter title, subject, person, or keyword hectic pace rss feeds entries rss comments rss © – . all rights reserved, hectic pace toggle navigation inkdroid about bookmarks photos music software social talks april , metadata coincidence? unless otherwise noted all the content here is licensed cc-by global chapters | legal hackers navigation our story blog global chapters press videos governance summit search our story blog global chapters press videos governance summit search global chapters global chapters want to be a part of the largest grassroots legal innovation movement in the world? join us! legal hackers is an open and collaborative community of individuals passionate about exploring and building creative solutions to some of the most pressing issues at the intersection of law and technology. since , we have used the hashtag #legalhack to share the activities of the global legal hackers community. legal hackers online communities twitter: @legalhackers facebook: www.facebook.com/groups/legalhackers slack: legalhackers.slack.com (invitation here) linkedin: https://www.linkedin.com/groups/ hashtag: #legalhack legal hackers local chapters legal hackers is the largest grassroots legal innovation community in the world, with chapters in many major cities. check out the list below to find your local legal hackers community. if you don’t see a community near you below, apply to start your own by clicking the appropriate link below: i want to start a traditional legal hackers chapter for my city or region i want to start a student-only legal hackers student group for my university questions? read more about the differences between chapters and student groups here, or email us at us at: info [at] legalhackers [dot] org. north americaeuropeafricaasiaaustralia & new zealandlatamstudent groups atlanta, georgia baltimore, maryland boston, massachusetts chicago, illinois cleveland, ohio dfw (dallas-fort worth), texas denver, colorado detroit, michigan houston, texas kansas city, missouri london, ontario miami, florida minneapolis-st. paul, minnesota montreal, québec nashville, tennessee new orleans, louisiana new york, new york north carolina orlando, florida ottawa, ontario philadelphia, pennsylvania portland, oregon puerto rico salt lake city, utah san diego, california san francisco, california seattle, washington toronto, ontario tulsa, oklahoma vancouver, british columbia washington, d.c. amsterdam, netherlands asturias, spain athens, greece barcelona, spain bari, italy belfast, northern ireland belgrade, serbia berlin, germany bern, switzerland bilbao, spain bologna, italy bristol, england brno, czech republic brussels, belgium bucharest, romania chișinău, moldova cologne/bonn, germany copenhagen, denmark dublin, ireland estonia firenze, italy frankfurt, germany geneva, switzerland genova, italy ghent, belgium the hague, netherlands hamburg, germany helsinki, finland istanbul, turkey kyiv, ukraine limassol, cyprus lisbon, portugal ljubljana, slovenia london, england luxembourg lviv, ukraine madrid, spain malaga, spain manchester, england mantova, italy milan, italy moscow, russia munich, germany napoli-campania, italy novi sad, serbia nürnberg, germany padova, italy paris, france perugia, italy pescara, italy pisa, italy porto, portugal preston, england roma, italy rijeka, croatia scotland sheffield, england skopje, macedonia sofia, bulgaria st. petersburg, russia stockholm, sweden timisoara, romania torino, italy toulouse, france trieste, italy valencia, spain venezia, italy verona, italy vienna, austria vilnius, lithuania warsaw, poland zagreb, croatia zurich, switzerland abuja, nigeria accra, ghana alexandria, egypt cape town, south africa casablanca, morocco douala, cameroon enugu, nigeria harare, zimbabwe imo, nigeria kampala, uganda lagos, nigeria luanda, angola nairobi, kenya almaty, kazakhstan ankara, turkey bhopal, india chandigarh, india delhi, india goa, india hong kong jakarta, indonesia jeddah, saudi arabia kuala lumpur, malaysia lahore, pakistan lucknow, india manila, philippines patna, india pune, india seoul, south korea singapore tokyo, japan melbourne, australia perth, australia sydney, australia wellington, new zealand aguascalientes, mexico arequipa, peru baja, mexico barranquilla, colombia belém, brazil belo horizonte, brazil bogota, colombia brasília, brazil buenos aires, argentina campinas, brazil cuiabá, brazil curitiba, brazil cusco, peru fortaleza, brazil goiânia, brazil guadalajara, mexico guatemala city, guatemala guayaquil, ecuador imperatriz. brazil jaraguá do sul, brazil lavras, brazil lima, peru manaus, brazil manizales, colombia maringá, brazil medellin, colombia mexico city, mexico mogi das cruzes, brazil monterrey, mexico montevideo, uruguay natal, brazil panama city, panama passo fundo, brazil pereira, colombia petrolina, brazil porto alegre, brazil porto velho, brazil puebla, mexico querétaro, mexico quito, ecuador recife, brazil rio de janiero, brazil salvador, brazil santa cruz, bolivia santo andré, brazil são paulo, brazil san salvador, el salvador sete lagoas, brazil tegucigalpa, hondouras tepic, mexico kansas, usa – kansas university new brunswick, canada – university of new brunswick new york, usa – brooklyn law school north carolina, usa – wake forest university school of law tennessee, usa – university of tennessee college of law toronto, canada – university of toronto coventry, england – university of warwick kyiv, ukraine – national university of kyiv-mohyla academy kyiv, ukraine – taras shevchenko national university of kyiv london, england – university college london sheffield, england – university of sheffield tarragona, spain – universitat rovira i virgili kharagpur, india – indian institute of technology quito, ecuador – pontificia universidad católica del ecuador (puce) tweets by legalhackers our story blog global chapters press videos governance summit type and press “enter” to search our story global chapters governance press none literary machines - digital libraries, books, archives literary machines digital libraries, books, archives. about me jul archiviiify a short guide to download digitized books from internet archive and rehost on your own infrastructure using iiif with full-text search. jan pywb . - docker quickstart four years have passed since i first wrote of pywb: it was a young tool at the time, but already usable and extremely simple to deploy. since then a lot of works has been done by ilya kreymer (and others), resulting in all the new features available with the . release. also, some very big webarchiving initiatives have moved and used pywb in these years: webrecorder itself, rhizome, perma, arquivo pt in portugal, the italian national library in florence (italy), (others i’m missing). oct anonymous webarchiving webarchiving activities, as any other activity where an http client is involved, leave marks of their steps: the web server you are visiting or crawling will save your ip address in its logs (or even worse it can decide to ban your ip). this is usually not a problem, there are plenty of good reasons for a webserver to keep logs of its visitors. but sometimes you may need to protect your own identity when you are visiting or saving something from a website, and there a lot of sensitive careers that need this protection: activists, journalist, political dissidents. tor has been invented for this, and today offer a good protection to browse anonymously the web. can we also archive the web through tor? sep open bni il maggio viene annunciato il rilascio libero della bibliografia nazionale italiana (bni). viene apprezzata l’apertura di questo catalogo (anche se con i limiti dei soli pdf), e da profano di biblioteconomia faccio anche una domanda sull’effettivo caso d’uso della bni. il agosto viene annunciato il rilascio delle annate e anche in formato unimarc e marcxml. incuriosito dal catalogo inizio ad esplorarlo, per pensare a possibili trasformazioni (triple rdf) o arricchimenti con/verso altri dati (wikidata). mar epub linkrot linkrot also affects epub files (who would have thought! :)). how to check the health of external links in epub books (required tools: a shell, atool, pup, gnu parallel). feb skos nuovo soggettario, api e autocomplete come creare una api per un form con autocompletamento usando i termini del nuovo soggettario, con i sorted sets di redis e nginx+lua. nov serve deepzoom images from a zip archive with openseadragon vips is a fast image processing system. version higher than . can generate static tiles of big images in deepzoom format, saving them directly into a zip archive. oct a wayback machine (pywb) on a cheap, shared host for a long time the only free (i’m unaware of commercial ones) implementation of a web archival replay software has been the wayback machine (now openwayback). it’s a stable and mature software, with a strong community behind. to use it you need to be confident with the deploy of a java web application; not so difficult, and documentation is exaustive. but there is a new player in the game, pywb, developed by ilya kramer, a former internet archive developer. built in python, relatively simpler than wayback, and now used in a pro archiving project at rhizome. sep opendata dell'anagrafe biblioteche come usare gli opendata dell’anagrafe delle biblioteche italiane e disegnare su una mappa web gli indirizzi delle biblioteche. sep api json dell'opac sbn alcuni mesi fa è stata rilasciata da iccu una app mobile per consultare l’opac sbn. anche se graficamente poco accattivante l’app funziona bene, e trovo molto utili le funzioni di ricerca di un libro scansionando il codice a barre con la camera del telefonino, e la possibilità di bookmarkare dei preferiti. incuriosito dal funzionamento ho pensato di analizzarne il traffico http. page of subscribe! all content is licensed under a creative commons attribution . international license made with jekyll + kasper theme posts | mark a. matienzo skip to content w c svg main navigation menu about now posts notes music projects publications presentations press posts iah forecast - disquiet junto project publish date: february , tags: music black tent by mark a. matienzo an experiment with recording a new single using vcv rack and reaper based on a compositional prompt. i ended up recording two tracks. perfecting a favorite: oatmeal chocolate chip cookies publish date: november , tags: recipes food by mark a. matienzo i have a horrible sweet tooth, and i absolutely love oatmeal chocolate chip cookies. i tend to bake as a means to cope with stress, and of course, more often then that means making these cookies. after making many iterations, i’ve settled upon this recipe as the ultimate version to which all compare. (read more …) in memoriam and appreciation of rob casson ( - ) publish date: october , tags: code lib personal by mark a. matienzo the world lost one of its brightest and most charming lights earlier this week, rob casson. many of us knew rob through the code lib community and conferences and his work at miami university libraries. we miss his generosity, patience, sense of humor, and genuine kindness. those of us who got the chance to socialize with him also remember his passion for music, and some of us were even lucky to see live shows in the evenings between conference sessions and other social activities. on sunday, october at : pm pacific/ : pm eastern, those of us who knew him through code lib and the world of libraries are encouraged to gather to share our memories of him and to appreciate his life and work. please join me and my co-organizers, mike giarlo and declan fleming on zoom (registration required). robert casson (robcasson), jan - sep . photo: declan fleming. (read more …) first sota activation publish date: september , tags: ham radio by mark a. matienzo about a month ago, i got my ham radio license, and soon after i got pretty curious about summits on the air (sota), an award scheme focused on safe and low impact portable operation from mountaintops. while i like to hike, i’m arguably a pretty casual hiker, and living in california provides a surprising number of options within minutes driving time for sota newbies. (read more …) optimizing friction publish date: august , tags: indieweb music plan food by mark a. matienzo over and in response to the last few months, i’ve been reflecting about intentionality, and how i spend my time creating things. i have tried to improve the indiewebbiness of my site, and understanding what it means to “scratch my own itch”. this resonates particularly lately because it’s leading me to mull over which parts should be hard and easy. unsurprisingly, much of that is personal preference, and figuring out how i want to optimize from the perspective of user experience. friction in ux can be a powerful tool, part of what i’m trying to find is where i want to retain friction as it helps me remain intentional. (read more …) a hugo shortcode for embedding mirador publish date: july , tags: iiif hugo by mark a. matienzo i spent a little time over the last day or so trying to bodge together a shortcode for hugo to embed an instance of mirador. while it’s not quite as simple (or full-featured) as i’d like, it’s nonetheless a starting point. the shortcode generates a snippet of html that gets loaded into hugo pages, but (unfortunately) most of the heavy lifting is done by a separate static page that gets included as an