White Paper Report Report ID: 100838 Application Number: HD5108810 Project Director: Cheryl Ball (cball@ilstu.edu) Institution: Illinois State University Reporting Period: 9/1/2010-2/28/2013 Report Due: 5/31/2013 Date Submitted: 3/18/2014 Building  a  Better  Back-­End:  Editor,  Author,  and  Reader  Tools  for  Scholarly  Multimedia     Final  White  Paper  (Grant  #HD-­‐51088-­‐10)     Cheryl  E.  Ball  (principal  investigator)   s2ceball@gmail.com   Illinois  State  University       Submitted  March  18,  2014   Final Performance Report: Building a Better Back-End 2 Project  Activities   The  original  proposal  for  this  Level  II  Digital  Humanities  Start-­‐Up  Grant  was  to  modify  the  open-­‐ source,  editorial-­‐management  system  Open  Journal  Systems  (OJS)  for  use  with  scholarly   multimedia.  The  goal  was  to  build  PHP-­‐based  plug-­‐ins  that  would  facilitate  synchronous  and   asynchronous  review  of  multimodal  webtexts,  which  includes  adding  metadata  to  the  author   upload  functions,  maintaining  linked  file  structures  of  webtexts  through  the  versioning  system  of   OJS,  and  capturing  nondiscursive  synchronous  review  data  such  as  sticky  notes  and  drawings  on   screencaptures  of  interactive  webtext  submissions.  A  second  set  of  goals,  to  build  remix  and   citation  tools  for  readers,  had  to  be  set  aside  early  on  due  to  the  scope  of  the  review  plug-­‐in   deliverable.   Brief  Background  on  Scholarly  Multimedia   Scholarly  multimedia  (also  called  webtexts)  are  article-­‐  or  book-­‐length  digital  pieces  of  peer-­‐ reviewed  scholarship  designed  using  hypertextual  and  media-­‐rich  elements  to  enact  an  author’s   argument.  They  incorporate  interactivity,  digital  media,  and  different  argumentation  strategies   such  as  visual  juxtaposition  and  associational  logic  and  are  composed  using  webpages  with  links,   animations,  images,  audio,  video,  scripts,  databases,  multimedia,  and  other  design  elements.  These   publications  are  unique  in  that  each  webtext  is  individually  designed,  which  makes  basic  editorial   processes  such  as  reviewing,  copy-­‐  and  design-­‐editing,  publishing,  and  indexing  significantly  more   complicated  than  print-­‐based  or  linear  (e.g.  PDF-­‐like)  scholarship.  The  oldest,  continuously   published  journal  for  webtexts  is  Kairos:  Rhetoric,  Technology,  and  Pedagogy   (http://kairos.technorhetoric.net).  The  PI  and  two  of  the  grant’s  consultants  are  Kairos  editors  and   drew  on  their  combined  30  years  of  expertise  with  the  journal  to  inform  the  deliverables  of  this   project.       Project  Purpose  and  Original  Scope   Editors,  authors,  readers,  and  publishers  need  media-­‐specific  tools  to  help  them  engage  with  and   promote  scholarly  multimedia,  but  the  unique  editorial  processes  for  scholarly  multimedia-­‐-­‐-­‐such   as  the  lack  of  feasibility  to  blind  review;  the  need  for  collaborative  review  processes;  and  the  added   layers  of  copy-­‐editing  that  attend  to  usability,  accessibility,  sustainability,  and  rhetorical   appropriateness  of  a  webtext’s  design-­‐-­‐-­‐inhibit  this  growth.  Creating  tools  that  display  a  webtext   submission  within  a  review  system  (instead  of  downloading  it  for  offline  review,  as  OJS  does)   allows  editors  to  offer  reviewers  the  opportunity  to     • synchronously  chat  about  a  webtext  as  they  interact  with  it,     • put  sticky  notes  on  areas  of  the  design  that  may  need  attention,     • dis/agree  with  other  reviewer’s  comments  in  a  similar  manner  to  Facebook’s  “like”  (and  the   • much-­‐called-­‐for  “dislike”)  button,     • vote  to  Accept/Accept  with  Revisions/Revise  and  Resubmit/Reject,  and     • track  which  reviewers  receive  feedback  from  their  co-­‐reviewers  (using  a  game-­‐like  badge-­‐ system   • for  their  logins/avatars  to  promote  the  creative  play  inherent  in  scholarly  multimedia)  and   to  see  which  kinds  of  webtext  content  they  prefer  responding  to,  which  would  help  editors   further  support  reviewers’  disciplinary  and  technical  expertise  when  assignments  are   needed.     Final Performance Report: Building a Better Back-End 3 The  team’s  goal  with  this  grant  was  to  build  an  a/synchronous  webtext  review  plug-­‐in  that  we   would  distribute  through  Open  Journal  Systems’s  Plug-­‐ins  Gallery.  (We  called  this  the  Kairos-­‐OJS   plug-­‐in.)  In  addition,  we  wanted  to  build  plug-­‐ins  for  increased  implementation  of  metadata  for   media  elements,  better  indexing  and  bibliography  management  tools  (i.e.,  cross-­‐support  of   scholarly  multimedia  with  Zotero),  and  citation  tools  for  individual  media  elements  or  portions  of   elements  (e.g.,  citing  a  30-­‐second  clip  from  within  a  2-­‐minute  podcast),  among  others.     Major  activities  completed   2010   Fourth  Quarter   • PI  (Cheryl  Ball)  and  primary  consultants  (Douglas  Eyman  and  Kathie  Gossett)  met  several   times,  and  once  with  programmer  (Steven  Potts).   • Team  (PI,  consultants)  created  technical  specs  and  wireframes  for  its  revised  version  of  OJS.   • Team  (PI,  consultants)  created  metadata  schema  with  crosswalk  between  OJS,  Dublin  Core,   and  Kairos  (the  scholarly  multimedia  journal  used  as  the  test-­‐case  for  this  NEH  project).  The   metadata  schema  helped  us  to  figure  out  what  new  fields  we  would  need  to  build  in  OJS  to   accommodate  the  reader  tools  we  had  proposed.   2011   First  Quarter   • PI  worked  with  her  digital  publishing  undergraduate  class  to  mine  metadata  from  all  the   back  issues  of  Kairos.     Second  Quarter   • Team  (PI,  consultants)  ran  user-­‐testing  with  Kairos  editors  for  potential  back-­‐end  changes   to  OJS  using  wireframes  and  interactive  mock-­‐ups.     Third  Quarter   • Team  presented  on  wireframes  at  PKP  (Public  Knowledge  Project  conference)  in  Berlin,  and   consulted  with  PKP  developers  on  OJS.     Fourth  Quarter   • Team  negotiated  for  installation  of  developmental  server  (from  NEH  grant  budget)  at  PI’s   home  institution.  OJS  installed.   2012   First  quarter   • PI  began  initial  set-­‐up  to  migrate  Kairos  to  OJS.   • NEH  grant  extended  for  one  year.     Second  quarter   • Team  conducted  user-­‐testing  with  Kairos  staff  of  a/synchronous  multimedia  review  system   mock-­‐up.   • PI  presented  metadata  schema  at  New  Media  Consortium  conference.       Final Performance Report: Building a Better Back-End 4 Third  quarter   • PI  and  consultant  Eyman  met  in  Lansing,  MI,  to  retrieve  prototype  from  programmer.   • PI  called  project  failed.   2013   First  quarter  [end  of  grant  period]   • PI  wrote  article  about  mining  metadata  as  a  pedagogical  tool.   • PI  formed  advisory  group  for  boutique  data  repository  (see  Long-­‐Term  Impact  section.)   • PI  and  consultant  (Eyman)  delivered  presentation  at  Networked  Humanities  conference  on   infrastructures  of  digital  media  publishing  (and  later  published  an  article  on  same)   Changes  in  proposed  project  activities   scope  &  deliverables   The  grant  team  changed  the  initial  scope  of  the  project  fairly  quickly  after  meeting  the  first  few   times,  to  exclude  the  reader  tools  (for  remix  and  citation  of  multimedia  elements  and  webtexts)   from  this  project,  with  the  hopes  of  returning  to  these  goals  in  a  follow-­‐up  grant.  The  decision  was   made  to  remove  these  tools  because  the  scope  of  completing  just  the  editorial  workflow  (back-­‐end)   portions  of  the  project  proved  to  be  too  large  to  complete  with  the  time,  money,  and  human   resources  the  grant  provided.  Basically,  we  would  have  had  to  totally  re-­‐write  OJS  to  get  it  to  do  all   of  these  things,  and  that  was  beyond  the  intended  scope  of  the  grant  project  (see  technological   changes,  below).     We  further  limited  the  scope  of  the  project,  after  our  initial  user-­‐testing  in  the  second  quarter  of   2011,  the  a/synchronous  multimedia  review  plug-­‐in.  We  did  this  because  when  we  tested  the   potential  changes  we  had  planned  for  the  author  and  editorial  workflow  tools  within  OJS,  we   discovered  that  with  slight  modifications  of  our  own  workflows,  we  could  fit  into  the  current  OJS   workflow  relatively  well  without  having  to  rework  the  system.  For  instance,  although  we  would   have  to  change  some  of  our  long-­‐standing  terminology,  like  “Design-­‐Editing”  to  “Layout  Editor,”  and   to  re-­‐arrange  the  workflow  pattern  in  OJS  (since  design-­‐editing  for  Kairos  comes  before  copy-­‐ editing  the  written  content),  changing  our  terminology  was  potentially  an  easier  fix  than  rewriting   a  major  part  of  OJS  to  accommodate  a  single  journal’s  current  workflow  (even  if  that  workflow  is   best  practice  for  webtextual  journals,  which  are  not  the  mainstay  audience  for  OJS).     Thus,  our  focus  for  the  grant  project  ended  up  being  almost  exclusively  on  writing  a  plug-­‐in  for  OJS   that  would  accommodate  a/synchronous  reviewing  of  webtexts.  It  is  unknown  whether  this   prototype  was  successful,  as  the  programmer  stopped  responding  to  all  grant-­‐related   communications  in  Fall  2012,  when  delivery  (after  a  year  delay)  was  intended  to  occur.  It  is   rumored  that  the  plug-­‐in  prototype  was  completed  and  did  successfully  run,  but  that  it  could  not  be   made  to  integrate  with  OJS  (see  technological  changes,  below).       The  team  did  add  a  deliverable,  however,  in  the  form  of  the  metadata  mining  project.  This   unintended  deliverable  was  created  by  the  PI  with  a  class  of  15  undergraduate  digital  publishing   students  at  Illinois  State  University.  We  mined  over  a  million  points  of  data  from  every  webtext  and   media  element  (filetype)  that  Kairos  had  ever  published,  in  its  then-­‐15-­‐year  history.  (We  have  since   expanded  the  collection  to  the  issues  published  since  this  part  of  the  project  was  completed  in  mid-­‐ Final Performance Report: Building a Better Back-End 5 2011.)  This  metadata  was  meant  to  be  used  to  populate  OJS  so  that  the  journal’s  archives  could  be   searchable  and  sharable  within  the  new  OJS  reader-­‐interface  we  had  originally  planned  to  build.     personnel   The  project  was  unable  to  be  completed  because  the  programmer  stopped  communicating  with  the   grant  team  right  before  delivery  of  the  prototype  was  to  have  been  made.  It  was  too  late  in  the   project,  at  that  point,  to  hire  a  new  programmer.     technological   The  team’s  technological  understanding  of  OJS  changed  the  project  from  its  original  intent  the  most.   Open  Journal  Systems  is  an  organically  coded  tool  built  up  through  the  love  and  grant-­‐getting  of  the   Public  Knowledge  Project’s  architectural  and  programming  team.  It  has  been  built  on  and  modified   over  the  last  decade  through  piecemeal  efforts,  acknowledged  by  the  PKP  team  as  somewhat   haphazard,  and  (as  indicated  at  the  PKP  conference  our  grant  team  attended  in  Berlin  in  2011)  left   to  its  own  devices  in  favor  of  the  more  nuanced,  modular,  and  lessons-­‐learned  coding  project  that   has  become  OJS’s  next  iteration:  Open  Monograph  Press.  While  OJS  functions  pretty  well  from  a   non-­‐technical  viewpoint,  programmers  looking  under  the  hood  have  repeatedly  come  back  with   very  realistic  evaluations  that  modifying  the  system  in  as  radical  a  way  as  this  grant  project  had   hoped  to  do  would  be  unsuccessful.  Several  programmers  we  have  spoken  to  have  suggested  that   OJS  needs  to  be  forked  or,  more  efficiently,  rewritten  from  the  ground  up  in  order  to  implement  the   changes  we  wanted  to  make,  which  would  make  it  an  entirely  new  platform.  Doing  so  was  outside   the  scope  of  this  NEH  grant,  as  we  had  neither  the  time  nor  the  resources  to  maintain  a  new  system,   nor  did  we  want  to  do  the  current  OJS  users  a  disservice  by  forking  and  then  not  being  able  to   provide  a  migration  tool.     Publicity  of  results  (summary)   The  major  publicity  efforts  regarding  the  multimedia  plug-­‐in  deliverable  were  based  in  conference   presentations  and  one  article.  The  major  publicity  efforts  regarding  the  metadata-­‐mining  project   were  based  in  conference  presentations,  keynotes,  an  article,  and  the  creation  of  a  boutique  data   repository,  which  is  also  publicized  in  conference  presentations  and  another  article.  See  the  Grant   Products  section  for  links  to  these  publicity  artifacts.   Accomplishments   (1)  Our  objective  to  explore  whether  Open  Journal  Systems  as  a  platform  would  be  usable,  with   modifications  via  plug-­‐ins,  for  multimedia  publishing  was  accomplished.  The  outcome  of  this   objective  indicated  that  OJS  is  not  currently  viable  for  multimedia  publishing.  This  is  probably  the   most  important  outcome  for  our  project,  as  well  as  for  any  person  working  with  and  in  digital   publishing  platforms  today.     (2)  Our  objective  to  create  plug-­‐ins  for  multimedia-­‐based  editorial  workflow  with  OJS  was  only   minimally  accomplished:   a. We  discovered  that  a  multimedia-­‐based  workflow  based  on  best  practices  at  Kairos  could  be   minimally  manipulated  to  work  within  OJS’s  current  production  workflow.  This  would   require  us  to  use  ZIP  files  of  webtexts  instead  of  transferring  files  within  folder  structures,   as  we  do  now  by  hand  (on  our  servers).   Final Performance Report: Building a Better Back-End 6 b. We  were  not  able  to  deliver  on  our  refocused  objective  to  create  an  a/synchronous  review   plug-­‐in  for  multimedia  texts  in  OJS.  Although  the  possibility  exists  that  such  a  plug-­‐in  could   be  created  with  more  funding  and  better  programming,  the  grant  team  has  elected  to  not   pursue  this  project  due  to  the  lack  of  overall  viability  for  using  OJS  for  multimedia   publishing.     (3)  Our  objective  to  create  a  robust  reader  interface  for  multimedia  journals  in  OJS  was  removed   from  the  project  as  being  too  large  of  a  technological  task  within  the  financial  scope  of  the  NEH   grant.     (4)  The  biggest,  unintended  accomplishment  with  this  grant  was  the  unexpected  deliverables   produced  by  the  metadata  mining  project,  which  elicited  over  a  million  points  of  data  about  the   history  of  webtext  publication  in  Kairos,  the  longest-­‐running  journal  of  its  kind.  The  PI  has   published  several  articles  relating  to  this  outcome  and  has  begun  a  new  digital  humanities  project,   rhetoric.io—a  boutique  data  repository—the  idea  for  which  was  an  outgrowth  of  the  lack  of   availability  of  venues  for  distributing  important,  albeit  small,  data  sets  in  the  humanities.  This  new   project  is  briefly  discussed  in  the  Long-­‐Term  Impact  section  below.   Audiences   The  primary  intended  audience  for  the  Kairos-­‐OJS  plug-­‐ins  were  OJS  users,  specifically  publishers   and  editors  who  already  use  OJS  and  wanted  to  publish  more  multimedia  content,  as  well  as  those   who  wanted  to  start  multimedia  journals  from  scratch.  The  secondary  intended  audience—and   those  who  were  user-­‐tested  during  this  grant—included  editorial  board  and  staff  members  from   Kairos,  who  already  have  a  working  knowledge  of  multimedia  publishing.  A  third,  unintended   audience  would  have  been  teachers,  who  could  use  a  multimedia  review  plug-­‐in,  like  the  one  we   had  planned,  for  conducting  peer-­‐review  workshops  and  multimedia  analyses  in  their  classes.   However,  the  project  had  little  actual  impact  on  any  of  these  audiences  since  the  major  deliverable   (the  review  plug-­‐in)  could  not  be  completed.     Despite  this  failure,  the  project  has  allowed  us  to  have  conversations  with  several  possible,  future   stakeholders  who  may  be  able  to  help  us  expand  our  collaborations  (and  our  audiences)  to  build  a   new  editorial-­‐management  system  that  is  multimedia-­‐specific.   Evaluation   Because  the  project  wasn’t  completed,  we  do  not  have  evaluation  statistics  to  provide.   Lessons  Learned   Instead  of  an  evaluation,  we  provide  the  following  list,  written  by  a  first-­‐time  PI  of  an  NEH  grant:   • Managing  a  grant,  even  a  relatively  “small”  $50,000  one,  takes  more  time  than  you’d   imagine.  It’s  equivalent,  at  least,  to  teaching  a  new  prep,  if  not  more.  Do  not  skimp  on   budgeting  for  personnel,  including  the  PI’s  time,  whether  it  be  through  a  course  re-­‐ assignment,  summer  salary,  or  paying  for  a  staff  person  to  manage  the  mountains  of   paperwork  for  you.  Check  with  your  institutional  research  office  to  see  whether  some  of  the   administrative  tasks  can  be  wrapped  into  their  office  and  the  overhead  you’re  already   paying  the  university.   • Although  it  adds  to  the  paperwork,  requiring  quarterly  (or  more  frequent)  reports  from   consultants  and  grant  team  members  will  assist  with  meeting  grant  project  milestones.  Use   Final Performance Report: Building a Better Back-End 7 project  management  software  from  the  start,  or  hire  someone  with  experience  as  a  project   manager  if  the  PI  can’t  do  it  themselves.   • Write-­‐in  travel  money  for  publicity  of  your  project.  Going  to  conferences  to  present   (particularly  ones  that  are  usually  outside  of  the  budget  of  most  humanities  scholars)  will   assist  with  your  networking  capabilities  and  will  usually  provide  you  with  a  forum  to   receive  insightful  feedback  on  your  in-­‐progress  project.   • Saving  money  by  conducting  the  majority  of  the  work  offsite  (and  at  a  lower  overhead  rate)   doesn’t  make  up  for  not  having  oversight  of  consultants.  Work  at  a  distance  only  with   people  you  know  well  and  trust  or  have  a  binding  contract  with.   • If  you  don’t  already  have  a  working  relationship  with  consultants,  conduct  formal   interviews  and/or  ask  for  references  and  CVs/résumés.  Don’t  rely  on  recommendations,   unless  those  recommenders  have  established  a  formal  working  relationship  with  the   consultant.  Also  ask  your  institutional  research  office  in  advance  whether  there  is  a   recuperation  process  if  the  consultant  breaks  his  or  her  contract.   • If  you  do  run  into  personnel  problems,  treat  everyone  involved  humanely  and  communicate   with  them  as  quickly  as  possible,  by  as  many  means  as  necessary  (f2f,  phone,  email,  Skype,   text,  etc.).  If  none  of  the  above  provides  a  successful  resolution,  seek  advice  from  your   research  office  or  the  NEH  program  officer.   • Be  welcome  to  unexpected  turns  in  the  project  that  might  produce  interesting  outcomes.  Be   cognizant  of  when  those  turns  become  unproductive,  though,  and  are  taking  you  too  far   afield.   • For  a  high-­‐risk  grant  such  as  the  NEH  Digital  Humanities  Start-­‐Up  grants,  failures  still   produce  outcomes  that  are  useful  to  you  and  the  field,  even  if  the  deliverables  you  intended   don’t  work  out.   Public  response   We  were  able  to  conduct  two  rounds  of  usability  tests  with  wireframes  and  mock-­‐ups,  as  well  as   present  those  wireframes  at  several  conference  panels.  We  have  anecdotal  evidence  from  both  of   these  scenarios  to  indicate  that,  if  the  multimedia  review  plug-­‐in  would  have  been  made  available,   people  would  have  definitely  wanted  to  use  it.  Several  key  members  of  the  OJS  team—PKP  founder   John  Willinsky  and  lead  OJS  technical  architect  Alec  Smecher,  in  particular—were  very  excited  by  it   when  we  discussed  it  with  them  via  Skype  early  on  in  the  grant  as  well  as  when  we  presented  the   wireframes  at  the  PKP  conference  in  Berlin  a  year  later.  We  also  had  Skype  calls  with  Stanford’s   High  Wire  press,  to  discuss  their  implementation  of  multimedia  in  OJS,  and  they  were  very   interested  in  what  we  were  working  on  as  they  were  working  on  a  complementary  project  at  the   time.         In  addition,  Kairos  staff  members  and  other  journal  editors  alike  thought  that  having  both   synchronous  and  asynchronous  review  possibilities  was  a  smart  idea,  given  the  lack  of  time   reviewers  have  for  providing  reviews.  Additionally,  being  able  to  individually  navigate  and  mark-­‐up   (draw  on,  attach  sticky  notes  with  written  text,  highlight,  etc.)  a  webtext  and  then  share  those   markers  with  other  reviewers  in  a  synchronous  space  was  one  of  the  key  features  editors  and   reviewers  said  they  liked.     We  deemed  from  this  project  that  editors  and  publishers  do  want  a  multimedia  journal  editing   system,  and  while  OJS  cannot  offer  that  in  its  current  instantiation,  it’s  still  an  idea  that  should  be   pursued  (just  with  a  LOT  more  funding  and  people  involved).   Final Performance Report: Building a Better Back-End 8 Continuation  of  the  Project   There  are  no  plans  to  continue  building  PHP  plug-­‐ins  for  OJS  to  make  it  multimedia  compatible.   Long  Term  Impact   This  project  allowed  for  conversations  to  begin  with  several  stakeholders  at  multiple,  international   universities  and  non-­‐profit  organizations  about  several  related  projects,  including  building  a   digital-­‐media  publishing  infrastructure  from  the  ground  up.  This  infrastructure  would  potentially   inform  work  on   • an  (open-­‐source)  editorial-­‐management  system  for  digital,  open-­‐access  publishers  that   includes  print-­‐based  and  multimedia  publishing  of  article-­‐  and  book-­‐length  scholarly   projects  as  well  as  data-­‐based  publishing,   • a  linked,  boutique  data  repository,  called  rhetoric.io,  which  would  provide  searchable,   visualizable  data  and  would  function  as  a  sustainable  data  management  storage  facility  (see   http://rhetoric.io),  and     • digital  authoring  and  publishing  institutes,  held  to  train  authors,  editors,  publishers,  and   evaluators  of  digital  (media)  scholarship  how  to  compose,  edit,  publish,  and  assess  such   work  using  best  practices.   Grant  Products   The  major  grant  product  was  the  unintended  deliverable  of  metadata,  created  from  mining  the  back   issues  of  Kairos  from  1996–2011  (with  additional  years,  through  2013,  supplied  by  research   assistants  not  affiliated  with  the  NEH  grant).  Although  we  did  not  use  it  for  its  original  intention  (as   data  for  the  OJS  database  that  would  have  run  Kairos),  the  metadata  is  important  because  it  is  a   wunderkammern  that  showcases  the  history  of  webtext  publishing  over  the  last  20  years.  With   over  a  million  points  of  data  categorized  at  both  the  webtext  (article)  level  and  the  media-­‐element   level  (for  every  single  file  associated  with  a  webtext),  this  data  can  provide  researchers  with  a   plethora  of  interesting  results,  such  as  the  possibility  to  trace  the  rise  and  fall  of  certain  filetypes,   mimetypes,  and  genres  within  webtext  publishing.  More  over,  much  of  this  data  speaks  to  the  Web’s   and  Web-­‐users’  understanding  of  accessibility  or  lack  thereof.  It’s  a  rich  data  source  that  should  be   made  public.  But  because  there  was  no  venue  to  publish  the  metadata  by  itself  and  the  idea  of  just   uploading  it  unmarked  or  uncommented  to  GitHub  seemed  like  asking  for  obsolescence,  the  PI– working  with  a  cohort  of  other  digital  writing  studies  scholars–started  a  boutique  data  repository,   called  rhetoric.io.  This  repository  is  in-­‐progress  as  of  this  writing  (although  the  initial  website  is  up:   http://rhetoric.io).   Publications   Ball,  Cheryl  E.;  Graban,  Tarez  Samra;  &  Sidler,  Michelle.  (forthcoming/under  review).  The  boutique   is  open:  Data  for  writing  studies.  In  Jeff  Rice  &  Brian  McNely  (Eds.),  Networked  humanities.   Minneapolis:  University  of  Minnesota  Press.  Pre-­‐print:  http://ceball.com/2013/11/17/the-­‐ boutique-­‐is-­‐open-­‐data-­‐for-­‐writing-­‐studies/   Eyman,  Douglas,  &  Ball,  Cheryl  E.  (forthcoming/2014).  Digital  humanities  scholarship  and   electronic  publication.  In  Jim  Ridolfo  &  William  Hart-­‐Davidson  (Eds.),  Rhetoric  and  the  digital   humanities.  Chicago:  University  of  Chicago  Press.  Pre-­‐print:   http://ceball.com/2013/07/11/digital-­‐humanities-­‐scholarship-­‐and-­‐electronic-­‐publication/   Final Performance Report: Building a Better Back-End 9 Ball,  Cheryl  E.  (2013).  Pirates  of  metadata  or,  The  true  adventures  of  how  one  editor,  fifteen   undergraduate  publishing  majors,  and  25,000  media  elements  survived  a  metadata  mining   project.  In  Stephanie  Davis-­‐Kahl  &  Merinda  Hensley  (Eds.),  Extend  and  unify:  Outreach  and   education  for  scholarly  communication  and  information  literacy  programs.  Chicago:  Association   of  College  and  Research  Libraries.  Free  copy:  http://ceball.com/2013/07/11/pirates-­‐of-­‐ metadata-­‐the-­‐true-­‐adventures-­‐of-­‐a-­‐harrowing-­‐metadata-­‐mining-­‐project/   Presentations   Ball,  Cheryl  E.  (2013,  December  6).  The  mixed  genres  of  Kairos  webtexts  [Invited  lecture].   Department  of  Media  and  Communication,  University  of  Oslo,  Norway.   Ball,  Cheryl  E.  (2013,  November  20).  The  kairos  of  scholarly  multimedia:  Examining  the  history  of   webtexts  through  metadata  [Invited  lecture].  Blekinge  Museum,  Karlskrona,  Sweden.   Ball,  Cheryl  E.  (2013,  June  9).  Preservation  &  access  for  scholarly  multimedia.  Computers  &  Writing,   Frostburg,  MD.   Ball,  Cheryl  E.  (2013,  June  7).  Futures  of  computers  and  writing:  Publishing  [Roundtable].   Computers  &  Writing,  Frostburg,  MD.   Ball,  Cheryl  E.  (2013,  May  30).  Boutique  data  in  writing  studies  [Keynote].  Technical   Communication  and  Rhetoric  PhD  Maymester,  Texas  Tech  University,  Lubbock.   Ball,  Cheryl  E.  (2012,  March  22).  The  Mid-­‐Life  (Crisis?)  of  Kairos:  Caring  for  the  Health  and  Welfare   of  Open-­‐Access  Digital  Media  Publishing.  Conference  on  College  Composition  and   Communication,  St.  Louis,  MO.   Eyman,  Douglas,  &  Ball,  Cheryl  E.  (2013,  February  15).  Networked  humanities  scholarship,  or  the   life  of  Kairos.  Networked  Humanities  Conference,  University  of  Kentucky,  Lexington,  KY.   Ball,  Cheryl  E.  (2012,  Jan.  5).  The  Future  of  Peer  Review  in  Scholarly  Multimedia.  Modern  Language   Association,  Seattle,  WA.   Ball,  Cheryl  E.  (2011,  Oct.  21).  The  Challenges  of  Publishing  Webtexts.  International  Society  for  the   Scholarship  of  Teaching  and  Learning,  Milwaukee,  WI.   Ball,  Cheryl  E.;  Gossett,  Kathie;  &  Eyman,  Douglas.  (2011,  Sept.  27).  Kairos  and  Multimedia  Digital   Scholarship:  The  Need  for  Better  PublishingTools.  The  Public  Knowledge  Project  (PKP)   Conference.  Berlin,  Germany.   Ball,  Cheryl  E.  (2011,  July  19).  Learning  Through  Leading:  Digital  Media  Scholarly  Publishing   [Poster  presentation].  New  Media  Consortium,  Madison,  WI.   Ball,  Cheryl  E.  (2011,  April  6).  Writing  proposals  and  getting  grants  [CCCC  Research  Committee   Roundtable].  Conference  on  College  Composition  and  Communication,  Atlanta,  GA.   Ball,  Cheryl  E.  (2011,  January  8).  Digital  media  scholarship:  Innovation  or  insanity?  Modern   Language  Association,  Los  Angeles,  CA.   Syllabi   English  354:  Digital  Publishing,  http://354s11.ceball.com/  Dr.  Cheryl  E.  Ball,  Illinois  State   University,  Spring  2011  Course  website  includes  100+  pages  of  instructions  for  mining   metadata  from  fifteen  years  of  Kairos  back  issues,  with  metadata  schema  and  crosswalks  to   OJS.   Final Performance Report: Building a Better Back-End 10 Appendices   To  keep  filesizes  down,  I  have  elected  to  include  links  in  the  section  above  to  all  relevant   publications  and  syllabi,  which  amount  to  nearly  200  pages  of  content.  Readers  can  access  all  PDFs   for  free  on  my  website.  The  appendix,  then,  only  includes  screenshots  of  the  interactive  prototype   for  the  a/synchronous  reviewing  system.       Figure  1.  A  screenshot  of  the  asynchronous,  multimedia  review  prototype  used  in  second-­‐round   user-­‐testing.  (The  prototype  was  intended  as  an  OJS  plug-­‐in).  This  shot  shows  a  reviewer  adding  a   sticky  note  with  written  commentary  on  top  of  a  webtext  (“Anna  Wintour”)  that  is  located  center-­‐ screen.  This  review  system  would  upload  a  webtext  to  the  review  database,  where  readers  could   interact  with  it  individually  online  during  an  open  window  of  three  weeks  (or  so,  as  scheduled  by   the  editor),  and  add  their  written  comments  and  annotated  webtext  screenshots  through  the   Submit  button  (bottom  right).  This  would  create  an  interactive  discussion  forum  over  the  course  of   several  weeks,  which  the  editor  could  then  retrieve  for  revision  purposes.     Final Performance Report: Building a Better Back-End 11   Figure  2.  In  the  synchronous  review  system,  several  editorial  board  members  could  meet  at  the   same  time  to  review  a  webtext  (center-­‐screen:  “Anna  Wintour”)  and  chat  about  using  the  Chat   feature  in  the  right  sidebar  of  the  screen.  Some  chat  features  are  shown  in  this  screenshot.  All   attendees  in  the  chat  are  listed  in  the  left  sidebar.  The  same  annotation  features  as  the   asynchronous  review  has  (note  bubbles,  sticky  notes,  highlighting,  pencil/drawing,  and  eraser)  are   shown  in  the  Comment  Tools  bar  (mid-­‐screen,  below  the  webtext).