Conference Paper
Analyzing the MISO
Data: Broader Perspectives on Library and Computing Trends
Laurie Allen
Coordinator for Digital Scholarship and Services
Haverford College Libraries
Haverford, Pennsylvania, United States of America
Email: lallen@haverford.edu
Neal Baker
Library Director
Earlham College
Richmond, Indiana, United States of America
Email: bakerne@earlham.edu
Josh Wilson
Director for Academic Support and User Services
Brandeis University
Waltham, Massachusetts, United States of America
Email: jwilson@brandeis.edu
Kevin Creamer
Director for Teaching, Learning and Technology
Boatwright Memorial Library
University of Richmond
Richmond, Virginia, United States of America
Email: kcreamer@richmond.edu
David Consiglio
Head of Research Support and Educational Technology
Bryn Mawr College
Bryn Mawr, Pennsylvania, United States of America
Email: dconsiglio@brynmawr.edu
2013 Allen, Baker, Wilson, Creamer, and Consiglio.
This is an Open Access article distributed under the terms of the Creative
Commons‐Attribution‐Noncommercial‐Share Alike License 2.5 Canada (http://creativecommons.org/licenses/by-nc-sa/2.5/ca/),
which permits unrestricted use, distribution, and reproduction in any medium,
provided the original work is properly attributed, not used for commercial
purposes, and, if transformed, the resulting work is redistributed under the
same or similar license to this one.
Abstract
Objective – To
analyze data collected by 38 colleges and universities that participated in the
Measuring Information Services Outcomes (MISO) survey between 2005 and 2010.
Methods
– The MISO survey is a Web-based
quantitative survey designed to measure how faculty, students, and staff view
library and computing services in higher education. Since 2005, over 10,000
faculty, 18,000 students, and 15,000 staff have completed the survey. To date,
the MISO survey team has analyzed the data by faculty age group and student
cohort. Much of the data analysis has focused on changes in the use,
importance, and satisfaction with services over time.
Results
– Analysis of the data collected during 2008-2010
reveals marked differences in how faculty and students use the library. The
most frequently used services by faculty are the online library catalog (3.39
on a 5-point scale), library databases (3.34), and the library website (3.29).
In contrast, the most frequently used services by students are public computers
in the library (3.61) and quiet work space in the library (3.29). Faculty
reported a much higher use of online resources from off campus. Analysis of
data from schools where the survey was administered more than once during
2005-2010 reveals that both faculty and students increased their utilization of
databases over time. All other significant faculty trends reflected declines in
usage, whereas, with the exception of use of the library website, all other
student trends reflected no change or increased usage.
Conclusion
–
As the MISO survey has continued and expanded over the years, the usefulness of
rich comparable data from a set of peer institutions over time has increased
tremendously. In addition to providing a rich source of data, MISO can serve as
a model for how a group of schools can collaborate on a share assessment tool
that meets the needs of individual institutions and provides a robust,
aggregated dataset for deeper analysis.
Introduction
As higher education changes in response to
budgetary, technological, and political pressures, library and technology
leaders increasingly look for meaningful ways to assess how and to what extent
our organizations support scholarship, teaching, and learning. The Measuring Information
Services Outcomes (MISO) Survey is a Web-based quantitative survey designed to
measure how faculty, students, and staff view library and computing services in
higher education.
The core of the MISO Survey consists of
questions designed to measure the use of library and IT services, their
importance to the campus community, and the level of satisfaction with which
the community views these services. The survey also measures the ownership of
technology tools and their use for academic and personal purposes, as well as
participants’ perceptions of their own technology skills and preferred learning
methods. In addition, it measures overall attitude toward library and
technology services on campus.
By looking at computing and library services
together, the MISO Survey provides a richer context for each set of services
while acknowledging the shared nature of many of the services as seen from the
perspective of our constituents. While there are many distinct services offered
by library and computing organizations on campuses, librarians and
technologists also frequently work together to support instructional and
academic computing needs on campus and to provide resources to off-campus
students and faculty. In addition, library buildings are, on many campuses, the
site of many computing resources.
Launched in 2005, the MISO Survey has been
taken more than 43,000 times at 38 participating institutions, 26 of whom have
responded to the Survey more than once and 8 more than twice. Overall, more
than 10,000 faculty, 18,000 students, and 15,000 staff have completed the
survey.
The precursor to the MISO Survey was designed
by David Consiglio and his colleagues at Bryn Mawr College to assess the
effectiveness of the College’s recently merged Information Services department.
When the Survey proved extremely useful, a group of chief information officers
from the Council of Library and Information Resources (CLIR) agreed to use the
Bryn Mawr Survey as the basis for a common survey to be administered across
schools. This would allow each school to learn from the data gathered on its
campus and also compare itself to a group of peer institutions. In addition, by
conducting the Survey every year, each institution would be able to evaluate
its services over time. Bates College, Middlebury College, the University of
Richmond, and Wellesley College graciously agreed to donate a significant
amount of a top manager’s time toward the project. In January of 2005, the team
members met for the first time at Bryn Mawr College to begin this process.
During the Spring and Summer of 2005, the MISO
Survey team prepared and tested the instrument. Their five institutions
participated in a pilot administration in Fall 2005. Additional schools
administered the MISO Survey in Spring 2006 and in each Spring since.
The MISO Survey Team works together to develop
long-term strategies, to conduct in-depth analysis of data, and to complete
biennial revisions to the instrument. The co-investigators also liaise with
participating institutions during the survey administration season to ensure
that the survey administration goes well. The team has developed high standards
for data quality by using tested questions, ensuring high response rates and
customizing the survey instrument so that participating institutions can
address local concerns. Each participating school receives a summary dataset
representing all institutions for the survey year for comparison.
This article will focus on analysis of the
larger dataset of all schools and years, offering deeper analysis of user needs
than any one school could conduct using its own data. To date, the MISO Survey
Team has analyzed the data by faculty age group and student cohort and is now
examining how views on services are affected by academic discipline. Finally,
the Survey Team combined use and importance trends to provide a richer look at
longitudinal changes and better predict how constituents will view services in
the future.
Survey Method, Structure, and Process
At each participating institution, the Survey is administered to all teaching
faculty, all staff members who are not members of the library or IT
organizations, and a stratified sample of students selected randomly from the
population. The Survey is generally administered starting on the fourth
Thursday of each institution’s Spring semester. This approach helps ensure that
each institution’s data can be compared to data gathered at other institutions.
The Survey’s strategy of regular outreach to
respondents enables each campus to achieve high response rates compared to
other surveys. In addition, surveying a sample of each institution’s student
body helps to avoid student survey fatigue and further increases the student
response rate. These methods helped to achieve response rates in 2010 of 48.8%
for faculty, 44.9% for students, and 50.3% for staff.
In addition to the core questions included in
the base instrument, most participating schools include an expanded set of
optional questions and many include custom questions that ask about local
services. Most of the core and optional questions can be customized to reflect
the service names in use at each institution (e.g., those about the online
catalog or the course management system).
Once a school has agreed to participate in the
Survey in the coming year, its leadership selects a Campus Survey Administrator
(CSA) from among the library or IT staff. This individual is responsible for
all aspects of survey administration at his or her institution. A member of the
MISO Survey Team liaises with each institution, helps its CSA prepare for
upcoming administration deliverables, guides the CSA in working with the
school’s Institutional Review Board, and answers questions as the process
unfolds. These preparations for survey administration take place largely during
the Fall semester so that the Survey is ready to go live early in the Spring
semester. A more detailed timeline for MISO Survey administration is available
on the MISO website (http://www.misosurvey.org).
Once all participating schools have concluded
their survey administration, the results are summarized and analyzed during the
Spring and Summer months. Each participating school receives a comprehensive
spreadsheet that includes the mean values for questions included in its Survey
instrument for each population surveyed (faculty, staff, and students), as well
as comparable mean values for all other participating schools. These
spreadsheets include results from the current year as well as all previous
years. The spreadsheets allow for easy comparison of schools and cohorts to
show where statistically significant differences exist. Each institution also
receives its raw data as well as an SPSS file for further data analysis.
What Is Unique about MISO?
While each institution has a rich collection
of data to analyze from their own survey and from the spreadsheet of mean data
for all schools, the MISO team has also spent considerable time analyzing
results from all schools and cohorts to see broader patterns from within the
data. This broader analysis is one of the unique features of the MISO Survey,
as it is done in a statistically rigorous way that allows us to differentiate
between patterns that seem emergent based on anecdotal evidence or changes at a
single school and those that are truly widespread. The team has been able to
view changes in student attitudes about services as they move from freshman
year to senior year, as well as some changes that are happening in student
attitudes over time without regard to class year. We have also looked closely
at trends in the use, importance, and satisfaction with our services as it
relates to the age of our faculty members. Beginning with the 2010 Survey
(Table 1), we will look at how faculty and students within the various
disciplines interact differently with our services as well. Below, we have
provided one example of the kind of trend analysis possible with the MISO
Survey instrument by taking a deeper look at how the use of library services has
changed over time, and how those changes are different for faculty as compared
with students.
Examples of How Analyzing the MISO Data Provides Broader Perspectives
on Library and Technology Services
Much of the data analysis has focused on
changes in the use, importance, and satisfaction with services over time. In
this section, we look more closely at trends in the reported use of library
services, without consideration of importance or satisfaction, as an example of
one kind of analysis possible with the data. This section first presents the
mean frequency of use for faculty and students 2008-2010 as a benchmark about
current use patterns, followed by time trends taken from all institutions which
participated in the Survey more than once from 2005-2010 (N=27). It is
important to underline at the outset that an analysis of frequency of use alone
is not a sufficient gauge of a service’s value to faculty and students. Such an
analysis does, however, provide one informative, broader perspective on the IT landscape
in higher education.
2010 Benchmarks: Faculty and Student Frequency
of Use
Frequency of use in the MISO Survey is set on
a five-point scale:
- 1 (never)
- 2 (once or twice a semester)
- 3 (one to three times a month)
- 4 (one to three times a week)
- 5 (more than three times a week)
It should be noted that while the numbers used
in the scale increase in a linear fashion the categories do not increase
linearly. Each successive category represents an increase in use that is three
or four times greater than the previous category. As a result, a person
selecting category four uses a service about 16 times as much as a person
selecting category two, even if the numbers “4” and “2” suggest there is only
twice as much use.
Below are tables illustrating the frequency of
use of all library and technology services 2008-2010 (Figures 1 & 2). No
attempt is made to isolate what constitutes a library service per se, so that
nominal “library” services can be viewed in the context of all services. It is
of course difficult to decouple such increasingly linked terms.
Selecting from the overall array of services,
various combinations can be grouped under a more focused rubric labeled “the
library.” Any attempt to do so is potentially problematic given local
conditions at each institution. Librarian position descriptions at some
colleges involve campus course management system duties, for example, while
other librarians elsewhere help to maintain access to online resources from
off-campus via software proxy servers.
Despite differences in local conditions, there
will likely be wide consensus as to what represents a typical library service.
These standard library functions are grouped together for comparative analysis (Table
2).
Comparison of the data reveals marked differences
in how faculty and students use “the library.”
The most frequently used services by faculty
are the online library catalog (3.39), library databases like JSTOR (3.34), and
the library website (3.29). These are the only library services that faculty
use at least one to three times a month, on average.
In contrast, the most frequently used services
by students are public computers in the library (3.61) and quiet work space in
the library (3.29). These are the only library services that students use at
least one to three times a month on average.
Table 1
Sample Sizes and Response Rates
|
Population |
Sample Size |
Responses |
Response
Rate |
Total
Institutions |
|
Faculty |
9,482 |
4,707 |
49.6% |
38 |
|
Students |
22,757 |
8,605 |
37.8% |
38 |
Figure 1
Faculty use benchmarks
Figure 2
Student use benchmarks
The implications for “library as place” are
worth serious consideration. Across the board, students
report using library facilities more than faculty (Table 3).
When planning library facilities upgrades,
decision makers might do well to consider design with students foremost in
mind. They could also synthesize MISO frequency of use data with other
empirical research that yields similar results about faculty and library
facilities (Schonfeld and Housewright, 2010).
Whereas students use a location-based library,
faculty turn to online library services with greater frequency (Table 4). In
addition, faculty report a much higher use of “Access to online resources from
off campus” (3.54 vs. 2.38), which presumably includes the use of proxy
services which allow access to library materials outside of library facilities.
Note that it is difficult to determine the
extent to which some library services are perceived as location-based or
online. For example, library reference services can occur at a physical desk on
campus or via email and/or chat. Likewise, the provision of interlibrary loan
services occurs via online forms embedded in proprietary databases and at
location-based service points. Furthermore, library patrons can typically use
circulation services either online (i.e., a “renew books” option available in
the online library catalog) or in a physical facility. Overall, faculty use
most of these hybrid online/place-based library services with greater frequency
than students. However, library reference services are used to basically the
same extent by students and faculty (Table 5).
Table 2
Comparison of All Library Services Use
Benchmarks
|
Service Name |
Faculty Mean |
Student Mean |
|
Interlibrary Loan |
2.32 |
1.76 |
|
Library Circulation services |
2.70 |
2.25 |
|
Library Reference services |
2.18 |
2.22 |
|
Library website |
3.29 |
2.93 |
|
Online library catalog |
3.39 |
2.80 |
|
Library collections |
2.92 |
2.49 |
|
Library databases (e.g. JSTOR) |
3.34 |
2.90 |
|
Digital image collections (e.g. ARTstor) |
1.48 |
1.49 |
|
Library liaison/contact |
1.91 |
Not asked |
|
Online course reserves |
Not asked |
2.96 |
|
Study carrels in the library |
1.28 |
2.78 |
|
Quiet work space in the library |
1.55 |
3.29 |
|
Group study spaces in the library |
1.24 |
2.76 |
|
The Library café |
2.32 |
2.93 |
|
Public computers in the library |
1.82 |
3.61 |
Table 3
Comparison of “Place-Based” Library Services
Use Benchmarks
|
Service Name |
Student Mean |
Faculty Mean |
|
Public computers in the library |
3.61 |
1.82 |
|
Quiet work space in the library |
3.29 |
1.55 |
|
The library café |
2.93 |
2.32 |
|
Study carrels in the library |
2.78 |
1.28 |
|
Group study spaces in the library |
2.76 |
1.24 |
Table 4
Comparison of Online Library Services Use
Benchmarks
|
Service Name |
Faculty Mean |
Student Mean |
|
Online library catalog |
3.39 |
2.80 |
|
Library databases (e.g. JSTOR) |
3.34 |
2.90 |
|
Library website |
3.20 |
2.93 |
2010 Trends: Faculty and Student Frequency of Use
To analyze trends in the use of library
services, the following analysis relies only on data from schools where the
Survey was administered more than once since 2005 (N=27). New questions have
been added to the MISO Survey since 2005, stemming from changes in the wider
library and technology services landscape. As a result, trend data is available
for a smaller number of services because not all survey questions have yet to
be answered more than once by enough institutions to provide generalizeable trends
(denoted by “N.A.” in Table 6).
This section only reports on services where
the change in use over time was statistically significant for faculty or
students and where the change was large enough (+/-.025) to merit attention.
Consequently, an “--“ value in the table below denotes
a slope (i.e., a possible change over time) that is not statistically significant
or not large enough to be of real practical significance.
The only library services use trend common to
both faculty and students is increased utilization of databases like JSTOR
(0.0300 and 0.0348, respectively).
With the exception of library database use,
all other significant faculty library services trends reflect declines in usage
for faculty: reference services (-0.0380), circulation services (-0.0430), and
the online library catalog (-0.0430).
With the exception of the library website
(-0.0338), all other student library services trends reflect no change, or
reflect an increased usage (a rise in digital images collections like ARTstor
[0.0711]) with less pronounced but still significant growth in interlibrary
loan (0.0338).
Taken as a whole, these divergent trends also
suggest important differences in faculty and student library use patterns.
To focus only on notional “library” services
is to occlude important developments of interest to librarians, and this is
where MISO data distinguishes itself relative to more circumscribed assessment
tools. By means of conclusion, one additional technology frequency of use trend
deserves careful attention. Both faculty and students increasingly turn to the
course management system (0.2110 and 0.1399). The usage slopes for products
like Blackboard and Moodle are much steeper than any increased library use
trend. Librarians ought to consider embedding their services in their course
management system since that is where their patrons are to be increasingly
found.
Table 5
Comparison of Hybrid Online/”Place-Based”
Library Services Use
|
Service Name |
Faculty Mean |
Student Mean |
|
Library collections |
2.92 |
2.49 |
|
Library Circulation services |
2.70 |
2.25 |
|
Interlibrary Loan |
2.32 |
1.76 |
|
Library Reference services |
2.22 |
2.18 |
Table 6
Comparison of Statistically Significant
Library Services Use Trends
|
Service Name |
Faculty Trend |
Student Trend |
|
Interlibrary Loan |
-- |
0.0338 |
|
Library Circulation services |
-0.0430 |
-- |
|
Library Reference services |
-0.0380 |
-- |
|
Library Website |
-- |
-0.0337 |
|
Online library catalog |
-0.0430 |
-- |
|
Library collections |
N.A. |
N.A. |
|
Library databases (e.g. JSTOR) |
0.0300 |
0.0348 |
|
Digital image collections (e.g. ARTstor) |
-- |
0.0711 |
|
Library liaison/contact |
-- |
Not asked |
|
Online course reserves |
Not asked |
-- |
|
Study carrels in the library |
N.A. |
N.A. |
|
Quiet work space in the library |
N.A. |
N.A. |
|
Group study spaces in the library |
N.A. |
N.A. |
|
The Library café |
N.A. |
N.A. |
|
Public computers in the library |
N.A. |
N.A. |
Conclusion
The data analyzed provide evidence of trends
in stakeholder interactions with libraries for 2010. Faculty, for example,
decreasingly use the online library catalog, library circulation services, and
library reference services, and view these three service categories as
decreasingly important. Of these three service categories, the online library
catalog and library circulation services experienced slight drops in perceived
importance among faculty while library reference services experienced a
somewhat larger drop. On the other hand, faculty increasingly use library
databases and are increasingly likely to access online resources from
off-campus, which potentially speaks to an increased importance of proxy
services. At the same time, faculty consider library research instruction,
library liaisons, the library website, and interlibrary loan to be increasingly
important, in that order. As for undergraduates, they are slightly less
inclined to use library reference services and much less inclined to use the
library website over time. Conversely, and more so than faculty, undergraduates
increasingly use interlibrary loan, library databases, and particularly digital
image collections. Like faculty but even more so, undergraduates consider
library research instruction and interlibrary loan to be increasingly important,
in that order. Unlike faculty, the undergraduate trend is to view the library
website as slightly less important. Consistent with faculty, undergraduates
view library databases and off-campus access as increasingly important.
The analysis above provides one look at the
MISO data. By examining the use values for the subset of variables representing
library services across time and institutions, we can see trends and patterns
that would not have been as meaningful if taken from a single school. As the
MISO Survey has continued and expanded over the years, the usefulness of rich
comparable data from a set of peer institutions over time has increased
tremendously. The MISO annual summary data help participant schools in
identifying their relative strengths and weaknesses, creating peer groups for
analysis, and determining whether a problem is a local concern or a nationwide
trend. The analysis of micro data provided by the Survey Team allows library
and technology decision makers a wider perspective on trends and relationships
between services.
In addition to providing a rich source of
data, MISO can serve as a model for how a group of schools can collaborate on a
shared assessment tool that meets the needs of individual institutions and
provides a robust, aggregated dataset for deeper analysis. The process of
designing, updating, and customizing the MISO Survey by a team of library and
computing leaders from within participating institutions ensures that the
instrument remains relevant to decision making, and that the Survey is easy to
conduct. As the dataset becomes larger, and a greater variety of institutions
participate, we will continue to plan for ways to increase the usefulness and
scope of analysis, while ensuring that all participating institutions continue
to find useful measures of their own service.
References
Schonfeld, R. C., & Housewright, R. (2010). Faculty survey 2009: Key strategic insights for libraries, publishers,
and societies. New York: Ithaka S+R. Retrieved 24 May 2013 from http://cyber.law.harvard.edu/communia2010/sites/communia2010/images/Faculty_Study_2009.pdf