Article
Evidence Based Practice Using Formative Assessment in
Library Research Support
Jackie Wolstenholme
Research Services Librarian
Information & Research
Services
Eddie Koiki Mabo Library
James Cook University
Townsville, Australia QLD
4811
Email: jackie.wolstenholme@jcu.edu.au
Received: 28 Jan. 2015 Accepted:
17 Jun 2015
2015 Wolstenholme.
This is an Open Access article distributed under the terms of the Creative
Commons‐Attribution‐Noncommercial‐Share Alike License 4.0
International (http://creativecommons.org/licenses/by-nc-sa/4.0/),
which permits unrestricted use, distribution, and reproduction in any medium,
provided the original work is properly attributed, not used for commercial
purposes, and, if transformed, the resulting work is redistributed under the
same or similar license to this one.
Abstract
Objective
–
The purpose of this study was to develop and review the effectiveness of a new
evidence-based approach for teaching library research support.
Methods
–
Formative assessment, through two variations of the One Minute Paper model, is
used to poll the experiences of university researchers in library research
support sessions. Prior to a session, Polling One Minute Papers (POMPs) assess
what researchers know about topics that will be covered in the session. After a
session, Reflective One Minute Papers (ROMPs) review whether university
researchers achieved the intended learning outcomes of the session. POMPs were
used for 16 sessions and ROMPs were used for a subset of 11 of these sessions.
Examples of responses from the POMPs and ROMPs were presented to describe and
analyse the effectiveness of this approach for library support of research.
Results
–
POMP and ROMP responses were remarkably informative given their simplicity and
the little effort required on the part of the instructing librarian or
researchers. The completion rate of POMPs was 72.7%. They gave researchers the
opportunity to self-assess their current level of knowledge or skills about the
topic to be covered in the upcoming session. The librarian could then tailor the
session content to this level of knowledge. POMP responses were shared as part
of the session content, enabling researchers to benchmark themselves against
their peers. Completion rate of ROMPs was 20.9%, with the level of reflection
in the individual researchers’ responses varying from shallow to insightful.
Deeper responses stated how the researchers would use what they learned or pose
new questions which emerged from their learning.
Conclusion
–
Polling One Minute Papers (POMPs) and Reflective One Minute Papers (ROMPs) are
an effective and efficient approach for guiding the learning of researchers and
closing the feedback loop for librarians. These tools extend the opportunity
for librarians to engage with researchers and, through tailoring of session content,
assist to maximise the benefit of library research support sessions for both
librarians and researchers. Sharing of POMP and ROMP responses can assist
librarians to coordinate the teaching of the researchers that they support. At
an institutional level, evidence in POMPs and ROMPs can be used to demonstrate
the value that the library has contributed to improving awareness and
performance of its researchers.
Introduction
Researchers
in universities are working in an increasingly complex and competitive
environment (e.g., Frances, Fletcher, & Harmer, 2011; Kennan, Corrall,
& Afzal, 2014; Richardson, Nolan-Brown, Loria, & Bradbury, 2012).
Factors driving these changes include Internet and digital technologies and
greater accountability through performance management and institutional
benchmarking. These changes are requiring researchers to adapt faster than most
would achieve through their traditional discipline-based networks, including
information sharing among colleagues.
The
Internet and digital technologies have transformed scholarly communication.
Research outputs, although still published as books and journals, are now also
made available in an array of other digital options including blogs and other
social media, multimedia formats, and data files which may be displayed through
sophisticated visualization tools. The numbers of research outputs have vastly
increased and are distributed through a growing range of publishing models,
many offering some form of Open Access. Researchers, as creators of research
outputs, need to consider copyright and licensing for managing their rights, in
balance with maximizing accessibility to their research outputs. The quality of
publishers also needs to be assessed, to ensure that researchers avoid
unethical publishers (e.g., see Beall, 2014).
To
measure and benchmark performance, researchers and their institutions rely on
citation ranking metrics. Researchers need to understand how these metrics are
calculated and how citation indexes (e.g., the h-Index) are calculated.
Researchers are also expected to have an online presence, ideally as a
professional profile to promote their research interests and achievements.
Altmetrics are emerging as an additional measure of impact, by measuring the
online activity of a researcher or their outputs (Priem, Taraborelli, Groth,
& Neylon, 2010).
University
libraries can assist researchers to work in this environment and make the most
of emerging opportunities. To provide this support, university libraries are
moving their core business from provision of information resources to provision
of information services and information solutions (Association of College and
Research Libraries, 2010; Kaufman, 2009; Parsons, 2010). Information resources
have traditionally involved the development and management of collections. In
contrast, information services and solutions include infrastructure such as
repositories (Simons & Richardson, 2013) and instructional support on
topics such as scholarly and open access publishing, managing research data,
maximizing research visibility, and measuring research performance (e.g.,
Auckland, 2012; Haddow, 2012; Kennan, et al., 2014).
Effective
communication skills are essential for building a rapport with researchers and
providing a valued service (Auckland, 2012; Creaser & Spezi, 2013; Parker,
2012). Research support librarians need to be confident in talking about the
range of topics that researchers need to learn, match the information they
provide with the skill level of the researcher, and explain the information in
a way that is understandable for the researcher. This study investigates a
teaching and learning approach which can assist librarians to achieve this.
Literature
Review
Teaching and
Learning in Library Research Support
Teaching
and learning for researchers is best suited to the learning theory of
andragogy. This theory is based on the assumption that adults are self-directed
learners who are interested in immediate application of knowledge (Merriam,
2001). According to this theory, adults take the initiative in diagnosing their
learning needs, including formulating objectives, identifying resources,
implementing strategies, and evaluating outcomes (Knowles, 1975). Other
learning theories of relevance to library research support include
problem-based learning (Knowles, 1975), experiential learning (Kolb, 1984), and
informed learning (Hughes & Bruce, 2012). In problem-based and experiential
learning, learners draw on their prior knowledge and experience (Brodie, 2012),
enabling learning to be built on a researcher’s existing practices. Informed
learning describes how learners develop flexibility and confidence to use
information in constantly evolving information environments, shifting the focus
of information literacy education from mastering skills to learning to use
information critically, ethically, and creatively (Hughes & Bruce, 2012).
In
addition to self-directed learning, another key aspect of researcher learning
is that researchers frequently learn from their peers. The peer is a defining
figure in research practice. For example, it is implicit in the institution of
“peer review” (Boud & Lee, 2005). As described for higher degree research
students, research learning can be usefully construed in terms of entry into
communities of practice, where peer learning becomes a powerful tool for describing
and developing a rich understanding of the learning resources available (Boud
& Lee, 2005). Peer interaction can enhance learning by stimulating the
production of deeper thought through the desire to know what a colleague knows,
prompting self-assessment and clarification of uncertainties (Draper, 2009).
Assessment in
Library Research Support
Library
research support needs to contribute to improving research performance without
adding additional burden to a researcher’s workload. Researchers operate in a
constant environment of research performance assessment (Parker, 2012), e.g.,
through funding or promotion applications, performance management acquittal, or
as part of institutional assessment exercises such as those in Australia and
the United Kingdom (Australian Research Council, 2014; REF2014, 2014). Research
librarians need to be acutely aware of this research assessment landscape
(Parker, 2012). The learning needs of researchers must form the core content of
library research support material, in terms of what researchers need to know as
well as their current status of understanding a particular topic.
Within
a teaching and learning framework, assessment of the performance of researchers
is analogous to summative assessment, and library support is analogous to
formative assessment. Summative assessment tasks focus a student’s learning on
“what counts,” while formative assessment provides a fine tuning mechanism
which guides the learner’s learning progress (Boud, 2000). Summative assessment
occurs after the learning process for the purpose of certification (Sadler,
1989). In contrast, formative assessment occurs as part of the learning
process. Through formative assessment, the learner gains feedback which is
intended to shape and improve their learning, leading to independent learners
who are able to self-monitor their learning needs (Sadler, 1989). To implement
this analogy of formative assessment in library research support, research
librarians will be most effective if they develop teaching materials which
incorporate the established practices of researchers’ self-directed and peer
learning.
Learning
is a cyclical process, as explained in experiential learning theory (Kolb, 1984).
In the context of library research support, there is no starting point in the
researchers’ learning process, but rather, they build on what they already know
or have experienced. Within the learning cycle of assessment (Figure 1, Crisp,
2009), there may be many feedback loops between the phases of diagnostic,
learning, and formative assessment (Sadler, 1989). Often, formative assessment
leads to summative assessment, as researchers take on tasks to advance their
careers or to meet institutional requirements.
Research
Framework: Formative Assessment of Researchers
The
One Minute Paper (OMP) is a formative assessment tool that has been successful
in improving the teaching of, and learning by, undergraduate students (Bartlett
& Morrow, 2001; Chizmar & Ostrosky, 1998). OMPs are effective for
gaining student feedback in return for a modest amount of student and
instructor effort (Bartlett & Morrow, 2001; Chizmar & Ostrosky, 1998;
Drummond, 2007; Stead, 2005). The OMP is a questionnaire which asks:
1. What was the most important thing you
learned today?
2. What was the most confusing point in
today’s lecture?
3. What was the most interesting fact that you
learned today?
Figure 1
Relationship between diagnostic, formative, and
summative assessments (redrawn from Crisp, 2009)
The
addition of a third question was recommended by Bartlett and Morrow (2001:
The
OMP benefits both instructors and students, regardless of their teaching or
learning ability (Chizmar & Ostrosky, 1998). OMPs can provide specific and
immediate feedback to the instructor about student learning, helping to set the
pace and content of future instruction. This is useful for inexperienced
instructors or instructors of new material (Stead, 2005), as is often the case
in library research support. Instructors can also use the feedback to identify
and then address misconceptions (Bartlett & Morrow, 2001). This closing of
the feedback loop demonstrates that the instructor values student opinion and
encourages students to actively contribute to their own learning experience
(Stead, 2005). Class discussion of issues raised in OMP’s have reassured
students by enabling them to benchmark their learning against their classmates,
often revealing that the problems that others are experiencing are the same as
their own (Bartlett & Morrow, 2001).
The
simplicity of the OMP makes it an ideal tool for identifying the learning needs
and learning outcomes of researchers. Content of library research support
sessions can then be tailored accordingly. The OMP is typically assigned at the
end of a class, but could also be adapted for implementation prior to a class
(Stead, 2005). Pre-class formative quizzes encourage students to think
critically about course content prior to a session (Dobson, 2008), offering the
benefits of identifying current learning needs or learning gaps, providing an
indication of what will be covered in the upcoming session, and creating an
opportunity for self-assessment.
Aims
This
study describes a method, adapted from formative assessment in teaching and
learning, to assist research support librarians to develop an evidence-based
foundation to support their teaching. Two variations of the One Minute Paper
(OMP), Polling OMPs and Reflective OMPs, are developed in this study. A case
study approach, from a series of multiple workshops on a range of topics, is
used to investigate whether the two variations of the OMP are an effective and
efficient approach for guiding the learning of researchers and closing the
feedback loop for librarians.
Questions
asked are:
1.
Do
POMPs stimulate researcher engagement and interest?
2.
Can
POMPs identify learning needs of researchers?
3.
Are
POMPs or ROMPs effective tools for gaining feedback about researcher learning?
Methods
This
study reports on the outcomes of a series of case studies, to explore the
effectiveness of using the One Minute Paper (OMP) model for the purpose of
library research support sessions. Two variations of the OMP were developed in
this study: POMPs, i.e., Polling One Minute Papers, and ROMPs, i.e., Reflective
One Minute Papers.
Both
the POMPs and ROMPs are intended as tools which guide the formative learning of
researchers. POMPs were distributed prior to a session and ROMPs were
distributed after a session. Sessions were organized in response to specific
requests from researchers on behalf of a research group, rather than according
to a specific schedule.
Overview of
the James Cook University Research Profile
The
OMPs described in this study were developed for library research support
sessions at James Cook University (JCU). The Strategic Intent of JCU is to
create a brighter future for life in the tropics world-wide (James Cook
University, 2015).The number, distribution and turnover of JCU researchers make
it challenging to identify and meet their evolving library research support
needs. There are approximately 2600 academic staff and more than 600 Higher
Degree Research students (James Cook University, 2014) across Townsville,
Cairns, Singapore and other smaller, regional centres.
Research
needs vary with disciplinary research practices and career stage. At JCU, the
largest and fastest growing area of research is in the medical disciplines,
with many of these researchers having a strong applied knowledge but limited
research experience. In contrast, internationally recognized researchers in the
biological and environmental sciences tend to have metrics-driven library
support needs. Humanities and social sciences, including Law and Creative Arts,
have the most discipline-centric research needs. Career stage also influences
library research support requirements: postgraduate students and early career
researchers need to develop their research skills, mid-career researchers may
be concerned about keeping up with technological changes, and senior
researchers may be under pressure to maintain their high research standing.
Polling One
Minute Papers (POMPs)
POMPs
are a self-assessment tool. The questions asked in a POMP were structured
around the topic of a library research support session, polling researchers to
gauge their understanding of the topic. Session content was then tailored for
this level of understanding. The questions and response options in POMPs are
listed in the Appendix. POMPs were also intended to promote a session and
stimulate interest about the content that would be covered in that session.
Sessions
were organized in collaboration with research leaders, e.g., key researchers or
research managers. This strategy helped to increase attendance and facilitate
discussion because participants shared common research interests and usually knew
each other prior to the session. POMPs were distributed approximately one week
prior to a session in an email. This email was sent by the research leader to
all researchers who he felt should attend the session. POMPs were voluntary,
with participants being encouraged to submit their responses prior to the
session. A summary of the POMP results were shared during the corresponding
session to enable peer benchmarking and therefore further self-assessment. POMP
responses were presented as descriptive data in tables, histograms or pie
charts.
Reflective One
Minute Papers (ROMPs)
ROMPs
are a feedback tool which encouraged voluntary reflection about a session. The
same three questions were asked in all ROMPs:
1.
What
was the most important thing you learnt?
2.
What
was the most confusing thing I covered?
3.
What
was the most interesting thing you learnt?
ROMPs
were completed on a voluntary basis. By responding to these three questions,
researchers were able to provide feedback about the session and their learning
to the librarian. ROMPs were developed partway through this project in order to
close the formative assessment loop. ROMPs were used for 11 sessions, as listed
in Table 1. A link to the three ROMP questions was usually distributed on the
last slide of a session presentation or immediately after a session. In one
exception (Session 16), the ROMP link was sent out 2 weeks after the session.
ROMP responses were in an unstructured, free text format. Thematic analysis,
also known as analytic coding, was used to interpret the responses and quantify
them according to themes (Guest, MacQueen, & Namey, 2012; Richards, 2015).
Data
Collection
POMPs
and ROMPs were created in Google Forms (Google, 2014). The selection of Google
Forms was based on a number of criteria: Google Forms is a free service with no
limit on the number of questions that can be asked; checklists of multiple
options per question can be selected, enabling quick standardized responses;
and, the OMPs could be shared through an online link, enabling participation by
both local and remote researchers.
POMPs
and ROMPs were anonymous, a decision based on the presumption that anonymity
would make researchers more likely to provide honest responses and therefore
enable more realistic benchmarking amongst peers (Dillman, Smyth, &
Christian, 2009). Respondents were also not required to log in using Google
Forms, building researcher confidence that the OMPs were anonymous.
Results
The
results report on a series of case studies to explore the effectiveness of
POMPs and ROMPs to respond to the three questions stated in the Aims.
Response Rates
of POMPs and ROMPs
The
numbers of researchers attending the sessions for which POMPs and ROMPs were distributed
are summarized in Table 1.
The
completion rate of POMPs was high, at 72.7%, i.e., 136 from a pool of 187
researchers who attended the 16 sessions. A likely factor contributing to this
high rate of completion was the simplicity of the POMP form. In one click from
a link in an email, it was immediately evident what the researcher needed to
respond to. The entire POMP could be viewed on a desktop screen without
scrolling, visually emphasizing that the form would be quick to complete, with
the format of all or most responses being checklists. Promoting the POMP as a
“1 minute quiz” with endorsement from a research leader was also effective. For
example, the Director of a research centre for one of the Life and
Environmental Science sessions included the following statement of support in
his email:
As
part of the prep for the planning day, and to help with our understanding of
the use of Research Profiles, can I ask you to take this 1 minute quiz? I took
it and it took even less than 1 minute.
The
completion rate of ROMPs was lower, at 20.9%, i.e., 23 from a pool of 110
researchers who attended the 11 sessions for which ROMPs were distributed. A
major factor contributing to this low response rate may have been that the
response format was free text.
Researchers
were from the disciplines of Life and Environmental Science, Health Science,
Social Science and Humanities, and Mixed Disciplines (Table 1). The librarian
was present in the same room with participants for sessions 1-10, 15 and 16.
Sessions 11-14 were conducted remotely via videoconference (Table 1). Sessions
1-15 were presented by the author. Session 16 was presented by another research
support librarian who provided peer feedback about the value of OMPs.
Table 1
Summary of Attendance, POMPs and ROMPs for each
Session
|
Discipline /
Client Group |
Session Topic |
Remote* |
Session date |
Attendance |
# POMPs submitted |
% POMPs submitted |
# ROMPs submitted |
% ROMPs submitted |
|
|
1 |
Life and Environmental Sciences: Post-Docs |
Altmetrics |
No |
9 October 2013 |
23 |
17 |
73.9 |
---- |
---- |
|
2 |
Life and Environmental Sciences |
Managing research profiles |
No |
11 October 2013 |
28 |
28 |
100.0 |
---- |
---- |
|
3 |
Life and Environmental Sciences |
Research profile update and management |
No |
12 August 2014 |
8 |
9 |
112.5 |
6 |
75.0 |
|
4 |
Life and Environmental Sciences |
Research profile update and management |
No |
29 August 2014 |
8 |
6 |
75.0 |
1 |
12.5 |
|
5 |
Life and Environmental Sciences |
Research profile update and management |
No |
19 September 2014 |
5 |
4 |
80.0 |
1 |
20.0 |
|
6 |
Life and Environmental Sciences |
Research profile update and management |
No |
9 October 2014 |
4 |
4 |
100.0 |
0 |
0.0 |
|
7 |
Life and Environmental Sciences |
Research profile update and management |
No |
10 October 2014 |
4 |
2 |
50.0 |
0 |
0.0 |
|
8 |
Life and Environmental Sciences |
Research profile update and management |
No |
24 November 2014 |
9 |
8 |
88.9 |
0 |
0.0 |
|
9 |
Health Sciences |
Publishing academic research |
No |
23 October 2013 |
11 |
6 |
54.5 |
---- |
---- |
|
10 |
Health Sciences |
Research and social media |
No |
27 August 2014 |
14 |
15 |
107.1 |
4 |
28.6 |
|
11 |
Health Sciences |
Quality publishing |
Yes |
28 August 2014 |
3 |
2 |
66.7 |
3 |
100.0 |
|
12 |
Social Sciences and Humanities |
Researcher identifiers |
Yes |
9 September 2014 |
2 |
3 |
150.0 |
2 |
100.0 |
|
13 |
Social Sciences and Humanities |
Altmetrics |
Yes |
18 September 2014 |
5 |
4 |
80.0 |
4 |
80.0 |
|
14 |
Social Sciences and Humanities |
Promoting and maximising research impact |
Yes |
2 December 2013 |
2 |
6 |
300.0 |
---- |
---- |
|
15 |
Mixed disciplines: Academic teaching staff |
Research impact and publishing |
No |
13 February 2014 |
13 |
9 |
69.2 |
---- |
---- |
|
16 |
Mixed disciplines: Higher Degree Research
students |
Intellectual Property & Copyright |
No |
2 and 9 October 2014** |
48 |
13 |
27.1 |
2 |
4.2 |
|
Total Attendance
and number of POMPs/ROMPs submitted for all sessions |
187 |
136 |
|
23 |
|
||||
Figure 2
Display of results for the POMP question "Have
you used any of the following to promote or discuss your research?" (Responses
from 78 submitted POMPs distributed to 89 researchers, compiled progressively
and presented in sessions 1-8).
Aim 1: Do
POMPs Stimulate Researcher Engagement and Interest?
POMPs
enhanced the engagement of researchers. Completion of a POMP and then the
sharing of POMP results provided researchers with two opportunities for
self-assessment. Firstly, POMPs could provide additional incentives to go to a
session if researchers identified gaps in their knowledge or skills based on
questions asked in the POMP. In at least one case, a researcher started to use
some of the tools listed in the POMP prior to the session. Secondly, in
sessions, researchers showed great interest in the activities and tools being
used by their peers. Figure 2 displays the combined responses for eight
sessions on the same topic (Managing research profiles) for researchers from
the same discipline (Life and Environmental Sciences). The cumulative responses
were shared in each of the eight sessions. The opportunity to benchmark against
peers and resulting discussion gave endorsement to the advice provided by the
librarian. For example, one researcher reported that he now has a new
international collaborator with substantial funding through a LinkedIn™
connection. Hearing how their peers were using such tools gave context-relevant
evidence, making it easier to sell the concept of using social media tools in a
research context. Researchers were also able to see which tools were widely
used by their peers, giving an indication of where to get started, or a
confidence boost if they were already using those tools. Learning about lesser
used tools such as The Conversation (http://theconversation.com/au) gave
researchers ideas for how they could increase awareness of their research.
Aim 2: Can
POMPs Identify Learning Needs of Researchers?
POMPs
were effective and efficient in identifying current learning needs of
researchers and what the focus of the session content should be prior to a
session. POMP responses indicated what researchers understood or were doing
well and also gaps in their current knowledge or activities.
Responses
from Life and Environmental Science researchers attending one of the eight
sessions represented in Figure 2 (Session 2, as listed in Table 1) indicated
that participants in this session were using some online tools and almost half
had used traditional media to communicate and promote their research (Figure
3). This suggested that they were generally aware of the importance of
communicating and promoting their research. Despite this apparent awareness,
the moderate to nil use of more than half the listed tools (Figure 3) also
highlighted areas for further instruction.
Figure 3
Display of results for the POMP question "Have
you used any of the following to promote or discuss your research?"
(Responses from 28 submitted POMPs distributed to 28 researchers, presented in
session 2).
As
a further example of how POMPs were useful for identifying learning needs,
Health Science and Life and Environmental Science researchers were mostly aware
of whether an article processing charge (APC) had been paid for their article,
but were less clear about whether they had signed copyright ownership over to
the publisher. Approximately one third of responding researchers from each
discipline had authored a paper in which an APC had been paid and less than 10%
were not sure (Figure 4). For authors who had paid an APC, a serious issue for
discussion was highlighted: eight of the Life and Environmental Science
researchers (Figure 5) and the four Health Science researchers who indicated
they had paid an APC were not sure if they had retained copyright ownership of
their work. This finding highlighted the need to focus on the importance of
understanding the conditions of a publisher copyright agreement in the session.
This was pertinent given that ResearchGate™, a site which facilitates the
sharing of research publications, was a frequently used social media tool
(Figures 2 and 3).
Figure 4
Relative proportions of researchers, based on POMP responses,
who have paid an Article Processing Charge (APC) to make an article Open
Access; Health Science (15 responses, Session 10) and Life and Environmental
Science (33 responses, Sessions 3-8)
Figure 5
Display of results for the POMP questions 5a:
"Have you or a co-author paid an Article Processing Charge to make any of
your articles open access?" and 5b "If you have paid an Article
Processing Charge, do you know if you retained copyright ownership of your
article?”. (Responses from 38 submitted POMPs distributed to 38 researchers,
compiled progressively and presented in sessions 3-8).
Despite
the anonymity of POMP responses, examining answers to each question from
individual responses was also useful for identifying learning needs. In
relation to scholarly publishing, researchers attending session 9 each had some
publishing experience, mostly with journals and conferences (Table 2). However,
only three researchers were familiar with the Australian Government Higher
Education Research Data Collection (HERDC) (Department of Education, 2014) and
Excellence in Research for Australia (ERA) (Australian Research Council, 2014).
Although this group of researchers was aware of the importance of publishing,
most were not using the HERDC and ERA specifications, a form of summative
assessment for researchers in Australian universities, to guide their
publishing decisions. Session content was therefore adapted to explain the
specifications at an introductory level. Similarly, of the 12 researchers in
Figure 3 who had used traditional media to promote their research, only 6
identified ResearchOnline@JCU, the institutional repository, as a tool for
communicating and promoting their research. This identified a point for
discussion in the session, revealing that some researchers had only considered
the institutional repository as an administrative reporting tool and not an
avenue for communicating and promoting their research.
Table 2
POMP Responses from Health Science Researchers about
Their Experience in Publishing Academic Research*
|
Have you published
any of the following types of journal articles? |
Have you published
any of the following types of conference works? |
Have you published
any of the following types of books or book chapters? |
Have you co-authored
other types of works that are derived from your research? |
Familiarity with
HERDC or ERA |
Use of journal
ranking tools |
|
Peer reviewed article |
Peer reviewed paper, Poster, Abstract or
summary |
|
|
ERA: Excellence in Research for Australia |
|
|
Peer reviewed article, Short note or
commentary |
Abstract or summary |
|
|
|
Journal Citation Reports |
|
Peer reviewed article, Non-refereed article,
Case study |
Peer reviewed paper, Non-refereed paper,
Poster |
|
|
|
|
|
Peer reviewed article, Non-refereed article,
Short note or commentary |
Poster, Abstract or summary, Edited a
conference proceedings |
Teaching material |
|
HERDC: Higher Education Research Data
Collection, ERA: Excellence in Research for Australia |
Journal Citation Reports |
|
Peer reviewed article, Non-refereed article,
Short note or commentary |
Abstract or summary |
Non-commercial |
|
HERDC: Higher Education Research Data
Collection |
Journal Citation Reports |
|
Non-refereed article |
|
|
|
|
Journal Citation Reports, Scopus analytics |
Aim 3: Are
OMPs an Effective Tool for Gaining Feedback about Researcher Learning?
Polling One Minute Papers (POMPs)
POMPs
created an opportunity to learn about the audience prior to an upcoming
session, freeing up time in a session for discussion and teaching additional
content. POMPs also created the opportunity to ‘hear’ from researchers who
would normally not speak up in a session or who were only prepared to share
information anonymously. Even when the number of responses was low, due to a
small number of researchers attending a session, POMPs were still useful
because all or most of the audience responded.
POMP
responses were useful for refining how sessions were taught. For example, only
a small proportion (14.7%) of researchers indicated that they had a good
understanding of the term altmetrics, but all who selected this option provided
an appropriate description of the term (Table 3). In contrast, of the 75
researchers who responded to this question, 42% indicated they had no
understanding and 22% indicated they had some understanding of altmetrics.
Given this range, discussion was encouraged in sessions to facilitate learning
from peers who provided explanations that other session participants could
easily relate to.
POMPs
created an unexpected insight from one group of remote researchers (session
14). Enquiry as to why only one researcher had responded to the POMP revealed
cultural differences as the underlying issue. The group’s research manager
explained that the researchers were embarrassed that they had no or few
publications and so were reluctant to respond to the POMP. This information was
useful in itself as it indicated that the session content needed to be directed
at getting the researchers started with publishing. Further explanation about
the POMP boosted the response rate to 6, although only 2 researchers attended
the session. These insights will be considered in the planning of future
sessions with this group of researchers.
Feedback
from a colleague who trialed the use of a POMP (session 16) reported that it
was very interesting and useful to see the range of disciplines and prior
knowledge of the researchers who had registered for the session. In this POMP,
respondents were asked to table any prior questions, which gave the librarian time to prepare for complex
questions.
Reflective One Minute Papers (ROMPs)
ROMP
responses gave an insight into what researchers gained from their session. They
were used by the librarian to determine whether intended learning outcomes were
achieved, and consider how the teaching or content of future sessions could be
improved. ROMPs were particularly effective in facilitating reflective feedback
from remote sessions with small numbers of participants. In discussions at the
end of sessions with 2-5 participants (Sessions 11-13), responses were received
from all or most participants. In some cases, immediate clarification was given
in the session by the librarian. For more complex issues, future sessions were
offered as a response. Although response rates were lower for larger sessions,
the ROMPs were still beneficial for encouraging reflection, receiving feedback,
and continuing the conversation with session participants. ROMPs were least
effective for Sessions 4-8, which were hands-on computer sessions with
substantial discussion and feedback throughout the sessions. When asked to
complete the ROMP, the researchers repeated comments they had made during the
session, but only 2 of 30 recorded their feedback in a ROMP. Feedback from
these hands-on sessions was mostly positive, with the exception of one
researcher who was frustrated with the work she needed to do to manage her
online presence.
Most
researchers responding to the ROMPs gave a response for each of the three
questions. Within the 23 ROMP responses, 17 researchers responded to all 3
questions, 4 responded to 2 questions, and 2 responded to only 1 question.
Using thematic analysis, responses were grouped into one of five categories
(Table 4). The number of responses to each of the three questions is presented
in Table 5. The categories “topic named” and “positive statement” suggest a
relatively shallow level of learning and limited engagement with the session
content. “Reflective statement” suggests some level of engagement, while
responses coded as “reflective statement with further insight” and “reflective
question” each provide evidence of deeper learning. Most responses to the
question about the most important things learned were reflective statements.
The majority of responses to the question about the most confusing thing
covered were positive, indicating that respondents felt that they understood
the session content. Responses to the question about the most interesting thing
learned indicated a range of levels of engagement from positive statements or a
reflective statement with some demonstrating further insight.
Table 3
POMP Responses from Researchers Who Indicated that
They Had Some or a Good Understanding of Altmetrics
|
Understanding of
altmetrics |
If you have heard of
the term altmetrics, briefly describe what you understand it to mean? |
|
|
Health Sciences |
Good |
stats of research acknowledgement |
|
Health Sciences |
Good |
It appears to be a count of twitter mentions
(although it might include more than that) |
|
Health Sciences |
Some |
heard it but not sure of it's meaning |
|
Health Sciences |
Some |
No much I understand it |
|
Health Sciences |
Good |
A measure of the social media impact of your
paper |
|
Health Sciences |
Good |
means of measuring research impact |
|
Life and Environmental Sciences |
Good |
Non-traditional metrics, number of mentions
on websites, social media, media etc |
|
Life and Environmental Sciences |
Good |
proposed/potential alternative to Impact
Factor, as a measure of influence |
|
Life and Environmental Sciences |
Good |
Another way of measuring research 'impact' |
|
Mixed Disciplines |
Good |
A number that represent the amount of attention
an article receives from blogs, twitter, etc. |
|
Mixed Disciplines |
Good |
measure of attention
an article has received relative to 'lifespan' |
Table 4
Categories Used to Group ROMP Responses
|
Response category |
Explanation of category |
Example responses from this study |
|
Topic named |
Simple listing of an
aspect covered in the session, with no insight to the researcher's learning |
· open access · altmetrics |
|
Positive statement |
Indicates session was
worthwhile, with no insight to the researcher's learning |
· Nothing was
confusing - excellently done · All of it.
Informative and interesting presentation. Thank you. No negative feedback was
recorded in any responses. |
|
Reflective statement |
Repeats content from
the session, highlighting specific aspects |
· labouring the points
about "dodgy" journals. I liked the tips to improve your Altmetric
score. · I also didn't know
that Twitter can be so useful. |
|
Reflective statement
with further insight |
What the researcher
learnt and how it applies to their personal situation or how they will use
what they have learnt |
· The most interesting
thing I learn was about the importance of open access. I am going to bring
some of this information to my lab group and postgrads. Overall, thank you so
much for taking the time and going over these things - you are so
knowledgeable and kind, it was a wonderful workshop! · The importance of
twitter to academics. I always thought it was meant for teenagers. I will
definitely sign up for a twitter account after this session. |
|
Reflective question |
Indicates deeper
thinking by new issues that the session content raised for the researcher |
· Probably not covered
entirely, but I am interested how publications in journal with lower impact,
but receive higher citations, might influence the indices we looked at during
the session · Nothing was
confusing. However would have liked an example of how to tweet a publication.
I have tweeted a publication but there are no doughnuts associated with the
corresponding author publication list in Research Portfolio. |
Table 5
Number of Responses to Each of the Three ROMP
Questions
|
|
ROMP Question and number of responses |
||
|
Response category |
What was the most important thing you
learnt? |
What was the most confusing thing I covered? |
What was the most interesting thing you
learnt? |
|
Topic named |
2 |
1 |
1 |
|
Positive statement |
1 |
10 |
6 |
|
Reflective statement |
13 |
3 |
8 |
|
Reflective statement
with further insight |
4 |
2 |
5 |
|
Reflective question |
0 |
4 |
1 |
Discussion
This
study describes a new approach to providing library support for researchers.
The defining feature of this approach is that it is simple yet informative. The
approach adapts and combines two strategies derived from a formative assessment
framework. The first strategy is polling researchers using the One Minute Paper
concept to (1) identify their learning needs and (2) increase levels of
engagement (e.g., Hoppenfeld, 2012). The Polling One Minute Paper (POMP) is
designed to be quick for researchers to complete, and easy for librarians to
interpret and gain a snapshot of current learning needs of the target group of
researchers. Completing the POMP prior to the session gives responding
researchers an indication of the session content, raising their interest and
allowing them to self-assess their understanding. During the session, a summary
of the anonymous POMP responses is presented.
Sharing
the POMP responses is effective in that it provides context relevant
information against which session participants can benchmark themselves, and
prompts peer to peer discussion within the context of the participants’
discipline. Increased levels of discussion provide informal evidence of the
effectiveness of this strategy, which would be unlikely to occur through
didactic delivery of the session content. At the end of the session,
participants are asked to complete a Reflective One Minute Paper (ROMP),
encouraging researchers to reflect on the session content and provide feedback
which enables the librarian to review what the researchers gained from the
session. This second strategy is adapted from the original
One Minute Paper (Chizmar & Ostrosky, 1998), which was first used in
library instruction by Choinski and Emmanuel (2006).
The
POMP-ROMP approach can improve the value of library research support sessions
for researchers. It offers a responsive approach to the current learning needs
of researchers. Variation in undergraduate students’ competencies represents a
significant challenge in information literacy pedagogy (Dunaway & Orblych,
2011). Researcher learning needs may be more diverse than that of undergraduate
students, varying with career stage, discipline, current research priorities or
activities, and previous training. The increasingly complex and competitive
environment that researchers now work in (Richardson, et al., 2012) also makes
it difficult to identify and track researcher learning needs. Given this
variability, the two complementary OMP tools described in this study are an
effective means of rapidly obtaining a snapshot of the current learning status
of a specific research group, immediately prior to and following a library
support training session. Librarians can use this snapshot to determine the
level of detail that they teach in a session, with the aim of meeting the
learning needs of researchers and providing the right amount of challenge to
spark engagement in the topic. Using a tailored approach also creates proactive rather than passive library
services, providing “just-in-time” and “just-for-me” assistance (Association of
College and Research Libraries, 2010).
The
POMP-ROMP approach is designed to slot into researchers’ workflow. The
simplicity of both the POMPs and ROMPs mean that they can be prepared at short
notice and used for small, large, remote, face-to-face, lecture, or hands-on
sessions. The flexibility of these tools enables the library to contribute to
improving researcher learning without adding additional burden to a
researcher’s workload (Parker, 2012). The POMPs and ROMPs also extend the
opportunity for discussion with researchers beyond the defined period of a
library research support session, creating further opportunity to build and
strengthen the researcher-librarian relationship (Auckland, 2012; Parker,
2012). In this study, ROMP feedback received in discussion at the end of
sessions was responded to directly, with the offer of future sessions where
relevant. Further consideration is needed for how to respond to written
responses. One option would be to ask researchers to include their name on the
ROMP if they would like further information. Another option would be to provide
a single response, shared with all session participants, which responds to all
issues raised in each ROMP from the corresponding session.
Many
academic libraries are now developing or offering support programs for their
researchers (Auckland, 2012; Richardson, et al., 2012). Maximizing the benefit
for researchers attending library research support sessions will also benefit
libraries by efficiently using the time that librarians spend in supporting
researchers. Efficient use of the time of librarians is important because it is
likely that libraries will develop research support services with no or little
additional resources (Kennan, Cole, Willard, Wilson, & Marion, 2006;
Kennan, et al., 2014). Therefore, as more librarians become involved in
supporting researchers in their institution, a coordinated approach will be
necessary to make efficient use of limited library resources. Aggregating POMP
and ROMP data from all library research sessions could be used as a
professional development tool for research support librarians, as also
described for peer review of teaching data (Drew & Klopper, 2014). From the
perspective of academic libraries, aggregated POMP and ROMP data could be used
for identifying learning gaps, sharing evolving perspectives in researcher
feedback, and optimizing content in library research support programs. Such a
community of practice could fast track the development of research librarians
(Drew & Klopper, 2014). As more librarians become involved, the opportunity
for peer review of library instruction could also lead to new insights (Drew
& Klopper, 2014) which may improve POMPs, ROMPs and other strategies for
teaching library support to researchers.
One
of the most important but often overlooked parts of the assessment cycle is for
teachers or instructors to close the loop by reflecting on results and making
appropriate changes, such as adjusting teaching methodologies or changing the
structure of a program (Oakleaf, 2009; Swoger, 2011). Ultimately, the goal of
all instruction and assessment efforts is to engage in reflective practice
(Oakleaf, 2014). Teaching programs can nearly always be improved (Swoger, 2011)
and should evolve to keep pace with new teaching strategies and content. POMPs
and ROMPs are a viable means of gaining regular, systematic feedback from
researchers to assist with developing and improving library research support
programs. If implemented as a routine practice, the POMP-ROMP model could
support a cyclical process of quality control and improvement.
Academic
libraries and librarians must demonstrate their value. Libraries and librarians
can no longer rely on an assumed belief by stakeholders that they are important
(Association of College and Research Libraries, 2010). This is particularly
true for the context of library research support. In this relatively new
enterprise of research support, libraries are trialing various models of operation
and entering spaces that were previously the domain of other sections of the
university, e.g., research offices. Assessing impact, to demonstrate value, is
made more complicated because academic libraries operate in a changing
environment in which people, services, and needs are constantly evolving (Association of College and Research Libraries, 2010).
It is also difficult to prove that actions taken by the library contributed to
improvements in the performance of researchers (Association of College and
Research Libraries, 2010). Implementing the POMP-ROMP approach as standard
operating practice for library research support will develop a data source
which provides evidence of a library’s value and how its contribution to the
support of the institution’s researchers has changed over time.
Limitations
The
strength but also limitation of the POMP-ROMP model is that collected
information applies to a specific situation, with respect to a topic and point
in time. POMP-ROMP responses should not be extrapolated to other contexts.
Each
POMP-ROMP dataset provides a snapshot of the status of the learning of a
specific group of researchers. No control groups are used and sample sizes are
often small and not randomly selected. All responses are analyzed and samples
are comprised of researchers with similar interests or skill levels. Questions
asked in POMPs are tailored to the topic of an upcoming session, so are not
intended to provide a detailed or comprehensive insight to library research
support issues.
It
should also be recognised that POMP and ROMP responses may not reflect the
learning or experience of all researchers in a corresponding session. The needs
of researchers who do not submit a POMP may not be considered in the
preparation of session content. Non-responses are most likely due to the
researcher having other priorities, not seeing a personal benefit in submitting
a response, or being reluctant to share his thoughts. In the most extreme
situation, non-response could be due to a lack of understanding of the
question. Given these assumptions, an unexpected finding was that the number of
POMP responses for sessions 3, 10, 12 and 14 was higher than session
attendance. As a result, the needs of the responding researchers may be
addressed but not actually apply to the researchers who attended the session.
In the current study, non-response to ROMPs seemed mainly due to researchers
not having anything to report that was not already expressed during the
session. Unfortunately, this verbal feedback was not recorded for longer term
analysis and comparison. In future sessions, the value of completing a ROMP
will be emphasized, to encourage researchers to reflect on the session content
and to record feedback to the librarian.
Conclusions
Polling
One Minute Papers (POMPs) and Reflective One Minute Papers (ROMPs) offer a new
approach for librarians to guide the learning process of the researchers they
support. This study demonstrates that POMPs and ROMPs have the potential to be
remarkably informative, despite requiring little effort on the part of
the researchers or librarians. POMPs and ROMPs are adaptations of
conventional formative assessment tools which extend the opportunity for
librarians to engage with researchers, both before and after a library research
support session. POMPs allow researchers to benchmark the status of their
learning needs and assist librarians to identify learning gaps. ROMPs encourage
researchers to reflect on what they learned in library research support
sessions and assist librarians to determine whether intended learning outcomes
were achieved. The simplicity of POMPs and ROMPs enable them to be slotted into
researchers’ workflow. As librarians take on research support duties, these
tools can be used to share recorded evidence of the evolving learning needs of
researchers. Responses to POMPs and ROMPs also document evidence of the value
that a library has contributed to supporting its researchers.
References
Association
of College and Research Libraries. (2010). The
value of academic libraries: a comprehensive research review and report. Researched
by Megan Oakleaf. Chicago: Association of College and Research Libraries.
Retrieved from http://www.acrl.ala.org/value/page_id=21
Auckland,
M. (2012). Re-skilling for research - An investigation into the role and skills of
subject and liaison librarians required to effectively support the evolving
information needs of researchers. Retrieved from http://www.rluk.ac.uk/files/RLUK%20Re-skilling.pdf
Australian
Research Council. (2014). Excellence in
Research for Australia (ERA). Retrieved from http://www.arc.gov.au/era/default.htm
Bartlett,
M. G., & Morrow, K. A. (2001). Method for assessing course knowledge in a
large classroom environment: an improved version of the minute paper. American Journal of Pharmaceutical
Education, 65(3), 264-267.
Beall,
J. (2014). Scholarly Open Access:
critical analysis of scholarly open-access publishing. Retrieved from http://scholarlyoa.com/
Boud,
D. (2000). Sustainable assessment: rethinking assessment for the learning
society. Studies in Continuing Education,
22(2), 151-167. http://dx.doi.org/10.1080/713695728
Boud,
D., & Lee, A. (2005). ‘Peer learning’ as pedagogic discourse for research
education. Studies in Higher Education,
30(5), 501-516. http://dx.doi.org/10.1080/03075070500249138
Chizmar,
J. F., & Ostrosky, A. L. (1998). The one-minute paper: Some empirical
findings. The Journal of Economic
Education, 29(1), 3-10. http://dx.doi.org/10.1080/00220489809596436
Choinski,
E., & Emmanuel, M. (2006). The one-minute paper and the one-hour class:
Outcomes assessment for one-shot library instruction. Reference Services Review, 34(1), 148-155. http://dx.doi.org/10.1108/00907320610648824
Creaser,
C., & Spezi, V. (2013). Improving perceptions of value to teaching and
research staff: The next challenge for academic libraries. Journal of librarianship and information science, 46(3), 191-206. http://dx.doi.org/10.1177/0961000613477678
Department
of Education. (2014). Higher Education
Research Data Collection (HERDC). Retrieved from http://education.gov.au/higher-education-research-data-collection
Dobson,
J. L. (2008). The use of formative online quizzes to enhance class preparation
and scores on summative exams. Advances
in Physiology Education, 32, 297-302. http://dx.doi.org/10.1152/advan.90162.2008
Draper,
S. W. (2009). Catalytic assessment: understanding how MCQs and EVS can foster
deep learning. British Journal of Educational
Technology, 40(2), 285-293. http://dx.doi.org/10.1111/j.1467-8535.2008.00920.x
Drew,
S., & Klopper, C. (2014). Evaluating faculty pedagogic practices to inform
strategic academic professional development: A case of cases. Higher education, 67(3), 349-367. http://dx.doi.org/10.1007/s10734-013-9657-1
Drummond,
G. (2007). Programme feedback: It only takes a minute! International Journal of Educational Management, 21(1), 29-36. http://dx.doi.org/10.1108/09513540710716803
Dunaway,
M. K., & Orblych, M. T. (2011). Formative assessment: transforming
information literacy instruction. Reference
Services Review, 39(1), 24-41. http://dx.doi.org/10.1108/00907321111108097
Frances,
M., Fletcher, J., & Harmer, S. (2011). Reshaping
and rescoping university libraries to fit changing academic requirements.
Paper presented at the 32nd IATUL Annual Conference: Libraries for An Open
Environment: strategies, technologies and partnerships, Warsaw, Poland.
Retrieved from http://www.bg.pw.edu.pl/iatul2011/proceedings/ft/Frances_M.pdf
Google.
(2014). Create a survey using Google
forms. Retrieved from http://support.google.com/drive/answer/87809?hl=en
Hoppenfeld,
J. (2012). Keeping students engaged with web-based polling in the library
instruction session. Library Hi Tech, 30(2),
235-252. http://dx.doi.org/10.1108/07378831211239933
Hughes,
H., & Bruce, C. (2012). Informed learning in online environments:
Supporting the higher education curriculum beyond Web 2.0. In P. P. Godwin
& J. Parker (Eds.), Information
literacy beyond library 2.0 (pp. 65-79). London: Facet.
James
Cook University. (2014). Facts &
figures 2014: Data snapshot. Retrieved from http://www.jcu.edu.au/about/public/groups/everyone/documents/annual_report/jcu_140345.pdf
James
Cook University. (2015). About JCU:
Strategic intent. Retrieved from http://www.jcu.edu.au/about/strategic-intent/
Kaufman,
P. T. (2009). Carpe Diem: Transforming services in academic libraries. In IDEALS. Retrieved from http://www.ideals.illinois.edu/handle/2142/12032
Kennan,
M. A., Cole, F., Willard, P., Wilson, C., & Marion, L. (2006). Changing
workplace demands: What job ads tell us. Aslib
proceedings: New information perspectives, 58(3), 179-196. http://dx.doi.org/10.1108/00012530610677228
Kennan,
M. A., Corrall, S., & Afzal, W. (2014). "Making space" in
practice and education: Research support services in academic libraries. Library Management, 35(8/9), 666-683. http://dx.doi.org/10.1108/LM-03-2014-0037
Merriam,
S. B. (2001). Andragogy and self-directed learning: Pillars of adult learning
theory. New directions for adult and
continuing education, 2001(89), 3-13. http://dx.doi.org/10.1002/ace.3
Oakleaf,
M. (2009). The information literacy instruction assessment cycle: A guide for
increasing student learning and improving librarian instructional skills. Journal of documentation, 65(4),
539-560. http://dx.doi.org/10.1108/00220410910970249
Oakleaf,
M. (2014). A roadmap for assessing student learning using the new framework for
information literacy for higher education. The
Journal of Academic Librarianship, 40(5), 510-514. http://dx.doi.org/10.1016/j.acalib.2014.08.001
Parker,
R. (2012). What the library did next:
Strengthening our visibility in research support. Paper presented at
eM-powering eFutures, the 16th Biennial VALA Conference and Exhibition,
Melbourne, Victoria, Australia.
Retrieved from http://hdl.handle.net/1959.3/214486
Parsons,
A. (2010). Academic liaison librarianship: Curatorial pedagogy or pedagogical
curation? Ariadne, 65, 1-11. Retrieved from
http://www.ariadne.ac.uk/issue65/parsons/
Priem,
J., Taraborelli, D., Groth, P., & Neylon, C. (2010). Altmetrics: A
manifesto. Retrieved from http://altmetrics.org/manifesto/
REF2014.
(2014). REF2014: Research Excellence
Framework. Retrieved from http://www.ref.ac.uk/
Richardson,
J., Nolan-Brown, T., Loria, P., & Bradbury, S. (2012). Library research support
in Queensland: a survey. Australian
Academic & Research Libraries, 43(4), 258-277. http://dx.doi.org/10.1080/00048623.2012.10722287
Sadler,
D. R. (1989). Formative assessment and the design of instructional systems. Instructional Science, 18(2), 119-144. http://dx.doi.org/10.1007/BF00117714
Stead,
D. R. (2005). A review of the one-minute paper. Active Learning in Higher Education, 6(2), 118-131. http://dx.doi.org/10.1177/1469787405054237
Appendix
|
Session # |
Discipline / Client Group |
Questions |
Response Options |
|
|
1 |
Life and Environmental Sciences: Post-Docs |
How would you rate your understanding of altmetrics? |
None; Low; Good |
|
|
1 |
Life and Environmental Sciences: Post-Docs |
Which of the following do you use to do your
research, or to talk about your research interests? |
Academia.edu; Blogging; CiteULike; Comments on other
sites; Facebook; Mendeley; Newspaper interviews; Radio interviews;
ResearchGate; Slideshare; Twitter; Wikipedia; YouTube; Other |
|
|
1 |
Life and Environmental Sciences: Post-Docs |
Which of the following sites do you use to manage
your research profile? |
JCU Research Portfolio; ResearchOnline@JCU;
ResearcherID; Scopus; ORCID; Google Scholar; Other |
|
|
2 |
Life and Environmental Sciences |
Is the information in your JCU Research Portfolio
profile up-to-date? |
Yes; No; Other |
|
|
2 |
Life and Environmental Sciences |
Which of the following do you use to do your
research, or to talk about your research interests? |
ResearchOnline@JCU; ResearcherID; Scopus; ORCID;
Google Scholar; Other |
|
|
2 |
Life and Environmental Sciences |
Which of the following sites do you use to manage
your research profile? |
Academia.edu; Blogging; CiteULike; Comments on other
sites; Facebook; Mendeley; Newspaper interviews; Radio interviews;
ResearchGate; Slideshare; Twitter; Wikipedia; YouTube; Other |
|
|
3, 4, 5, 6, 7, 8 |
Life and Environmental Sciences |
Have you used any of the following to promote or
discuss your research? |
ResearcherID; Scopus; Google Scholar; ORCID; The
Conversation; Traditional media - newspaper, radio, TV; Blogging; Comments on
the blogs or articles by others; Twitter, ResearchGate; YouTube; LinkedIn;
Mendeley; Other |
|
|
3, 4, 5, 6, 7, 8 |
Life and Environmental Sciences |
Have you heard of the term altmetrics? |
Yes; No; Not sure |
|
|
3, 4, 5, 6, 7, 8 |
Life and Environmental Sciences |
If you have heard of the term altmetrics, briefly
describe what you understand it to mean? |
[Free text] |
|
|
3, 4, 5, 6, 7, 8 |
Life and Environmental Sciences |
If you have paid an Article Processing Charge, do
you know if you retained copyright ownership of your article? |
Yes; No; Not sure |
|
|
3, 4, 5, 6, 7, 8 |
Life and Environmental Sciences |
Have you or a co-author paid an Article Processing
Charge to make any of your articles Open Access? |
Yes; No; Not sure |
|
|
9 |
Health Sciences |
Have you published any of the following types of
journal articles? |
Peer reviewed article; Non-refereed article; Case
study; Short note or commentary; Other |
|
|
9 |
Health Sciences |
Have you published any of the following types of
conference works? |
Peer reviewed paper; Non-refereed paper; Poster;
Abstract or summary; Edited a conference proceedings; Other |
|
|
9 |
Health Sciences |
Have you published any of the following types of
books or book chapters? |
Research; Non-research; Teaching material; Non-commercial;
Reference; Later edition; Report |
|
|
9 |
Health Sciences |
Tick the box if you are you familiar with the
following government reporting or assessment exercises: |
HERDC: Higher Education Research Data Collection;
ERA: Excellence in Research for Australia; Other |
|
|
9 |
Health Sciences |
Tick the box if you have used any of the following
for ranking the value of a journal: |
Journal Citation Reports; Scopus analytics; Beall's
list of predatory publishers; Other |
|
|
10 |
Health Sciences |
Have you used any of the following to promote or
discuss your research? |
ResearcherID; Scopus; Google Scholar; ORCID; The
Conversation; Traditional media - newspaper, radio, TV; Blogging; Comments on
the blogs or articles by others; Twitter, ResearchGate; YouTube; LinkedIn;
Mendeley; Other |
|
|
10 |
Health Sciences |
Have you heard of the term altmetrics? |
Yes; No; Not sure |
|
|
10 |
Health Sciences |
If you have heard of the term altmetrics, briefly
describe what you understand it to mean? |
[Free text] |
|
|
10 |
Health Sciences |
Have you or a co-author paid an Article Processing
Charge to make any of your articles Open Access? |
Yes; No; Not sure |
|
|
10 |
Health Sciences |
If you have paid an Article Processing Charge, do
you know if you retained copyright ownership of your article? |
Yes; No; Not sure |
|
|
11 |
Health Sciences |
Have you published any of the following types of
journal articles? |
Peer reviewed article; Non-refereed article; Case
study; Short note or commentary; Other |
|
|
|
|
|
|
|
|
11 |
Health Sciences |
Have you published any of the following types of
conference works? |
Peer reviewed paper; Non-refereed paper; Poster;
Abstract or summary; Edited a conference proceedings; Other |
|
|
11 |
Health Sciences |
Have you published any of the following types of
books or book chapters? |
Research; Non-research; Teaching material;
Non-commercial; Reference; Later edition; Report |
|
|
11 |
Health Sciences |
Would you consider that you have a fair
understanding of: |
HERDC: Higher Education Research Data Collection; ERA:
Excellence in Research for Australia; Other |
|
|
11 |
Health Sciences |
Are any of your publications Open Access: |
In ResearchOnline@JCU?; On the publisher's website?;
Other |
|
|
12 |
Social Sciences and Humanities |
Are you the author of a peer reviewed journal article? |
No; Yes |
|
|
12 |
Social Sciences and Humanities |
Are you the author of a peer reviewed conference
paper? |
No; Yes |
|
|
12 |
Social Sciences and Humanities |
Are you the author of a commercially published book
or book chapter about a research topic? |
No; Yes |
|
|
12 |
Social Sciences and Humanities |
Do you have publications available in
ResearchOnline@JCU? |
No; Yes |
|
|
12 |
Social Sciences and Humanities |
Do you have a Google Scholar profile that lists your
research outputs? |
No; Yes |
|
|
12 |
Social Sciences and Humanities |
If you have answered yes to Questions 1,2 or 3, have
you searched for your publications in Web of Science? |
No; Yes |
|
|
12 |
Social Sciences and Humanities |
If you have answered yes to Questions 1,2 or 3, have
you searched for your publications in Scopus? |
No; Yes |
|
|
12 |
Social Sciences and Humanities |
Is there any topic or question that you would like
me to talk about in the session? |
[Free text] |
|
|
13 |
Social Sciences and Humanities |
Have you used any of the following to promote or
discuss your research? |
ResearcherID; Scopus; Google Scholar; ORCID; The
Conversation; Traditional media - newspaper, radio, TV; Blogging; Comments on
the blogs or articles by others; Twitter, ResearchGate; YouTube; LinkedIn;
Mendeley; Other |
|
|
13 |
Social Sciences and Humanities |
Have you heard of the term altmetrics? |
Yes; No; Not sure |
|
|
13 |
Social Sciences and Humanities |
If you have heard of the term altmetrics, briefly
describe what you understand it to mean? |
[Free text] |
|
|
13 |
Social Sciences and Humanities |
Have you or a co-author paid an Article Processing
Charge to make any of your articles Open Access? |
Yes; No; Not sure |
|
|
13 |
Social Sciences and Humanities |
If you have paid an Article Processing Charge, do
you know if you retained copyright ownership of your article? |
Yes; No; Not sure |
|
|
14 |
Social Sciences and Humanities |
How many HERDC eligible publications do you have?
This includes peer reviewed publications or commercially published books or
book chapters. |
[give number] |
|
|
14 |
Social Sciences and Humanities |
Have you heard of the term altmetrics, and how would
you rate your understanding of altmetrics? |
I have not heard of the term; I have some
understanding; I have a good understanding |
|
|
14 |
Social Sciences and Humanities |
Which of the following do you use to manage your
research profile? |
JCU Research Portfolio; ResearchOnline@JCU;
ResearcherID; Scopus; ORCID; Google Scholar; Other |
|
|
14 |
Social Sciences and Humanities |
Which of the following do you use to do your
research, or to talk about your research? |
Academia.edu; Blogs; Comments on any sites;
Facebook; Mendeley; Newspaper interviews; Radio interviews; ResearchGate;
Slideshare; Twitter; Wikipedia; YouTube; Other |
|
|
15 |
Mixed disciplines: Academic teaching staff |
Have you published any of the following outputs
about your research? |
Peer reviewed journal article; Non-peer reviewed
journal article; Non-peer reviewed conference paper; Book or book chapter;
Report; Other |
|
|
15 |
Mixed disciplines: Academic teaching staff |
Have you published any of the following outputs
about your teaching? |
Peer reviewed journal article; Non-peer reviewed
journal article; Non-peer reviewed conference paper; Book or book chapter;
Report; Other |
|
|
15 |
Mixed disciplines: Academic teaching staff |
Do you use any of the following social media tools
to talk about or promote your teaching and learning? |
Blogs; Comments on other people's posts; Twitter;
Facebook; Academic.edu or ResearchGate; Slideshare; Wikipedia; YouTube;
(Your) JCU Research Portfolio; (Your) Google Scholar profile; Other |
|
|
16 |
Mixed disciplines: Higher Degree Research students |
What is the discipline or subject area of your
research? |
|
|
|
16 |
Mixed disciplines: Higher Degree Research students |
Are you a: |
Research Masters student; PhD student; Early Career
Researcher i.e. are you within 5 years of starting your post-PhD research
career?; Other |
|
|
16 |
Mixed disciplines: Higher Degree Research students |
Are you using (or planning to use) material in your
thesis for which the copyright may have expired? Select Other if you would
like to provide more information. |
Yes; No; Not sure; Other |
|
|
16 |
Mixed disciplines: Higher Degree Research students |
Are you using (or planning to use) material in your
thesis which is still under copyright? Select Other if you would like to
provide more information. |
Yes; No; Not sure; Other |
|
|
16 |
Mixed disciplines: Higher Degree Research students |
Have you ever requested copyright permission to use
someone else's work in your research? |
Yes; No; Not sure; Other |
|
|
16 |
Mixed disciplines: Higher Degree Research students |
Are you planning to publish your research in: |
Journal article(s); Conference paper(s); book or
book chapter(s); Other |
|
|
16 |
Mixed disciplines: Higher Degree Research students |
Have you heard of Creative Commons? |
Yes; No; Not sure |
|
|
16 |
Mixed disciplines: Higher Degree Research students |
Do you have a copyright question that you would like
answered in the workshop? Please provide more details here. |
[Free text] |
|