Testing Hypothesis

1. Introduction
Educational technology plays an important role in developing and
supporting instructional activities that promote learning and facilitate
students’ engagement in a learning process. Social annotation (SA) tools are an emerging educational technology that has been
drawing more of educators’ attention over the last several years. SA
technology is an online social bookmarking tool that allows annotating
(adding comments, highlights, sticky notes, etc.) of an electronic
resource and supports easy online information sharing. SA technology
offers knowledge sharing solutions and a social platform for interactions
and discussions.
SA tools provide a means for users to modify or attach any type of
content to any electronic resource (usually a text) where users can interact.
Social annotation tools enable users to collaboratively underline,
highlight, and annotate an electronic text, in addition to
providing a mechanism to write additional comments on the margins
of the electronic document (Huang, Huang, & Hsieh, 2008; Kawase,
Herder, & Nejdl, 2009). The embedded annotations such as tags, comments,
and highlights can be shared with other users at a time later
on. A SA technology is a multi-purpose system that facilitates adding,
editing, and modifying information in an electronic resource without
changing the resource itself. It also allows users to share their annotations
and web resources with friends or group/project members and
discuss the content of a certain resource, which fosters new level of
knowledge through aggregating information from many users. The
annotations can be thought of as an addition layer of information on
top of the existing resource that can be shared with other users
using the same SA system.
SA tools are sometimes confused with Text Annotation tools such
as MS Word, Adobe PDF, Google Docs. Similar to SA tools, it is possible
to make comments and highlight electronic text in MS Word or
Adobe PDF, for example. However, Text Annotation tools are not considered
SA technology, since they do not provide an online social platform
for information sharing. A user has to share manually an MS
Word or Adobe PDF annotated document via email or other communication
methods. Other tools provide an online social platform for information
sharing but lack some of the SA technology characteristics.
For example, Google Docs provides an online social support platform
but does not allow annotating of newly uploaded electronic materials/
documents created via other tools.
In order to facilitate discussion of educational uses of the SA technology,
the following SA features have been identified. First, SA technology
allows users to make written annotations such as notes or
comments. Annotating is implemented by creating and attaching
comments to specific sentences or areas within an electronic resource.
Second, within a SA system a user can highlight any part of
electronic text using either different colors or various types of notations
and marks including underlining, circling, or boxing selected
passages within the text. Third, SA technology provides an online
platform for social collaboration. It enables creating private groups
where students can collect electronic resources, annotate and highlight
these resources, and share all this information with others.
Moreover, students can collaboratively discuss electronic content on
the margins of a digital paper and tie it back to a specific text section
(s). This information sharing could be done either synchronously (i.e.,
instant sharing of information with others who are present/available
online) or asynchronously (i.e., sharing of information while others
are not available online). When information sharing is done asynchronously,
the SA technology notifies about the changes made. Some SA
tools enable public notes sharing and text formatting.However, the majority
of the SA tools do not support the text-formatting feature yet.
The first three SA features (i.e., annotations, highlights, and collaboration/
information sharing online platform) described above, define
the concept of SA educational technology, and are common among
all SA tools. What distinguishes one SA tool from another is its design
and technical features. Glover, Xu, and Hardaker (2007) provide
a list of literature-supported requirements and desirable
conceptual and technical factors for SA educational systems (see
Table 1). These requirements have been drawn based on the examination
of the literature related to web annotation and assessment of publicly available systems. According to Glover et al. (2007), the list is
neither exhaustive nor absolute since differences exist among various
SA tools. Nevertheless, the listed requirements provide a generic overview
of the considerations that might be encountered when selecting
a SA tool for education purposes.
Many annotation online tools have been developed over the last decade,
including Gibeo (Bateman, Farzan, Brusilovsky, & McCalla, 2006),
Annotea (Wu-Yuin, Chin-Yu, & Mike, 2007), EDUCOSM (Nokelainen,
Kurhila, Miettinen, Floreen, & Tirri, 2003), Diigo (Kawase et al.,
2009), HyLighter (Lebow & Lick, 2004), The Fluid Annotations projects
(Zellweger,Mangen, & Newman, 2002), andMADCOW(Bottoni
et al., 2004, 2006).
The growing number of social annotation technologies can be
explained by the need for using such tools in various settings and
the benefits this technology offers. Collaborative annotating allows
collecting and organizing of resources, and accessing and sharing
them with others easily from anywhere. The collaborative aspect of
these tools creates a sense of community among the users in a
given system and allows them to become more involved in the community
(Bateman et al., 2006). The SA tools are particularly useful for
teams working on projects.
SA tools have been successfully used in educational settings. Researchers
claim that annotation technologies increase participation
and engagement (Lebow & Lick, 2004; LeeTiernan & Grudin, 2001, as
cited in Wu-Yuin et al., 2007), improve instruction (Lebow & Lick,
2004), promote attention, communication, and organization (Yang et
al., 2004, as cited in Huang, et al., 2008; Davis & Huttenlocher, 1995,
as cited in Wu-Yuin et al., 2007). Annotations can improve reading
comprehension and peer-critique skills (Archibald, 2010; Johnson,
Archibald, & Tenenbaum, 2010; Mendenhall & Author, 2010). In addition
to benefiting the annotator, other readers may also benefit
from reviewing shared annotated documents (Kawase et al., 2009).
Readers may benefit by gaining ideas, seeing others’ different perspectives,
and building knowledge about the annotated resource.
However, most of the mentioned benefits have not been supported
by a rigorous empirical research. There are many challenges inherent
to the use of SA technologies in educational settings (Johnson
et al., 2010). It is unclear how to use these tools to improve academic
achievement. Do students really benefit from using SA tools? No
comprehensive literature review investigating the SA use in educational
settings and SA effect on academic achievement has been
The purpose of the paper is to review existing literature related to
the (1) uses of SA in educational settings and (2) effects of SA technology
on learning. A literature review of empirical studies concerning
the use and the effect of social annotation tools in higher educational
settings is presented in this paper. The questions underlying this review are: (a) Does SA technology lead to any learning gains in higher education,
and under what conditions does SA promote learning? (b) What
are the general learning benefits and learning outcomes linked to SA?
(c) How has SA technology been embedded into instructional activities
andwhat types of learning have been facilitated themost through using
SA technology?
2. Method
2.1. Procedure
A large body of literature focusing on online social annotation
tools was gathered and reviewed. The following online databases
were employed for the literature search: ERIC, PsychInfo, and ScienceDirect.
In addition, Google Scholar was used to search for and acquire
specific references.
2.2. Inclusion criteria
The focus of the search was to gather full-text articles presenting
empirical studies employing social annotation tools in educational settings.
In order tomake the number of the reviewed articlesmanageable
but comprehensive, exhaustive review with selected citation approach
(Cooper, 1988)was implemented. Specifically, the studies were included
in the literature review if they met each of the following criteria:
1. The study employed an online SA tool that embodied at least the
following three features: (a) annotations, (b) highlights, and
(c) information sharing/collaboration online platform.
2. The study examined either (a) usability of SA technology in educational
settings or (b) effects of SA technology on general learning
benefits or learning outcomes.
3. The study focused on higher education audience. Studies that
employed K-12 population were excluded.
4. The study reported research methods (i.e., participants, instructional
interventions, instruments, research design, etc.) and results.
Studies that employed quantitative research method
reported statistical analysis methods and results.
5. The research presented in one study did not overlap research from
another study.
6. The article was written in English.
The search was conducted using various search terms and keywords
such as online collaboration tools, annotation tools, online team
learning tools, learning outcomes, and achievement. Since the SA tools
are an emerging technology that has not yet been adopted widely,
the search was not limited to a particular date range, research
methods, specific learning outcomes, or publication type. From over
90 initially collected articles, eight experimental and quasi-experimental
studies and eight evaluation/survey studies met the inclusion
criteria for this literature review.