Preview before Submission[NSF Project Report - Version 1.2] ANNUAL REPORT FOR 
AWARD # 0126265 
      Jennifer Wiley ; U of Illinois Chicago 
      ROLE Proposal: Understanding in Science: Eyetracking Studies 


Participant Individuals: 
CoPrincipal Investigator(s) : Arthur C Graesser; Susan R Goldman
Graduate student(s) : Ivan K Ash; Christopher A Sanchez; Greg Colflesh; Joshua A 
Hemmerich
Other -- specify(s) : Jason A Braasch
Undergraduate student(s) : Kamila M Brodowinska; Melissa Bland; Reality M Canty
Other -- specify(s) : Cara M Jolly
Graduate student(s) : Shannon Whitten
Undergraduate student(s) : Elisa Cooper; Michelle Edwards; Bryan Haynes; Sarah 
Petschonek; Nycole Treier; Wende Wagner; James Wallace
Post-doc(s) : Kris Moreno
Graduate student(s) : Heather Mitchell; Tyrun Rokadiya

Partner Organizations:

Activities and findings:
Research and Education Activities: 
ROLE Understanding in Science (Wiley, Goldman, Graesser)
Year 1 Research Activity Report

The overall goals of this project are to understand how and when
students adopt a critical stance toward scientific information, and
how such a stance can be supported.  To this end, we proposed two
strands of research that focus on how young adult learners interact
with science content obtained from the internet.  Our goal in the
first strand is to examine when and how readers spontaneously adopt a
critical stance toward scientific information, and in particular,
whether different purposes or goals for reading can impact the
adoption of a critical stance.  In the second strand of research, we
will investigate how the use of an intelligent conversational agent
assists learners in approaching scientific content.

The goals for the first year of work included the development of the
scientific content on internet sites, learning assessments, and
standardized experimental procedures, which involved several pilot
studies.  Upon completion of the pilot phase, data collection was
begun for the first strand of studies as proposed. Preparatory
projects, pilot studies, and progress on proposed studies are detailed
below.

Initial Web Sites/Learning Assessment Development Project
Developing the web sites for this project required extensive internet
and literature searches, as well as the re-analysis of some previously
collected data from UIC undergraduates. Originally 10 sites were
selected for the first pilot studies.  These sites were selected from
the output of a Google search using the key words “causes
volcanic eruptions” with the constraint in mind that the set of
sites needed to reflect a range of reliable information about
volcanoes.  Based on a literature review and analysis of data
previously collected on UIC undergraduates’ understanding of
volcanoes, we identified various categories of conceptual models and
misconceptions related to volcanic activity.  A correct causal model
was also outlined. Web sites were selected so that at least one
included a reference to each of the important steps in a correct
causal model of volcanic activity. Several web sites that made
reference to inaccurate “causes” were also included so
that the sources were varied in their accuracy as well as their
reliability.  The reliable sites contained accurate information and
provided converging evidence for each other (i.e. the USGS site, the
NASA site and the PBS sites all provided information that could be
integrated into the same coherent causal model of volcanic activity).
 The unreliable, inaccurate sites all provided unique causal
information that could not be integrated into the model suggested by
the reliable sites.  

True/False inference questions were developed for a pre-test and
post-test based on the correct causal model as well as common
misconceptions.  The reliable sources that were used in piloting were
from USGS, NASA, Scientific American, 2 PBS related sites and 1 from
CPB/Annenberg. The unreliable sites were an Astrology site, an
inventor’s site, and a site written by a corporate forecaster. 
The final site was VolcanoLive.com, which despite its .com URL,
contains reliable information cited from a USGS site.  We class this
site as ambiguous.  

Reliability Pilot Study
This pilot study had a few purposes: 1) to get ratings of reliability
and general impressions from students on the 10 pilot sources, in
order to prune down to 7 sources (to fit on a single Google output
page), 2) to check that students recognize the range of reliability,
and 3) to see what factors students report as important for
reliability.

26 students were told that their task was to figure out what caused
the eruption of Mt. St. Helens from the output of a GOOGLE search
using the keywords 'causes volcanic eruptions.' They were given the
first two pages of those results, with 10 links. The students were
told they needed to decide which sources were reliable, that is, which
sources contained information that you could trust. Students were
asked to take each link and look through these sites and think about
how much they trusted the information.  Four students were run in an
individual condition and were asked to say aloud what they thought of
each site.

After the readers went through all 10 sites, they were asked to rank
the 10 sites in terms of how much they trusted the information
provided. After the rankings, subjects were asked: Thinking about all
of your rankings -- how did you decide on the reliability of each
site? At the end, all subjects filled out closed-ended survey ratings
on the importance of factors that they used to determine reliability.
 They were asked whether they read/skimmed the pages and if they read
the URL.

Students rated the 6 sources that we considered “reliable”
as more reliable than the 3 “unreliable” sources,
t(23)=7.69, p<.001.  The ambiguous volcanolive.com site fell between
reliable and unreliable sources, but appeared to be perceived as
primarily unreliable.  Students rated the following factors as being
most important for reliability: evidence supported theory (8.25),
provided evidence for theory (7.62), similarity to other things I have
learned (7.00), similar content across multiple sites (6.92), quality
or reputation of publishing outlet (6.91), but also looked interesting
(6.75); they rated speed of loading (2.96), order in search results
(3.29), and color scheme (3.63) as least important for their judgments
of reliability. While these results are encouraging, and suggest that
students have a good understanding of reliability, the think-aloud
protocols told a different story.

Protocol analysis revealed that students had a great deal of
variability in what they thought it meant to evaluate the reliability
of a site.  Each subject tended to engage in the task at hand
differently. All seemed to have a different idea of what did or did
not make a site reliable. The comments that were made regarding the
reliability of each site were primarily focused on four factors: 1) a
website’s extension (i.e. .edu, .gov, .org, .com), 2) the
credibility of the source of the website (i.e. NASA), 3) interest
in/usefulness of the content of the information on the website, and 4)
the layout/design of the website.  Further, thinking about reliability
was clearly not a well-defined task for these subjects, as all ended
up talking about their interest in sites or their visual appeal

Preliminary Conclusions from Reliability Pilot Study:
Students in general do not seem to have a common or stable
understanding of what makes some sources more “reliable”
than others. Although they seemed to report the “right”
reasons on the rating task, there was a large discrepancy between
explicit ratings of their basis for reliability judgments and what
they talked about during the think-aloud phase. There was also
variability across students.

In practice, the interestingness or usefulness of the site seemed to
be the basis for most evaluative activity. A tricky issue will be to
figure out whether prior knowledge of content is critical for
understanding reliability.  In this small sample, the students who
came closest to showing an appreciation for reliability also had some
prior knowledge about volcanoes.

This suggests that one potential area that we may address with
intelligent agents is helping students develop a more
scientifically-based understanding of reliability, and supporting the
use of reliability judgments as a basis for deciding which sites to
attend to.

Implications for Materials Development
The results of the ratings did show that students recognized that
these sources represented a range of reliability.  In general, even
though they could not say why, they did rate the 3
“unreliable” sources as less reliable than the reliable
sources.  These rating data, as well as other student comments on the
usefulness and appeal of the sites, were used to select 3
“reliable” sources (NASA, PBS – Savage Earth, and
Scientific American) and 3 “unreliable” sources for
inclusion in Strand 1.  The volcanolive.com site was retained
precisely because of its ambiguity.  How students deal with a .com
site that includes information from a reliable source will tell us
much about the criteria they use for evaluation of sites.

Think Aloud Pilot Study
This pilot study had a three purposes: 1) to develop a standardized
set of prompts for the think-aloud condition, 2) to determine how much
time would be needed for the tasks and 3) to pilot test the three
possible writing conditions (Argument, Descriptive Report, and
Evaluation) to ensure they prompted the kinds of behaviors we wanted
to investigate.

Nineteen students were told that their task was to figure out what
caused the eruption of Mt. St. Helens from the output of a Google
search using the keywords 'causes volcanic eruptions.' They were given
one Google results page with links to seven sites, and an hour to do
their research while thinking-aloud.  Seven students were told they
were reading in order to write a “description of what you have
found out about what caused the Mt. St. Helens eruption”, 9 were
told they were reading to write an argument about what caused the
eruption, and 3 were told they would have to evaluate an account of
what caused the eruption of Mt. St. Helens.  After the writing task,
students were asked to complete a post-test, and rate the reliability
of all seven sites.  Reading times on each page were collected
automatically and stored on a server.

After the first three participants were run in the Evaluation
condition, we decided to discontinue this condition.  Although we had
hoped that this condition would prompt students to generate their own
account of volcanic eruptions, and evaluate another account based on
their own understanding, these essays instead evaluated mainly
surface-level features like grammar and word choice.  Their learning
outcomes confirmed that students were getting little out of this
condition (an average of 50% correct on post-test).

Pilot data were analyzed for the amount of time spent on each site,
their reliability ratings, and the relationship between these measures
and learning.  As in the reliability pilot study, students ranked the
reliable sites as more reliable than unreliable sites, t(18)=8.35,
p<.01. Students also spent more time reading the reliable sites,
t(18)=2.69, p<.02. Thus, there was a relationship between the
reliability rankings and how long readers spent on sites; that is, the
more reliable the students perceived the site to be, the longer
students read it.  Most importantly, the larger the proportion of
reading time that was spent on relevant sources, the better students
did on the post-test (r=.73, p<.01).  This suggests that an
understanding of reliability, and the selection of reliable
information, correlates with learning science from web sites.

The think-aloud protocols of the 16 subjects in the description and
argument conditions were analyzed to determine their approaches to
navigating the sites. There were no differences in approaches related
to task condition: In both conditions, 31% of the students (5 of 16)
used the Click and Go approach, 50% used the Read Some and Select; and
19% used the Read Most and Select approach. There was also no
difference between the argument and description groups with respect to
which sites they accessed, although subjects in the descriptive group
had a higher mean number of sites accessed per person (9.7 sites) than
the argument group (6.7 sites). (For this analysis, going back to a
site previously visited counted as another access, hence a mean
greater than 7 is possible.)  However, students in the argument
condition tended to spend a larger proportion of their reading time on
reliable sources. On the learning posttest, students in the argument
condition scored a mean of two items higher than those in the
descriptive condition (17.38 compared to 15.43 items), although the
difference was not significant. However, despite no differences
between description and argument tasks, across both groups there were
clear relationships between reading, reliability judgments and
learning.
 
Preliminary Conclusions from Think Aloud Pilot Study:
The think-aloud pilot studies indicated that most students would
prefer longer than a half hour for the research task, while an hour
seems like a natural length of time for most participants.  Early
results with the evaluation task suggested that it was not yielding
research behaviors similar to the argument and description writing
tasks, so it was eliminated from the proposed design of Strand 1.  The
reading time, ranking, and learning results replicate and extend what
we found in the first pilot study. Students ranked the
“reliable” sources as more reliable than the unreliable
sources.   They also tended to spend more time reading reliable
sources, and that related to better learning.  These are exactly the
patterns we were hoping to be able to investigate further in Strand 1,
using think-aloud and eyetracking methodology to assess when and how
students adopt a critical stance toward scientific information they
encounter on websites.

Progress on Strand 1 Proposed Studies
These experiments were begun in the Fall 2002 semester, after the
materials and procedure for the study were finalized. Using the web
sites, materials and procedures developed through piloting, we began
our three-part approach to investigating when and how students develop
a critical stance toward scientific information.  Three parallel
studies have been begun using the same exact materials and design. 
Study 1 measures reading behavior as students perform their research
with a Dual-Purkinje eyetracker, Study 2 asks students to think-aloud
as they perform their research, and Study 3 combines both
methodologies using a head-mounted eyetracker as well as think-aloud
instructions.  All participants are told that their task is to figure
out what caused the eruption of Mt. St. Helens from the output of a
GOOGLE search using the keywords 'causes volcanic eruptions.' They are
given the first page of those results, with 7 links.  Of the 7
sources, 3 are meant to be reliable, 3 are intended as unreliable and
1 is reliable but ambiguous.  Students are told that their purpose for
reading is to either write a Descriptive Report or an Argument of what
caused the eruption of Mt. St. Helens. The order of the sites is
counterbalanced across two versions of the Google page, yielding a 2x2
design for these studies.  They are given about an hour for their
research, and about a half hour to write their essays. After essay
writing, all students complete learning posttests, ranking/rating
reliability surveys for each web site, evaluate a peer essay with an
inaccurate causal model, and complete demographic and computer
use/familiarity surveys. All students are given a pretest as part of
mass testing.

Mass Testing Results
All students in both the Memphis and UIC Psychology Department Subject
Pools completed the pre-test as part of mass testing.  The results of
this testing were used to screen participants (so that only students
who did not already have a mature concept of volcanic activity were
run in the studies). In addition, the mass testing data provided
important normative information about undergraduates’
understanding of volcanoes.  Mean score on the test at UIC was a 19/30
and scores were fairly normally distributed.  On average, students did
not demonstrate mastery of earth science concepts related to volcanic
activity on the pretest.  Item analyses confirm that few
undergraduates in our population possessed a good understanding of the
causes of volcanoes.  The most pervasive misconceptions seem to be
that volcanic eruptions are related to the earth’s spin and core
temperature.  There is also a fundamental misconception that the
continental plates are floating on top of the ocean plates like
islands.

In terms of an understanding of the correct model, there seems to be
some appreciation that the surface of the earth is composed of
multiple plates, that convection in the mantle plays a role in moving
the plates, and that gas and pressure are important for eruptions.
However, students are not at all clear about what kinds of plate
interactions lead to volcanic eruptions, nor do they understand the
crustal cycle (how crust is created and destroyed). This analysis
confirms that volcanic eruptions are a scientific topic for which many
undergraduates possess misconceptions and an incomplete understanding,
especially of the dynamics of the Earth’s crust, thus making
this an appropriate content area for our investigation. For Strand 1,
only students who scored below average on the pretest were be used.

Study 1: Tracking the research process with a Dual Purkinje
Eyetracker.
The purpose of this study is to get highly accurate reading
time/navigation data from readers as they attempt to figure out
“what caused the eruption of Mt. St. Helens” from web
pages, in order to correlate reading behavior with conceptual
understanding and the adoption of a critical stance. Eye movements
were monitored with a Dual Purkinje ET as the students read. They were
able to navigate on their own through the sites using a mouse.

As of December 2002, 19 usable subjects had been run in this study (10
in Descriptive Report and 9 in Argument).  Preliminary results suggest
that the amount of reading time that students chose to spend on
reliable sources predicts their reliability ratings of the sources,
and also whether they demonstrate an advanced conceptual understanding
of volcanoes in their essays.
			
Results that are particularly encouraging are the correlations between
reading behavior, reliability rankings and essay measures.  These
preliminary results suggest that the larger the proportion of reading
time that is spent on reliable sources, and the more discriminating
students are in their rankings of reliable and unreliable sources,
then the more correct concepts and the fewer misconceptions are
included in their essays. We are proceeding next with completing the
design, and performing eyetracking analyses to build a better
understanding of exactly what information the learners are attending
to, and how they navigate through the information in the sites.

Study 2: Tracking the research process with think-aloud protocols.
The purpose of this study is to get on-line comments from students as
they attempt to figure out “what caused the eruption of Mt. St.
Helens” from web pages, in order to correlate these comments
with navigating behavior, conceptual understanding, and the adoption
of a critical stance. The prompts used for this study were
deliberately chosen to be of a neutral, non-directive nature along the
lines of “Can you tell me what you are thinking?” 
Think-aloud sessions are videotaped for later transcription and
analysis.

As of December 2002, 27 usable subjects had been run in this study (13
in Description and 14 in Argument).  The protocols are currently being
transcribed and analyses on the other dependent measures are
underway.

Study 3: Tracking the research process with both eyetracking and
think-aloud protocols.
The purpose of this study is to collect think aloud protocols and eye
tracking data while college students read the web sites for the
purpose of writing an argument essay versus a descriptive essay.  This
study followed the same procedure as the two studies conducted at the
UIC. The eye tracker at UM is an Applied Science Laboratory Model 501
eye tracker that has a head-mounted device for collecting the eye
tracking data; it allows the participants to speak at the same time
the eye tracking behavior is recorded. We also videotaped the subject
interacting with the web page, picking up both auditory and visual
channels.  

As of December 2002, a total of 34 college students from the
psychology subject pool participated in the experiment, with half
being randomly assigned to the Argument essay condition and half to
the Description essay condition.  There was some attrition in useable
data, however, which routinely occurs in eye tracking and think-aloud
experiments.  We had to excuse 6 subjects because we had trouble
calibrating their eyes.  Another 3 subjects had unusable data because
the audio signal was not of sufficient volume and quality to
transcribe the think aloud protocols.   Therefore, only 25 college
students yielded usable data: 13 were in the Argument condition and 12
were in the Description condition.   

We finished collecting the data in late November, so we have only
begun mining the eye tracking data and think aloud protocols.  There
are a few highlights of the data to report.  College students visited
approximately the same number of web pages (included repeated visits)
in the Argument and Description conditions, with means of 38.2 and
41.5 pages, respectively.   The Argument essay had fewer words than
the Description essay, 192 vs. 269 words, and fewer sentences, 9.2
versus 17.0; these trends are not statistically significant but we are
convinced that differences will emerge with a larger sample of
subjects.   We are in the process of scoring the learning posttests,
the conceptual content of the essays, auxiliary tests and
questionnaires, and the eye tracking data.  The verbal protocols are
also in the process of being transcribed.       
 
Supplemental Studies
In addition to our proposed studies and the pilot studies that were
performed to refine the materials and procedures, several additional
studies have been performed that will inform our future work on this
grant.  The first study (Evaluation Pilot) concerns a new 
“Evaluation” condition to try to prompt students into a
critical stance. Two additional think-aloud studies have been run, one
on non-native English speakers (Non-native Think aloud), and one on
high knowledge students (High knowledge Think Aloud). The fourth study
(Agent Interface Pilot) is relevant to developing an effective tutor
interface in the later phases of our research.

Evaluation Pilot
The purpose of this study was to test if expecting to evaluate another
student’s essay will have an effect on learning. Students were
run under the same conditions as the Strand 1 studies, expecting to
write either an argument or a description. However in addition to the
writing instruction, half the participants in each condition were
warned that they would need to evaluate another student’s essay
based on their research.  No think-aloud or eyetracking data were
collected.  24 undergraduates at UIC participated in this study (6 in
each condition). Preliminary analyses on learning gains and
reliability rankings suggest that the evaluation warning seems to make
everyone in the Argument condition perform uniformly better than
students in the Descriptive condition (i.e. the warning tightens up
the variance in the Argument condition).  An explicit warning that
students may need to evaluate a peer essay AFTER they write an
argument may help students be more critical/selective as they read,
and may be an instruction to consider as part of the intervention in
Strand 2.  Based on preliminary results, an explicit prompt to
evaluate another essay appears to be more effective than simply the
instruction to write an evaluation (as in the Think Aloud pilot
study).

Non-native English Speaker Think-Aloud Study
16 non-native English speakers were run through the same condition as
the Strand 1 think-aloud study to investigate how these students learn
science from web-based sources. These protocols are currently being
transcribed and the other dependent measures analyzed.

High Knowledge Think-Aloud Study
The purpose of this study was to ask high knowledge students to
identify and describe the strategies, advice, and guidance they would
give to less knowledgeable students; in order to generate potential
ideas for dialogue moves that could be made by the conversational
agent in Strand 2.  Twelve students who were identified as
high-knowledge by their pretest scores were given the following
tasks:
a) Think aloud about the sites regarding what strategies, advice, or
guidance you would give a less knowledgeable person about the way to
research this topic.
b) Evaluate a student essay (with an incorrect causal model of
volcanic activity)
c) Rate the reliability of each site.
These protocols are currently being transcribed and analyzed.

Agent Interface Pilot Study
Years 2 and 3 of the grant will be developing a conversational agent
that assists the student in taking a critical stance while reading the
web pages.  The question arises as to the type of agent that we should
use.  Animated conversational agents have facial features synchronized
with speech and in some cases appropriate gestures (Graesser et al.,
2001; Johnson, Rickel, & Lester, 2000).  The gestures point to
important material on display and emphasize ideas.  In essence, the
agents provide an anthropomorphic human-computer interface that
simulates having a conversation with a human.  One worry about having
an agent, however, is that the agent will be a magnet of attention and
distract the student from viewing the material on the web page.  

We conducted a study that collected eye tracking data while 12 college
students interacted with AutoTutor in a tutorial dialogue on computer
literacy.  Eye tracking data were collected during a 20-minute time
span while the participants first interacted with AutoTutor.  We
explored how the learners allocated their attention to four windows in
the display during the tutoring session:   (1) The animated
conversational agent (which took up 28% of the computer screen), (2)
the main deep-reasoning question (e.g., why, how, what-if, 13% of the
screen), (3) a graphic display of the computer system components that
are relevant to the question (46% of the display), and (4) the
student’s input (a window that shows what the student types in,
13% of the display).   The animated agent could conceivably affect eye
tracking in a number of different ways.   The agent could conceivably
be an “attention magnet” that draws the learner’s
attention more than would be expected by chance, if eye movements were
randomly allocated to regions on the display.  Another possibility is
that that there is a “novelty effect.”  That is, the agent
might start out being a magnet of attention, but attention on it
decreases over time.   In order to assess this possibility, we
measured the fixation times on the four information sources as a
function of four 5-minute time intervals in the 20-minute tutoring
session.  We computed a metric called an adjusted fixation ratio
(AFR), which is 1 if the eyes wander randomly within the computer
screen, greater than 1 to the extent that an agent is an attention
magnet, and less than 1 to the extent the agent is ignored.  

We measured the amount of time the eyes fixated on the four windows of
the display as a function of the four 5-minute time intervals.  There
were statistically significant differences in the percentage of time
allocated to the four information sources, with means of 54, 5, 32,
and 9%, for the agent, main question, graphic display, and student
input regions, respectively, F(3,33) = 31.11, p < .01, but there was
no main effect or interaction with time interval.  The AFR scores were
1.81, .43, .78, and .68 for the corresponding four information
sources.  These results robustly support the attention magnet
hypothesis but not the novelty effect hypothesis.  The animated agent
is clearly a strong magnet of attention, and this effect does not
decrease over time.  This result raises important questions about the
sort of tutor we should have in assisting the student in to take a
critical stance.  Perhaps voice alone would be better than a talking
head.  Alternatively, an animated conversational agent may occur at
the beginning of the session while instructions are given, whereas
voice alone will be used when college students interact with the
websites.


Findings:
An analysis of volcano knowledge tests gathered on a large sample of
undergraduates confirms that volcanic eruptions are a scientific 
topic about which many undergraduates possess misconceptions and an
incomplete understanding. Thus, volcanic eruptions and their causes
are an important and appropriate content area for an investigation 
of contexts that will improve scientific understanding.

Although we are still in the early stages of our research program,
there are some preliminary results that are particularly encouraging.
In pilot studies and preliminary analyses of Strand 1 experiments,
correlations were found among reading behavior, information
reliability rankings, and learning measures. This suggests that an
understanding of reliability, and the strategic selection of reliable
information during reading, predicts learning from scientific web
sites. These are exactly the patterns we were hoping to be able to
investigate further in Strand 1, using think-aloud and eyetracking
methodology to assess when and how students adopt a critical stance
toward scientific information.



Training and Development:
The most notable training opportunity on this project is that it
involves two highly skill-intensive methodologies: eyetracking and
think-aloud interviews.  In order to start collecting data on this
project, several graduate students and a dozen undergraduates have
been trained in eyetracking and/or think-aloud methodology and
analyses.  Both methods require attention to detail so that
calibration and prompting are standardized across subjects and across
sites.  Hence, rigorous training in these procedures is imperative.
The combined think-aloud/eyetracking conditions conducted at the
Memphis site presents a major opportunity for learning how two
coordinate these two methodologies.  The project staff at both sites
have needed to take a proactive approach to figuring out
how these methodologies can be best combined.

For undergraduates and new research assistants, additional training
was given in the practical aspects of running experimental sessions. 
 This includes the rudiments of experimental design (importance of
counterbalancing, random assignment, coding blind,etc.), data entry,
data coding, and file manipulation, as
well as in getting basic distributions and statistical analyses in
SPSS.  All project staff were also required to attend an IRB training
for new investigators.  Those with computer science skills needed to
learn how to write programs for collecting eye tracking data and
integrating such data collection with web facilities.   

Graduate students are being trained in these skills themselves, but
are also responsible for playing a role in the training of
undergraduates.  The development of the web sources at the start of
this project required the training of several graduate and
undergraduate students in web page design and formatting; while the
development of the learning assessment required extensive research
into prior studies on earth science instruction, some studies
examining learning from scientific text, and identification of the
misconceptions that students hold about volcanoes.  Finally, the need
for data reduction from navigation and eyetracking logs has required
several students to acquire programming skills, as they develop
programs to streamline the re-formatting of data so that it can be
analyzed more immediately.

In the next phase of the research, students will also receive 
training in protocol and essay coding, using rubrics developed by the
PIs from a subset of the participants.



Outreach Activities:
The eye tracking equipment and AutoTutor is being featured as a major
image for the new FedEx Technology Institute.  The Institute will be
housed in a new building on the University of Memphis campus.  A
picture has been taken with someone using the eye tracking equipment
and looking at a page with an animated agent.  This picture will occur
in the Memphis International Airport and other locations in the city.

There was a presentation by Art Graesser to the public in the city of
Memphis on the eye tracking research.  This occurred in the Fogelman
Executive Center on October 25, 2002.  

There was an interview with Dr. Art Graesser by the Newsday’s
Sunday Journal in New York State (December 8, 2002).  The interview
addressed the importance of taking a critical stance while students
read web pages because there is a large volume of incorrect
information on the web.    

Journal Publications:Graesser, A.C., Person, N., & Hu, X., "Improving 
      comprehension through discourse processes.", New Directions in Teaching 
      and Learning, vol. 89, (2002), p. 33-44. Published 
      Graesser, A.C., & Olde, B.A., "How does one know whether a person 
      understands a device? The quality of the questions the person asks when 
      the device breaks down.", Journal of Educational Psychology, vol. , (), p. 
      . Accepted 

Book(s) of other one-time publications(s): Hemmerich, J. & Wiley, J., "Do 
      argumentation tasks promote conceptual change about volcanoes?" , bibl. 
      Hillsdale, NJ: Erlbaum., (2002). Conference Proceedings Published 
      of Collection: W. G. Gray and C.D. Schunn (Eds.), "Proceedings of the 
      Twenty-Fourth Annual Conference of the Cognitive Science Society." 
      Wiley, J. & Bailey, J., "Effects of Collaboration and Argumentation on 
      Learning from Web Pages." , bibl. Hillsdale, NJ: Erlbaum., (). Book 
      Chapter Accepted 
      of Collection: A. O'Donnell & C. Hmelo, "Collaborative Learning, 
      Reasoning, And Technology" 
      Wiley, J., Goldman, S.R., & Graesser, A.C., "Promoting critical inquiry 
      from web sources." , bibl. Mahwah, NJ: Erlbaum. , (2002). Conference 
      Proceedings Published 
      of Collection: W. G. Gray and C.D. Schunn (Eds.), "Proceedings of the 24rd 
      Annual Conference of the Cognitive Science Society" 
      Otero, J., Leon, J.A., & Graesser, A.C., "The psychology of science text 
      comprehension" , bibl. Mahwah, NJ: Erlbaum., (2002). Book Published 
      Graesser, A.C., Gernsbacher, M.A., & Goldman, S., "Handbook of Discourse 
      Processes" , bibl. Mahwah, NJ: Erlbaum., (2003). Book Accepted 
      Graesser,A.C., McNamara,D.S.,& Louwerse, M.M, "What do readers need to 
      learn in order to process coherence relations in narrative and expository 
      text." , bibl. New York: Guilford Publications, (). Book Chapter Accepted 
      of Collection: A.P. Sweet and C.E. Snow, "Rethinking reading 
      comprehension" 
      DiPaolo, R.E., Graesser, A.C., Hacker, D.J., White, H.A., & TRG, "Hints in 
      human and computer tutoring." , bibl. Mahwah, NJ: Erlbaum, (). Book 
      Chapter Accepted 
      of Collection: M. Rabinowitz, "The impact of media on technology of 
      instruction." 


Other Specific Products:



Special Requirements for Annual Project Report:
  Unobligated funds: less than 20 percent of current funds 

  Categories for which nothing is reported:
  Participants: Partner organizations
  Participants: Other Collaborators
  Products: Other Specific Product
  Products: Internet Dissemination
  Contributions Within Discipline
  Contributions to Other Disciplines
  Contributions to Education and Human Resources
  Contributions to Resources for Science and Technology
  Contributions Beyond Science and Engineering
  Special Reporting Requirements
  Animal, Human Subjects, Biohazards


            
    

              We welcome comments on this system