Preliminary Results on Evaluation Warning Pilot Study, Fall 2002

Learning Gain scores (post test score - pretest score)
Expect Evaluation in Green, No Expectation in Red
Argument on left, Descriptive report on right
(main effect for writing, p<.08; no other effects)

DIfference in Ranking for Reliable vs Unreliable Sources (bigger difference means better discrimination)
Expect Evaluation in Green, No Expectation in Red
Argument on left, Descriptive report on right
Main effect for writing (p<.03) and significant interaction (p<.006)


Tentative Conclusions:
Gain scores and Differences between Rankings for reliable and unreliable sources suggest that the evaluation warning seems to make everyone in the Argument condition perform more uniformly well (i.e. cleans up the variance)

Still to be analyzed:
essays
peer evaluations
reading times on reliable/unreliable sites
navigation patterns

Possible Implications:
An explicit warning that students may need to evaluate a peer essay AFTER they write an argument may help students be more critical/selective as they read, and may be an instruction to consider as part of the intervention in Strand 2.  Based on preliminary results, in terms of an explicit prompt to evaluate, this manipulation seems to be more effective than the instruction to write an evaluation was in Summer TA pilot.