Developing an observational rubric of writing: Preliminary reliability and validity evidence

The purpose of this paper is (1) to report on the design of the early writing observational writing rubric designed to observe and describe change over time in the writing of children emerging into conventional literacy (ages 6-7) within an instructional setting and (2) to investigate the initial reliability and validity of the rubric. We used an extant data set that included 52 videos of writing instruction in Reading Recovery lessons (approximately 520 minutes) and pre- and post-intervention test data, for 24 students, taken at multiple time points across a 20-week period. Dependent sample t-tests and HLM were used to ascertain if the rubric was sensitive to change over occasions. We also considered if the scores correlated with external literacy measures. The findings suggest that the rubric has good initial reliability and validity and is a useful tool for researchers to observe and measure change over time as young children write in an instructional setting; further validation work is required for use in other settings.