Evaluative Rubrics – Helping you to make sense of your evaluation data

Three times in one week I’ve now found myself explaining the use of evaluation rubrics to potential evaluation users. I usually start with an example like this, that people can relate to:

When your high school creative writing paper was graded, your teacher most likely gave you an evaluative rubric which specified that you do well if you 1) used good grammar and spelling, 2) structured your arguments well, and 3) found an innovative and interesting angle on your topic. In essence, this rubric helped you to know what is “good” and what is “not good”.

In an evaluation, a rubric does exactly the same. What is a good outcome if you judge a post- school science and maths bridging programme? How does the outcomes of “being employed” or  “busy with a third year  B Sc. Degree at university” compare to an outcome like “being a self-employed university drop-out with three registered patents” or to an outcome like “being unemployed and not sure what to do about the future”. A rubric can help you to figure this out.

E. Jane Davidson has some excellent resources on rubrics here and here. If you need a rubric on evaluating value for investment, Julian King has a good resource here.  And of course, there is the usual great content on better evaluation here.

I love how Jane describes why we need evaluation rubrics:

Evaluative rubrics make transparent how quality and value are defined and applied. I sometimes refer to rubrics as the antidote to both ‘Rorschach inkblot’ (“You work it out”) and ‘divine judgment’ (“I looked upon it and saw that it was good”)-type evaluations.

How to specify your needs if you require a case study

A client is interested in contracting us to write up a case study for one of their programmes, but they don’t really know which information will be necessary. Since there are no terms of reference yet for the case study, I suggested that the client clarifies the following, in order for us to be able to assess the level of effort required.

1. What will the case study be used for? (To document lessons learnt, to help with marketing, to document evidence of a successful initiative)
2. What is the final product that you have in mind, and how long does it need to be? (A written report, or a presentation, or a glossy publication)
3. Who will be reading the Case Study?
4. How much background documents do you have available? (Project descriptions, evaluation findings, participation data, survey data)
5. What kind of additional data collection will be necessary? (Interviews, photo’s, site observations)
6. Would you want to meet with the evaluation team before the assignment starts, and after it is completed? 

I came across this useful little guide on how to use Case Studies to do Program Evaluation.It helps one to assess whether a case study should be used, and how to do it.

Edith D. Balbach, Tufts University
March 1999
Copyright © 1999 California Department of Health Services

Developed by the Stanford Center for Research in Disease Prevention

The Better Evaluation page on Case Studies can be found here.