Royal Alberta Museum - Concepts of a Criterion-based Evaluation

Concepts of a Criterion-based Evaluation Image

There are several basic concepts that underlie the development of an objective and reproducible assessment system. These concepts (Braidwood, 1987) were used in the development of the evaluations in this report. They are as follows:

Categories - Only broad categories such as low/moderate/high and yes/no are used to keep the ranking system relatively simple.

Criterion Scoring - Definite numerical values are assigned to criteria. Although these values may be viewed as subjective, reproducible results are more likely to be acheived where a limited number of choices is given than where each evaluator can assign a range of scores.

Weighting - Criteria are weighted differently to reflect their relative importance. A multiplicative scoring system was not used as it can accentuate rank differences.

Number of Criteria - Literature states that between two and twelve criteria should be applied. Too many criteria can result in poor accuracy and a reduction of reliability. As well, where a large number of criteria is employed, the average score will mask very high or very low individual scores that may be significant.

Relevance to all Sites - Criteria should be relevant to all potential sites. The realm of geology/Palaeontology is so broad that, occasionally, questions in the evaluation may not be applicable. The total score will be adjusted in these cases.

Total Score - The scores chosen to reflect levels of geological significance, degree of threat, and interpretive value were selected by analyzing the scores of a large number of known sites. The total scores are not absolute values indicating importance of the site, but are only relative, to facilitate comparison between sites.

Criterion Overlap- It is important that criteria overlap as little as possible, otherwise the same criteria will be measured more than once.