This research investigated strategies for assigning a priori difficulty levels to questions used on online tests. Two techniques were investigated: assignment based on subject-specific terms involved in answering the question, and assignment based on Bloom's taxonomy level. A prototype tool, DIFFICILE, has been developed to recommend an a priori difficulty level based on Bloom's Taxonomy and subject-specific terms. The professor is able to accept the recommendation or assign a difficulty level.

The DIFFICILE tool has been applied to a subset of the question bank used in the CS1 course. A correlation study comparing assigned a priori difficulty against actual student performance on the questions was made to assess how well the a priori difficulty levels reflect or correlate with after-the-fact measurement of difficulty based on student performance.

A major contribution of this research is the automation of the assignment of question difficulty level based on Bloom's Taxonomy. For the questions used in the study, estimation based on Bloom's Taxonomy performed comparably with estimation based on subject-specific terms, and difficulty estimation was found to be easiest for multiple-choice questions.