Types of Rubrics
Rubrics can be categorized into three broad groups. The first group distinguishes generic from task-specific rubrics. The second group distinguishes analytic from holistic descriptors of performance. The third group distinguishes primary trait from multiple trait rubrics.
Generic and Task-Specific Rubrics
Rubrics can be categorized as generic or task-specific. As is so often the case in assessment, the line between the two categories may overlap creating a combination or hybrid model. Many task-based rubrics are adaptations of generic scales, and many are combinations of generic elements and elements specific to a particular performance task.
Generic rubrics can be applied to a number of different tasks within a single mode of communication (Interpersonal, Interpretive, Presentational). ACTFL recommends the following domains be considered when designing rubrics for the three modes of communication: Functions, Contexts/Content, Text Type, Language Control, Vocabulary, Communication Strategies, Cultural Awareness (ACTFL Performance Descriptors for Language Learners). The rubric in Figure 1 is a generic Interpersonal rubric designed to be learner-friendly. Learners can ask themselves the questions that will be used to judge their performance in a conversation at the novice level. The questions reflect all of the domains except Contexts/Content which is represented in the choice of topic for the learners to discuss. This generic rubric serves as a guide for novice learners to reference any time they are engaged in an Interpersonal task. They know the characteristics of a strong performance, and know that they must demonstrate a “strong performance” consistently over multiple novice-level topics in order to be considered a strong novice interpersonal communicator.
Task-specific rubrics are used with particular tasks, and their criteria and descriptors reflect specific features of the elicited performance. Tedick (2002) writes: "While some rubrics are created in such a way as to be generic in scope for use with any number of writing or speaking tasks, it is best to consider the task first and make sure that the rubric represents a good fit with the task and your instructional objectives. Just as a variety of task-types should be used in language classrooms, so should a variety of rubrics and checklists be used for assessing performance on those tasks" (p. 37). For example, this Presentational Writing task requires learners to refer to a series of pictures to tell a story in the past about a visit to the monuments in Paris, France. The rubric (Figure 2) focuses on control of past tenses, use of story form, and factual information about the monuments.
Hybrid rubrics that combine features of generic and task-specific rubrics are very useful in classroom assessment because they provide feedback to learners on broad dimensions of language production along with their performance on the particular competencies and knowledge targeted by a specific task within a specific unit of instruction. Teachers may keep the generic language production elements as they are and change one or two categories to focus on requirements for a specific task. For example, the rubric in Figure 3 for a Presentational Writing task includes categories that are generic (used for all writing tasks) and categories that are task-specific. The task is to write a 5-paragraph essay comparing the importance of biodiversity in France to the United States. In this case, the categories of Organization, Vocabulary, Accuracy, and Culture are elements that are evaluated in all Presentational Writing tasks. In addition to these four categories, a fifth category about the use of a variety of authentic resources is added for this task. It is important to remember that presentational tasks benefit from feedback and subsequent revisions to achieve a polished product.
Holistic and Analytic Rubrics
Rubrics may also be categorized as holistic or analytic. Holistic rubrics describe the characteristics of a performance to give an overall judgment of the quality of the performance. An analytic rubric looks at the individual characteristics of a performance and judges each characteristic separately.
In holistic evaluation, raters make judgments by forming an overall impression of a performance and matching it to the best fit from among the descriptions on the scale. Each band on the scale describes performance on several criteria (e.g. text type + vocabulary + language control). Three or four levels of performance are commonly found in holistic rubrics. Holistic scales may be either generic or task-specific. Figure 4 is a sample of a holistic rubric for a Presentational Writing task.
Advantages of holistic rubrics:
- They are often written generically and can be used with many tasks.
- They save time by minimizing the number of decisions raters must make.
- Trained raters tend to apply them consistently, resulting in more reliable measurement.
- They are good for summative assessments as they give an overall judgment of performance.
Disadvantages of holistic rubrics:
- They do not provide specific feedback to learners about the strengths and how to improve performance.
- Because they lack specific details, they are not useful for formative assessments.
(Tedick (2002), Mueller (2002), and TeacherVision.com (2000-2002).)
Analytic rubrics often combine performance categories from a generic rubric with categories directly related to a task, such as demonstrating understanding of specific lesson content (Moskal, 2000). In practice, the names "analytic rubric" and "multiple trait rubric" may be used interchangeably.
Performance dimensions commonly found in analytic rubrics include the domains in the ACTFL Perfomance Descriptors:
- Text type
- Language control
- Communication strategies
- Cultural Awareness
These rubrics may also include task specific items such as:
- Pronunciation and Intonation
- Effective use of technology
- Successful collaboration
- Evidence of integration of multiple sources of factual information
Figure 5 presents an adaptation of a well-known analytic scale for evaluating ESL writing performance. Describing this rubric, Tedick (2002) writes: "Note that the scale assigns different weights to different features. This allows a teacher to give more emphasis to content than to grammar or mechanics, for example. The option to weigh characteristics on the scale represents an advantage to analytic scoring." (p. 35).
Figure 6 shows an analytic scale for role plays and interviews used with students in first-year French courses at the University of Minnesota. This rubric can be used with other languages. In this example, all criteria are weighted equally.
Advantages of analytic rubrics:
- They provide useful feedback to learners on areas of strength and weakness.
- Their dimensions can be weighted to reflect relative importance of individual criteria.
- They can show learners that they have made progress over time in some or all dimensions when the same rubric categories are used repeatedly (Moskal, 2000).
Disadvantages of analytic rubrics:
- "The whole is greater than the sum of its parts." Tedick (2002) notes: "Separate scores for different aspects of a student’s writing or speaking performance may be considered artificial in that it does not give the teacher (or student) a good assessment of the "whole" of a performance." (p. 36).
- They take more time to create and use.
- There are more possibilities for raters to disagree. It is more difficult to achieve intra- and inter-rater reliability on all of the dimensions in an analytic rubric than on a single score yielded by a holistic rubric.
- There is some evidence that raters tend to evaluate grammar-related categories more harshly than they do other categories (McNamara, 1996), thereby overemphasizing the role of accuracy in providing a profile of learners' proficiency.
- There is some evidence that "when raters are asked to make multiple judgments, they really make one..." (Fulcher, 2009). Care must be taken to avoid a "halo effect" and focus on the individual criteria to assure that diverse information about the learner's performance is not lost.
Primary and Multiple Trait Rubrics
Rubrics can also be categorized as primary trait or multiple trait rubrics. A primary trait rubric is task-specific and evaluates performance based on only one characteristic. A multiple trait rubric evaluates performance based on several characteristics of a specific task.
Primary trait rubrics
Primary trait scoring, as developed by Lloyd-Jones and Carl Klaus (Lloyd-Jones, 1977) was designed to evaluate the primary language function or rhetorical trait elicited by a given writing task or prompt. "Primary trait assessment in its initial formulations focused on the specific approach that a writer might take to be successful on a specific writing task; every task required its own unique scoring guide" (Applebee, 2000, p. 4). In its original form, primary trait scoring would be strictly classified as task-specific, and performance would be evaluated on only one trait, such as the "Persuading an audience" example from Tedick (2002, p. 36) for a task requiring learners to write a persuasive letter to the editor of the school newspaper:
|Primary Trait: Persuading an audience|
|Fails to persuade the audience.|
|Attempts to persuade but does not provide sufficient support.|
Presents a somewhat persuasive argument but without consistent development and support
Develops a persuasive argument that is well developed and supported.
Today, you may find that primary trait rubrics vary markedly from their original design and intended use. Applebee notes: "Over the years as primary trait approaches were used more widely, they evolved into a more generic approach which recognized the similarities in approach within broad uses or purposes. The basic question addressed in scoring, however, remained, 'Did the writer successfully accomplish the purpose of this task?' To insure that raters maintained this focus, scoring guidelines usually instructed raters to ignore errors in conventions of written language, and to focus on overall rhetorical effectiveness" (p. 4).
Primary trait rubrics are useful in formative assessments designed to determine how well learners perform a particular language function they have been working on in class. For example, if several lessons have been devoted to working on descriptive language, a culminating writing task might be scored solely on its effectiveness as a description.
Multiple trait rubrics
Hamp-Lyons (1991) coined the term multiple trait scoring for rubrics that she designed, based on the concepts of primary trait scoring, to provide diagnostic feedback to learners and other stakeholders about performance on "context-appropriate and task-appropriate criteria" for a specified topic/text type. She designed her multiple trait rubrics to be applicable across a range of similar tasks. Currently, multiple trait (or multitrait) rubrics are commonlyconsidered to be task-specific, although one or more of their dimensions might also be found in generic, analytic rubrics. Figure 7 illustrates a task and multitrait scoring rubric from a resource for language teachers (Petersen, 1999).
Multiple trait rubrics look like analytic rubrics in that performance is evaluated in several categories, and, in practice, you may find the terms used interchangeably. However, analytic rubrics usually evaluate the more traditional and generic dimensions of language production, while the criteria in multiple trait rubrics focus on specific features of performance necessary for successful fulfillment of a given task or tasks.
Advantages of primary and multiple trait rubrics
- The rubrics are aligned with the task and curriculum.
- Feedback is focused on one or more dimensions that are important in the current learning context.
- With a multiple trait rubric, learners receive information about their strengths and weaknesses.
- Primary and multiple trait rubrics are generally written in language that students understand.
- Teachers are able to rate performances quickly.
Disadvantages of primary and multiple trait rubrics
- Information provided by primary trait rubrics is limited and may not easily translate into grades.
- Task-specific rubrics cannot be applied to other tasks without adaptation of at least one or more dimensions.
(Hamp-Lyons, 1991; McNamara, 1996; Tedick, 2002.)
Types of Assessment Methods
What is an Essay?
An Essay is an assessment question that requires an answer in a sentence, paragraph, or short composition. Essay assessments are usually classified as subjective assessments as there are normally a variety of responses.
According to Trigwell, there are 3 standard forms of essays:
- Role Play Essays
Students respond to the essay as if he/she is performing a specific role in the essay.
For example: Write a letter to the local county council, explaining the environmental issues in the area, and requesting them to produce some measures; giving evidences and social arguments from government reports.
This type of essays allows the students to become involved and see the relevance of the task.
- Structured Essays
Structured Essays are essays which have specific questions or topics that require answers.
For example: In Shakespeares play Hamlet, discuss and compare some of the soliloquies in terms of its style, syntax and imagery.
This type of essays is useful if the assessors wish to test specific knowledge and techniques, it is also easier to mark as the assessors know what type of answers to expect.
- Interpretation of Data Evidence Essays
Students are asked to write an essay based on data from a report/experiment they produced or from an external source.
For example: Using the measurements found in the laboratory, explain and discuss the chemical reactions between the two main elements found.
This type of essays is greatly pragmatic, using data the students collected, allowing students to reflect and analyze.
An essay (depending on the types of essays) is usually expected to consist of an
- Major points and ideas explained and summarized
- Results/Related points/Issues/or others depending on the topic
- Conclusion future work
|Take Time to Set|
|Y||Take Time to Answer|
|Y||Take Time to Correct|
|Y||Take Time to provide Feedback|
|Y||Suitable for Large Class|
|Can substitute with Computers|
|Process Oriented Method|
|Y||Product Oriented Method|
|P = Possibly Y =Yes|
Advantages of Essay Assessment
- Essays have the ability to assess all levels of learning objectives.
- It encourages original and creative thinking.
- Due to the subjective nature of essay assessments, grading is very unreliable even for the same assessor at different periods.
- Grading may be influenced by other factors such as handwriting and length of response.
- As essays are very time-consuming to answer and to correct, they are not recommended if only low-level of learning outcomes are assessed which can be assessed by multiple choices or short answer questions.
- Although guessing is not possible in essay assessments, but bluffing is.
- It is also not advisable to give the topic of the essay to the students at an early date. This may give rise to superficial learning where students concentrate all their efforts in completing the essay only.
- Let students know the assessment criteria and marking scheme, including grammar, spellings and other issues.
- Try to reduce ambiguity in the essay questions, clearly define the expected response such as compare, evaluate, summarize, critique etc.
- Do not use essays to measure knowledge or understanding that can be assessed using less time consuming assessment methods.
There are two general grading approaches holistic and analytic grading. Holistic approach is grading the essay as a whole. Analytic approach grades the important components of the essay and assigns marks to each component.
|Introduction:||Attitude is defined; thesis is clearly focused; subject is significant||Thesis is clear; provides direction for essay||Unclear; formulaic; not creative||Introduction is incomplete, ineffective, or missing|
|Idea Development:||Interesting; sophisticated; insightful||Clear and Thoughtful||Simplistic; uneven in quality; lacking in relevance||Absent or ineffective|
|Support or Evidence:||Detailed; accurate; convincing||Sufficient and accurate||Uneven||Vague, missing, or inaccurate|
|Word Choice:||Engaging and powerful choice of words||Appropriate to task||Uneven||Limited, monotonous, inappropriate|
|Conclusion:||Extends; connects; comments on topics||Purposeful and perceptive||Summarizes previously stated information||Absent, incomplete, or unfocused|
|Topic Sentences:||Clearly related to thesis; comprehensive; incorporates effective transitions||Comprehensive and logical||Provides bland restatement of thesis; narrow or inaccurate||Absent|
|Paragraph Order:||Contributes to an effective argument; reinforces the content||Demonstrates a clear plan||Ineffective or inconsistent||Random|
|Transitions:||Effective and varied||Clear and functional||Mechanical||Absent|
|Sentence Structure:||Complete; varied; interesting||Complete and correct||Variety is present; some errors are evident||Repetitious; fragments and run-ons are frequent|
|Punctuation/Spelling:||Error-free||Present but do not interfere with meaning||Careless or distracting||Block meaning|
Web Reference and Resources
Tips for Students PresentingTo Reference these pages
Copy and paste the text below:
Chan C.(2009) Assessment: Essay, Assessment Resources@HKU, University of Hong Kong [http://ar.cetl.hku.hk]: Available: Accessed: DATE