Skip to main content
Intranet

Assessing graduate qualities

In 2017, the Assessment Working Group recommended using a common approach in measuring the extent to which our students have attained the graduate qualities through their study at the University.

Endorsed by the Academic Board, this approach involved the development of a suite of University-level assessment rubrics to measure each student's achievement in their progression toward the graduate qualities.

Nine working parties (one for each graduate quality) were established to develop the generic rubrics, composed of staff with relevant expertise from across the University. Draft rubrics were developed. Following feedback from international and local experts, the draft rubrics were revised and approved by the Academic Board at its final meeting of 2018.

In 2019, the Assessment Advisory Committee and Education Portfolio are working closely with faculties and University schools to pilot the rubrics in units of study to validate their effectiveness.

The rubrics will be used to:

  • measure student attainment of the graduate qualities in assessment tasks and curriculums
  • support faculties and University schools in developing assessment plans at the course and curriculum component levels.

The Assessment Advisory Committee is conducting a suite of validation activities to ensure that the common University rubrics measure what they aim to measure. These activities include:

  • testing the descriptors, to ensure logical progression and user understanding
  • measuring inter-rater reliability, to ensure the degree of agreement between markers is consistent over repeated testing
  • conducting focus groups and sourcing feedback from a variety of internal and external stakeholders, including students, staff, industry, alumni and worldwide experts
  • piloting the graduate quality rubrics in existing units of study.

Pilots

The Education Portfolio is working closely with unit coordinators to help them pilot the graduate quality rubrics in their units of study. This will help the Assessment Advisory Committee to build a strong evidence base over time to ensure the validity of the graduate quality rubrics. At the same time, the pilot allows rubric users to provide feedback, and make revisions as necessary.

Specific questions we are asking pilots to feedback to us include:

  • how well the rubrics matched and were able to measure the assessment task(s)
  • if the rubrics needed to be adapted to match the discipline
  • what staff and students thought of using the rubrics
  • the logical progression of the developmental stages in the rubric
  • the wording of the descriptors
  • the degree to which markers agreed on measurements in the rubrics.

Where are we currently piloting?

There are plans to pilot the rubrics in over 30 units of study across the University in 2019 and 2020, with the list growing by the day. A list of broad categories are listed below (as at January 2020):

  • Bachelor of Education electives
  • Education electives
  • First-year media studies
  • First-year physics units
  • Health collaboration challenge
  • Health sciences
  • Industry community project units
  • Language majors
  • Online learning environments
  • Performance seminar in music
  • Professional engineering program
  • Second and third-year Bachelor of Arts electives

Are you interested in piloting the graduate quality rubrics in your unit of study?

If you would like to get involved, please submit an Expression of Interest and we will be in touch with you to discuss further.

We are particularly interested in sourcing pilots from one of the following categories:

  • units of study that are identified on assessment plans to measure a student’s attainment of one or more of the graduate qualities
  • units of study interested in trialling inter-rater reliability (support may be available to assist with this)
  • OLE units of study that specifically target one graduate quality
  • units of study in STEM, health sciences, law or architecture and design.

As part of the work in refreshing assessment to reduce burden on staff and students, an ongoing workstream of the Assessment Advisory Committee is to develop a compendium of assessments that may be measured by the common University rubrics.

By starting with the targeted graduate quality, assessment tasks are designed to produce evidence that can be measured by the components and stage descriptors in the graduate quality rubrics.

This is the reverse approach to the typical development of assessments, where assessment rubrics are developed based on the assessment task and criteria.

Toward a compendium of assessment tasks that measure graduate qualities

The aim of the compendium is to provide a resource for staff to help them develop and design assessment tasks will provide evidence that may be measured by the graduate quality rubrics.

In May 2019, the Education Portfolio held a workshop to commence this work. Over 100 assessment tasks were brainstormed at this workshop.

Further developing assessment tasks that measure the graduate qualities

The Education Portfolio has developed a Canvas page where staff can collaborate to brainstorm and develop assessment tasks that measure the graduate qualities. The page uses tools to facilitate collaboration and aims to eventually list fully developed tasks into a compendium of assessment tasks that measure the graduate qualities.

What is an assessment plan?

An assessment plan is a document which:

  • articulates the pedagogical approach to assessment for the discipline
  • maps curriculum learning outcomes and graduate qualities against units and assessment tasks
  • provides a whole-of-curriculum overview of assessment across different units, to coordinate progressive feedback to students, balance types of tasks and manage workload.

Assessment plans may apply to an entire degree (eg, a tightly structured professional degree) or within a component (eg, a major in a liberal studies degree). This planning will make it possible to ensure that the balance of assessment tasks gives appropriate priority to the learning outcomes most valued by students, staff and employers.

Assessment plans ensure that every student has the opportunity to develop the graduate qualities which are assessed during their degree, with feedback along the way.

Assessment plans aim to assist academic staff, faculty committees, and boards to manage and coordinate assessment across an entire curriculum. They assure faculty and academic staff that students are achieving learning outcomes and graduate qualities.

Additionally, the plans provide a means of balancing assessment types to avoid duplication between units. Overall, they are a planning tool to maximise assessment effectiveness across an entire curriculum while managing and, where appropriate, reducing assessment volume.

Benefits of assessment plans

Assessment plans:

  • apply to each undergraduate student
  • provide a strategic overview of the development and assessment of graduate qualities in a curriculum
  • indicate how student performance in graduate qualities is assessed through 'measurement points' across a curriculum
  • provide a means of measuring the scale of assessment from both student and staff's perspective
  • monitor the amount, spread and timing of assessment across a course or curriculum component
  • help identify opportunities to manage assessment volume and reduce duplication
  • provide opportunities to consider how authentic tasks and innovative approaches can enhance a learning program
  • enable academics to become aware of the knowledge and skills being assessed in other units.

Requirements for assessment plans

In general, assessment plans should:

  • articulate the learning outcomes in terms that give expression to the graduate qualities for the degree and relevant curriculum components
  • for each curriculum, indicate in which units and assessment tasks each of the graduate qualities are developed and measured
  • describe in narrative terms how students’ achievement of the graduate qualities will be developed, the tasks typically used and how they will be assessed by completion of the degree.

The Assessment Advisory Committee will provide ongoing feedback to faculties that produce draft assessment plans prior to the deadline. If you would like some feedback on a draft assessment plan, please contact the committee's secreatariat Christine Lacey on [email protected].

Who is responsible for assessment plans in my faculty?

All course and component coordinators will be engaged in this process and may reach out to unit coordinators for relevant input in the design of assessment plans for their curriculums.

Faculties have nominated staff to coordinate the process internally and if you have any questions, you should reach out these staff. If you are unsure about who to contact in your faculty, you should enquire with your Faculty Curriculum Manager or equivalent, or your Associate Dean (Education).

Exemptions

Curriculums exempt from the development of assessment plans are:

  • postgraduate degrees
  • double degrees (individual degrees will already have an assessment plan)
  • degrees which will no longer be offered in 2021.

From January 2020, the Assessment Advisory Committee will begin reviewing and recommending assessment plans for all curriculums to the Academic Board for approval.