skip to main content

Research Developments from ACER

Higher Education

Image ©

Taking higher education assessment from the boutique to eBay

The move to massive open online courses reflects the shift of higher education to an eBay approach to curriculum and teaching, but assessment remains a boutique enterprise, as Hamish Coates explains.

Want to study flight vehicle aerodynamics at MIT? Maybe fly to Yaoundé in Cameroon.  Animal behaviour at the University of Melbourne? You might be in Sydney, Nova Scotia. Calculus at Ohio State University? Not a problem, even if you’re on the International Space Station.

Beyond our bricks-and-mortar universities, large fields of study in higher education have, over the last 20 years, become quite freely accessible online. Sophisticated virtual robots driven by powerful people analytics now help experienced humans teach millions of students. In just the blink of an eye, the key challenge for higher education has switched from selecting who gets in, to grading who gets out.

In just the blink of an eye, the key challenge for higher education has switched from selecting who gets in, to grading who gets out.

Some of the issues are political. Who owns and governs this ‘open’ learning? And what do competing stakeholders mean by ‘open’? Some of the issues are practical. How will innovative educators find effective and credible new ways for assessing what learners know and can do? Can the cloud or crowd address higher education’s deep productivity and quality challenges?

Innovators, institutions, and larger networks in higher education now proffer a suite of ideas and practices for curriculum and teaching. Progress in the important matter of assessment, however, appears modest. Unlike the eBay approach to curriculum and teaching through massive open online courses, or MOOCs, assessment remains a boutique enterprise.

Unlike the eBay approach to curriculum and teaching through massive open online courses, or MOOCs, assessment remains a boutique enterprise.

In Australia alone, we lack pretty basic details on assessment resources and practices, even in highly regulated professional fields. Projects have been advanced in recent years to moderate conceptions of what learners should know and how they are performing. Related work has stimulated the creation and sharing of assessment materials. Quality reviews of assessment tasks show that about a third of that in common use meets acceptable technical standards. This important work treads gently and gradually, weaving around tacit academic cultures and conventions, and more tangible industrial and professional matters.

Imagine a more scalable model of assessment. What might it look like? First, learning outcomes and assessment materials would be defined and created by broad collaborations of scholars, practitioners and technical experts. The materials would be carefully reviewed, with about a third being discarded. Institutions would share materials, underpinning standards and benchmarking. Technology would be used for production, storage, review, delivery and reporting. Educators would moderate processes and outcomes.

This powerful vision is triggered by online education, but its implications are far wider.

The rudiments of the architecture for such a scalable model of assessment are already established. Through the OECD’s Assessment of Higher Education Learning Outcomes study, researchers at the Australian Council for Educational Research showed it is feasible to reach consensus internationally on what undergraduate students should know, and to invoke large international collaborations to review materials. Running an online assessment across 250 universities internationally also revealed that technology is easily managed compared with complex matters like human ethics, individual identification and organisational compacts. Through the Australian Medical Assessment Collaboration we’ve shown how medical schools can work together to produce very large numbers of shared materials for filtering into practice or for use as a stand-alone test.

Yes, research projects like these take years to progress, but they enable us to transform tomorrow given proof of concept and feasibility, and not least given cost pressures eating into university budgets. Let’s get moving, fuelled by all the urgency of new massive online participation. Let’s at least test the waters to find out what’s going on. â– 

This article was published in The Australian on Wednesday 14 August 2013.

Related articles

Higher Education
University completions and equity | RD

University completions and equity

06 May 2015

University students from disadvantaged groups have a lower completion rate than their more advantaged peers, but most disadvantaged students do complete their degrees, research reveals.

Evaluation, Quality & Standards, Higher Education, Indigenous

Education & Development
Teacher absenteeism in Indonesia | RD

Teacher absenteeism in Indonesia

16 March 2015

A comprehensive new study reveals that teacher absenteeism in Indonesia is declining, and provides evidence for policy makers focused on improving teaching and learning, as Phil McKenzie explains.

Evaluation, Quality & Standards, Survey, School, Education & Development

Assessing learning of students with disabilities | RD

Assessing learning of students with disabilities

13 February 2020

ACER and NEQMAP reviewed existing inclusive education policies and practices in the Asia-Pacific region with a particular focus on learning assessment of children with disabilities.

Assessment, School, Education & Development, Featured school, Australia, India, Indonesia, Malaysia