skip to main content

Research Developments from ACER

Subscribe
International
{rd-image-caption}

Image ©Shutterstock/ wavebreakmedia

International assessments in developing countries

Many countries now participate in PISA, and developing countries recognise the value of participation, but using PISA in the developing world presents some technical issues, as Ray Adams explains.

The Programme for International Student Assessment (PISA) triennial international survey, by the Organisation for Economic Cooperation and Development (OECD), tests the skills and knowledge of 15-year-old students.

It enables participating countries and economies to evaluate their education systems and the impact of education reforms, and compare the achievement of their students with other participating countries and economies.

While the original assessment frameworks for PISA have been developed for OECD countries, the expanding participation of lower-income or ‘developing’ countries has led to demand for PISA for Development (PISA-D).

In essence, the PISA-D project aims to implement a version of PISA that better enables developing countries to accurately assess and describe their 15-year-olds’ competencies, knowledge and skills in the key areas of reading, mathematics and science, while at the same time providing participating countries with an opportunity to build their capacity to manage student assessment and apply the results for system improvement.

The development of PISA-D has required further analysis of the fitness of the PISA tests and questionnaires for informing participating countries about student achievement and related background factors.

To that end, our analysis for the OECD, reported in PISA For Development Technical Strand A: Enhancement of PISA cognitive instruments, includes an investigation of:

  • the assessment framework for PISA
  • the validity of described scales
  • comparability, to ensure items are invariant in terms of the nationality, culture, language, gender and socioeconomic background of sampled students
  • the appropriateness of the existing PISA item pool in terms of item difficulty, measurement precision and the resulting information profile.

The aim of PISA-D, in coordination with the regular triennial PISA surveys, is to provide participating countries with a more precise measure of their students’ capacity and, through an appropriately targeted questionnaire, a better understanding of the factors that contribute to their students’ success.

Each country expects the results of PISA-D assessments to be fully comparable to the results of all countries that undertake PISA. This means in essence that the principles and the technical standards that apply to test design, administration, data entry, translation and sampling applied to PISA-D must be the same as for PISA.

Our analysis for the OECD investigates the technical issues in drawing on the large set of existing PISA items for use in an appropriate PISA-D test to give maximum information to participating countries.

While our analysis in PISA For Development Technical Strand A: Enhancement of PISA cognitive instruments reveals a number of issues in the rollout of PISA-D it also indicates that, with care and extensive consultation with participating countries, it remains feasible. ■

Further information:

To read the full report, Pisa For Development Technical Strand A: Enhancement of PISA cognitive instruments, visit the OECD website.

RD

About the author

Professor Ray Adams is the Director of ACER’s Centre for Global Education Monitoring and a Professorial Fellow of the University of Melbourne, specialising in psychometrics, educational statistics, large-scale testing and international comparative studies.

More [rd] articles by Ray Adams

View selected works of Ray Adams

Related articles

Higher Education
University completions and equity | RD

University completions and equity

06 May 2015

University students from disadvantaged groups have a lower completion rate than their more advantaged peers, but most disadvantaged students do complete their degrees, research reveals.

Evaluation, Quality & Standards, Higher Education, Indigenous, Featured higher education

International
Teacher absenteeism in Indonesia | RD

Teacher absenteeism in Indonesia

16 March 2015

A comprehensive new study reveals that teacher absenteeism in Indonesia is declining, and provides evidence for policy makers focused on improving teaching and learning, as Phil McKenzie explains.

Evaluation, Quality & Standards, Survey, School, International, Featured international

Higher Education
Designing assessment for radiopharmaceutical scientist training | RD

Designing assessment for radiopharmaceutical scientist training

16 February 2017

ACER is working with radiopharmaceutical scientists to develop fit-for-purpose progressive assessment practices for a new training program, as Jacob Pearce explains.

Assessment, Quality & Standards, Higher Education, Featured higher education