Issue 31 (3/2010)
10.12.2010

ISSN 1457-7682

Front page
Previous issues
Editorial staff

FSD Bulletin is the electronic newsletter of the Finnish Social Science Data Archive. The Bulletin provides information and news related to the data archive and social science research.

Finnish Social Science Data Archive
E-mail: fsd@tuni.fi

Cross-national Research on Education

Helena Laaksonen

The Finnish Institute for Educational Research (FIER) has long experience in producing high-quality research on education. Based at the University of Jyväskylä, the institute specializes in comparative assessment of learning outcomes, educational cultures and systems, the most well-know example being the PISA study.

FIER Researchers Researchers working at the institute are not too pleased about the enthusiasm produced by the comparative assessment results.
– Country rankings are a visible part of the results but we are talking about just a few measurements, conducted from a certain viewpoint. The differences between countries are small. Country rankings tend to lose information. Education is not competitive sports, says a staff member, professor Pekka Kupari.
– There is a risk that countries which do well on the rankings become satisfied with the prevailing situation, thinking there is no need for change. The goal of international comparative assessments is to look for important factors influencing learning results both on the national level and by country groups, and to try to figure out what can be learned from them. Consequently, the studies collect a lot of background information on pupils, school environment and teaching.

Forty years of comparative assess

The Finnish Institute for Educational Research was established in 1968. It is responsible for the Finnish part of the PISA study (Programme for International Student Assessment), as well as for other OECD-funded study programmes such as the Program for the International Assessment of Adult Competencies (PIACC) and the Assessment of Higher Education Learning Outcomes (AHELO).

The institute has been a member of the International Association for the Evaluation of Educational Achievement (IEA) ever since its establishment. Being a IEA member has led to the institute conducting also the International Civic and Citizenship Education Study (ICCS), Progress in International Reading Literacy Study (PIRLS) and the Trends in International Mathematics and Science Study (TIMSS) in Finland.

Increase in evaluation studies

Pekka Kupari is the most practiced researcher working with cross-national comparative assessments, having been involved with such studies ever since the 1970s. He is currently involved in all cross-national assessments conducted by the institute. Kupari says that the number of evaluation studies has increased rapidly over the past ten years.

This has led to a situation where the institute is preparing, carrying out or reporting several assessment studies at the same time. In 2010, for instance, the institute reported on the 2009 PISA findings, prepared the 2012 PISA study, and pretested measurements for both TIMS and PIRLS. All this work was executed by a small group of ten researchers. Senior Researcher Sari Sulkunen emphasizes that this would be impossible without very experienced staff. With such expertise, there is always the risk that someone leaves, taking their know-how with them.
– There is no-one we could hire to as a quick replacement of a person leaving.

Cross-national assessment process

Cross-national assessments are all collected in waves, generally every 3 - 5 years. If the cycle is frequent, there may be problems with non-response and refusals to participate. To ensure comparability, all research projects have an international consortium which lays down the rules regarding the schedules, instructions and quality criteria which the national level operators must abide by.
– For example, the PIACC guidelines are 300 pages long, says Senior Researcher Antero Malin.

All cross-national studies proceed in a similar manner. First the assignments and measurements are planned and pre-tested. At this stage the national operators have a say in methods and measurements.
– Once the actual data collection stage is reached, there is no longer any room for changes, Kupari explains.

National cultures influence results

The researchers working at the Finnish Institute for Educational Research stress that measurements used in cross-national assessment studies are always compromises. This should be taken into account when considering their validity. Cultural environment always affects the results.
– Still, the main goal is to achieve as high comparability as possible. Researchers know how to take problematic factors into consideration, Malinen points out.

Even with strict guidelines, schedules and quality requirements, researchers working at the institute find international cooperation rewarding. Both Kupari and Sulkunen feel it is educational to write joint papers.
–One achieves knowledge and skills it would be hard to get otherwise, Sulkunen says.