Validating professional standards for teachers

Many education systems around the world are developing professional standards for teachers. In order to be effective, the teaching profession and their stakeholders must be confident that such standards provide a true representation of what they claim to measure.

Validation studies can strengthen the evidence base for these standards, and demonstrate that they are appropriate descriptions of teaching practice in their context. But what does validation mean, and how can validation studies be conducted rigorously and cost-effectively?

A recent ACER literature review examines international examples of validation studies of teaching standards, to show the many possibilities for validation study design. Drawing on Samuel Messick's components of construct validity, the review identifies the different types of validity that such studies may demonstrate.

The review shows how the design of a validation study relates to how validity is defined:

• Consequential validity concerns the benefits of teaching standards, relative to their risks. It is often demonstrated with stakeholder surveys or consultations, or documentation of impact.
• Content validity and substantive validity concerns whether teaching standards describe quality teaching practice, as it is demonstrated in the classroom, and articulated in theory and research. Content validity is often demonstrated by subjecting standards to expert review.
• Structural validity concerns the internal coherence of the standards, as demonstrated in empirical data. It is often demonstrated using psychometric methods, including item response theory.
• External validity concerns how teaching standards relate to other indicators of teacher effectiveness. Several studies have explored the relationship between teaching practice and student learning outcomes, and found that the relationship varies widely.
• Generalisability concerns whether standards are equally applicable to different types of teachers, regardless of their characteristics and contexts. Studies often address this through representative sampling, or disaggregation of validation data for key demographic groups.

A significant theme identified in the literature review is that validation is both a political and methodological process. The review notes that all teachers – regardless of the schools in which they teach, and the communities that they serve – should feel that the standards are a fair, accurate and useful representation of their work.

ACER’s analysis generated eight recommendations to guide researchers in developing an effective validation studies of teaching standards. Among these recommendations were the importance of involving subject-matter experts and designing reliable methods of teacher assessment against the standards.

The review was originally undertaken to inform the design of a validation study for the draft Teacher Competency Standards Framework (TCSF) in Myanmar, but may also benefit other countries that are considering any similar process of teacher professional standards development.

Read the full report:
Validating professional standards for teachers: A practical guide for research design by Jennifer Jackson and Yung Nietschke, Australian Council for Educational Research (2018)

Many education systems around the world are developing professional standards for teachers. In order to be effective, the teaching profession and their stakeholders must be confident that such standards provide a true representation of what they claim to measure.

Validation studies can strengthen the evidence base for these standards, and demonstrate that they are appropriate descriptions of teaching practice in their context. But what does validation mean, and how can validation studies be conducted rigorously and cost-effectively?

A recent ACER literature review examines international examples of validation studies of teaching standards, to show the many possibilities for validation study design. Drawing on Samuel Messick's components of construct validity, the review identifies the different types of validity that such studies may demonstrate.

The review shows how the design of a validation study relates to how validity is defined:

• Consequential validity concerns the benefits of teaching standards, relative to their risks. It is often demonstrated with stakeholder surveys or consultations, or documentation of impact.
• Content validity and substantive validity concerns whether teaching standards describe quality teaching practice, as it is demonstrated in the classroom, and articulated in theory and research. Content validity is often demonstrated by subjecting standards to expert review.
• Structural validity concerns the internal coherence of the standards, as demonstrated in empirical data. It is often demonstrated using psychometric methods, including item response theory.
• External validity concerns how teaching standards relate to other indicators of teacher effectiveness. Several studies have explored the relationship between teaching practice and student learning outcomes, and found that the relationship varies widely.
• Generalisability concerns whether standards are equally applicable to different types of teachers, regardless of their characteristics and contexts. Studies often address this through representative sampling, or disaggregation of validation data for key demographic groups.

A significant theme identified in the literature review is that validation is both a political and methodological process. The review notes that all teachers – regardless of the schools in which they teach, and the communities that they serve – should feel that the standards are a fair, accurate and useful representation of their work.

ACER’s analysis generated eight recommendations to guide researchers in developing an effective validation studies of teaching standards. Among these recommendations were the importance of involving subject-matter experts and designing reliable methods of teacher assessment against the standards.

The review was originally undertaken to inform the design of a validation study for the draft Teacher Competency Standards Framework (TCSF) in Myanmar, but may also benefit other countries that are considering any similar process of teacher professional standards development.

Read the full report:
Validating professional standards for teachers: A practical guide for research design by Jennifer Jackson and Yung Nietschke, Australian Council for Educational Research (2018)