Psychology Board of Australia to introduce a national psychology examination

The Psychology Board of Australia recently announced that a national psychology examination will be implemented from 1 July 2013 for applicants seeking general registration. The Consultation Paper on Guidelines for the National Psychology Examination, published by the Psychology Board in April this year, did not seek views as to whether the psychology profession needed an examination, but instead provided an overview of the format, curriculum and other aspects of the examination process for comment. The APS made a submission to the Consultation Paper, highlighting its concerns and providing recommendations. This article provides information on what is known about the intended Psychology Board examination thus far and outlines the APS advocacy and recommendations in relation to it.

Why the exam is being introduced

The Psychology Board's Consultation Paper states that the fifth and sixth years of education in psychology can now be obtained via multiple pathways, some of which are not accredited, and that the examination will provide “a mechanism for measurement of a minimum level of applied professional knowledge of psychology, regardless of the various training backgrounds” (p.13).

Who the exam will apply to

The Consultation Paper indicates that the examination will be used for four purposes: (1) Final assessment for applicants seeking to move from provisional to general registration; (2) Assessment of overseas-trained psychologists; (3) Assessment of knowledge “where questions of performance have been notified”; and (4) Return-to-work assessments after lengthy periods without practice. The Psychology Board has indicated that it is considering applying an exemption from sitting the examination to applicants for general registration who have completed an accredited Australian Masters or Doctoral degree within the past five years.

Content and form of the exam

The examination is designed to test only the knowledge component of each of eight areas of capability “at entry level” (Table 1). This will be measured by “four examination domains” (Table 2), each with a series of sub-domains (for a list of sub-domains, see the Consultation Paper, available at www.psychologyboard.gov.au/News/Past-Consultations.aspx ). The eight capability areas are the same as the core capabilities and attributes outlined in the Psychology Board’s two-year internship program.

The examination is proposed to be a test of knowledge in a multiple choice question format, consisting of some 150 items and with an estimated completion time of three hours.

Sample items provided in the Consultation Paper indicate that the intention is to design questions to assess the application of knowledge to hypothetical problems encountered in the practice of psychology.

APS advocacy in relation to the exam

The APS response to the Consultation Paper (available at: www.psychology.org.au/APS/media/pdfs/Legacy/May-2011-Submission-to-PBA-Consultation-Paper9-Guidelines-for-National-Psychology-Examinations.pdf ) was supportive of the notion of assuring competency outcomes but highlighted the very limited scope of the Psychology Board’s consultation process and alluded to a number of problems with the approach proposed.

The decision to introduce a national examination as a compulsory hurdle before general registration is without precedent in Australia and it is clear that it will have far reaching implications for all aspects of psychology’s professional education and training pathways.

Considering this, it is hard to understand why there was no opportunity for the discipline and profession to express views regarding whether an examination is needed or appropriate for the psychology profession, or to consider and debate the relative merits of alternative approaches. This problem was compounded by the very limited time frame allowed for stakeholders to respond to the Consultation Paper. It is imperative that the design of the examination now includes full consultation with the profession and discipline.

The key APS recommendations in relation to the psychology exam, which were set out in the APS submission, are summarised opposite.

Table 1. Eight entry-level capabilities in the national examination

Table 2. Four examination domains

Exempt graduates of accredited fifth and sixth year professional postgraduate degrees from re-examination

Postgraduate professional training at the Master or Doctoral level is an accredited pathway which must meet and maintain Australian Psychology Accreditation Council (APAC) standards. The APAC Standards explicitly require higher education providers to demonstrate how each of the core competencies in Table 1 are formally assessed, and require that each student must pass each of these assessments before he or she can graduate. To require re-examination of the knowledge of graduates of this pathway would be an unnecessary and unfair imposition, with additional financial and time burdens for students.

Ensure an identical standard for knowledge assessment as that required in the postgraduate pathway

The minimum required standard of performance needed to pass should be consistent with that required by accredited postgraduate professional Master and Doctoral level courses for core knowledge. This will avoid any risk that the examination will create a different standard for knowledge between the two pathways to registration.

It is worthwhile reflecting on the standardising capacity of an examination at the completion of supervision programs for four- or five-year graduates. This could allow for some increased flexibility around the requirements for provisional psychologists in such programs. The current Psychology Board procedures are having significant workforce implications, as both employers and supervisors are finding the onerous administrative demands unreasonable and are abandoning their commitment to training through this pathway. If reduced procedural demands are a consequence of the examination, then this certainly assists in justifying an examination.

Design the examination according to evidence-based standards of professional competency assessment

Considering that psychology is a scientific discipline, it is imperative that the design of any national competency examination is based on the published empirical literature regarding the assessment of professional knowledge and competency. A key feature of that literature is the principle that assessment of competence is most valid when multiple forms of assessment are employed (Kaslow et al., 2007).

Multiple choice methods alone are also subject to a number of problems in a competency environment and require very careful contextualisation and design to avoid validity problems when being used to assess judgement in solving clinical problems (e.g., Kaslow, 2009; Veloski et al., 1999). These problems are well illustrated by the evaluation of a similar and highly developed multiple choice examination in the USA, the Examination for Professional Practice in Psychology (EPPP), which has been used by the United States and Canada province psychologist licensing boards. That evaluation concluded that the EEEP’s psychometric properties, particularly its criterion and predictive validity, had not been substantiated despite very great investment in its continuing development (Sharpless & Barber, 2009).

Design multiple assessment instruments to match the four diverse purposes of the examination

To design one assessment instrument employing only one assessment method and to expect it to be a valid and reliable approach for a diverse set of purposes runs counter to principles of good assessment design. For example, overseas trained psychologists are certain to need a more comprehensive assessment process given the very limited international mutual recognition arrangements in psychology. The pathway to registration for overseas trained psychologists cannot assume the common accredited four-year undergraduate foundation which the other pathways in Australia all have in common. Multiple tailored assessment instruments testing to the same standard are required. Broaden the content to capture the diversity of psychology

The examination domains are the same as the core capabilities and attributes outlined in the Psychology Board’s two-year internship program and as such represent a very narrow “clinical” and “health care” conception of entry-level professional knowledge considering the diversity of the psychology profession. Many have expressed concern that because these domains fail to adequately capture the full spectrum of knowledge required for psychologists to practice safely in sectors other than health, an examination which tests knowledge only in these health-centric domains may not provide adequate protection for the public against practitioners with poor levels of knowledge in other domains. Involve psychology educators and the accrediting body in development and delivery of the examination.

The highly regarded work of Nadine Kaslow and her colleagues on the American Psychological Association’s Task Force on the Assessment of Competence in Professional Psychology (e.g., Kaslow et al., 2007) states that a key assessment principle is the integration of formative and summative evaluations. The national examination will be perhaps the most important summative evaluation which provisional psychologists will undertake and it is therefore crucially important that there is good integration between the courses which teach the required knowledge and its assessment by the national examination.

Though the examination is currently being created, it is hoped that the Psychology Board will consult more extensively before its introduction in 2013. The expertise and views of the higher education providers, students and the accrediting body in particular need to be engaged to ensure that the examination will be well integrated into Australia’s psychology education pathways and will serve the public well.