It has been said that testing is the seed from which psychology grew1. Psychological testing continues to be a key feature of the work of many psychologists and is a core component of psychology training. A group within the Australian Psychological Society – The Test and Testing Expert group (TTEG) – continually navigate the professions use of psychological testing. TTEG are very focused on the ethical aspects of testing, training and supervision, and the future development of online testing to name but a few current issues. In the coming months we hope to survey our profession on the use of psychological testing and their training needs. But before we do this, we have gone back to higher education to interview a current postgraduate student from Monash University, Michael Kreminski, who offers his insights from an emerging clinicians’ perspective about testing and the recent training he experienced. Many thanks Michael for your thoughts on this topic.
How did your undergraduate training prepare you for psychological testing courses at the postgraduate level?
I completed my undergraduate degree at the Australian National University. In my experience, I found this training to be weighted heavily towards the theoretical aspects of psychology. In terms of psychometric testing, this involved comprehensively covering somewhat abstract issues such as types of reliability, response biases, objective versus projective testing, presuppositions and statistical methods. Instruction on the practical application of testing was minimal, with exposure to the tests themselves being largely informal in nature. I do note, however, that many of my peers did have the opportunity to learn and practise testing skills on research participants during their honours year. However, due to the nature of my research project I did not have this experience.
Learning these tests in postgraduate training was therefore a bit of a mixed bag. The theoretical training I had received meant that I was able to quickly grasp how the tests sought to measure the various constructs, (although I cursed having to do the courses at the time!) and the statistical training I had received meant that calculation and interpretation of the obtained scores was quite straightforward.
However, as I am sure any psychologist will attest to, the obtained scores are only part of the picture – the process of how I engage with the client, how the client engages with the test and how they respond to test items makes up just as much (if not more) of my understanding of that client. Consequently, I had to quickly overcome this gap, mainly through observing and practising with my peers.
How were you assessed in the subject and what ‘gaps’ did you identify in the curriculum?
The tests to which I have been formally trained and assessed include the WAIS-IV, the WMS-IV, the WISC-V and the Personality Assessment Inventory (PAI). I have also received training on skills in general behavioural assessment and case formulation. Assessment of our skills was done in a variety of ways, but the most strenuous assessment was done for the WAIS-IV. Assessment here was done over multiple periods and consisted of:
- scoring (and norming) a pre-filled response booklet and record form
- applying the WAIS on an unsuspecting partner/friend and submitting the response booklet and the scored record form
- an Objective and Structured Clinical Examination (OSCE), consisting of applying two randomly-selected WAIS-IV subtests on a confederate, with an examiner observing and being recorded on video.
The marking criteria for these assessments was extremely strict (many of us had to re-test), however it did ingrain in us the importance of being meticulous when applying and interpreting these tests.
Throughout our training, it was reinforced that it would be impossible for a psychologist to be trained on the application and interpretation of every single psychological test. The focus of training was therefore to impart us with the skills which underpin the majority of these tests. A goal I feel was well achieved.
“Obtained scores are only part of the picture – the process of how I engage with the client, how the client engages with the test and how they respond to test items makes up just as much (if not more) of my understanding”
Where have you had the opportunity to apply your knowledge and skills in testing?
Application of these testing skills started essentially on day one in my current placement at the Turner Clinic (Monash University’s in-house training clinic). The Turner Clinic is able to offer a highly subsidised service (as little as $300 for a full cognitive assessment). Consequently, we do have a steady flow of clients seeking these assessments and offering us as clinical candidates the opportunity to hone these skills.
This application of knowledge and skills practised in the clinic goes beyond simply applying and calculating the test scores. Choosing the correct test for a particular client (there are often multiple ones to choose from), modifying the application of that test for the client as warranted, interpretation of the scores based upon both how the client presented on test-day (e.g., were they fatigued?) but also how the obtained scores relate to other tests and the obtained client history, are all critical components of test utilisation.
Further, being able to consult with both my supervisor and my peers, obtaining their input and subsequently modifying my approach throughout the assessment process, has all assisted with obtaining the clearest understanding of my clients as possible.
In what way has your approach/thinking about testing changed since commencing your studies? What adjustments did you have to make since the emergence of COVID–19?
Before commencing my postgraduate training, although I understood the wholistic approach to formulation, I do confess to holding a somewhat naïve view that by becoming proficient in a handful of ‘gold standard’ tests I would be able to rely upon these obtained scores to either make or decline any one diagnosis. Since undertaking this training I understand that this approach is inadequate, no matter how well-reviewed the test is, I cannot rely upon that score in isolation. Rather, much in the same way as developing a formulation, the diagnoses I give must be based upon the full clinical picture presented to me.
COVID-19 presents a serious challenge to the Australian community, and this includes impacting on the provision of psychological services. As I write this, Stage 4 restrictions here in Melbourne mean that the Turner Clinic has shifted to a purely online service. Although we have been able to conduct some tests online (mainly those where we are merely seeking verbal information), our concerns on being able to obtain deeper process-based information (e.g., the manner in which a client responds to a test) means that we feel our ability to conduct these assessments would be compromised by utilising digital means only. Accordingly, we have delayed some of these tests until we can physically bring the clients into the clinic and we are endeavouring to support our clients until this time.
How do you think testing is viewed by (a) your psychology colleagues in general and (b) the actual test-takers?
My colleagues do appear to hold a positive view to psychological testing, seeing this as a useful tool with which to assist in our understanding of a client. Of note, one issue which has been raised is how quickly our testing skills decline, even considering the short time since we were deemed competent on those tests. This has required us to pull the tests out in advance and reacquainting ourselves with them before applying them with a client.
The clients I have interacted with seem to have mixed views on testing. Some clients are quite skeptical of tests and may see it as a waste of their time (and dare I say money) when they would much rather focus immediately upon their presenting issues. Conversely, other clients may present purely for the sake of assessment, and may hold out high-hopes for the obtained score (e.g., when assessing a potential developmental disorder) that may be a determinant for receiving funding/support. I therefore ensure that I provide a proper rationale for testing, but also managing expectations as to what the tests may reveal (e.g., the idea of a disorder being ‘sub-threshold’).
Would you see benefit in the APS offering something beyond the training received in higher education and from test publishers? If so, what would this look like?
The skills and techniques to conduct psychometric testing do appear to be quite perishable and quickly atrophy if not utilised on regular intervals. If I did not conduct testing over a sustained period, I would certainly not feel comfortable in administering certain tests. This would be especially so where there could be secondary implications of the obtained scores (e.g., in a forensic situation). I am aware that there are workshops available for some of these tests, however, they are relatively rare as well as being quite expensive. I imagine that the infrequency and costs of these courses could present a barrier to me attending in later years.
A potential solution would be for the APS to offer these workshops in an online medium. The provision of the initial information could be done via pre-recorded videos/handouts, which could help to reduce costs. I believe that some form of objective assessment would be recommended here though to ensure that the clinician is competent at a level to administer and interpret a particular psychometric test. To this end, the clinician could record themselves administering a test/part of a test, recording this (with the client’s consent of course) and submitting both the video and their written observations to an external examiner. This would allow a clinician to obtain an unbiased measure of their own test-skills, and allow that clinician to see where they there is room for improvement.
1Weiner B Irving, Past President of American Psychological Association’s Division 5 (Evaluation, Measurement, and Statistics)