Overview

Refereees often provide lenient assessments with low inter-rater reliability and predictive validity. The challenge for recruiters is to use referee reports effectively without encouraging overly positive evaluations that fail to differentiate between candidates or predict their future performance accurately. This research examines two types of standardised rating scales used for postgraduate medical specialist program applications in Australia.

In 2023, Referee Scale #1 placed candidates into one of four percentile bands, comparing them to their peers. Most candidates were rated as above average, which is unrealistic. In 2024, Referee Scale #2 introduced the Fuzzy Graphic Rating Technique (FGRT), where referees rated candidates' typical, best, and poorest performance on each attribute. This method provided more rigorous evaluations, highlighting variability in real-world performance.

It is hypothesised that Scale #1 results would be negatively skewed due to leniency, while Scale #2's typical ratings would follow a normal distribution. Findings, limitations, conclusions, and recommendations are discussed.

Learning Outcomes

• Insight into how to improve the validity and reliability of referee ratings in selection.
• An improved understanding of the utility of the Fuzzy Graphic Rating Technique (FGRT) to gather referee ratings.

Presenter(s)

About the presenter(s)

Dr. Julie West is a senior organisational psychologist with over 30 years experience working as a specialist consultant and, prior to this, in the Australian Public Service. She is the Principal and founder of Workplace Research Associates, a management consultancy established in 1996, which specialises in the delivery of organisational psychology services.

Notes

Venue is wheelchair accessible - It is the ground floor and there is slight ramp from the street designed for wheelchairs.

Parking: There is parking on the street, on Eastlake Pde and outside the Canberra Glassworks.

Target Audience

This activity is aimed at psychologists and health professionals.

Duration of Access

This event will be recorded. The recording will be emailed to all registered within 2 weeks post event and available for viewing up to 90 days.

CPD

It is up to attendees to assess and determine how learning from this event aligns with the requirements of their learning plan. The providers accordingly do not make any representation that the event counts towards attendees’ CPD learning requirements. If an attendee determines themselves that the learning they complete aligns to their learning plan, then they can decide to count those hours towards their CPD requirements for the registration cycle.

The information in this presentation has been prepared in good faith and for educational purposes only. Therefore, the information is general in nature and should not be relied upon in the treatment of any condition and you should seek your own independent professional and/or legal advice concerning any specific issue. The APS accepts no responsibility for any errors, omissions or decisions relating to the information. The content should not be reproduced without permission or unless permitted by law.

If you are experiencing registration difficulties please contact APS Events Support.