Resident In-Training Assessment Reports(ITARs) and In-Training Evaluation Reports(ITERs)

Date of original approval: April 27, 2012
Date of last revision and approval: January 20, 2018
Date for next review: 

PGME Minimum Standards¹

OVERVIEW:

The following roles for the ITAR and ITER are consistent with the requirements outlined in the Guidelines for the Assessment of Postgraduate Residents of the Faculty of Medicine at the University of Toronto².

• ITARs and ITERs sit within a program of assessment
• ITARs and ITERs are linked to the program of assessment (e.g. goals and objectives, rotation plans, Required Training Experiences (RTEs), and Entrustable Professional Activities (EPAs))
• ITARs and ITERs are completed at defined intervals, such as at the end of a rotation or as per progress review timelines, and at least every 6 months
• ITARs and ITERs are part of a transparent approach to monitor resident progress and enable learner handover
• Prompt flags for unacceptable performance enables prompt follow-up related to performance gaps.

STANDARDS:

1. ITARs and ITERs must be integrated as one assessment method within the residency programs’ in-training evaluation system, which must:
   1. be based on the goals and objectives and/or curriculum map for the program,
   2. clearly identify the methods by which residents are to be evaluated³, and
   3. clearly identify the level of performance expected of residents in the achievement of these objectives⁴.
2. ITARs should:
   1. reflect additional assessment items that are not captured in EPAs
   2. be of reasonable length (i.e. maximum of 20 items)
   3. reflect an explicit and integrated mapping of: rotation-specific goals and objectives, Entrustable Professional Activities (EPAs), Required Training Experiences (RTEs), and specialty specific competencies and graded responsibility (i.e. appropriately varying expectations between years of training and/or development from junior to senior trainees).
3. ITERs should:
   1. be of reasonable length (max 20 items)
   2. reflect an explicit and integrated mapping of:
      1. i. rotation specific goals and objectives different practice contexts (i.e. patient populations, clinical/practice, settings)
      2. ii. graded responsibility (i.e. appropriately varying expectations between years of training and/or development from junior to senior trainees).
4. ITARs and ITERs should be coded with items pertaining to the CanMEDs framework.
5. All ratings items will be on a 5-point scale with appropriate anchors⁵ or yes/no items⁶.
   1. Each scale will provide clarity on what is an acceptable level of performance and what rating indicates a need for improvement.
   2. The use of numbers is not required on the form:
      1. i. Where questions use a 5-point numerical scale, 1 will be the lowest and 5 will be the highest.
      2. ii. Where there is a descriptive 5-point scale, the left most anchor will be the lowest and the right most anchor will be the highest.
   3. Numbers will be employed for statistical and summary purposes.
6. All forms will have 1 item that serves as the overall global performance item.
   1. This overall item will be rated on a 5-point scale.
   2. Where items use a 5-point numerical scale, 1 will be the lowest and 5 will be the highest.
   3. Where there is a descriptive 5-point scale, the left most anchor will be the lowest and the right most anchor will be the highest.
   4. This item will stand alone from other general performance questions and for systems in which the ITAR or ITER is the single definitive assessment tool for a rotation, be considered the definitive score for global evaluation analysis.
   5. The scale will provide clarity on what is an acceptable performance and what rating indicates a need for improvement.
7. Any unacceptable rating on the ITAR or ITER will trigger a flag for the Program Director or their designate.
8. All ITARs will have a section where the residents will indicate they have seen the ITAR and also have the opportunity to provide comments.

APPENDICES:

• Appendix 1: Sample ITAR
• Appendix 2: Template for In-Training Assessment Reports (ITAR)
• Appendix 3: Frequently Asked Questions (FAQ)

¹The ITAR guidelines apply to programs when they formally implement Competence by Design (CBD). Other programs may use either the ITER or ITAR guideline.

²Link: Guidelines for the Assessment of PG Residents

³General Standards of Accreditation for Residency Programs, v. 1.1, July 2017: 3.4.1.1 & 3.4.1.2

⁴General Standards of Accreditation for Residency Programs, v. 1.1, July 2017: 3.1.4.3

⁵Non-ratings questions would include questions such as the number of procedures performed, yes/no questions, and all others where the user is not asked to rate or evaluate using a set of values and anchors

⁶Approved, POWER Steering Committee, Nov 2008

⁷Link: PGME Minimum Standards: Resident In-Training Assessment Reports (ITARs) and In-Training Evaluation Reports (ITERs)

⁸Link: Learner Handover and Appropriate Disclosure of Learner Needs