Program Evaluation

Initial teacher preparation programs for AY 2022-2023:

*Indicates programs that were reviewed and accredited by CAEP during the last accreditation cycle in 2017.

The degree program names listed below are linked to their respective pages in the NAU Catalog. The Details tab in the catalog outlines requirements that must be met to earn the degree and qualify for a state Institutional Recommendation. Completion of these requirements is confirmed by an academic advisor before the teacher candidate can receive final clearance to student teach as well as PEP (Professional Education Programs) staff prior to issuing a state Institutional Recommendation. To access each program’s Progression Plan in the catalog, open the Overview tab and click on the Progression Plan link.

College of Arts and Letters

• Art Education (BSEd)*
• Music Secondary Education (BMEd)*
  • Choral Emphasis
  • Instrumental Emphasis
• Secondary Education – English (BSEd)*
• Secondary Education – History and Social Studies (BSEd)*
• Secondary Education – Spanish (BSEd)*

College of Education

• Early Childhood Education and Early Childhood Special Education (BSEd)
• Elementary Education (BSEd)*
• Elementary Education – Certification (MEd)*
• Special and Elementary Education (BSEd)*
• Special Education – Early Childhood Special Education with Certification (MEd)*
• Special Education – Mild/Moderate Disabilities Certified (MEd)*
• Teaching Science With Certification (MAT-S)*

College of the Environment, Forestry and Natural Sciences

• Secondary Education – Biology (BSEd)*
• Secondary Education – Chemistry (BSEd)*
• Secondary Education – Earth Science (BSEd)*
• Secondary Education – General Science (BSEd)*
• Secondary Education – Mathematics (BSEd)*
• Secondary Education – Physics (BSEd)*

College of Health and Human Services

• Health Sciences – Physical Education (BSEd)*
• Physical Education Path to Certification (UCert) – New program starting in Summer 2023

Historical listing of initial teacher preparation programs

Advanced professional education programs for AY 2022-2023:

*Indicates programs that were reviewed and accredited by CAEP during the last accreditation cycle in 2017.

The degree program names listed below are linked to their respective pages in the NAU catalog. The Details tab in the catalog outlines requirements that must be met to earn the degree/certificate and qualify for a state Institutional Recommendation. To access the Program of Study, open the Overview tab and click on the Program of Study link. Completion of program requirements is confirmed by an academic advisor or faculty mentors from School Psychology or Education Leadership before the candidate is cleared to enroll in the final internship course. A second check of requirements is completed by Graduate College staff as part of the candidate’s application for graduation. Finally, the programs complete a final check of requirements prior to issuing a state Institutional Recommendation.

Educational Leadership

• Principal (GCert)*
• Educational Leadership – Principal K-12 (MEd)*
• Superintendent (GCert)*
• Educational Leadership – K-12 Administration (EdD)*

Educational Psychology

• School Psychology (EdS)*

Historical listing of advanced professional education programs

NAU’s enrollment and graduation data for all degree programs are publicly available through the Office of Strategic Planning, Institutional Research and Analytics. Listings of the initial and advanced professional education programs are included in the previous accordion, “Listings of initial and advanced professional education programs and progression plans outlining program expectations.” Candidates enrolled in initial teacher preparation programs (and other programs intended for in-service teachers) are eligible to apply for the Arizona Teachers Academy (ATA) at NAU Scholarship and Program, which is designed to support recruitment and retention of teachers for Arizona’s public schools; see NAU’s ATA website for more information about this scholarship and program.

Table 1 provides an overview of enrollment and graduation counts for initial teacher preparation programs for the past three years. Figures 1-4 provide ATA at NAU enrollment and graduation trends, respectively, by academic year and campus location. Table 2 provides enrollment and graduation counts for advanced professional education programs. Following these data charts, a narrative providing analyses, interpretations, and use of data is provided.

Table 1: Initial Teacher Preparation Programs Enrollment and Graduation Data

Figure 1: ATA at NAU Enrollment Trends by Academic Year

Figure 2: ATA at NAU Enrollment Trends by Academic Year and Campus Location

Figure 3: ATA at NAU Graduation Trends by Academic Year

Figure 4: ATA at NAU Graduation Trends by Academic Year and Campus Location

Data Sources: NAU’s website for enrollment data; PEP’s Title II reports for graduation rates; PEP’s ATA at NAU reports.

Table 2: Advanced Professional Education Programs Enrollment and Graduation Data

*There was an additional advanced professional education program in 2020-2021 (i.e., the PK-12 College and Career Counseling (MEd) program) in comparison to the previous academic year (i.e., 2019-2020).

**The number of advanced professional education programs reduced to five programs starting in 2021-2022; current and historical listings of NAU’s advanced professional education programs are included in the previous accordion, “Listings of initial and advanced professional education programs and progression plans outlining program expectations.”

Analyses, Interpretations & Use of Data

Initial teacher preparation programs:

Enrollment for initial teacher preparation programs has been relatively stable for the past three academic years with 2,136 candidates in AY 2019-2020, and 2,182 candidates in AY 2020-2021, with a slight decline in AY 2021-2022 to 2,018 candidates. NAU has also experienced enrollment declines in relation to the COVID-19 pandemic. The number of initial teacher preparation program graduates has been relatively stable across these three years (556, 521, and 599 program graduates, respectively); however a decline was noted in AY 2020-2021. PEP staff overseeing student teaching placements noted a decline in student teaching applications for AY 2020-2021 due to the COVID-19 pandemic. PEP staff have also noted a substantial increase to approximately 630 student teachers for AY 2021-2022.

Enrollment and graduation data, including ATA at NAU enrollment and graduation data, are shared internally with program faculty and leadership through a PEP Initial Teacher Preparation Coordinating Council meeting and through the COE/PEP Recruitment and Retention faculty and staff committee. The past three years of enrollment and graduation data are also shared in programs’ internal biennial reports completed during the fall semesters of even years. Additionally, these data were shared with external stakeholders as part of the NAU PEP’s Annual Career Fair, which is attended by 70 to 90 districts and schools from across Arizona.

There has been significant growth in ATA at NAU due to state approved funding of the program beginning in AY 2019-2020. ATA at NAU enrollment grew from 59 and 161 ATA at NAU members in AYs 2017-2018 and 2018-2019, respectively, to 900, 951, and 1,014 ATA at NAU members in AYs 2019-2020, 2020-2021, and 2021-2022, respectively. Interest in the ATA continues to be high, and NAU distributes the full award allocation provided to NAU by the Arizona Board of Regents. In AY 2020-2021, NAU initially expended all funding and maintained an ATA waitlist. When funding was increased in Spring 2021 because of recent, voter-approved ballot initiatives, students on the waitlist were approved for ATA at NAU and provided appropriate scholarship funding.

Similar to the enrollment numbers, the ATA at NAU graduation trend data show an increase. The number of ATA at NAU graduates grew from 26 ATA at NAU graduates for the first cohort graduating in AY 2018-2019 to 265, 305, and 371 ATA graduates in AYs 2019-2020, 2020-2021, and 2021-2022, respectively. ATA data reports are shared internally with program faculty and leadership through the PEP Initial Teacher Preparation Coordinating Council as well as through public reports published by the Arizona Board of Regents. ATA’s data reports are available on the ATA at NAU website.

Advanced professional education programs:

Enrollment for NAU’s advanced professional education programs has declined from 914 enrolled in AY 2019-2020 to 877 enrolled in AY 2020-2021. Despite the decline in enrollment, the number of candidates graduating from NAU’s advanced programs remained relatively stable with 273 graduates in AY 2019-2020 and 266 graduates in AY 2020-2021. One explanation for the decrease in enrollment is a technical reporting issue in that NAU needed to move from a single academic plan with multiple emphases to separate academic plans. This change impacted enrollment numbers for two of our advanced programs. That is, the Educational Leadership (EdD) – K-12 Administration Emphasis and Elementary Education (MEd) Reading K-8 Emphasis became separate academic programs in AY 2020-2021, which allows us to report enrollment and graduation data for only these emphases. In AYs 2018-2019 and 2019-2020, these emphases were part of a larger academic plan with multiple emphases, which resulted in only being able to report enrollment and graduation data at the program level and not for these specific emphases. NAU has also experienced enrollment declines in relation to the COVID-19 pandemic. Further, the number of advanced professional education programs reduced to five programs starting in 2021-2022 based on CAEP’s Scope of Accreditation; current and historical listings of NAU’s advanced professional education programs are included in the previous accordion, “Listings of initial and advanced professional education programs and progression plans outlining program expectations.” For these five advanced professional education programs in AY 2021-2022, there were 685 candidates enrolled and 141 candidates graduated from these programs. Enrollment and graduation data are shared internally with program faculty and leadership through the COE/PEP Recruitment and Retention faculty and staff committee. The past three years of enrollment and graduation data are also shared in programs’ internal biennial reports completed during the fall semesters of even years.

NAU recognizes the value of evaluating how graduates contribute to P-12 student learning and growth, and apply the professional knowledge, skills, and dispositions from their program of study as a new teacher. To meet this challenging expectation, NAU PEP works in conjunction with the AZ K12 Center, a center sponsored by NAU’s College of Education, to collect meaningful data regarding program completer effectiveness from NAU graduates participating in the Arizona Teachers Academy (ATA).

ATA provides a scholarship for any remaining tuition and mandatory fees once all gift aid is applied for students enrolled in an eligible teacher education program. ATA participants commit to teach one year in an Arizona public school district, charter school, or Bureau of Indian Education school for any portion of a year they are an ATA member. In addition to providing scholarship funding, the State of Arizona also funded mentoring/induction services for ATA graduates during their first year of teaching in an Arizona school. NAU’s ATA graduates receive mentoring/induction services from the AZ K12 Center. ATA graduates are allowed a one year grace period before beginning their teaching service commitment. This delay may influence if or when the graduate applies for mentoring/induction services with the AZ K12 Center.

Given the significant number of initial teacher preparation program candidates enrolled in ATA, NAU is gathering data from this sub-population in relation to CAEP Standard R4.1 . NAU PEP and AZ K-12 Center leadership and staff meet regularly to increase coordination of services and implement effective communication to student teachers and graduates. Some strategies implemented to more effectively enroll ATA graduates into the AZ K12 Center mentoring/induction services include adding marketing messages outlining the range of mentoring/induction options to the ATA Welcome Letter, and including representatives of the AZ K-12 Center at the NAU PEP’s Annual Career Fair. The data collection instruments are aligned to:

R4.1 Completer Effectiveness – The provider demonstrates that program completers:

• effectively contribute to P-12 student-learning growth AND
• apply in P-12 classrooms the professional knowledge, skills, and dispositions that the preparation experiences were designed to achieve.

To provide evidence that NAU graduates “effectively contribute to P-12 student-learning growth,” NAU PEP is exploring the use of the New Teacher Center’s Analyzing Student Learning instrument, which is one tool the AZ K-12 Center provides for mentors to use in their work with beginning teachers. The Analyzing Student Learning tool was developed to create habits of mind when reviewing student work in relation to student learning or performance goals and content standards. This tool allows teachers to reflect on the strengths their students have in relation to a set criteria based on standards. Next, teachers identify challenge areas and consider ways to support students to make categorical growth. The Analyzing Student Learning tool supports the scaffolding process for reviewing assessment data related to student learning, creating immediate next steps to address the various student learning needs and setting expectations for the lesson plan criteria. Mentors use coaching language to find entry points into a conversation with beginning teachers to show how to make connections between teacher efficacy and student performance. The AZ K-12 Center is collecting the Analyzing Student Learning from mentors who worked with NAU graduates. Due to the COVID-19 pandemic, many Arizona schools/districts restricted on site visits in classrooms, which limited the use of the Analyzing Student Learning tool by mentors. The AZ K12 Center was able to share four Analyzing Student Learning samples for our graduates from AY 2021-2022. For AY 2022-2023, the AZ K12 Center was able to collect more Analyzing Student Learning samples (for 10 NAU graduates) as COVID-19 restrictions by districts/schools regarding on site visits to classrooms are lifted. A narrative providing analyses, interpretations, and use of these data is provided below.

To seek evidence related to NAU graduates applying “the professional knowledge, skills, and dispositions that the preparation experiences were designed to achieve,” NAU PEP and AZ K12 Center Staff have developed a Beginning Teacher Effectiveness Observation Instrument aligned to this CAEP Standard component. This instrument was designed to be completed by mentors in collaboration with their mentees/NAU graduates. A pilot administration of the instrument was conducted in Fall 2021. There was a very low response rate to the survey in Spring 2022, and therefore these results are not shared nor summarized below. Measures were taken by NAU PEP and the AZ K12 Center to increase response rates for this survey by mentors in collaboration with their mentees/NAU graduates in Fall 2022 and Spring 2023. These results are available through the links below. Following these result files, a narrative providing analyses, interpretations, and use of data is listed.

• Beginning Teacher Effectiveness Observation Results from Fall 2021 (Pilot), Fall 2022, and Spring 2023

Analyses, Interpretations & Use of Data

Analyzing Student Learning Results from AY 2021-2022 (Pilot Year) and AY 2022-2023

In AY 2021-2022 (pilot year), the Analyzing Student Learning tool was used by mentors with four NAU AY 2020-2021 graduates. The four Analyzing Student Learning tool samples were for mathematics lessons at the 1st, 3rd, 5th, and 7th grade levels. In completing the Analyzing Student Learning tool, all four NAU graduates completed the Standards-Based Content and Informing Instructional Practice sections of the tool describing the standard(s) including the knowledge, skills, and actions targeted in the lesson, how the targeted standard(s) are addressed in the lesson, and what student performance would look like for meeting standard(s). Two of the samples also demonstrated that NAU graduates completed the Analyzing Student Learning: Student Performance Levels section of the tool where they rated each of their students’ level of performance as well as calculated the number and percentage of students meeting standard(s), approaching standard(s), and far below standard(s). When completing this analysis of their students’ performance, both NAU graduates found that the majority of their students met the lesson standards (25-48%) or earned a score(s) related to approaching the lesson standards (12-37.5%). Very few students were found to be far below standards (8-12.5%). Additionally, it was noted that a performance rating could not be provided for some students at the time the Analyzing Student Learning tool was completed.

In AY 2022-2023, the Analyzing Student Learning tool was used by mentors with 10 NAU AY 2021-2022 graduates who graduated from the following NAU initial teacher preparation programs: Elementary Education (BSEd), Secondary Education – English (BSEd), Secondary Education – Mathematics (BSEd), and Special and Elementary Education (BSEd). One NAU graduate completed the Analyzing Student Learning tool four times with their mentor, two times for a mathematics lesson and two additional times for a reading lesson. Overall, the 13 Analyzing Student Learning tool samples were completed for lessons for the following grade levels/subject areas: kindergarten mathematics, kindergarten ELA, kindergarten reading, 1st grade mathematics, 3rd grade ELA, 4th grade ELA, 5th grade ELA, 9th grade Algebra I, and 10th grade English. Twelve of the samples demonstrated that 9 of the NAU graduates completed the Student Performance Levels section of the tool, and overall found that the majority of their students exceeded the lesson standards (6-56%), met the lesson standards (13-37%) or earned a score(s) related to approaching the lesson standards (11-59%). Overall, 13-97% of their students were found to be far below standards; it should be noted that the 67% was for one NAU graduate that completed the tool with a small group of three students as opposed to the whole class. Additionally, it was noted that a performance rating could not be provided for some students at the time the Analyzing Student Learning tool was completed. Finally, all 10 NAU graduates for all 9 samples completed the Analyze Student Learning: Areas of Strength, Analyze Student Learning: Areas of Need, Reflect and Plan, and Next Steps portions of the tool where they reflected on assessment data related to student learning as well as immediate next steps to address the various student learning needs and set expectations for the lesson plan criteria.

The AZ K12 Center will continue to collect, annually, the Analyzing Student Learning from mentors that use it with our ATA at NAU graduates. In AY 2022-2023, the AZ K-12 Center was able to collect significantly more Analyzing Student Learning samples (as COVID-19 restrictions by districts/schools regarding on site visits to classrooms were removed) as well as encourage mentors to utilize the Analyzing Student Learning: Student Performance Levels, Analyze Student Learning: Areas of Strength, Analyze Student Learning: Areas of Need, Reflect and Plan, and Next Steps portions of the tool.

Beginning Teacher Effectiveness Observation Survey Results from Fall 2021 (Pilot), Fall 2022, and Spring 2023

As noted above, the Beginning Teacher Effectiveness Observation Instrument was completed by mentors in collaboration with their mentee teachers (i.e., ATA at NAU graduates) to measure “the professional knowledge, skills, and dispositions that the preparation experiences were designed to achieve.” When completing this survey, mentors had several opportunities to observe mentees’ classroom practices and debrief with their mentees. Additionally, mentees had time in the classroom as a beginning teacher to recognize their strengths and areas for improvement. For the Fall 2021 pilot, survey responses were received from 12 mentors and mentees (i.e., NAU graduates) from AY 2020-2021. These mentees/graduates represent NAU’s undergraduate teacher preparation programs in Elementary Education, Secondary Education – English, Special and Elementary Education, Health Sciences – Physical Education, and Music Secondary Education. Based on these pilot survey results, Learning Environments (InTASC Standard 3), Planning for Instruction (InTASC Standard 7), and Learning Differences (InTASC Standard 2) were areas our graduates demonstrated effective teaching practices. The area that respondents indicated our graduates were struggling or lacked confidence in relation to their teaching practices was Classroom Management.

For the Fall 2021 pilot, survey responses were received from 12 mentors and mentees (i.e., NAU graduates) from AY 2020-2021. These mentees/graduates represent NAU’s undergraduate teacher preparation programs in Elementary Education, Secondary Education – English, Special and Elementary Education, Health Sciences – Physical Education, and Music Secondary Education. In Fall 2022, survey responses were received for 19 ATA at NAU graduates in AY 2021-2022 from the following NAU initial teacher preparation programs: Elementary Education BSEd (42%), Special and Elementary Education BSEd (26%), Elementary Education – Certification MEd (5%), Music Secondary Education BMEd (5%), Secondary Education – General Science BSEd (5%), Secondary Education – History and Social Studies BSEd (5%), Secondary Education – Mathematics BSEd (5%), and Spanish Education MAT (5.3%). Finally, in Spring 2023 survey responses represent 24 ATA at NAU graduates in AY 2021-2022 from the following NAU teacher preparation programs: Elementary Education BSEd (42%), Special and Elementary Education BSEd (29%), Early Childhood Education (4%), Elementary Education – Certification MEd (4%), English -Secondary Education MA (4%), Secondary Education – General Science BSEd (4%), Secondary Education – History and Social Studies BSEd (4%), Spanish Education MAT (4%), and Special Education – Mild/Moderate Disabilities Certified MEd (4%). The Beginning Teacher Effectiveness Observation Survey in Spring 2023 was completed for 10 ATA at NAU graduates that this survey was also completed for in Fall 2022.

Across these three data cycles, respondents indicated that Learning Environments (InTASC Standard 3) was by far the strongest teaching practice area of our graduates (75-92%). In Fall 2021, the next strongest area indicated by respondents was Planning for Instruction (InTASC Standard 7; 67%); while in Fall 2022 and/or Spring 2022, Content Knowledge (InTASC Standard 4; 58% in Fall 2022; 63% in Spring 2023) and Dispositions (58% in Fall 2022) were additional areas our graduates demonstrated effective teaching practices. The area that respondents indicated, across these three data cycles, our graduates were struggling or lacked confidence in relation to their teaching practices was Classroom Management (46-67%).

The Beginning Teacher Effectiveness Observation Instrument will continue to be administered by the AZ K12 Center in October and March. The decision to administer this survey during these months was a collaborative decision by NAU PEP and the AZ K12 Center to allow the mentor and mentee enough time working together to provide meaningful responses. Additionally, in conjunction with completion of this survey, mentors collaborate with their mentees on goals based on a combination of InTASC standards, the district’s evaluation rubric, and priority indicators through the New Teacher Center of the AZ K12 Center. The AZ K12 Center has found that waiting to administer this survey allows for richer conversation that is authentic to the knowledge, skills, and disposition of that beginning teacher. In addition to publicly posting results on this NAU PEP Program Evaluation: CAEP Accountability Measures web page, these results along with data from other instruments (e.g., student teaching evaluation measures) and input from stakeholders (e.g., Cooperating Teachers feedback) are shared internally with program faculty and leadership through the PEP Initial Teacher Preparation Coordinating Council meetings. Program faculty and leadership review these data to determine if any program improvements are needed and if so, what curriculum changes should be made.

Initial teacher preparation programs:

NAU PEP is gathering employer satisfaction data from school and district personnel who attend NAU PEP’s Annual Career Fair. Results are available through the links below. Following these results files, a narrative providing analysis, interpretation, and use of data is listed.

• Career Fair Survey for District Participants, Results from Spring 2023
• Career Fair Survey for District Participants, Results from Spring 2021-Spring 2022
• Career Fair Survey for District Participants, Results from Spring 2018-Spring 2020

Analysis, Interpretation & Use of Data

As noted above, employer satisfaction data for initial programs are collected from school/district personnel who attend NAU PEP’s Annual Career Fair. Due to the COVID-19 pandemic, the Spring 2021 Career Fair was held virtually using Handshake. The percentage of respondents indicating that our candidates were prepared for their interviews with them has been consistently high at 94% (in Spring 2019) to 100% (in Spring 2020), though the percentage decreased to 78% in Spring 2021. The change from an in-person event to a virtual format may account for the decreased percentage of respondents indicating that our teacher candidates were prepared for their interviews. This may be due in part to the on site coaching regarding how to discuss and describe professional terms and instructional strategies, etc. that is provided to candidates by the PEP Director and other faculty, advisors, and staff that occurs prior to and between interviews. The percentage of respondents indicating that our candidates were prepared for their interviews with them increased to 83% in Spring 2022.

For the past four years, based on interviews with candidates, respondents have indicated dispositions (e.g., attitude, openness for students to learn, confidence, ethics, etc.) and pedagogy were strengths of our candidates’ preparation. Technology and content were also identified as candidates’ strengths. Areas for improvement identified by respondents across the past four years were working with special populations (e.g., IEPs, laws, differentiation, etc.), assessment (e.g., collect, manage, and analyze assessment data), and classroom management. In addition to publicly posting results on this NAU PEP Program Evaluation web page, results are shared internally with program faculty and leadership through a PEP Initial Teacher Preparation Coordinating Council meeting.

NAU transitioned to using the NExT (Network for Excellence in Teaching) Common Metrics Instruments starting in AY 2021-2022. In addition to an Exit Survey and a Transition to Teaching Survey (administered to program completers approximately one-year post-graduation), the NExT Common Metrics Instruments include a Supervisor Survey to collect employer satisfaction data about program completers for their first-year teaching. NAU will utilize the Supervisor Survey for the first time in Summer 2022. All of these NExT surveys are aligned with each other as well as the InTASC standards. The NExT Surveys are proprietary instruments and have evidence of validity and reliability.

Advanced professional education programs:

With the approval of the Advanced CAEP Standards, the NAU PEP Advanced Coordinating Council members developed an Employer Survey for advanced programs. The Employer Survey was piloted in Spring 2022, though the response rate for this pilot administration was low. These pilot results are available through the link below. Following this results file, a narrative providing analysis, interpretation, and use of data is listed. Due to the COVID-19 pandemic, the Advanced Programs Employer Survey was not administered in Spring 2021. For the Spring 2022 administration of the survey, program leadership and faculty employed other strategies for locating advanced program graduates’ employers in an effort to improve response rates as well as implementing systematic reminders. Spring 2022 data results will be extracted in summer and analyzed by programs in Fall 2022.

• Employer Survey Results Spring 2020

The EDL program leadership and faculty are also exploring other methods for collecting employer satisfaction data, including conducting focus group interviews with partner districts. One focus group interview was held in Fall 2021; ( see Focus Group Protoc0l); preliminary analysis of data results is provided below.

Analysis, Interpretation & Use of Data

Spring 2020 Survey Data Analysis

Across NAU’s advanced professional education programs, 54 employers completed the Employer Survey during the pilot administration in Spring 2020. Of the 54 employers that responded, only 29 employers (or 56%) indicated they had hired NAU advanced professional education program graduates during the past three years. Based on the survey items, these 29 employers appear to have hired NAU graduates representing all advanced professional education programs. However, the number of responses per program were low ranging from 1 to 11 responses. The percentage of responses from these 29 employers indicating that they find NAU advanced program completers competent and worthy of hiring was high with 52% of respondents indicating “always” and 39% indicating “most of the time.” Employers also indicated they would hire another NAU advanced program graduate with 86% of respondents indicating they were “extremely likely” and 24% indicating they were “likely.” The percentage of employers indicating they were satisfied with the overall preparation of NAU advanced program graduates for their position was high with 70% indicating they were “extremely satisfied” and 26% indicating they were “satisfied.” Further results related to advanced program completers’ knowledge, skills, and dispositions by position/program can be found in the results file listed above. Findings from this pilot employer survey were shared internally with program faculty and leadership.

Fall 2021 Educational Leadership Focus Group Data Analysis

Employers from one district who participated in a Fall 2021 focus group were complimentary of the hands-on, relevant degree program which enabled graduates to enter district positions with a degree of familiarity of processes and procedures. They indicated that the graduates hired in their district were well-prepared and “have a good understanding of what’s expected of them as an administrator”. The district participants in the focus group encouraged the continuance of the partnership and offering of the degree program locally, which is specifically geared toward producing administrators with the knowledge and skills germane to the district while maintaining the high preparation standards of NELP. Focus groups will be conducted in Spring 2022 with other districts to determine their level of satisfaction with our graduates.

Initial teacher preparation programs:

NAU PEP has established two key approaches to improve our collection of feedback from key stakeholders, cooperating teachers (CTs) and principals/directors.

Student Teaching Cooperating Teacher Survey: For AY 2021-2022, additional feedback items (see General CT Additional Items; Math/Science Additional Items) were added to the mid-term administration of the Student Teaching CT Survey instruments. These new items share data results from the respective student teaching evaluation instruments from the previous academic year and request feedback regarding these data results in relation to the candidate the CT is mentoring. The items also request general feedback regarding communication, training materials, and placement processes/procedures.

• Data Collection: Fall 2021 was the pilot administration with CTs who were mentoring student teachers in 16-week placements (N = 187); in Spring 2022, additional items were administered to all CTs as part of the mid-term evaluation of student teachers in 8-week and 16-week placements. For AY 2022-2023, the additional items were administered to all CTs as part of the mid-term evaluation of student teachers in 8-week and 16-week placements. Links to the data results from Fall 2021, Spring 2022 and Fall 2011 are provided below. The data collected during AY 2022-2023 will be analyzed and interpreted in Summer 2023.
  • • Summary Data Results from Fall 2022:
      • Early Childhood Education/Elementary Education
      • Special Education
      • Secondary Education
    • Summary Data Results from Spring 2022:
      • Early Childhood Education/Elementary Education
      • Special Education
      • Secondary Education
    • Summary Data Results from Fall 2021:
      • Early Childhood Education/Elementary Education
      • Special Education
      • Secondary Education

Teacher Preparation Advisory Council: A Teacher Preparation Advisory Council was formed in AY 2021-2022 with the first meeting convening in Spring 2022. CTs and principals/directors were invited to participate through the additional instrument items and through direct emails to CTs and principals/directors who frequently accept NAU student teachers. Participation in the Teacher Preparation Advisory Council was approved by the Arizona Department of Education as qualifying for “Serving in a leadership role of a professional organization related to the profession of teaching or the field of public education” (see Educator Certification: Renew Your Certification | Arizona Department of Education (azed.gov), Professional Development Activities).

• Spring 2023: In February/March 2023, a total of 9 cooperating teachers and 1 principal/director participated in the meeting aligned to their position and/or discipline area. Participants represented schools and districts in Flagstaff as well as other Arizona communities where NAU delivers teacher preparation programs including Chandler, Lakeside and Apache County, Prescott, and Holbrook. The data presentations from Spring 2023 for each group are linked below.
  • Early Childhood/Elementary/Special Education
  • Secondary Education
• Fall 2022: In October 2022, the Teacher Preparation Advisory Council met, and a total of 10 cooperating teachers and 2 principals/directors participated in the meeting aligned to their position and/or discipline area. Participants represented schools and districts in Flagstaff as well as other Arizona communities where NAU delivers teacher preparation programs including Williams, Tucson, Deer Valley, Lakeside and Apache County, Yuma, and Gilbert. The data presentations from Fall 2022 for each group are linked below.
  • Early Childhood/Elementary/Special Education
  • Secondary Education
• Spring 2022: In February/March 2022, a total of 6 cooperating teachers and 3 principals/directors participated in the meeting aligned to their position and/or discipline area. Participants represented schools and districts in Flagstaff as well as other Arizona communities where NAU delivers teacher preparation programs, such as Yuma and Glendale. The data presentations from Spring 2022 for each group are linked below.
  • Early Childhood Education/Elementary Education
  • Special Education
  • Secondary Education

 Analyses, Interpretations & Use of Data

Although we have consistently connected with school and district partners through NAU PEP’s Annual Career Fair, NAU PEP sought to improve our collection of feedback from cooperating teachers and principals/directors for program improvement. Results from the Cooperating Teacher Survey as well as feedback noted through the Teacher Preparation Advisory Council meetings have been shared with programs.

At the unit level, the PEP Initial Teacher Preparation (ITP) Coordinating Council acted upon feedback from cooperating teachers who indicated that the scale for the Cooperating Teacher Survey items did not allow for the option of communicating to teacher candidates that while they observed a candidate’s growth/development, there was still room for improvement. This feedback was shared with the Teacher Preparation Advisory Council members as noted in the presentations. The members were in support of this recommended change to the scale to improve feedback to teacher candidates and quality of data collected and reported. At the March 2022 ITP Coordinating Council Meeting, the change to the scale for the Cooperating Teacher Survey instruments was approved for implementation beginning in AY 2022-2023.

Given the data results, the PEP staff who oversee student teaching are working on ways to improve communication regarding expectations to cooperating teachers/principals/directors. A concise handout summarizing key aspects of the mentor teacher role was developed and provided to cooperating teachers/principals/directors beginning in Fall 2022. Expanded details regarding the cooperating teacher role is described in the Teacher Candidate handbook.

At the Fall 2022 meeting of the Teacher Preparation Advisory Council, a cooperating teacher commented that the orientation that they received for mentoring a student teacher in a mid-term-starting 8-week placement was less comprehensive than for the 16-week, start-of-term placement. In response, the PEP Director of Student Teaching emphasized the importance of the orientation meeting between the candidate, university supervisor, and cooperating teacher at the training session for Spring 2023 supervisors. At the Spring 2023 meeting of the Teacher Preparation Advisory Council, PEP shared a proposed change of the Alerts Instrument used for NAU’s initial teacher preparation programs including in student teaching. The attendees offered very positive feedback about a potential instrument, and they suggested some additional considerations for the instrument. These suggestions have been shared with the ITP Coordinating Council Alerts Subcommittee.

Advanced educator programs:

Principal (GCert & MEd) and Superintendent (GCert & EdD)

Educational Leadership faculty members who coordinate the Principal and Superintendent programs participated in several data-gathering efforts regarding satisfaction of employers and stakeholder involvement. These included a focus group in Fall 2021, as well as a stakeholder survey in Spring 2022.

A focus group held in Fall 2021 provided rich data from stakeholders with one partner school district offering an accelerated MEd Education Leadership – Principal K-12, which was co-constructed and implemented between NAU and the district. The stakeholders from the partner school district noted the professionalism of the program, the rigor, as well as its effectiveness. As stakeholders, they expressed interest in keeping the program relevant, real-life, and hands-on. They noted hiring program graduates as a mutual benefit. Other benefits discussed included the ability to augment syllabi to include what candidates want to know more about, the camaraderie of the program, and the practicality of the degree. Given their familiarity with the EdD Educational Leadership-K-12 Administration degree program, focus group participants also discussed this program. Participants described the cohort nature of the program, as well as the ability to draw on one’s experiences while working through the program as two strengths.

A stakeholder survey in Spring 2022 (N = 36) yielded a 44% response rate. Responses were received from school and district leaders, community partners, faculty members, candidates, past graduates, and emeriti faculty. Through the survey results, respondents indicated they had been involved in program design (37.5%), continuous improvement (31.25%), and program evaluation (31.25%). Program design examples included reformatting and implementing new syllabi, developing the American Indian School Leadership Program, supporting accreditation work, and designing new courses. Examples of continuous improvement efforts included providing programmatic feedback, acknowledging the role of readily accessible mentors for part-time faculty who teach in the programs, and improving upon programs such as the American Indian School Leadership Program.

A focus group (N = 6) with another Arizona district was held in Summer 2022. This focus group consisted of elementary and high school principals and members of the superintendency. Additionally, some members were program completers and employers. The Summer 2022 stakeholder responses yielded information regarding their appreciation about the flexibility of the programs and the importance of realizing that candidates are juggling lives of the principalship and superintendency, as well as ways to solicit information for improvement. One program completer stakeholder noted, “I could be able to do all the things I wanted to do in my profession, but then still be able to take the program.” The importance of the cohort model was also emphasized in assisting one another supportively. One participant indicated that asking for program improvement suggestions from district mentors in the program may be beneficial. Program completers as stakeholders mentioned the length of classes in the accelerated programs, both the pros and cons. Most moving, though, may have been the candidate (now administrator) stakeholder who said, “I like the fact that they didn’t give up on me when I had given it up on myself.”

School Psychology (EdS)

In AY 2021-2022, a stakeholder survey was developed and administered to school/district mentors who work with EdS School Psychology candidates. The results of this survey yielded 33 responses, an 18% response rate. Most school/district respondents had participated as a mentor for internship candidates completing an embedded assessment, the Competency Evaluation of Practicum Field/Internship Experiences (CEFE) (85.71%) and one respondent had participated in program evaluation (14.29%). The CEFE is a comprehensive appraisal of the development of candidates’ skills in practice and is organized by each of the 10 NASP 2020 Domains of Practice. Under each of the domains, there are key skill areas that relate to each of the domains. CEFE data, along with ratings on other internship signature assignment submissions and other available data, guide decisions regarding a candidate towards recommending progression to internship semester two or program completion. Survey respondents indicated participating in evaluating interns, supporting placements of practica candidates with supervisors who in turn evaluated the intern, supervised interns in schools, coordinated interns and practicum candidates, had interns in the past, and/or provided university proctors with feedback regarding candidate preparation. Program evaluation comments included evaluating the preparation of interns and practicum candidates.

Following the pilot administration in AY 2021-2022, and to maximize the quality of stakeholder feedback, in AY 2022-2023 the survey protocol was administered to stakeholders and employers through focus groups rather than through emailed surveys. In Summer 2022, the stakeholder survey was administered in a focus group at the June meeting of the Arizona Association of School Psychology. The group included individuals who had supervised practicum candidates and interns as well as employers who had hired graduates of the NAU School Psychology program. In Fall 2022 and Spring 2023, another focus group was conducted with lead school psychologists from the following Arizona school districts: Scottsdale, Deer Valley, Roosevelt, Cottonwood, Gilbert, and Humbolt. These districts represent urban and rural districts as well as Title I schools and affluent suburban districts. The stakeholders indicated that the candidates were extremely well prepared, have high integrity, and are responsible individuals. The stakeholders also noted that the cohort model adopted by the School Psychology program was very effective.

Initial teacher preparation programs:

NAU PEP monitors candidate progression and competency from admission to a teacher preparation program (i.e., meeting Additional Admission Requirements specified in the catalog) through completion through the following practices and measures (CAEP Standards R3.2 and R3.3):

Established, documented policies and procedures that outline the steps followed by academic advisors when working with teacher preparation candidates. These policies and procedures align with the PEP Data Management System (in Salesforce, a university supported application) and the associated online applications and automated email communications for PEP admission and student teaching.

• Summary: In relation to the policies and procedures and specified requirements, three examples of changes include refining catalog language related to the calculation of admission GPA(s), implemented in the AY 2020-2021 catalog; modifying student teaching requirements to address multiple state and university trainings that are now required, implemented in the AY 2021-2022 catalog (see https://nau.edu/pep/required-training-modules/); and refining catalog language related to the English composition admission requirements for graduate programs for initial teacher preparation, approved in AY 2022-2023 to be implemented in the AY 2023-2024 catalog. Work related to established policies and procedures are tracked by the EPP in an evidence file (see Evidence File: Policies & Procedures for Monitoring Candidate Progression and Competency from Admission through Completion). Section 5 Data of this file provides updated and approved policies and procedures. Data documents continued verification of students teachers’ meeting catalog and student teaching course requirements. When student teachers do not meet course requirements, an appropriate grade (i.e., Incomplete, Fail, or Withdraw) is entered in accordance with university policies.

Student teaching requires successful completion of several instruments that measure candidate competency related to InTASC Model Core Teaching Standards. The primary student teaching observation instruments are:

• National Institute for Excellence in Teaching (NIET) Aspiring Teacher Rubric (used for NAU’s teacher preparation programs except the Secondary Education Math and Sciences programs)
  • Summary: The Aspiring Teacher Rubric (ATR) is a proprietary instrument developed by the National Institute for Excellence in Teaching (NIET). NAU PEP selected the ATR as the student teaching evaluation instrument due to the explicit alignment to the InTASC Standards (CAEP Standard R1). It is a validated instrument that requires training and certification to support the consistent implementation and reliability of the data collected. The AY 2020-2021 and 2021-2022 ATR data results were shared internally with the Initial Teacher Preparation (ITP) Coordinating Council and with Cooperating Teachers (key stakeholders) through new feedback items added to the instrument that mentor teachers complete. These items regarding these evaluation results were piloted in Fall 2021 and fully implemented in Spring 2022. The raw data results and data analysis from these additional items as well as open ended comments provided by Cooperating Teachers were provided to ITP Coordinating Council members and program coordinators at the March 2022 ITP CC meeting to review further and determine if any instructional or program improvements are needed and if so, what curriculum changes should be made. Additionally, ATR data are reported at the program level. Work related to the ATR (including examples of use of data from program level assessment report files) are documented by the EPP in an evidence file (see Evidence File: Student Teaching Evaluation: ATR Summary & Data ).
• University of Texas Austin UTeach Observation Protocol (used for NAU’s Secondary Education Mathematics and Sciences initial teacher preparation programs)
  • The UTeach Observation Protocol (UTOP) is a proprietary instrument developed by the University of Texas at Austin. NAU PEP selected the UTOP as the student teaching evaluation instrument for its secondary education mathematics and sciences programs due to the explicit alignment to the InTASC Standards (CAEP Standard R1). It is a validated instrument that requires inter-rater reliability training to support the consistent implementation and reliability of the data collected. The AY 2019-2020, 2020-2021, and 2021-2022 UTOP data results were shared internally with the Initial Teacher Preparation (ITP) Coordinating Council and with Cooperating Teachers (key stakeholders) through new feedback items added to the instrument mentor teachers complete. These additional items regarding these evaluation results were piloted in Fall 2021 and fully implemented in Spring 2022. The raw data results and data analysis from these additional items as well as open ended comments provided by Cooperating Teachers were provided to ITP Coordinating Council members and program coordinators at the March 2022 ITP CC meeting to review further and determine if any instructional or program improvements are needed and if so, what curriculum changes should be made. Summary information regarding the UTOP and data analysis are documented by the EPP in an evidence file (see Evidence File: Student Teaching Evaluation: UTOP Summary & Data Analysis).

Arizona has several requirements for teacher preparation programs that are intended to ensure candidate competency as well.These program requirements align with CAEP Annual Reporting Measure 3, Ability of completers to meet licensing and state requirements, and Title II Reports (Initial R3.3/Advanced RA3.3). These requirements include:

• Passing score on appropriate Professional Knowledge licensure exam(s) (see https://nau.edu/coe/test-preparation-resources/);
• Completion of Youth Suicide Prevention Training (see https://nau.edu/pep/required-training-modules/;
• Completion of a state approved Structured English Immersion (SEI) course; NAU’s state approved SEI courses include: 1) BME 331W: Structured English Immersion in Early Childhood Settings, 2) BME 430: Methods and Materials in Second Language Teaching and Structure English Immersion, 3) BME 631: Structured English Immersion and Sheltered English Content Instruction, 4) BME 437: Structured English Immersion Methods for Secondary School, 5) BME 537: Structured English Immersion Methods for Secondary School, and 6) ENG 406: ELL Methods and Materials for Secondary Teachers; for courses listed as a Major Requirement for NAU’s initial teacher preparation programs, such as these state approved SEI courses, a Grade of “C” or better is required; completion of the SEI requirement is also marked on the graduate’s Institutional Recommendation;
• Completion of 6 credits in literacy and the science of reading for some of NAU’s initial teacher preparation programs (i.e., BSEd Early Childhood & Early Childhood Special Education; BSEd Elementary Education; BSEd Special & Elementary Education; MEd Elementary Education – Certification; MEd Special Education – Mild/Moderate Disabilities Certified; and MEd Special Education – Early Childhood Special Education), which lead to a Literacy, K-5 endorsement in elementary, special education, and early childhood initial teacher preparation programs;
• Completion of a course or state-approved exam on the US and Arizona Constitutions: NAU offers POS 220: Federal and Arizona Constitutions, which is available for both undergraduate and graduate teacher preparation candidates; evidence of completion of a course to meet the US and Arizona Constitutions requirement is confirmed by the academic advisor and then marked on the candidate’s Institutional Recommendation, which is provided to the Arizona Department of Education for the graduate’s application for teacher certification.

Advanced educator programs:

NAU’s advanced professional education programs that currently fall within CAEP’s scope policy include Educational Leadership (Principal GCert, Educational Leadership – Principal K-12 MEd, Superintendent GCert, Educational Leadership – K-12 Administration EdD) and School Psychology (School Psychology EdS). These programs follow prescribed curricula tied to respective national standards (School Psychology program/National Association of School Psychologists [NASP]; Principal and Superintendent programs/National Educational Leadership Preparation [NELP]).

Entry into these programs requires adherence to NAU’s Graduate College admission criteria, as well as individual program admission requirements: School Psychology EdS; Principal GCert, Educational Leadership – Principal K-12 MEd, Superintendent GCert, Educational Leadership – K-12 Administration EdD).

Candidates are guided by faculty and advisors throughout their program of study (POS), which ensures proper class sequencing and mentoring. Programs provide handbooks (e.g. School Psychology Student Handbook) and/or other appropriate guiding documents that are shared internally with their candidates to assist with their progression. Additionally, NAU’s Graduate College monitors candidate progress through grade reports. When candidates fall below grade thresholds, academic improvement plans are constructed to assist the candidates toward success. For example, in the School Psychology EdS program, a review of academic and professional progress occurs for each candidate at the end of each semester. Candidates also formally meet with advisors on an annual basis to discuss their progress in the program. At that meeting, the candidate reviews the rating form and comments related to their progress in the areas of didactic coursework; research skills and progress; clinical skills and progress; assistantship performance; interpersonal skills and professionalism; self-awareness, self-evaluation, and critical thinking skills; other accomplishments and/or concerns; and status on graduation requirements. Similar processes and procedures are followed for the EDL Principal and Superintendent programs. In particular, the EDL Principal and Superintendent programs check for successful completion of program requirements prior to final clearance to begin the internship course and again at the conclusion of the program of study before the state Institutional Recommendation is issued to the graduate.

Initial teacher preparation programs:

Licensure exam pass rates

Note: These data are based on Title II reports filed by the EPP. Title II reports are due in April or May each year and report the previous academic year’s data. Licensure exam pass rates are not yet available for AY 2021-2022 program graduates, we expect these data to become available in Summer 2023. Statewide Average Pass Rate data become available two years later, typically in the spring semester. Statewide Average Pass Rate data for AY 2021-2022 are not yet available, we expect these data to become available in Spring 2024. Scores are only reported when there are 12 or more candidates.

Table 1. Overall Assessment Pass Rates reported to Title II (Traditional Programs)

Table 2. Overall Assessment Pass Rates reported to Title II (Alternative Programs)

Table 3. NAU’s Average Pass Rates for NES Professional Knowledge Exams (Required Exam for Teacher Certification), AY 2019-2020 thru AY 2021-2022

Analyses, Interpretations & Use of Data

For the past four reported years (i.e., AYs 2018-2019, 2019-2020, 2020-2021 and 2021-2022), licensure exam pass rates for initial teacher preparation programs reported to Title II have been high at 96-99% for traditional programs. These high licensure exam pass rates for NAU’s initial program candidates are higher than the state’s average pass rate of 91% for traditional programs for data available to date.

For alternative pathway programs, there were fewer than 12 candidates completing the licensure exams in AYs 2017-2018 and 2018-2019 therefore pass rates are not reported publicly. The pass rate data have been shared with the program faculty. For AY 2019-2020, NAU had no alternative pathway program graduates. There were 34 NAU alternative pathway program graduates in AY 2020-2021 with 85% of these alternative pathway program graduates passing their licensure exams; the statewide average pass rate for alternative pathway program graduates was 90% in AY 2020-2021. In AY 2021-2022, there were 35 NAU alternative pathway program graduates with 94% of these alternative pathway program graduates passing their licensure exam; Statewide Average Pass Rate data for AY 2021-2022 are not yet available from Title II, we expect these data to become available in Spring 2024.

Table 3 documents NAU’s average pass rates for the NES Professional Knowledge Exams for AY 2019-2020 thru AY 2021-2022. The Arizona Department of Education requires a passing score on the appropriate Professional Knowledge Exam for teacher certification. As noted in the chart, NAU’s average pass rate for the three years reported met or exceeded the state’s average pass rate for the same three reporting years.

PEP’s licensure exam pass rates are reviewed each spring, especially in relation to available state pass rate data, for all EPP programs as well as at the individual program level. The results are shared internally with program faculty and leadership through the PEP Initial Teacher Preparation Coordinating Council and program review assessment report files. Additionally, the average pass rates for the Professional Knowledge Exams (Table 3) are shared with candidates through an Arizona Teacher Licensure Exams: Information and Preparation Resources webinar and web page (see https://nau.edu/coe/test-preparation-resources/). The webinar has been offered twice each year since it was piloted in AY 2021-2022.

State requirement

Youth suicide prevention training is a state requirement passed by the Arizona legislature for teacher training programs to provide suicide awareness and prevention instruction to all teacher candidates (see Statue 15-1656. Suicide prevention training). The awareness teacher candidates gain through this suicide prevention training is important preparation for assisting their future students. This state required training also aligns to CAEP Standards R1.4, “candidates engage in professional learning” and R3.2, “the provider creates and monitors transition points from admission through completion that indicate candidates’ professional responsibilities.” Additionally, this evidence aligns to CAEP Standard R3.3, “the provider ensures candidates possess academic competency to teach effectively with positive impacts on diverse P-12 student learning and development” and as noted in the CAEP Annual Report for Measure 3 (R3.3) “data that reflect the ability of EPP candidates to meet state requirements.” Information and data reporting regarding this state requirement is being tracked and maintained in an evidence file maintained internally by the EPP.

Summary Spring 2021 and AY 2021-2022: Arizona EPP’s were asked to implement this new state requirement during AY 2020-2021. During Fall 2020, NAU researched various options and selected a state approved, free, online training option developed by the Society for the Prevention of Teen Suicide (see https://sptsuniversity.org/). It was determined that the “Making Educators Partners in Youth Suicide Prevention: ACT on FACTS” was the best option in terms of being a state approved training and accessible by NAU teacher preparation students located at campus sites throughout the state. This required professional development training was launched in Spring 2021 and assigned to all teacher candidates officially admitted to an initial teacher preparation program. Reminder emails to complete the required training were sent throughout the semester and additional efforts were made to follow-up through phone calls as well as email messages.

In Spring 2021, our data results demonstrated a 96% completion rate. In Fall 2021, we had a 100% completion rate and in Spring 2022, we had a 97% completion rate. These data results were shared with the PEP Initial Teacher Preparation Coordinating Council and the determination was made to continue to administer this state requirement centrally with the support of PEP staff. Additionally, EPP leadership worked with university staff who oversee updates to the catalog, and the catalog language beginning in AY 2021-2022 was modified for all of the EPP’s initial teacher preparation programs to indicate the Youth Suicide Prevention Training as well as the Professional Dispositions Modules are required for clearance for student teaching.

Title II Reports

Traditional Programs, 2021-2022

Alternative Programs, 2021-2022

Traditional Programs, 2020-2021

Alternative Programs, 2020-2021

Traditional Programs, 2019-2020

Alternative Programs, 2019-2020

Traditional Programs, 2018-2019

Alternative Programs, 2018-2019

Traditional Programs, 2017-2018

Alternative Programs, 2017-2018

Advanced professional education programs:

Principal (GCert & MEd) and Superintendent (GCert & EdD)

The Principal and Superintendent programs are the only advanced programs that require completion of a licensure exam for certification. Listed below are the number of program graduates by Principal and Superintendent programs. This information is provided for comparison with aggregate pass rate results for the total number of NAU candidates in Tables 3, 4, 5 and 6.

AEPA/NES principal and superintendent exams pass rates

AY 2021-2022, Program Graduates

• Principal Graduate Certificate & MEd Educational Leadership – Principal K-12 programs = 92
• Superintendent Graduate Certificate & EdD Educational Leadership – K-12 Administration programs = 29

AY 2020-2021, Program Graduates

• Principal Graduate Certificate & MEd Educational Leadership – Principal K-12 programs = 134
• Superintendent Graduate Certificate & EdD Educational Leadership – K-12 Administration programs = 15

AY 2019-2020, Program Graduates

• Principal Graduate Certificate & MEd Educational Leadership – Principal K-12 programs = 132
• Superintendent Graduate Certificate & EdD Educational Leadership – K-12 Administration programs = 42 *one student completed both programs simultaneously and is counted twice

Table 3. Overall Assessment Pass Rates, aggregated for the Principal Sub-test 1

Table 4. Overall Assessment Pass Rates, aggregated for the Principal Sub-test 2

Table 5. Overall Assessment Pass Rates, aggregated for the Superintendent Sub-test 1

Table 6. Overall Assessment Pass Rates, aggregated for the Superintendent Sub-test 2

*Note: Scores are only reported when there are 12 or more candidates. Additionally, NAU only has access to results if candidates note that NAU may receive the results when registering for the exam.

Analyses, Interpretations, & Use of Data

For advanced professional education programs, only the principal and superintendent programs require a licensure exam for certification. The number of principal candidates passing these licensure exams has been high with a 84-97% pass rate for both principal licensure sub-tests over the last three years (i.e., AYs 2019-2020, 2020-2021 and 2021-2022). For the superintendent advanced programs, there have been fewer than 12 candidates completing the licensure exams therefore pass rates are not reported publicly. The pass rate data have been shared with the program faculty. The principal and superintendent program’s faculty analyzed and interpreted these data as well as worked on next steps in relation to these findings as part of their NELP (National Educational Leadership Preparation) program review reports submitted in September 2021. For example, based on the pass rates for program graduates for the principal and superintendent degree programs, Educational Leadership faculty are continuing to monitor the licensure exam data, especially as more data becomes available for the MEd and Graduate Certificate Principal program graduates. Further, to respond to the lower response rates (i.e., data results faculty are able to access), faculty remind candidates about taking the AEPA exam as soon as possible post-graduation and ask they select NAU when completing the exam registration form. The reminders use a variety of methods such as including this information in the internship syllabus and by asking university supervisors to share this information again during the final meeting for the candidate’s internship portfolio.

School Psychology (EdS)

All candidates in the EdS School Psychology program must complete the National School Psychology Examination for School Psychologists (i.e., the ETS PRAXIS II #5402 exam). Although Arizona does not require a licensure exam, NAU candidates are required to take the PRAXIS ll exam prior to completing internship and evidence of the exam completion must be received by the Educational Psychology Department for a candidate to be issued an Arizona Institutional Recommendation (IR) upon graduation. Additionally, to become a National Certified School Psychologist (NCSP), applicants must achieve a passing score (147) on this exam. NAU’s AYs 2018-2019, 2019-2020 and 2021-2021 results (N = 46) indicate that 100% of graduates passed the PRAXIS II at the NCSP level with some graduates far exceeding the minimum criterion. These results support candidate attainment of the relevant standards and are a strong indication that NAU’s graduates have mastered the information presented throughout the program.

Initial teacher preparation programs:

Employment rate

Arizona continues to experience a severe teacher shortage and ranks first in the United States in terms of turnover rate of teachers (Carver-Thomas & Darling-Hammond, 2017). In 2017, Arizona’s turnover rate was calculated as 24% compared to the average turnover rate among all states which was 10% (Carver-Thomas & Darling-Hammond, 2017). For the teacher openings in Arizona needing to be filled for the 2023-2024 school year, as of September 2023, approximately 30% of these Arizona teacher positions across the state remain unfilled and an additional 53% of these Arizona teacher positions were “filled by individuals not meeting standard teacher requirements (using alternative methods)” (Wing, Wilson, Rossini, & Goodwin, 2023). The annual US Department of Education Teacher Shortage Areas report further documents teacher shortages for the state of Arizona. Given the significant teacher shortage Arizona schools/districts continue to face, NAU PEP hosts an annual Career Fair to connect employees with NAU candidates during the final semester of their program of study as well as with recent NAU graduates. Below are marketing flyers for the past three years.

• March 24, 2023 Career Fair
• February 18, 2022 Career Fair
• February 12, 2021 Virtual Career Fair

NAU also collects self-report employment data from candidates through the T-PREP Exit Survey administered at the conclusion of student teaching. The survey items ask if the candidate has already been offered a teaching position for the next school year. Starting in AY 2021-2022, NAU transitioned to using the NExT (Network for Excellence in Teaching) Common Metrics Instruments, which include an Exit Survey (to collect satisfaction data from candidates near the end of their final/student teaching semester) and a Transition to Teaching Survey (administered to program completers approximately one-year post-graduation); the NExT Exit Survey replaced the T-PREP Exit Survey. The NExT Common Metrics Instruments also includes a Supervisor Survey to collect employer satisfaction data about program completers for their first-year teaching. All of these NExT surveys are aligned with each other as well as the InTASC standards. These NExT Surveys are proprietary instruments and have evidence of validity and reliability. Additionally, NAU and the state’s third party vendor Heartland ECSI (Educational Computer Systems, Inc.) collect employment verification data for the subset of NAU candidates who were members of the Arizona Teachers Academy (ATA) at NAU. These employment data are collected for ATA at NAU members in order to verify completion of their ATA teaching service commitment, and they started being collected in December 2020 for AY 2019-2020 ATA at NAU program completers.

Analysis, Interpretation & Use of Data

T-PREP Exit Survey data for the ability of our candidates to be hired have been relatively consistent. At the end of student teaching, the percentage of candidates responding to that question indicating they had already been offered a teaching position for the next school year was 46%, 48%, and 53%, and in AYs 2018-2019, 2019-2020, and 2020-2021, respectively. Results are shared internally with program faculty and leadership through a PEP Initial Teacher Preparation Coordinating Council meeting. Due to a data collection issue with the implementation of the NExT Exit Survey, candidates were not asked if they had already been offered a teaching position for the next school year on the AY 2021-2022 NExT Exit Survey administered in Fall 2021 and Spring 2022; this was corrected for the AY 2022-2023 NExT Exit Survey with questions about whether candidates already have teaching positions for the next school year being added to the survey. NExT Exit Survey results from AY 2022-2023 will be available to be reviewed and shared publicly via this webpage in Summer 2023.

Additionally, NAU’s 2016 Self-Study Report provided employment data requested from the Arizona Department of Education (ADE). In March 2021, NAU worked with ADE to approve a new data sharing request contract. In Spring 2021, NAU PEP completed a new data request for employment data for initial teacher preparation program graduates from the past 5 years. The requested employment data for initial teacher preparation program graduates was received from ADE in Fall 2021. NAU PEP is currently working with NAU’s Strategic Planning, Institutional Research, and Analytics (SPIRA) Office on aggregating and reporting for this massive, raw data file. NAU PEP plans to analyze and interpret these employment data and share findings with stakeholders during AY 2023-2024. Finally, in September 2023, ADE is scheduled to launch a new data management program; this program will allow the state to track employment data of teachers that is tagged to higher education institutions and teacher education programs. When this program is launched, PEP will be able to request accessible and usable state employment data.

Advanced professional education programs:

NAU PEP collects self-report employment data from advanced professional education program candidates through the Exit Survey administered near the end of the candidate’s final semester in the program. The survey items ask if the candidates from advanced programs have already been hired in education positions for which they were prepared.

Analysis, Interpretation, & Use of Data

The results for AYs 2019-2020 and 2021-2022 showed 76% and 70% of advanced program candidates, respectively, indicated they had a teaching or other education professional position that they would begin or continue upon graduation. The latest results from AY 2021-2022 showed an increase to 80% of advanced program candidates with positions. The results are shared internally with program faculty and leadership.

As noted above, in March 2021, NAU worked with ADE to approve a new data sharing request contract. NAU PEP has submitted an employment data request to ADE to obtain Arizona employment information for NAU’s advanced program completers for the past five years; these data will allow for tracking of the employment of our advanced program graduates working for a public school or district in Arizona. The requested employment data for advanced program graduates was received from ADE in Fall 2021. NAU PEP is currently working with NAU’s Strategic Planning, Institutional Research, and Analytics (SPIRA) Office on aggregating and reporting for this massive, raw data file. NAU PEP plans to analyze these employment data and share findings with stakeholders during AY 2023-2024.

Initial teacher preparation programs:

NAU PEP surveys teacher preparation program candidates regarding their satisfaction with their degree program at the conclusion of the student teaching course. The Exit Survey instrument (T-PREP) was developed by a group of teacher preparation researchers at Arizona’s three state universities. NAU PEP also surveys graduates using a New Teacher Survey instrument adapted from the Exit Survey. It is distributed to program completers once, approximately one year after graduation. The links below are for results files for the past four data cycles; administrations of the New Teacher Survey data are presented with the corresponding Exit Survey results for comparison purposes. Following these results files, a narrative is presented providing analysis, interpretation, and use of data.

• Exit Survey Results (AY 2019-2020 Graduates) & New Teacher Survey Results (Spring 2021 New Teachers)
• Exit Survey Results (AY 2018-2019 Graduates) & New Teacher Survey Results (Spring 2020 New Teachers)
• Exit Survey Results (AY 2017-2018 Graduates) & New Teacher Survey Results (Spring 2019 New Teachers)

Analysis, Interpretation & Use of Data

For the initial programs Exit Survey administered in the candidates’ final semester, the response rates were 63%, 78%, and 72% for 2017-2018, 2018-2019, and 2019-2020, respectively. For the New Teacher Survey administered approximately one year after graduation, the response rates were 16%, 13%, and 20% for Spring 2019, 2020, and 2021 respectively. Satisfaction data from the Exit and New Teacher Surveys have been consistently high for these three cycles of data. The percentage of respondents indicating they would recommend their program to a future teacher has been 85-91% of respondents indicating they strongly agreed or agreed. During these three data cycles, the percentage of respondents grading initial program experiences with a grade of A or B have also been consistently high in relation to the quality of classes (89-93% of Exit Survey respondents and 72-85% of New Teacher Survey respondents), practicum experiences (80-91% of respondents), and student teaching experiences (85% or more of respondents). Additional narrative related to the high program ratings as well as areas for improvement identified can be found in the results files linked above.

These results are shared internally with program faculty and leadership through the PEP Initial Teacher Preparation Coordinating Council meeting and with external stakeholders participating in the Annual Teacher Career Fair. Additionally, Exit Survey and New Teacher Survey results from AY 2020-2021 related to areas where our candidates indicated they felt somewhat unprepared (i.e., navigating school politics; working with school administrators; and explaining student performance data to parents/guardians, students, and families) were shared through the Teacher Preparation Advisory Council meetings held in February/March 2022 to gather input from external stakeholders (i.e., Cooperating Teachers and Principals/Directors). Feedback noted through the Teacher Preparation Advisory Council meetings have been shared with programs to review further and determine if any instructional or program improvements are needed and if so, what curriculum changes should be made.

NAU transitioned to using the NExT (Network for Excellence in Teaching) Common Metrics Instruments starting in AY 2021-2022. The NExT Common Metrics Instruments include an Exit Survey (to collect satisfaction data from candidates near the end of their final/student teaching semester) and a Transition to Teaching Survey (administered to program completers approximately one-year post-graduation). The NExT Common Metrics Instruments also includes a Supervisor Survey (to collect employer satisfaction data about program completers for their first-year teaching). All of these NExT surveys are aligned with each other as well as the InTASC standards. The NExT Exit and Transition to Teaching Surveys are proprietary instruments and have evidence of validity and reliability.

Advanced professional education programs:

The NAU PEP Advanced Coordinating Council members developed and piloted an Exit Survey with candidates in their final semester in Spring 2018. This common instrument is now being administered centrally across all advanced programs to candidates at the conclusion of both the fall and spring semesters. Beginning in Spring 2019, the Exit Survey was adapted to create a Graduates Survey, a common instrument to survey graduates one year after graduation across all advanced programs. Due to the COVID-19 pandemic, the Advanced Programs Employer Survey was not administered in Spring 2021. For the Spring 2022 administration of the Graduates Survey, leadership and faculty for advanced programs are employing other strategies for locating advanced program graduates’ contact and employment information in an effort to improve response rates; one strategy being used is utilizing the employer information provided by ADE for advanced program graduates. Spring 2022 data results will be extracted in summer and analyzed by programs in Fall 2022. In the links below, administrations of the Graduates Survey data are presented with the corresponding Exit Survey results for comparison purposes. Following these results files, a narrative providing analysis, interpretation, and use of data is listed.

• Exit Survey Results 2020-2021
• Exit Survey Results 2019-2020
• Exit Survey Pilot Results 2018-2019, Graduates Survey Results Spring 2020
• Exit Survey Pilot Results Spring 2018, Graduates Survey Pilot Results Spring 2019

Analysis, Interpretation & Use of Data

For the pilot administrations of the Exit and Graduates surveys from Spring 2018 and Spring 2019, the response rates were 34% and 5%, respectively. The Exit Survey response rate was 36% for AY 2018-2019, and the response rate for the Graduates Survey was 8% in Spring 2020 (for advanced program graduates from AY 2018-2019). In AYs 2019-2020 and 2020-2021, the Exit Survey response rates were 35% and 43%, respectively. The percentage of respondents indicating they would recommend their NAU advanced program to others has been consistently high for these data cycles with 90-100% of respondents indicating they were “extremely likely” or “likely.” During these data cycles, quality of program ratings have also been consistently high in relation to the overall quality of their NAU advanced program (84-98% of respondents) and satisfaction with their preparation at NAU to pursue/continue their chosen profession (82-95% of respondents). Additional narrative related to the high quality of advanced program ratings as well as areas for improvement identified can be found in the results files listed above.

Beyond publicly posting results on this NAU PEP Program Evaluation web page, results are shared internally with program faculty and leadership.

The NAU PEP website includes an Exploring the Profession web page. This page provides information regarding degree programs offered, cost of attendance, student loan default rates, average teacher salaries, and resources for future teachers.

Federal Student Aid, an office of the US Department of Education, provides the following report of the most recent three year trend data for Northern Arizona University:

Analysis, Interpretation & Use of Data

Federal Student Aid reports the FY 2018 national cohort default rate is 7.3 percent. NAU’s FY 2018 default rate was 4.6 percent or 2.7 percentage points below the national cohort rate. NAU’s student loan default rate as compared to the national cohort default rate will continue to be reviewed each spring and updated results reported publicly through this web page.