Driver testing officer stood down by NZTA says all her tests were 'fair' and 'transparent'

A driver testing officer (DTO) who has been prohibited from testing drivers amid a major investigation by the transport agency says every test she's done was "fair" and "transparent".She says the New Zealand Transport Agency (NZTA) is "relying purely on cold statistics", and is adamant she has never "engaged in any improper activity whatsoever to pass drivers".RNZ earlier revealed 16 DTOs had been suspended with a further seven stood down following an investigation by the NZTA.The investigation relates to abnormal test results for both passenger and heavy vehicles across the country, with about 650 drivers required to re-sit tests.Do you know more? Email sam.sherwood@rnz.co.nzAmong the DTOs who were suspended was Auckland-based testing officer Michaela Howe who worked at VTNZ ABC Manukau and VTNZ Sylvia Park.Howe received notice from NZTA in July of her immediate suspension as a testing officer.In the letter, seen by RNZ, Howe was told NZTA had identified issues in relation to "significant pass rate deviations" from national averages."Your average pass rates are significantly higher than the average for Driver Testing Officers (specifically, at a significant level for candidates whose country of birth is listed as the Philippines, and at a material level for candidates whose country of birth is listed as China)."We assessed the probability of such high pass rates arising solely by chance as being very unlikely. This indicates to us that they are more likely the result of how you are conducting your tests."NZTA said Howe had the right to provide a written response by the end of July, following which NZTA would make a final decision on whether or not they would revoke her endorsement.Then, last week, NZTA contacted Howe telling her they had decided to revoke her testing officer (O) endorsement, and she was prohibited from carrying out duties as a DTO.The letter, by NZTA's head of regulatory risk, performance and insights Jodi Mitchell, said the statistical assessment was considered together with other evidence."On the information available, NZTA considers that the results are more likely to be explained by differences in how the tests were conducted rather than by chance alone; unless reliable evidence supports another explanation."NZTA believed there was a "material risk" that some driver licensing tests had not been conducted in accordance with the requirements for such tests."As a result, some candidates likely received a licence without demonstrating the required skill level," Mitchell said."This means there are potentially drivers on the road who should not be. We considered this poses a significant risk to public safety which required your immediate suspension."Mitchell said driver licence tests were designed to be "consistent throughout the country"."Because of this consistency, differences in pass rates between Driver Testing Officers should mainly reflect differences in candidate performance - not differences in how tests are conducted."Candidates from the same country are "likely to have similar driving behaviours," NZTA said."So comparing pass rates by country of birth provides a more accurate benchmark than comparing across all candidates."A "robust statistical analysis" had been applied to identify unusual pass rates.Mitchell said in the letter that Howe explained her higher pass rates were attributable to factors associated with the sites she worked at, "including the volume and demographic characteristics of applicants at those sites".NZTA did not accept her explanation, Mitchell said."If site-specific factors were the primary reason for the observed pass rate deviations, similar outcomes would be expected across Driver Testing Offices operating at the same sites and assessing the same applicant populations."However, the available data shows that your pass rates were substantially higher than the overall site pass rates at both VTNZ ABC Manukau and VTNZ Sylvia Park."For applicants born in China, Howe's pass rates were 72 percent compared with the site rate of 35 percent at ABC Manukau, and 76 percent compared with the site rate of 57 percent at Sylvia Park.In relation to applicants born in the Philippines, Howe's pass rate was 52 percent compared with the site rate of 28 percent at ABC Manukau and 77 percent compared with the site rate of 42 percent at Sylvia Park."These significant differences indicate that the elevated pass rates are not adequately explained by the characteristics of the sites or applicant cohorts alone, as those same factors applied to other Driver Testing Officers working at those locations."Accordingly, NZTA is not satisfied that the explanations provided account for the extent of the pass rate deviations identified."Mitchell said Howe had been unable to provide NZTA with an assurance she had conducted tests to the standards required, and her testing officer endorsement would be revoked.Howe's driving instructor endorsement remains suspended.Mitchell said NZTA noted Howe's "commitment to teaching people to drive", and believed she had demonstrated "genuine intent to improve if needed". Her current employer had also provided positive references in relation to her work as a driving instructor.NZTA said the suspension would be withdrawn if Howe did several things including completing an NZTA approved instructor endorsement course, provided a plan for managing conflicts of interest that may occur given her relationships and associations with DTOs, and an assurance she would abide by the manual for driving instructors.Mitchell also said Howe has the right to appeal the decision.Howe told RNZ that when she first got the letter from NZTA in July she was "completely shocked and gutted"."I've always taken my role and responsibilities incredibly seriously, so seeing that suspension come through felt like a sudden, massive blow out of nowhere."She said receiving the news last week that her testing officer endorsement had been revoked was "heartbreaking"."Knowing I am completely prohibited from carrying out my duties as a Driver Testing Officer is devastating to process. I genuinely believed I had found my calling - helping people and making a positive difference in their lives through safe driving instruction."To have that stripped away without NZTA ever sitting down to talk to me, conducting a proper audit, or giving me a single opportunity to demonstrate how I actually work with people during tests and lessons, is incredibly painful and unfair."Howe said NZTA was "relying purely on cold statistics" relating to country of birth."But those numbers completely ignore the reality of what happens on the ground. Working in high-volume Auckland testing stations like Manukau and Sylvia Park exposes you to a vastly different demographic mix than other regions, and natural statistical variations are going to happen."They are looking at raw data points while completely blanking out the practical, everyday context of our busiest sites."Howe said she had never "engaged in any improper activity whatsoever to pass drivers"."Every single test I conducted was fair, transparent, and based strictly on the driver's actual performance on the day."In response to questions from RNZ, an NZTA spokesperson said the analysis of Howe's tests found the pass rates for tests conducted by Howe were "significantly higher" than national averages for applicants from two countries."For applicants from the Philippines, those tested by Ms Howe had a pass rate of 70.2 percent, compared to a national average pass rate of 40.1 percent."For applicants from China, those tested by Ms Howe had a pass rate of 75.9 percent, compared to a national average of 47.3 percent."In relation to Howe's questioning of NZTA's use of statistical analysis, the spokesperson said NZTA considered it to be "appropriate and robust"."NZTA is not concerned with small differences in pass rates that could result from factors such as candidate performance, sample size or testing conditions."The analysis identifies pass rates that are statistically uncommon and increasingly difficult to explain through normal variation alone."The spokesperson said NZTA also engaged an external statistician to assess the likelihood of pass rates such as Howe's occurring by chance."This assessment found that Ms Howe's pass rates for applicants from the Philippines were a significant deviation from the national average, while her pass rates for applicants from China represented a material deviation from the national average."Combined with the differences between Ms Howe's pass rates and the site averages, NZTA was not satisfied that Ms Howe met the fit and proper person requirements for a testing officer, and her O endorsement has been revoked."