The City of New York Office of the Comptroller
Bureau of Management Audit

Audit Report on the Administration of New York State Standardized Tests by the New York City Department of Education

MD08-102A
July 22, 2009

AUDIT REPORT IN BRIEF

Download the Complete Audit Report (pdf 1,333 kb)

The New York City Department of Education (DOE) provides primary and secondary education to more than 1 million pre-kindergarten to grade 12 students in over 1,400 schools.  DOE prepares students to meet grade level standards in reading, writing, and math and tests students to determine how well they are meeting these mandated learning standards.

Students in grades 3 through 8 take both the New York State standardized English Language Arts (ELA) Test and the New York State standardized Mathematics (Math) Test.  This audit focuses on the administration of ELA and Math tests for students in elementary school grades 3, 4, and 5 only.  The audit determined whether DOE has adequate internal controls over the administering of New York State standardized tests for grades 3, 4, and 5. 

Audit Findings and Conclusions

DOE has adequate internal controls with respect to ensuring that schools are familiar with established procedures when administering the New York State standardized tests at elementary schools.  In addition, the schools that we visited generally complied with the State testing guidelines, the DOE Handbook, and testing memoranda.  However, DOE lacks sufficient preventive and detective controls aimed at deterring inappropriate manipulation of test scores, which would help to ensure the overall integrity of the assessment process. 

DOE has established procedures for the administration of New York State standardized ELA and Math tests at elementary schools.  DOE provides a Handbook and distributes test memoranda to its staff in an effort to keep them informed of all required procedures in administering State and Citywide tests.  DOE also offers its staff annual training on proper methods in administering the tests as well as training of scoring staff to help identify testing irregularities when grading the long-answer portions of the exam.

We also found that for the most part, the schools that we visited complied with the State guidelines and the guidelines outlined in the Handbook. Our own review of the data and documentation collected by DOE for the 2007–2008 ELA and Math tests and our observations conducted at the sampled schools on the day of testing did not reveal any instances of cheating.  However, as more fully explained in the audit report, we cannot be assured that cheating did not occur. 

Since achieving a positive school performance rating provides an added incentive for school officials to ensure that students perform well on standardized tests, there is a potential risk for inappropriate test manipulation.  Based on our observations, we identified significant weaknesses that DOE has not addressed to help prevent or detect the manipulation of test scores.  Specifically, DOE should improve its oversight of testing monitors to ensure that they are carrying out their duties properly and are using monitoring checklists more effectively.  In addition, DOE should re-implement the use of analytics to identify possible testing irregularities and tampering and should institute stronger controls over the second and third sections of the tests.  Finally, DOE should formalize a process to ensure that substantiated allegations of cheating are shared with the Office of Accountability (OA), the office primarily responsible for coordinating yearly testing and for compliance with New York State Education Department (NYSED) testing guidelines and DOE controls over the tests.

Audit Recommendations

Based on our findings, we make 14 recommendations, 5 of which are listed below.

DOE should:

• Accurately track the assignment of testing monitors to ensure that they are being used effectively.

• Discuss with NYSED the possibility of obtaining the answer keys promptly after the administration of each test to enable DOE to perform a timely erasure analysis.  However, DOE should perform erasure analysis to identify possible improprieties regardless of when it receives the answer key.

• Compile, maintain, and track data on the number of make-up exams that are taken for the Day Two and Day Three ELA and Math exams. 

• Identify indicators to use in detecting unusual patterns that may be indicative of test tampering or irregularities and collect sufficient data to adequately track those indicators.   Based on the information collected, DOE should target those schools with unusual patterns for further follow-up.

• Ensure that the Office of Special Investigations (OSI) formalizes a process to make certain that all instances of substantiated cheating are shared with OA, so that OA can strengthen existing controls or develop new ones in an effort to prevent cheating from occurring in the future.

Agency Response

DOE officials generally agreed with the audit’s recommendations but disagreed with one of them and did not address one of them.  They also disagreed with the tone of the report.  After carefully reviewing their comments, however, we found them to be without merit.