In which case is it necessary to submit a data request?
Data requests are necessary in order to receive extant student-level data; receipt of data requires a Non-Disclosure Agreement once the request has been approved. Researchers should not ask schools directly for student-level data. Research involving human subjects or data collection requires additional IRB approval.
Why do I need to go through this process to receive data?
The NYC DOE is governed by the following relevant laws and regulations:
- Family Education Rights and Privacy Act (FERPA)
- New York Education Law § 3012-c(10)
- New York State Education Law § 2-d
- DOE Chancellor’s Regulation A-820
- NYC DOE Parents’ Bill of Rights
Can I request data from school staff?
No. All request for NYCDOE data must be submitted to RPSG. School staff (teachers, principals, administrators, etc.) cannot provide data to requestors.
Can you provide me with DOE staff contact information?
No. We do not provide researchers with school staff contact information for recruitment purposes.
Publicly Available Data
Where can I find publicly available data?
Check the NYC Data webpage to get the most recent publicly available data.
Which datasets are available to researchers?
Datasets available to researchers can be found in the list below. All datasets are linkable. Please note that student data at the DOE are housed in multiple databases across multiple offices. Some data are owned by offices other than the Research & Policy Support Group (RPSG), which is the team that manages data requests. Data from DOE offices outside of RPSG may take longer to compile or may not be available at all.
Datasets for the 2017-18* school year and earlier years are currently available for request.
*Note: Not all datasets for the 2017-18 school year are available. Some data, like graduation results, will not be available until later in Winter 2019.
Data Type to Sample File Name ("School Year" is unique to your inquiry)
- Audited Register to "School Year_biog1031.csv"
- June Biographic to "School Year_june-biog_PK-12.csv"
- Advanced Placement (AP) to "School Year_AP-Exams.csv"
- PSAT to "School Year_PSAT.csv"
- SAT to "School Year_SAT.csv"
- ELA and Math to "School Year_student_test-Biog_All_G38_NYC.csv"
- Fitnessgram to "School Year_Fitnessgram.csv"
- Graduation (NYC) to "CLSchool Year"
- Graduation (NYS) to "School Year_Grad_StateCalc.csv"
- NYSESLAT to "School Year_NYSESLAT.csv"
- NYS Science to "School Year_Science_G4&8.csv"
- NYS Social Studies to "School Year_SocStudies_G4&8.csv"
- Regents to "School Year_Regents.csv"
- Zoned DBN to "School Year_ZonedDBN_ALL.csv"
- Pedagogue to "School Year_PedagInfo_All.csv"
Process for Requesting Data
What information do I need to provide in my data request?
The completed data request form must provide a clear and concise overview of the research design and the intended educational value of the proposal, as well as the specific benefit to the NYC educational community (i.e., the DOE). Data requests requiring identifiable data must contain:
- Consent forms signed by parent/guardian
- List of unique identifiers for data matching, including:
- Student name
- Date of Birth
- Student ID Number
Requests missing above information will be automatically rejected. In addition, requests for data not yet available will not be approved. Requestors will need to resubmit once data are available.
Where do I find the data request form?
Why are some requests approved while others are rejected?
Approval of requests is based on the completeness of the data request; expected educational value or benefit to the NYC education community as a result of the study; the availability and regulations governing access to data; and the amount of staff time and other DOE resources required to execute the request.
Matching DOE student-level data
Which unique identifiers are required to match student-level data?
Researchers interested in matching DOE student-level data for program evaluations are required to provide the following identifiers:
- Student Identification Number (OSIS) OR
- Student first and last name and;
- Date of Birth and;
- School DBN
Matches will only be conducted once. If requestors are dissatisfied with matching results, they will have to initiate a new data request.
Identifiable Data and Consent
Do I need consent forms?
Consent forms are required for identifiable data. Identifiable data are data that include student name, student ID, and/or date of birth. Some data also become identifiable based on number of students in a specific sub-demographic group (typically, the DOE considers students in a group of five or fewer to be identifiable).
What do I need to include when a project requires consent forms?
Consent forms need to specify all data and years being requested. Consent forms that contain generic, unspecified requests for student-level data will not be accepted. Please see our sample Data Release Consent Form for guidance. Please note that consent forms approved by the NYCDOE IRB are not automatically approved by the data request committee.
Once you receive approval of the data request, you need to provide cataloged consent forms, following the format below:
- Each consent form must be labeled with a number.
- Provide a spreadsheet with:
- Consent form number
- Student last name
- Student first name
- Student date of birth
- DBN of student's school
- Unique identifiers (such as program flag, if applicable)
- Consent forms must be sorted in the same order as they are numbered in the spreadsheet
For an example, see our Consent Form Catalog.
Scanned PDF copies of every consent form along with the spreadsheet catalog need to be transmitted via FTP before data can be transferred. Consent forms and catalog not provided in the format above will be denied.
Timing and Ways to Receive Data
When will I receive my data?
Processing time can take 4-6 weeks and varies depending on the complexity of the request. Please be aware this is an estimated timeline and we cannot make any guarantees to meet external researchers’ deadlines.
Requests requiring any of the following will take an additional 4-6 weeks:
- Consent forms
- Data matching (appending new data to a researcher’s existing data)
- Creation of new datasets
- Data from other DOE offices (datasets not listed above)
- Multiple datasets
Data will be transferred once all requested datasets are available, not piecemeal.
Can you send me files via DropBox?
No. Data can only be transmitted via a secure File Transfer Protocol (FTP), which is provided by the NYCDOE. Data must be encrypted and the computers accessing the data must be secure. Instructions will be provided during the data request process, once the request has been approved. The DOE can provide researchers with an FTP connection if necessary.
Do I need to be a U.S.-based researcher to receive data from the NYC DOE?
Yes. Our legal jurisdiction does not extend beyond the United States of America.
Who should I contact with more questions?
Additional questions should be sent to RPSGresearch@schools.nyc.gov. A response will be provided within five business days.