This lesson will provide a guide to data validation with your data imports.
Overview
If you're here, you're probably seen some data successfully imported into your Illuminate instance and you're on your way through the data integration process! There is still work to be done. Now that data has been successfully extracted and imported into Illuminate, you still need to check the data accuracy. While the data might have been successfully imported, files may be missing key fields or needed data transformations. Since Illuminate is a different system, there could be past data entry practices that may not match with the corresponding Illuminate field. This next step, Data Accuracy Validation, is a key part of the data migration and implementation as a whole.
Whether you're implementing ISI, DnA, or ISE, data accuracy is very important. The accuracy of the data in Illuminate will directly impact reporting through your systems. This is critica lin ISI because of state reporting implications.
Data Validation Guides to Check Accuracy
To help with this process, we've created a series of documents that will guide you through common data validation processes for each commonly imported data file. While these documents will attempt to guide you through the accuracy validation of many key data points within each file, these documents should not be viewed as the one-stop-shop for validation. In addition to following the recommendations found in the documents, you will want to add your own validation techniques, such as creating your own data check reports or having key end-users visually validate data.
Data Validation Process
The data validation process can be broken into two different sets of data: Core Data Sets and ISI Data Sets.
Below is a bit more information on the sets of data that are imported and links to each individual data validation document that will guide you through the process.
If you're new to the implementation process and would like some more context to how data integration in Illuminate works, check out this article about the Data Integration Overview.
The Core Data files consist of the key files which must be brought over into Illuminate in order to use the system in any capacity. As such, these files will contain a majority of the key state reported fields as well as other commonly used fields at sites (homeroom teacher, bell schedules, etc.). Confirming the accuracy of this data will be key to both the success of your data migration and the implementation as a whole. This accuracy validation process should begin early in the data process as adjustments to your extract and importing file might be required to correct various errors or data anomalies. Depending upon the file, there will be more data accuracy checks required.
- Sites
- Terms
- Courses
- Users
- Studemo
- Enrollment
- Mastschd
- Roster
Best Practice Scenario For example, the Studemo.txt will have substantial more checks need than Sites.txt due to the fact that Studemo.txt has more data points brought over than Sites.txt. Take your time when performing all validation checks as time spent now to find an error will save the headache of discovering the data inaccuracy later in the implementation.
The Additional ISI Data Files, sometimes referred to as the "ISI Data Spec" or the "Extra Data Spec" are a series of files that add a wealth of information to your Illuminate installation. These data files are not required for DnA or ISE installations, so they are sometimes considered "Extra." They can be imported into any installation but they are absolutely critical in ISI. Again, like the Core Data Files, confirming the validity of this data is very important. Files like Programs.txt and Transcripts.txt have significant impact on student graduation and program participation. Below is a list of the validation documents for various ISI data files:
- Roles
- Programs
- Contacts
- Transcripts
- Language
- Specialed
- Attendance
- Behavior
Validation is Verification
Keep in mind, that this process is not always a one-size-fits-all solution. The key is that you're looking at this data and validating it. You'll work with the data team to address issues, you'll work with the implementation team, you'll work internally with your own teams to address things ... it's a fluid process. Make sure to undertake it early and take it very seriously. Good data is the foundation for a good implementation!
Next Steps
Ready for some more file imports? Check out Data Import Tools to review other data files.