In Illuminate, the Core Data Automation Profiles (Admin > Core Data Automation Profile) identifies the files that are run with the nightly automation, the time the automation is supposed to run, and where to get the files for import, among other things. One other element of the automation profile is the "notification address." In the event that an import completes successfully but the system suspects that something might be wrong, Illuminate will send an email to the notification address.
At least 50% of the records were rejected
You may be importing a handful of files with your nightly automation. If at least one of those files has a 50% or higher rejection rate, then we'll send an email indicating that you may want to take a look at the you Core Data Import Logs (Admin > Core Data Import Logs). This means that if a file has 100 rows of data, you'll only be notified if at least 50 of the records were rejected. Only the rejections matter here. We are not worrying about warnings.
- incoming records: the number of records in the incoming data file
- processed records: the number of records that were actually imported after the incoming file is compared to yesterday's file (the "diff" process)
- rejected records: the number of records that were not successfully imported
- incoming file with 1000 rows is compared to yesterday's file and we identify that we need to process 10 records because of differences.
- imports process the 10 records and 6 of them are rejected
- incoming records: 1000
- processed records: 10
- rejected records: 6
- rejected / processed = 6/10 = 60% failures
- rejected / incoming = 6/1000 = .6% failures
- in reality, only .6% of the records failed ... so we will NOT send an email.