...
- If loading warehouse data, existing data will be deleted based on the BATCH_KEY and the new data will re-inserted. This will prevent duplicate data going in.
- If reference data, the UPDATES may occur twice. For ADD and DELETE type of entries, errors may be noticed on the Spoke when loading because the data is already present or deleted.
- Note: Any run can miss some data being inserted into the table during the extract time but the situation is corrected by the next run.
- In order to completely avoid this situation, it would be necessary to suspend any process that writes to the tables from where the I2I Extract pulls the data while the extract process is in progress so that a static set is provided to the query.
...
Processing Dates in Delta Mode
On the first step of this workflow Calculate_date is resolved. A record in the PACE_MASTERDBO.ORCH_REQUEST_DEF and PACE_MASTERDBO.ORCH_QUEUE tables is created with the following value:
...
| Code Block | ||
|---|---|---|
| ||
PARAMETER_NAME = :FeedType:+'_INSTANCE' | PARAMETER_VALUE = 202812041 |
| Info |
|---|
This process is different for common extracts. For more information, see the Delta-Based Filter in the Available Filters section of EagleML and EJM wiki. |
Orchestration can be switched off for delta mode by setting in w_config_custom.inc:
...