Page Comparison

This article explains how timestamps work with DeltaMode in I2I. 

With I2I run in delta mode, the prior run's timestamp is retrieved and cached.  When the query is complete, the timestamp being cached is saved for the next run.

Therefore, any records updated will be picked up as a part of the following run, thus ensuring that all data gets to Local PACE.

Note: As a result of this change, some data extracted in the first run will also be pulled in the second run. However, at the time of loading into Spoke:

• If loading warehouse data, existing data will be deleted based on the BATCH_KEY and the new data will re-inserted. This will prevent duplicate data going in.

• If reference data, the UPDATES may occur twice. For ADD and DELETE type of entries, errors may be noticed on the Spoke when loading because the data is already present or deleted.
  Note:  Any run can miss some data being inserted into the table during the extract time but the situation is corrected by the next run.

• In order to completely avoid this situation, it would be necessary to suspend any process that writes to the tables from where the I2I Extract pulls the data while the extract process is in progress so that a static set is provided to the query.

...

Processing Dates in Delta Mode

On the first step of this workflow Calculate_date is resolved. A record in the PACE_MASTERDBO.ORCH_REQUEST_DEF and PACE_MASTERDBO.ORCH_QUEUE tables table is created with the following value:

PACE_MASTERDBO.ORCH_REQUEST_DEF.CORRELATION_ID = 'GWDELTA_'+:DeltaBaseUID:

• GWDELTA_ is the common  prefix for delta processes

• :DeltaBaseUID: is a unique ID for the process

Fromdate value is  generated or extracted from the ORCH table

...

PARAMETER_NAME = :FeedType:+'_INSTANCE' | PARAMETER_VALUE = 202812041 

Orchestration can be switched off for delta mode by setting in w_config_custom.inc:

:W_DO_NOT_USE_ORCH_FOR_DELTA: = 'Y'

And old logic will be used. 

Expand
title Click to read about Last Run Date logic for I2I (obsolete):
What is the Last Run Date? The last run date is the value of the datetime stamp of the last launch of a certain I2I process.  title Info Note Despite a similar name, the delta last run time for I2I and a deltaBased filter for extracts are two different concepts. The last run date value is populated from system date on the Spoke at the moment the TSR from the Hub has arrived (this way it understands that the extraction is complete (it does not matter whether any data was extracted or not)). The following format is applied: Code Block Convert( Today(), 'yyyy-mm-dd') +' '+Convert( CurrentTime(), 'hh-mm-ss' ) Where is the Last Run Date Stored? EJM core takes the current date and time on the Spoke and writes it into CodeValue.   You can check it in the List Code Value panel (/eagle/star/reference/pan-lcodeval.htm) as shown below: Info title Note The code category is EAGLE I2I DELTA. Image RemovedImage Added The Code Value is 'i2i_d' (const prefix) + SpokeID + I2IProcessUID (set in the related i2i solution in IWS). As an example, the result of the lookup looks like this: Image Modified The Code Value Description (1177) is the value of the datetime stamp of last launch of the I2I process. The datetime stamp is linked to the I2I workflow (specifically, to the I2I Process UID), and to the UID of the environment (the same as the Spoke ID). It is not related to any users. If two users ran the same I2I workflow at the same time, it would be the same situation as if one user runs the same I2I workflow twice at the same time. I2I Process UID: Note Info title This IWS workflow is named eagle_i2i_delta_extract. Image ModifiedNote Info title Last run datetime is not on the record level. It is related to the I2I workflow Process UID and to the environment in general. In this example, if an SMF delta extract is done with a filter for one source at 1:00pm and then another SMF delta extract is done with a filter for another source at 2:00pm, the last run datetime will be updated twice: after the 1:00pm run and after 2:00pm run. What if I Need to Change or Reset the Last Run Date? To change or reset last run datetime stamp, you can use standard Change code value or Delete code value panels to do that.