Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Instance to Instance (I2I) EagleML is an EJM feature that provides data interaction between several Eagle environments.

Info
titleNote

I2I was introduced in the 2015 September monthend EagleML/EJM Release. You need this release or higher in order to leverage I2I functionality.

The following are some helpful FAQs, click a question to see the answer.

...

Expand
titleWhat are the default filters and what are their default values? (for example, effectivedate defaults to current day, etc.)

There is only one default

filter –

filter – maxrows (count of max returned records in the extract file)

 –

the default value of this parameter is 100.000 records. For more information,

 see

see the Available Filters page in the EagleML wiki.

Expand
titleWhat was the rationale for having the workflows not "stack up" on each other? When one workflow is run with the same I2IProcessUID, it will kill the prior extract.

There are two different cases:

Case #1 is for the STOP-START steps in the EJM Monitoring Tool. In the STOP step, the EJM core tries to stop the previously launched instance of the same workflow. This means that if we submit the eagle_i2i_full_extract_event twice, only one (1) instance will start.

Case #2 is for the I2IProcessUID parameter. This parameter is used only for the I2I delta mode and is related to the last run time. We can store several I2I delta workflows in one environment, so to separate the last time values for each delta workflow, this parameter will be used in the EagleML code.

Expand
titleIs it possible to set a max amount of retries or timeout period for a task when FTP is down or the task is in a hung status?

We have 5 tries to connect to FTP and the default time sleep between tries is 1 second. In the current EagleML version, we can't change these parameters in the config file.

Expand
titleIs it possible to extract only from the Composite table SMF?

Yes, it is possible. Do not set "historyOnlyFlag" filter value to "Y" – skip it in RTR or set to "N". In this case, Composite SMF table will be used. For more information, see the Available Filters page in the EagleML wiki.

Expand
titleIs it possible to use entityselectiontype2=EntityList as filter for I2I workflow in the Spoke?

Yes, it is possible. In this case, two (2) EntityList filters will be applied: first in the Hub and second in the Spoke.

Expand
titleWhere is the datetime stamp stored that is used to determine the cutoffs for the delta?

Datetime stamp value is stored in the Spoke database as a Code Value. You can review it in the List Code Value panel (/eagle/star/reference/pan-lcodeval.htm) as shown below:Image Removed

List Code Value Core TabImage Added

The Code Value is 'i2i_d' (const prefix)_ + SpokeID + I2IProcessUID (set in the related i2i solution in IWS).

Result of the lookup is:Image Removed

Code Value Description ColumnImage Added


The Code Value Description (1177) is the value of the datetime stamp of last launch of the I2I process.

Expand
titleIs there a way to seed or override the last extract datetime stamp?

Yes, it is possible. Currently, you can use the Change Code Value panel (/eagle/star/reference/pan-clcodeval.htm). In the future, we are planning to improve the EJM Monitoring Tool or/and I2I Dashboard and add this functionality to change/reset the last run datetime stamp.

Expand
titleWhat action is the datetime stamp associated with? In other words, if two users manually kick off a delta extract, is the cutoff a global cutoff or is it associated with the last time that particular user created an extract? Is the same true per server, engine, or service account?

The datetime stamp is linked to the I2I workflow (specifically, to the I2I Process UID), and to the UID of the environment (the same as the Spoke ID). It is not related to any users. Thus, if two users run the same I2I workflow at the same time, it will be the same situation as if one user runs the same I2I workflow twice at the same time.

For example, here is the I2I Process UID:

Image RemovedImage Added
Expand
titleIs the last run datetime captured on the record level? For example, if an SMF delta extract is done with a filter for one source at 1:00pm and then another SMF delta extract is done with a filter for another source at 2:00pm, will the extract properly pull data prior to 1:00pm for the second source?

No, last run datetime is not on the record level. It is related to the I2I workflow Process UID and to the environment in general. In this example, the last run datetime will be updated twice: after the 1:00pm run and after 2:00pm run.

...

Expand
titleIf the held position source is not selected, what is the default value? Does the extract look at all position sources?

If Held Position Source parameter is empty, the Held Position SQL query will not be executed. This means that the extract looks thru all positions sources (doesn't filter them).

Expand
titleIf the held position source selected, e.g. heldpositionsources = STARDIRECT, does the extract looks at all positions with source = STARDIRECT?

If Held Position Source parameter is set (for example: heldpositionsources = STARDIRECT), then all positions with source = STARDIRECT will be extracted.

Expand
titleSame questions for Held Cash Activity Source and Held Corporate Action Source filters.

This is also true for for Held Cash Activity Source. If Held Cash Activity Source is empty, then the Held Cash Activity SQL query will not be executed. If Held Cash Activity Source parameter is set (for example: heldcashsources = CASHTEST), then all cash activities with source = CASHTEST will be extracted.

Held Corporate Action Source is not used.

Date Filters Questions

Expand
titleCan this be used in the SMF request? And if so, what is the meaning of Date filters for SMF extract?

Yes, they can be used in SMF extracts. But, the date filters are different for Date filters without Held Mode and Date filters with Held Mode.

    • HOLDINGDBO.POSITION.EFFECTIVE_DATE = to_date([held effective date filter value],'yyyy-mm-dd') – for positions
      CASHDBO.CASH_ACTIVITY.ACCOUNTING_DATE >= to_date('2015-11-25','yyyy-mm-dd') – for cash activity

    • If ProcessCenter parameter is not used and "heldeffectivedate" filter value does not exist in RTR, then the value of "effectivedate" filter will be found as MAX of EFFECTIVE_DATE in the POSITION table. Conditions used in this query are: Date filter for UpdateDate, HeldPositionSource, and Entitylist.

  • Expand
    titleIf effectivedate is selected, I expect the extract to look at the holding for the effective date specified in this parameter and extract all securities held for this effective date and entity list specified. Is this the case?

    Date filters without Held Mode:
    There is no "EffectiveDate" field in the composite SMF table (SECURITY_MASTER). This means that the "EffectiveDate" filter will be ignored if you extract the SMF from the composite table. If you extract the SMF from the history table (historyonlyflag parameter is set to Y), the "EffectiveDate" filter will be used in the following way:

    Code Block
    SECMASTER_HISTORY.EFFECTIVE_DATE >= TO_DATE([effective date filter value], 'YYYY-MM-DD')

    This means that all SMF records from the history table with EFFECTIVE_DATE value greater than or equal to the effective date filter value will be extracted.
    The "updatedate" filter can be used to extract SMF records from the composite or history tables in the following way:

    Code Block
    [Composite or History table].UPDATE_DATE >= TO_DATE([update date filter value], 'YYYY-MM-DD')

    This means that all SMF records from the composite or history table with the UPDATE_DATE value greater than or equal to the update date filter value will be extracted.

    Date filters with Held Mode
    Effective Date, which will be used as a restriction in the search for SMFs from Position and Cash Activity Sources, the value for Held mode is set by the following steps:

    • If parameter "ProcessCenter" is used (entityselectiontype=ProcessCenter, entityselectionvalue=[Process Center filter value]), then the value of "effectivedate" filter will be found from the RULESDBO.PROCESS_CENTER table, for example:

      Code Block
      select to_char(pc.curr_dt,'yyyy-mm-dd') EFFECTIVE_DATE
      from RULESDBO.PROCESS_CENTER pc
      where pc.process_center = [Process Center filter value]

      To use two Entity filters, the following parameters are available:
      entityselectiontype / entityselectionvalue
      entityselectiontype2 / entityselectionvalue2

    • If we want to use ProcessCenter and EntityList filters together, we can use following structure:

      Code Block
      languagexml
      <taskParameter>
          <name>entityselectiontype</name>
          <dataType>S</dataType>
          <value>EntityList</value>
      </taskParameter>
      <taskParameter>
          <name>entityselectionvalue</name>
          <dataType>S</dataType>
          <value>SCPAREN3</value>
      </taskParameter>
      <taskParameter>
          <name>entityselectiontype2</name>
          <dataType>S</dataType>
          <value>ProcessCenter</value>
      </taskParameter>
      <taskParameter>
          <name>entityselectionvalue2</name>
          <dataType>S</dataType>
          <value>US</value>
      </taskParameter>

      Order of filters is not important. "entityselectiontype2" parameter can be used for both, ProcessCenter or EntityList as follows:

      • If the value of ProcessCenter parameter is incorrect, then the value of "effectivedate" filter will be set as null.

      • If ProcessCenter parameter is not used and "heldeffectivedate" filter value exists in RTR, then "heldeffectivedate" filter value will be used for filtering by EFFECTIVE_DATE field, for example:

    Panel
    Expand
    titleIf effectivedate is not selected, what is the default, and does it look at the effective date of the holdings?

    If "effectivedate" filter is not set, then filtering by EFFECITVE_DATE field will not be used, there is no default filter value. In general, "effectivedate" filter is not used in Held Mode.

    Expand
    titleIf the effective date is not related to the holding, what is the correct filter to use to point to a specific holding date?

    Need to use "heldeffectivedate" filter and do not use ProcessCenter parameter in RTR.

    Expand
    titleFROM effectivedate and TO effectivedate. Similar questions as per effective date.

    For Date filters without Held Mode: "fromeffectivedate" and "toeffectivedate" filters work as usual:

    • for SMF extraction from composite table - aren't available

    • for SMF extraction from history table - are available, for example:

    Code Block
    SECMASTER_HISTORY.EFFECTIVE_DATE >= TO_DATE([from effective date filter value], 'YYYY-MM-DD')
    SECMASTER_HISTORY.EFFECTIVE_DATE <= TO_DATE([to effective date filter value], 'YYYY-MM-DD')

    For "Date filters with Held Mode": "fromeffectivedate" and "toeffectivedate" filters aren't available.

    Expand
    titlePlease give some details on how these filters interact. For example, if only FROMeffective date is selected, what is the TO effective date default?

    The "effectivedate", "fromeffectivedate" and "toeffectivedate" filters are available only for SMF history extract and not available in Held Mode as follows:

    • If we set only "effectivedate" filter value, then following SQL restriction will be added:

      Code Block
      SECMASTER_HISTORY.EFFECTIVE_DATE = TO_DATE([effective date filter value], 'YYYY-MM-DD')
    • If we set only "fromeffectivedate" filter value, then following SQL restriction will be added:

      Code Block
      SECMASTER_HISTORY.EFFECTIVE_DATE >= TO_DATE([from effective date filter value], 'YYYY-MM-DD')
    • If we set only "toeffectivedate" filter value, then following SQL restriction will be added:

      Code Block
      SECMASTER_HISTORY.EFFECTIVE_DATE <= TO_DATE([to effective date filter value], 'YYYY-MM-DD')
    • If we set "fromeffectivedate" and "toeffectivedate" filter values, then  following SQL restriction will be added:

      Code Block
      SECMASTER_HISTORY.EFFECTIVE_DATE >= TO_DATE([from effective date filter value], 'YYYY-MM-DD')

      AND

      Code Block
      SECMASTER_HISTORY.EFFECTIVE_DATE <= TO_DATE([to effective date filter value], 'YYYY-MM-DD')

    ...

    Expand
    titleIf EffectiveDate is set to the current day, a full extract is returned. If there are no holdings as of a specific date, will we see an error?

    This was a problem in the EagleML code, we have fixed it. It works correctly if we use the HeldEffectiveDate parameter instead of EffectiveDate in the Date filter. EffectiveDate in the Date filter is not used in the Held mode. However, current version of I2I doesn't support error descriptions in the final TSR, it shows only Status (SUCCESS, ERROR, NO_DATA).

    We are working on it right now and in one of the following EagleML updates, we will add this new featur

    Expand
    titleIf Sourcename is set to a fake source (for example, FAKESOURCE) a full file is returned. Will we see an error? For reference, P2P gives the following error: "Level:1, SQL Call Failed. Check Stored Procedure or SQL Statement that caused the error[SERVERERROR] ORA-20002: Invalid Position Source FAKESRC".

    Yes, this was a problem and we have fixed it.

    Expand
    title Is it possible to show errors from the Hub in the Spoke Monitoring Tool? For example, if an entity ID does not exist.

    Yes, it is possible. We are working on it right now. In one of the following EagleML and EJM Monitoring Tool updates, we will add this new feature.