Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 3 Next »

Instance to Instance (I2I) EagleML is an EJM feature that provides data interaction between several Eagle environments.

Note

I2I was introduced in the 2015 September monthend EagleML/EJM Release. You need this release or higher in order to leverage I2I functionality.

The following are some helpful FAQs, click a question to see the answer.

General Questions

 What are the default filters and what are their default values? (for example, effectivedate defaults to current day, etc.)
There is only one default filter – maxrows (count of max returned records in the extract file) – the default value of this parameter is 100.000 records (refer to http://eagle.helpdocsonline.com/available-filters$Common%20Filter).
 What was the rationale for having the workflows not "stack up" on each other? When one workflow is run with the same I2IProcessUID, it will kill the prior extract.

There are two different cases:

Case #1 is for the STOP-START steps in the EJM Monitoring Tool. In the STOP step, the EJM core tries to stop the previously launched instance of the same workflow. This means that if we submit the eagle_i2i_full_extract_event twice, only one (1) instance will start.

Case #2 is for the I2IProcessUID parameter. This parameter is used only for the I2I delta mode and is related to the last run time. We can store several I2I delta workflows in one environment, so to separate the last time values for each delta workflow, this parameter will be used in the EagleML code.

 Is it possible to set a max amount of retries or timeout period for a task when FTP is down or the task is in a hung status?
We have 5 tries to connect to FTP and the default time sleep between tries is 1 second. In the current EagleML version, we can't change these parameters in the config file.
 Is it possible to extract only from the Composite table SMF?
Yes, it is possible. Do not set "historyOnlyFlag" filter value to "Y" – skip it in RTR or set to "N". In this case, Composite SMF table will be used (for more details, refer to http://eagle.helpdocsonline.com/available-filters$History%20Only%20Filter).
 Is it possible to use entityselectiontype2=EntityList as filter for I2I workflow in the Spoke?
Yes, it is possible. In this case, two (2) EntityList filters will be applied: first in the Hub and second in the Spoke.
 Where is the datetime stamp stored that is used to determine the cutoffs for the delta?

Datetime stamp value is stored in the Spoke database as a Code Value. You can review it in the List Code Value panel (/eagle/star/reference/pan-lcodeval.htm) as shown below:


The Code Value is 'i2i_d' (const prefix)_ + SpokeID + I2IProcessUID (set in the related i2i solution in IWS).

Result of the lookup is:


The Code Value Description (1177) is the value of the datetime stamp of last launch of the I2I process.

 Is there a way to seed or override the last extract datetime stamp?
Yes, it is possible. Currently, you can use the Change Code Value panel (/eagle/star/reference/pan-clcodeval.htm). In the future, we are planning to improve the EJM Monitoring Tool or/and I2I Dashboard and add this functionality to change/reset the last run datetime stamp.
 What action is the datetime stamp associated with? In other words, if two users manually kick off a delta extract, is the cutoff a global cutoff or is it associated with the last time that particular user created an extract? Is the same true per server, engine, or service account?

The datetime stamp is linked to the I2I workflow (specifically, to the I2I Process UID), and to the UID of the environment (the same as the Spoke ID). It is not related to any users. Thus, if two users run the same I2I workflow at the same time, it will be the same situation as if one user runs the same I2I workflow twice at the same time.

For example, here is the I2I Process UID:

 Is the last run datetime captured on the record level? For example, if an SMF delta extract is done with a filter for one source at 1:00pm and then another SMF delta extract is done with a filter for another source at 2:00pm, will the extract properly pull data prior to 1:00pm for the second source?

No, last run datetime is not on the record level. It is related to the I2I workflow Process UID and to the environment in general. In this example, the last run datetime will be updated twice: after the 1:00pm run and after 2:00pm run.

 How can we find out what version of I2I we are running?

I2I is part of EagleML, we don't have a separate version for I2I. EagleML/I2I have the same version.

To determine which EagleML version you have in your environment, open Message Center Editor and open the following file: /eagle_ml-2-0_cm/w_config.inc.

In the top line of this file, you will find information about the installed EagleML version. For example: <!-BR-EAGLEML_PRD_20151007.1-> 

In this example, you see BR-EAGLEML_PRD_20151007.1. This indicates that this environment contains the September 2015 monthend release. The date for the monthend release is typically the first Wednesday after the first Tuesday of the following month. For example, September 2015 EagleML monthend package was released on October 7th 2015. Refer to the related wiki page: http://eagle.helpdocsonline.com/checking-your-eagleml-version.

You can find detailed information about each EagleML release in the EagleML Release Notes section.

Filter Questions

 Security Master File (SMF) Filters Questions -- As shown in the wiki for SMF filters, the Include form is the default. If I don't specify any filters, should I assume that no filters will be applied and thus all security types will be extracted, all process security type will be extracted, etc.
Yes, if you don't specify any filters for SMF, then all SMF will be extracted. There are no default filters, with one exception: maxrows (count of max returned records in the extract file). The default value of this parameter is 100,000 records (refer to Available Filters wiki page).
But, on the level of I2I processes, there is also the EntityList filter which can be considered default if it is set in the spoke map. The spoke can also use one of the available Sources from the Sources List to filter SMFs – this is defined in the spoke map.

Held Security Filters Questions 1: If the held position source  is not selected, what is the default value? Does the extract look at all position sources? 2: If the held position source selected, e.g. heldpositionsources = STARDIRECT, does the extract looks at all positions with source = STARDIRECT? 3 and 4: Same questions for Held Cash Activity Source and Held Corporate Action Source filters.|8892]
A:
1: If Held Position Source parameter is empty, the Held Position SQL query will not be executed. This means that the extract looks thru all positions sources (doesn't filter them).
2: If Held Position Source parameter is set (for example: heldpositionsources = STARDIRECT), then all positions with source = STARDIRECT will be extracted.
3: This is also true for for Held Cash Activity Source. If Held Cash Activity Source is empty, then the Held Cash Activity SQL query will not be executed. If Held Cash Activity Source parameter is set (for example: heldcashsources = CASHTEST), then all cash activities with source = CASHTEST will be extracted.
4: Held Corporate Action Source is not used.

Q: Date Filters Questions 1: Can this be used in the SMF request? And if so, what is the meaning of Date filters for SMF extract? 2: If effectivedate is selected, I expect the extract to look at the holding for the effective date specified in this parameter and extract all securities held for this effective date and entity list specified. Is this the case? 3: If effectivedate is not selected, what is the default, and does it look at the effective date of the holdings? 4: If the effective date is not related to the holding, what is the correct filter to use to point to a specific holding date? 5: FROM effectivedate and TO effectivedate. Similar questions as per effective date. 6: Please give some details on how these filters interact. For example, if only FROMeffective date is selected, what is the TO effective date default?
A: 1: Yes, they can be used in SMF extracts. But, the date filters are different for Date filters without Held Mode and Date filters with Held Mode.
2: Date filters without Held Mode:
There is no "EffectiveDate" field in the composite SMF table (SECURITY_MASTER). This means that the "EffectiveDate" filter will be ignored if you extract the SMF from the composite table. If you extract the SMF from the history table (historyonlyflag parameter is set to Y), the "EffectiveDate" filter will be used in the following way:
SECMASTER_HISTORY.EFFECTIVE_DATE >= TO_DATE([effective date filter value], 'YYYY-MM-DD')
This means that all SMF records from the history table with EFFECTIVE_DATE value greater than or equal to the effective date filter value will be extracted.
"updatedate" filter can be used to extract SMF records from the composite or history tables in the following way:
[Composite or History table].UPDATE_DATE >= TO_DATE([update date filter value], 'YYYY-MM-DD')
This means that all SMF records from the composite or history table with the UPDATE_DATE value greater than or equal to the update date filter value will be extracted.
Date filters with Held Mode: 
Effective Date, which will be used as a restriction in the search for SMFs from Position and Cash Activity Sources, the value for Held mode is set by the following steps:

  • if parameter "ProcessCenter" is used (entityselectiontype=ProcessCenter, entityselectionvalue=[Process Center filter value]), then the value of "effectivedate" filter will be found from the RULESDBO.PROCESS_CENTER table, for example:
    select to_char(pc.curr_dt,'yyyy-mm-dd') EFFECTIVE_DATEfrom RULESDBO.PROCESS_CENTER pcwhere pc.process_center = [Process Center filter value]
    To use two Entity filters, the following parameters are available:
    entityselectiontype / entityselectionvalue
    entityselectiontype2 / entityselectionvalue2
    If we want to use ProcessCenter and EntityList filters together, we can use following structure:
           <taskParameter>             <name>entityselectiontype</name>             <dataType>S</dataType>             <value>EntityList</value>       </taskParameter>       <taskParameter>             <name>entityselectionvalue</name>             <dataType>S</dataType>             <value>SCPAREN3</value>       </taskParameter>       <taskParameter>             <name>entityselectiontype2</name>             <dataType>S</dataType>             <value>ProcessCenter</value>       </taskParameter>       <taskParameter>             <name>entityselectionvalue2</name>             <dataType>S</dataType>             <value>US</value>       </taskParameter>
    Order of filters is not important. "entityselectiontype2" parameter can be used for both, ProcessCenter or EntityList as follows:
  • If the value of ProcessCenter parameter is incorrect, then the value of "effectivedate" filter will be set as null.
  • If ProcessCenter parameter is not used and "heldeffectivedate" filter value exists in RTR, then "heldeffectivedate" filter value will be used for filtering by EFFECTIVE_DATE field, for example:
    HOLDINGDBO.POSITION.EFFECTIVE_DATE = to_date([held effective date filter value],'yyyy-mm-dd') - for positions
    CASHDBO.CASH_ACTIVITY.ACCOUNTING_DATE >= to_date('2015-11-25','yyyy-mm-dd') - for cash activity
  • If ProcessCenter parameter is not used and "heldeffectivedate" filter value does not exist in RTR, then the value of "effectivedate" filter will be found as MAX of EFFECTIVE_DATE in the POSITION table. Conditions used in this query are: Date filter for UpdateDate, HeldPositionSource, and Entitylist.
    3: If "effectivedate" filter is not set, then filtering by EFFECITVE_DATE field will not be used, there is no default filter value. In general, "effectivedate" filter is not used in Held Mode.
    4: Need to use "heldeffectivedate" filter and do not use ProcessCenter parameter in RTR.
    5: For Date filters without Held Mode: "fromeffectivedate" and "toeffectivedate" filters work as usual: for SMF extraction from composite table - aren't available, for SMF extraction from history table - are available, for example:
    SECMASTER_HISTORY.EFFECTIVE_DATE >= TO_DATE([from effective date filter value], 'YYYY-MM-DD')
    SECMASTER_HISTORY.EFFECTIVE_DATE <= TO_DATE([to effective date filter value], 'YYYY-MM-DD')
    For "Date filters with Held Mode": "fromeffectivedate" and "toeffectivedate" filters aren't available.
    6: "effectivedate", "fromeffectivedate" and "toeffectivedate" filters are available only for SMF history extract and not available in Held Mode as follows:
    -If we set only "effectivedate" filter value, then following SQL restriction will be added:
    SECMASTER_HISTORY.EFFECTIVE_DATE = TO_DATE([effective date filter value], 'YYYY-MM-DD')
    -If we set only "fromeffectivedate" filter value, then following SQL restriction will be added:
    SECMASTER_HISTORY.EFFECTIVE_DATE >= TO_DATE([from effective date filter value], 'YYYY-MM-DD')
    -If we set only "toeffectivedate" filter value, then following SQL restriction will be added:
    SECMASTER_HISTORY.EFFECTIVE_DATE <= TO_DATE([to effective date filter value], 'YYYY-MM-DD')
    -If we set "fromeffectivedate" and "toeffectivedate" filter values, then  following SQL restriction will be added:
    SECMASTER_HISTORY.EFFECTIVE_DATE >= TO_DATE([from effective date filter value], 'YYYY-MM-DD') AND SECMASTER_HISTORY.EFFECTIVE_DATE <= TO_DATE([to effective date filter value], 'YYYY-MM-DD')

    Planned Enhancements and Resolved Issues

    Q: If EffectiveDate is set to the current day, a full extract is returned. If there are no holdings as of a specific date, will we see an error?
    A: This was a problem in the EagleML code, we have fixed it. It works correctly if we use the HeldEffectiveDate parameter instead of EffectiveDate in the Date filter. EffectiveDate in the Date filter is not used in the Held mode. However, current version of I2I doesn't support error descriptions in the final TSR, it shows only Status (SUCCESS, ERROR, NO_DATA). We are working on it right now and in one of the following EagleML updates, we will add this new feature.
    Q: If Sourcename is set to a fake source (for example, FAKESOURCE) a full file is returned. Will we see an error? For reference, P2P gives the following error: "Level:1, SQL Call Failed. Check Stored Procedure or SQL Statement that caused the error[SERVERERROR] ORA-20002: Invalid Position Source FAKESRC".
    A: Yes, this was a problem and we have fixed it.
    Q: Is it possible to show errors from the Hub in the Spoke Monitoring Tool? For example, if an entity ID does not exist.
    A: Yes, it is possible. We are working on it right now. In one of the following EagleML and EJM Monitoring Tool updates, we will add this new feature.
  • No labels