Processing Guidelines for Data Files

Owned by Natalie Gesin (Deactivated)
Last updated: Jan 31, 2024 by Kinga Huszno
5 min read

You must observe several guidelines when processing data files using the Eagle generic default message streams and follow recommended steps when processing records that update rather than insert to the Eagle data warehouse tables.

Process with Default Message Streams

To process data files using Eagle default message streams, follow these guidelines:

  1. All Corporate Action announcements must first process through the default message stream in a Pending status. The announcements must then be reviewed and updated to a Released status. Each Corporate Action announcement must be in a Released status before Global/CA sweep processing can occur in STAR.
    NOTE: You must complete this step to ensure that processing does not occur if the events are not executed. For example, Calls and Puts that are not executed. Or, when necessary to ensure that the announcements received are accurate to the final announcements. This is important because after Corporate Action announcement have been placed in a Released status, the system does not allow the announcement to be changed.

  2. For all schedule information for Calls, Puts, and Sinks, the message stream places the information in a Released status, regardless of the value sent in the data file. The schedule data is used in the STAR Earnings process.

  3. Load all reference data, such as SMF, Prices, FX Rates, Corporate Actions, and so on, prior to submitting trades for processing. This approach minimizes the number of possible errors and improves performance.

  4. For all trade processing, sort the data files by Trade Date (tag 35), and load the Trades in the following sequence: 
    1. Buy
    2. Receive
    3. Sell
    4. Deliver
    5. Cancel 
    This approach minimizes the amount of rollback/replay and errors during processing.

  5. With the exception of Calls and Puts, all Corporate Actions are set up for processing through one stream. For Calls and Puts, the records must be processed through both the Corporate Action stream and the Call/Put stream – secondary schedule tables are used for earnings processing.

Process a Change to an Existing SMF Record

To process a change to an existing SMF record, follow these steps:

  1. Add an additional tag/value to the records that you wish to change.

  2. Add the Update Indicator field (tag 1076) to the record with a value of Y or B.

  3. Review the setting of the Conversion Status field (tag 1085):

    • If tag 1085 is set to VENDOR_MODE, include all fields that you wish to capture in the SECURITY database.

    • If tag 1085 is not set to VENDOR_MODE, include Processing Security Type (tag 3931), Primary Asset ID (tag 14), and Primary Asset ID Type (tag 1432) with the change record. You can then include all previous fields to captures the changes, or include just the fields that you wish to change.

  4. Review the setting of the Validation Level field (tag 1081):

    • If the Validation Level field (tag 1081) is set to NONE, include Vendor Cash Flows Source Name (tag 1102), Effective Date (tag 1109), Processing Security Type (tag 3931), Primary Asset ID (tag 14), Primary Asset ID Type (tag 1432), and Issue Name (tag 961).

    • If tag 1081 is NULL, Eagle Accounting validations are applied via the core security panels as described in the data maps.

  5. Set the Event Type field (tag 55) to ADDSECMASTER.

  6. Deliver the file to the same location as the Add SMF files.

Process a Change to an Existing Entity or Entity History Record

To process a change to an existing entity or entity history record, follow these steps:

  1. Include all the fields required for the Add message and include the tag 1109, the EFFECTIVE_DATE field. If Effective Date (tag 1109) is NULL and matches a record in the ENTITY table, then the database entity record is updated for all fields in the input record. If tag 1109 is populated and matches a record in the ENTITY_HIST table, then the database entity history is updated for all fields in the input record. Fields not supplied in the input record are not updated.

  2. Review the setting of the Validation Level field (tag 1081):

    • If tag 1081 is set to NONE, include tag 1109, tag 1163, and tag 1105.

    • If tag 1081 is NULL, Eagle Accounting validations are applied via the core security panels as described in the data maps.

  3. Set the Event Type field (tag 55) to ADDENTITY.

  4. Deliver the file to the same location as the Add message.

Process a Change to an Existing FX Record

To process a change to an existing FX record, follow these steps:

  1. Include all the fields required for the Add message and include the Update Indicator (tag 1076) set to Y. The FX Rates being sent must be the new FX Rates.

  2. Set the Event Type field (tag 55) to ADDFXRATES.

  3. Deliver the file to the same location as the Add message

Process a Change to an Existing Price Record

To process a change to an existing Price record, follow these steps:

  1. Include all the fields required for the Add message and include the Update Indicator (tag 1076) set to Y, except for price, you must send the new price.

  2. Set Event Type field (tag 55) to ADDPRICE.

  3. Deliver the file to the same location as the Add message.

Process a Change to an Existing Sink Schedule

To process a change to an existing Sink schedule, follow these steps:

  1. Include all the fields required for the Add message and include the Update Indicator (tag 1076) set to Y.

  2. Set the Event Type field (tag 55) ADDSINK.

  3. Deliver the file to the same location as the Add message.

Process a Change to an Existing Variable Rate

To process a change to an existing Variable Rate, follow these steps:

  1. Include all the fields required for the Add message and include the Update Indicator (tag 1076) set to Y.

  2. Set the Event Type field (tag 55) to ADDVARRATE.

  3. Deliver the file to the same location as the Add message.

Process Trades

To process Trades, follow these steps:

  1. Process a Cancel Trade event.

  2. Submit a new trade.

Process a Change to an Existing Corporate Action

To process a change to an existing Corporate Action, follow these steps:

  1. Include all the fields required for the Add message and include the Update Indicator (tag 1076) set to Y.

  2. Set the Event Type field (tag 55) to the same value required for the Add message.

  3. Deliver the file to the same location as the Add message.

Process a Change to an Existing Cash Activity Record

To process a change to an existing Cash Activity record, follow these steps:

  1. Set the Update Indicator (tag 1076) to a value of U (update), I (insert), or D (delete). If the field is NULL, the default is U (update).

  2. For CANCEL transactions, add the Cancel Flag (tag 10045) with a value of Y to mark as cancels. The transaction type is the original transaction type suffixed with CANCEL, that is, a BUYCANCEL is the cancel for an original BUY record and a BUY$CANCEL is the cancel for the reflexive row. Reflexive rows should be created for any transaction that impacts the cash balance of the portfolio.