EagleML Release Notes - May 3, 2023
- Eric Fardig (Deactivated)
Enhancements
SDP-73221 Cloud workflow was updated with additional default parameter ASISCOPY
Extract files generated via cloud workflow will have a new file name pattern - @CORRID_ASISCOPY_@ExtractMode_@FeedType_@DateStamp.@EXT - going forward. The DefaultAsIsCopy parameter is set to Y by default.
EDS service will upload data to Snowflake as it was extracted or in ASISCOPY mode.
Files changes:
eagle_ml-2-0_workflows/eagle_wrf_cloud_extract.wrf
eagle_ml-2-0_workflows/eagle_wrf_cloud_extract_schd.wrf
eagle_ml-2-0_workflows/eagle_wrf_generic_extract.wrf
eis/ejmwf/eagle_wrf_cloud_extract.ejmwf
eis/ejmwf/eagle_wrf_generic_extract.ejmwf
SDP-73140 Updated signage logic for fields in the WRHSTRADEEXTRACT
Signage logic was adjusted for the following fields for the 250MTCLOSELYNSELLCANCELY / 180MTCLOSELNYSELLY / 180MTCLOSELNYDELN signage keys.
COST_BASE
COST_LOCAL
ORIG_ACQ_COST_BASE
ORIG_ACQ_COST_LOCAL
PAR_OR_SHARES
BOOK_VALUE
BOOK_VALUE_LOCAL
PRINCIPAL
PRINCIPAL_BASE
DIRTY_PRINCIPAL
DIRTY_PRINCIPAL_BASE
ORIG_ACQ_COST_BASE and ORIG_ACQ_COST_LOCAL will provide the COST_BASE / COST_LOCAL values for the 180MTCLOSELNYSELLY and 180MTCLOSELNYDELN signage keys to provide a net zero against the cancel record during a rollback replay scenario.
BOOK_VALUE and BOOK_VALUE_LOCAL for signage key 250MTCLOSELYNSELLCANCELY required the cancel record to be set to ‘zero’ in order to match the original 150 record during a rollback replay scenario.
All other changes will provide a positive value on the cancel records to provide a net zero with the appropriate original record.
Files changed:
eagle_ml-2-0_cm/out/tagvalue/extract_wrhs_trade_signage_logic.inc
SDP-72737 EDS process center resource was enhanced to support the data load to Snowflake and Oracle
The EDS process center resource was enhanced to support both database providers. The RULESDBO.PROCESS_CENTER table is supported for Load and Extract:
Supported elements for THE extract and the load:
RULESDBO.PROCESS_CENTER.UPDATE_DT - EagleML/referenceTransaction/processCenter/updateTimestamp
RULESDBO.PROCESS_CENTER.UPDATE_SOURCE - EagleML/referenceTransaction/processCenter/updateSource
RULESDBO.PROCESS_CENTER.PROCESS_CENTER - EagleML/referenceTransaction/processCenter/processCenter
RULESDBO.PROCESS_CENTER.PROCESS_CENTER_DESC - EagleML/referenceTransaction/processCenter/processCenterDescription
RULESDBO.PROCESS_CENTER.CURR_DT - EagleML/referenceTransaction/processCenter/currentDate
RULESDBO.PROCESS_CENTER.CC_PROC_DT - EagleML/referenceTransaction/processCenter/ccCurrentDate
RULESDBO.PROCESS_CENTER.CC_ME_PROC_DT - EagleML/referenceTransaction/processCenter/ccMonthEndDate
RULESDBO.PROCESS_CENTER.CURR_DT_TRAN_CNT - EagleML/referenceTransaction/processCenter/currentDateTransactionCount
RULESDBO.PROCESS_CENTER.ROLL_PRIOR_PERIOD - EagleML/referenceTransaction/processCenter/rollPriorPeriod
Files changed:
metadata/core/ontology/reference/processcenter.json
metadata/core/ontology/reference/processcenter-models.json
Files added:
metadata/core/ontology/reference/processcenter_in.json
metadata/core/ontology/templates/eagle_ent_processcenter_insertupdate.txt
SDP-72424 Encode Output Option to avoid QUOTES in CSV format
A new parameter was added to support the format_dialect:
"quoting": "QUOTE_ALL" - quotes all lines
"quoting": "QUOTE_NONE" - excludes quoting
Files changed:
string_utils.py
SDP-72273 Multileg SMF now supports a list of parameters to load into history table
The below elements are now supported by EagleML for multilegSMF load in history mode:
floatingPriceInstrumentIndicator
floatingPriceSource
reitFlag
ndfIndicator
naicCapStructureCode
projectRate
fundRtgClassFocus
fundMarketCapFocus
fundIndustryFocus
fundManagementStyle
fundGeoFocus
fundMaturityBandFocus
rateConversionRule
Files changed:
eagle_default/in/xml/xml-smf_multileg_swap.xml
eagle_default/in/xml/include/contract_leg_alliases.inc
eagle_default/in/xml/include/contract_mapping.inc
eagle_default/in/xml/include/ins_upd_smf_req.inc
eagle_default/in/xml/include/pay_leg_alliases.inc
SDP-72247 EDS entitycharacteristic resource
The EDS entitycharacteristic resource was created to support both database providers. The RULESDBO.ENTITY_CHARACTERISTICS and RULESDBO.ENTITY_CHARACTERISTICS_HIST are supported for both the Load and the Extract.
Key for data inset/update: ENTITY_ID,CODE / ENTITY_ID,EFFECTIVE_DATE,CODE
Supported filters for extract:
entityfilter
entitytype
fromdate
todate
effectivedate
fromeffectivedate
toeffectivedate
Files changed:
metadata/core/ontology/eagle-vocab-common.json
metadata/core/ontology/eagle-vocab-entity-shared.json
metadata/core/ontology/eagle-vocab-ref-shared.json
Files added:
metadata/core/ontology/entity/entitycharacteristic.json
metadata/core/ontology/entity/entitycharacteristic-models.json
metadata/core/ontology/entity/entitycharacteristic_in.json
metadata/core/ontology/templates/eagle_ent_entitycharacteristic_insertupdate.txt
SDP-71959 Added Support of Realtime Processing for Accounting Impairments Interface
The accounting Impairments interface was enhanced to support realtime (in memory) processing which eliminates unnecessary behind the scenes work and file creation used outside realtime processing. Impairments records can be read from the MQ and processed in memory without intermediate files creation. Also, the creation of a TSR for Impairments realtime load processes was added.
Files changed:
eagle_ml-2-0_cm/in/xmlrealtime_rtr.xml
eagle_ml-2-0_cm/w_config.inc
eagle_default/in/xml/xml-impairments.rsf
eagle_default/in/xml/xml-impairments.xml
SDP-71682 New elements supported in the entityAccountingBasis and genericEntity both for load and extract
The below elements are now supported by EagleML within the entityAccountingBasis and genericEntity both for load and extract:
derivativeFeeRuleId
iLBGainLossFlag
pIKURGLFlag
controllingBasis
expenseElection
ledgerGranularityRuleName
lgFirstEffDate
holdingTermMethod
posCashSegregationBasisInd
cashBasedDivRecognitionFlag
fXElectionForCashDivRecognition
equityMethodTreatment
incPostTreatment
mtmAccountingFlag
basisIncRepayCode
taxExemptProcessingFlag
eclMethod
settlementDateSwitch
settlementBeginDate
monthEndFxSource
monthEndFwdPriceSource
Files changed:
eagle_default/in/xml/xml-ent_entityaccountingbasis.xml
eagle_default/in/include/entity_acct.inc
eagle_default/in/include/entity_aliases.inc
eagle_default/in/include/entity_panels.inc
eagle_ml-2-0_cm/out/xml/t_entity_accounting_basis_streaming.inc
eagle_ml-2-0_cm/out/xml/t_entity.xml
eagle_ml-2-0_cm/out/xml/t_entity_streaming.inc
eagle_ml-2-0_cm/out/xml/t_entity_streaming_eml.inc
SDP-71183 Added more supported fields in the Naic Designation load and extract interface
Support for the below fields was added in the load and extract side.
Load changes
companyCodeType Tag 990
svoCompanyCode Tag 4428
naicCompanyCodeRMBS Tag 9608
naicDesignationModifier Tag 18104
designation and naicDesignation elements → NAIC_DESIGNATION field in DB, both elements are supported now, preference to use designation element.
naicDesignationPrefix and naicDesignationPreffix elements → NAIC_DESIGNATION_PREFIX field in DB, both elements are supported now, preference to use the naicDesignationPrefix element.
naicReviewDate element formatting added to remove
-symbols from itAdded resolution logic for Tag 1632
Added logic to update the Naic Designation record in case if it already exists.
Extract changes
Added NAIC_DESIGNATION_MODIFIER support
naicDesignationModifier element added
naicDesignationPreffix element fixed to naicDesignationPrefix
Files Changed:
eagle_default/in/xml/xml-ref_naicdesignation.xml
eagle_ml-2-0_cm/out/tagvalue/extract_naic_designation.xml
eagle_ml-2-0_cm/out/xml/t_ref_naic_designation.xml
eagle_ml-2-0_cm/out/xml/t_ref_naic_designation_streaming.inc
SDP-70924 Updated eagle_wrf_generic_extract_restart pkg
Technical debt was resolved as part of this Jira to enable clients to benefit from the new and enhanced rule files. The eagle_wrf_generic_extract_restart stream has been enhanced to use a new ruleset and new rules (workflow_restart.*) which will now allow for workflow cancellations and profiles inheritance. This functionality had been deployed during implementations and this change incorporates it into the core product.
Files changed:
eagle/estar/eagle_wrf_generic_extract_restart.pkg
Below are the newly used rules files:
workflow_restart.sch
get_stream_name.inc
workflow_restart.sch
workflow_restart.rsf
workflow_restart.xml
workflow_restart.wrf
SDP-67478 The EDS entityanalytic resource was enhanced to support data load to Snowflake and Oracle
EDS entityanalytic resource was enhanced to support both database providers. The RULESDBO.ENTITY_ANALYTICS and RULESDBO.ENTITY_PEER_GRP_ANALYTICS are supported for the Load and the Extract.
Additional elements are also supported now for extract and load:
RULESDBO.ENTITY_ANALYTICS.DERV_EXPOSURE - EagleML/entityTransaction/entityAnalyticModel/derivativeExposure
RULESDBO.ENTITY_ANALYTICS.CURR_DERV_EXPOSURE - EagleML/entityTransaction/entityAnalyticModel/currencyDerivativeExposure
RULESDBO.ENTITY_ANALYTICS.INT_DERV_EXPOSURE - EagleML/entityTransaction/entityAnalyticModel/interestDerivativeExposure
RULESDBO.ENTITY_ANALYTICS.DERV_EXP_BREACH_DAYS - EagleML/entityTransaction/entityAnalyticModel/derivativeExposureBreachDays
RULESDBO.ENTITY_ANALYTICS.VAR_EXCEPTIONS - EagleML/entityTransaction/entityAnalyticModel/valueAtRiskExceptions
Files changed:
metadata/core/ontology/entity/entityanalytic.json
metadata/core/ontology/entity/entityanalytic-models.json
metadata/core/ontology/eagle-vocab-entity-shared.json
metadata/core/ontology/definition/eagle-vocab-definition-entity.json
Files added:
metadata/core/ontology/entity/entityanalytic_in.json
metadata/core/ontology/templates/eagle_ent_entityanalytic_insertupdate.txt
Production Bugs
SDP-73169 Inbound exchangePrice interface fixed to use pan-field_id.htm
The exchange Price object was updated to retrieve the field id for each price type in the securitydbo.price_exchange table. The rule will now only call pan-field_id.htm without conditions for Eagle in Market core version and apply the useRDCPricing flag when routing prices through RDC.
Files changed:
eagle_default/in/xml/xml-ref_exchangeprice.xml
SDP-72526 EDS entitymanagerrelationship resource was improved
An incorrect extract file was generated which caused an issue on the data load to Snowflake resulting in the following Exception being raised: Account was not resolved: there are no entityId or identifiers provided.
The mapping for the entitymanagerrelationship resource was corrected and the issue was resolved:
Added direct mappings for entityId, managerId, entityDetailId
Added direct mapping for shareClass
Files changed:
entitymanagerrelationship-models.json
SDP-71961 The Processing statistic logic was corrected to always populate the end processing date and time
The processing statistic logic was improved to always update the end_processing_date as part of the statistic record in Message Center Console.
Files changed:
src/common_core/processing_env/processing_context.py