As a result, the data set is enriched with additional details. Core, custom and derived fields are supported. Prerequisite steps for the extract to work:
Metadata Lineage from Eagle MDC to Ontology - field attributes and element definitions
Set Up Temporary Table for fieldExpressions
Please use eagle_ml-2-0_default_cm_exec_eds stream to run EDS fieldattribute extract:
| Expand | ||
|---|---|---|
|
EQL REST API is supported, for example: https://o171-e001-ww01.eagleinvsys.com/eagle/v2/eql
| Code Block |
|---|
{
"ResourceName": "fieldattribute",
"ServiceName": "EBS",
"ServiceMethodName": "extract",
"savesqlquery": "Y"
}
|
The following filters are supported:
fieldattributeidfieldtypefieldname
| Code Block |
|---|
<taskParameter> <name>fieldattributeid</name> <dataType>S</dataType> <value>264</value> </taskParameter> <taskParameter> <name>fieldtype</name> <dataType>S</dataType> <value>fieldtype_VALUE</value> </taskParameter> <taskParameter> <name>fieldname</name> <dataType>S</dataType> <value>fieldname_value</value> </taskParameter> |
The processing rule for EDS fieldattribute extract was improved to support new element fieldAttributeOntologyDescription for all fields with direct or simple mapping.
| Code Block |
|---|
<referenceTransaction> <header> <objectType>FieldAttribute</objectType> </header> <fieldAttribute> <calcType>di</calcType> <databaseName>cash</databaseName> <fieldAttributeDescription>i Local Currency</fieldAttributeDescription> <fieldAttributeId>138</fieldAttributeId> <fieldIndicator>cf</fieldIndicator> <fieldName>LOCAL_CURRENCY</fieldName> <tableName>CASH_ACTIVITY</tableName> <fieldIdentifier>I LOCAL CURRENCY</fieldIdentifier> <scope>2</scope> <readOnly>Y</readOnly> <fieldAttributeOntologyDescription>Local currency. Three-letter currency code.</fieldAttributeOntologyDescription> </fieldAttribute> </referenceTransaction> |
| Info |
|---|
This field will be part of the EDS extract by default. In order to create a mashup file the manual step is required. Described here |
A new logic was implemented for derived or calculated fields. Now all user defined expressions for the calculated fields will be part of the extract. Two additional elements are part of the extract:
fieldExpression element includes user friendly expression available in MDC
userData element contains RAW definition of the value stored in the database
New fields are included by default, but they can be excluded by setting RTR parameter includederivedattributes to N.
| Code Block |
|---|
<referenceTransaction> <header> <objectType>FieldAttribute</objectType> </header> <fieldAttribute> <statField>Y</statField> <fieldAttributeDescription>DC for subtraction</fieldAttributeDescription> <fieldAttributeId>1429</fieldAttributeId> <fieldAttributeProcessId>1014</fieldAttributeProcessId> <fieldIndicator>DE</fieldIndicator> <fieldType>NUM</fieldType> <fieldFormat>2</fieldFormat> <fieldCategory><none></fieldCategory> <currencyProcess>NOT</currencyProcess> <fieldValueType>NOT</fieldValueType> <fieldIdentifier>DETAIL CAL FOR SUBTRACTION - SUPERUSERS</fieldIdentifier> <scope>2</scope> <readOnly>N</readOnly> <processDescription>Expression Calculator</processDescription> <processType>USERDEFINED</processType> <fieldExpression>H-PD-Market_Value_Income - H-PD-Local_Market_Value</fieldExpression> <userData>#`FP1014~#`FD1170~-~#`FD1155~</userData> </fieldAttribute> </referenceTransaction> |
| Info |
|---|
The initial configuration steps are required. Described here |
This functionality is limited for the calculated fields below with field_attribute_process_id:
1010 Arithmetic Addition Calculator
1011 Arithmetic Subtraction Calculator
1012 Arithmetic Division Calculator
1013 Arithmetic Multiplication Calculator
1014 Expression Calculator
1015 Adjustments Calculator
1025 Trade Adjustment Calculator
1030 IRR Calculator
1031 Modified Dietz Calculator
1032 Dollar Value Added Calculator
1033 Average Invested Balance Calculator
Example extract is attached.
| View file | ||
|---|---|---|
|
RDC Enrichments
Five additional fields are supported:
fieldAttributeReference
enrichmentRule
rdcValidation
rdcFieldGroup
rdcDataStrategy
Additional logic was implemented to group values and use TEMP table for join. Grouped values are separated by ; symbol.
fieldAttributeReference - field attributes which are used in calculations
Extract file:
enrichmentRule - the enrichment rule for the field
Extract file:
RDC
validation - list of used validations
Extract file:
Value in the cell:
First Compounding Date Cannot Be Greater than Maturity Date; First Compounding Date Cannot Be Greater than Termination Date; First Compounding Date Cannot Be Less than Dated Date; First Compounding Date Cannot Be Less than Effective Date
RDC
rdcFieldGroup - list of selected groups
Extract file:
RDC
Single value:
Multiple values:
rdcDataStrategy - list of applied data strtategies
Extract file:
Custom Mashup
In case you want to use custom file (CSV, JSON, etc.) to mashup it with fieldattributes extract.
Create custom dataset definition for the file in the folder: estar\tpe\dynamic\metadata\core\datasets
For example:
| Expand | ||
|---|---|---|
| ||
{ |
This file describe custom CSV file:
| View file | ||
|---|---|---|
|
NOTE: pay attention on file path and adjust it correspondently.
Place file for mashup to path specified in dataset definition:
For example in dataset definition, path is defined as:
"file_name": " /apps/eagle/estar/tpe/dynamic/metadata/core/ontology/templates/EADataMapping.csv",
To make this works create custom processing rule in cache:
KEY: eagle.ebs.resource:O17_FIELDATTRIBUTE_2.0.57
VALUE:
View file name Invalid file id - 3022b10c-aae4-47f3-aad0-ec9ebbc03976
| Expand | ||
|---|---|---|
| ||
{ |
Pay attention on the following:
Add definition with key in the register_for_mashups block
Add mashup in mashups block
Add mapping in taxonomy block
KEY: eagle.ebs.resource:O17_FIELDATTRIBUTE_VERSIONS
VALUE: 2.0.57
NOTE: O17 this is tenant prefix. it can be different for your case.
Run EQL extract:
| Code Block |
|---|
{
"ServiceName": "EBS",
"ServiceMethodName": "extract",
"resourcename": "fieldattribute",
"outputformat": "EagleJSON",
"disabledatasetcache": "Y",
"resourceversion": "2.0.57",
"filters": {
"fieldattributeid": "5300,4876,4877,2014,4263,4265,371,4266,5864,2110,108,109"
}
}
|
| Info |
|---|
|
Output example with custom mashup:
| Code Block |
|---|
{
"eagleJSONVersion": "2-0",
"eagleJSONType": "ReferenceTransactionMessage",
"eagleJSONRevision": "294",
"referenceTransactionMessage": {
"header": {},
"referenceTransaction": [
{
"header": {
"objectType": "FieldAttribute"
},
"fieldAttribute": [
{
"objectType": "FieldAttribute",
"statField": "Y",
"calcType": "di",
"databaseName": "security",
"fieldAttributeDescription": "i Cusip",
"fieldAttributeId": 108,
"fieldIndicator": "S",
"fieldName": "PRIMARY_ASSET_ID",
"tableName": "SECURITY_MASTER",
"refCodeId": 99,
"comments": "Cusip",
"fieldIdentifier": "I CUSIP",
"scope": 2,
"readOnly": "N",
"fieldAttributeOntologyDescription": "Stores the most appropriate (primary) market security identifier of the financial instrument.",
"rdcFieldGroup": "IIP_AF_5",
"ea_mapping.LOB": "NYL Investors",
"ea_mapping.SUBJECT_AREA_DOMAIN": "SECURITY",
"ea_mapping.SUB_DOMAIN": "SECURITY",
"ea_mapping.BUSINESS_NAME": "PRIMARY_ASSET_ID",
"ea_mapping.BUSINESS_DESCRIPTION": "Current Primary Identifier of a security",
"ea_mapping.PHASE_1_NYLIM": "-NA-",
"ea_mapping.REQUIRED_FOR_FUTURE_STATE": "Y",
"ea_mapping.SOURCE_PLATFORM": "CAMRA",
"ea_mapping.SOURCE_TABLE": "SECURITY",
"ea_mapping.SOURCE_COLUMN_SOURCE_COLUMN_STAT": "CUSIP",
"ea_mapping.FORMAT": "CHAR(100)",
"ea_mapping.COMMENTS_DERIVATIONS_TRANSFORMATIONS_FOR_SOURCE": "This is CUSIP",
"ea_mapping.EXTRACT_FIELD_EXTRACT_FIELD_STAT": "CUSIP",
"ea_mapping.FORMAT": "CHAR(100)",
"ea_mapping.ML_ELEMENT": "EagleML/referenceTransaction/genericSMF/primaryAssetId",
"ea_mapping.SUBJECT_AREA_FOR_INTERNAL": "cam_egl_smf",
"ea_mapping.TARGET_FIELD_NAME_FOR_INTERNAL": "primaryAssetId",
"ea_mapping.MANDATORY_FIELD": "Y"
}
]
}
]
}
} |