MetaData lineage: EDS fieldattribute extract

EDS fieldatrribute extract is similar to EagleML extract. This extract contains meta information about Eagle fields like field type, name, database table, description and etc. This implementation links two metadata storages - MDC which is the Eagle metadata storage and Ontology, EDS metadata storage.

As a result, the data set is enriched with additional details. Core, custom and derived fields are supported.

Please use eagle_ml-2-0_default_cm_exec_eds stream to run EDS fieldattribute extract:

Click here to expand...

<EagleML xmlns="http://www.eagleinvsys.com/2011/EagleML-2-0"  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="RunTaskRequest" eaglemlVersion="2-0"  xsi:schemaLocation="http://www.eagleinvsys.com/2011/EagleML-2-0 eagleml-main-2-0.xsd" eaglemlType="RunTaskRequest">
	<header>
		<messageId>E30X5FTN4N317JFF</messageId>
		<sentBy>MC2EJM</sentBy>
		<sendTo>http://www.eagleinvsys.com/eagle_default_ruleservice</sendTo>
		<creationTimestamp>2018-04-13T09:52:39.73-05:00</creationTimestamp>
		<expiryTimestamp/>
		<expiryLoadFlag/>
	</header>
	<taskIdentifier>
		<processingSequenceNumber/>
		<correlationId>EDS_FIELDATTRIBUTE_EXTRACT</correlationId>
		<businessTaskId>EDS_FIELDATTRIBUTE_EXTRACT</businessTaskId>
		<sequenceNumber/>
	</taskIdentifier>
	<taskTypeEnum>NEW</taskTypeEnum>
	<synchronousExecution>yes</synchronousExecution>
	<taskParameters>
		<taskParameter>
			<name>StreamName</name>
			<dataType>S</dataType>
			<value>eagle_default_ruleservice</value>
		</taskParameter>
		<taskParameter>
			<name>ServiceName</name>
			<value>EBS</value>
		</taskParameter>
		<taskParameter>
			<name>ServiceMethodName</name>
			<dataType>S</dataType>
			<value>extract</value>
		</taskParameter>
		<taskParameter>
			<name>ResourceName</name>
			<value>fieldattribute</value>
		</taskParameter>
		<taskParameter>
			<name>OutputFormat</name>
			<dataType>S</dataType>
			<value>EagleJSON</value>
		</taskParameter>
		<taskParameter>
			<name>SaveSQLQuery</name>
			<dataType>S</dataType>
			<value>Y</value>
		</taskParameter>
	</taskParameters>
</EagleML>

EQL REST API is supported, for example: https://o171-e001-ww01.eagleinvsys.com/eagle/v2/eql

{
    "ResourceName": "fieldattribute",
    "ServiceName": "EBS",
    "ServiceMethodName": "extract",
    "savesqlquery": "Y"
}

The following filters are supported:

fieldattributeid
fieldtype
fieldname

		<taskParameter>
			<name>fieldattributeid</name>
			<dataType>S</dataType>
			<value>264</value>
		</taskParameter>
		<taskParameter>
			<name>fieldtype</name>
			<dataType>S</dataType>
			<value>fieldtype_VALUE</value>
		</taskParameter>
		<taskParameter>
			<name>fieldname</name>
			<dataType>S</dataType>
			<value>fieldname_value</value>
		</taskParameter>

The processing rule for EDS fieldattribute extract was improved to support new element fieldAttributeOntologyDescription for all fields with direct or simple mapping.

	<referenceTransaction>
		<header>
			<objectType>FieldAttribute</objectType>
		</header>
		<fieldAttribute>
			<calcType>di</calcType>
			<databaseName>cash</databaseName>
			<fieldAttributeDescription>i Local Currency</fieldAttributeDescription>
			<fieldAttributeId>138</fieldAttributeId>
			<fieldIndicator>cf</fieldIndicator>
			<fieldName>LOCAL_CURRENCY</fieldName>
			<tableName>CASH_ACTIVITY</tableName>
			<fieldIdentifier>I LOCAL CURRENCY</fieldIdentifier>
			<scope>2</scope>
			<readOnly>Y</readOnly>
			<fieldAttributeOntologyDescription>Local currency. Three-letter currency code.</fieldAttributeOntologyDescription>
		</fieldAttribute>
	</referenceTransaction>

This field will be part of the EDS extract by default. In order to create a mashup file the manual step is required. Described here

A new logic was implemented for derived or calculated fields. Now all user defined expressions for the calculated fields will be part of the extract. Two additional elements are part of the extract:

fieldExpression element includes user friendly expression available in MDC
userData element contains RAW definition of the value stored in the database

New fields are included by default, but they can be excluded by setting RTR parameter includederivedattributes to N.

<referenceTransaction>
	<header>
		<objectType>FieldAttribute</objectType>
	</header>
	<fieldAttribute>
		<statField>Y</statField>
		<fieldAttributeDescription>DC for subtraction</fieldAttributeDescription>
		<fieldAttributeId>1429</fieldAttributeId>
		<fieldAttributeProcessId>1014</fieldAttributeProcessId>
		<fieldIndicator>DE</fieldIndicator>
		<fieldType>NUM</fieldType>
		<fieldFormat>2</fieldFormat>
		<fieldCategory>&lt;none&gt;</fieldCategory>
		<currencyProcess>NOT</currencyProcess>
		<fieldValueType>NOT</fieldValueType>
		<fieldIdentifier>DETAIL CAL FOR SUBTRACTION - SUPERUSERS</fieldIdentifier>
		<scope>2</scope>
		<readOnly>N</readOnly>
		<processDescription>Expression Calculator</processDescription>
		<processType>USERDEFINED</processType>
		<fieldExpression>H-PD-Market_Value_Income - H-PD-Local_Market_Value</fieldExpression>
		<userData>#`FP1014~#`FD1170~-~#`FD1155~</userData>
	</fieldAttribute>
</referenceTransaction>

The initial configuration steps are required. Described here

This functionality is limited for the calculated fields below with field_attribute_process_id:

1010 Arithmetic Addition Calculator
1011 Arithmetic Subtraction Calculator
1012 Arithmetic Division Calculator
1013 Arithmetic Multiplication Calculator
1014 Expression Calculator
1015 Adjustments Calculator
1025 Trade Adjustment Calculator
1030 IRR Calculator
1031 Modified Dietz Calculator
1032 Dollar Value Added Calculator
1033 Average Invested Balance Calculator

Example extract is attached.