SDP-45416 Change Default EDS CSV Extract Behavior to Include Null Columns
Changed Default EDS CSV Extract Behavior to Include Null Columns
We enhanced the EDS code to include null columns by default when running a custom extract with CSV. If null columns are wanted on the output for a core EDS resource then you will need to use the parameter includenull=Y.
SDP-45367 Improve xml parser to handle nodes with the same node as object node
Improve xml parser to handle nodes with the same node as object node
During development of load rule for codevalue resource we found out that xml parser can't parse row in case there is a node with same name as object node:<codeValue><codeValue><sourceName>EAGLE PACE</sourceName><updateSource>MCADMIN</updateSource><codeCategoryName>TEST CODE1</codeCategoryName><codeCategoryDescription>TEST CODE1 DESCR</codeCategoryDescription><codeValue>TEST VALUE1</codeValue><codeValueDescription>TEST VALUE1 DESCR</codeValueDescription></codeValue>
For this reason we introduced a new attribute in the dialect, default value = False:"is_single_row_txn": true
So it will give opportunity for the client to resolve this issue as most of the client files has single row in transaction node.
SDP-45249 Allow Compressed Input Files in Profiled Loader for EDS Loads, as is available for EML Loads
Enabled compressed input files in Profiled Loads for EDS inbound
A load profile can now specify a compressed data file for EDS inbound files,in addition to inbound EML files.
SDP-45132 Ontology Vocab Endpoint Not working
Ontology Vocab Endpoint Not working
We fixed error when running eagle/v2/metadata/vocab endpoint. This endpoint will expose all fields in Ontology.
SDP-45069 Fix incorrect idle steps clean up for warehouse sub-process of generic load
Corrected issue with defining start steps for multiple steps in a single load profile
Corrected issue with defining start steps for multiple steps in a single load profile
SDP-44959 Revise fromdateBind and todateBind for reportdictionarylevel Ontology
Changed format of fromdate and todate filters
The format of the fromdate and todate filters has been changed. Filters can be used in timestamp format to filter data.
Date and time can be defined for filtering as follows:
<taskParameter><name>fromdate</name><dataType>S</dataType><value>2020-12-14 07-04-09</value></taskParameter><taskParameter><name>todate</name><dataType>S</dataType><value>2020-12-15 09-04-09</value></taskParameter>
SDP-44779 SMFEXTRACT failed with PY script error in case of EagleML only installation
PY script error in case of EagleML only installation
Exceptions were handled and error messages were improved:
If exception is thrown during XML parsing, the error message is:
XML parsing error. {error_message}')
If exception is thrown during JSON parsing, the error message is:
JSON parsing error. {error_message}')
SDP-44542 Add DistributionProfile as a Delivery Method option for generic extract in MC2
Added support for DistributionProfiles to MC2 Generic Extractor
Added support for DistributionProfiles to MC2 Generic Extractor, to enable defining multiple destinations, supporting SFTP, Cloud, Email, and File delivery types.
SDP-43886 EDS Interfaces - New Metadata - custombenchmark
Created custombenchmark ontology
The Ontology for custombenchmark interface was created.
Files added:
- estar/tpe/dynamic/metadata/core/ontology/entity/custombenchmark.json
- estar/tpe/dynamic/metadata/core/ontology/entity/custombenchmark-models.json
- estar/tpe/dynamic/metadata/core/ontology/entity/custombenchmark_in.json
Data is stored in the following tables:RULESDBO.CUSTOM_INDEX_ATTRIBUTES and RULESDBO.CUSTOM_INDEX_ATTR_DETAIL
Records with customIndexType = CNST or EXCL will also have data in CUSTOM_INDEX_ATTR_DETAIL table, for other customIndexType values data is not stored in CUSTOM_INDEX_ATTR_DETAIL
Records with customIndexType = CNST expect contrainAsset complex node filled + elements constrainFlag, constrainWeight, constrainLevel, constrainDescription.Records with customIndexType = EXCL expect exclusionAsset complex node filled + elements exclusionLevel, exclusionDescription.
Extract method is supported for custombenchmark resource.Available filters :
- effectivedate/fromeffectivedate/toeffectivedate
- entityfilter
- fromdate/todate
- sourcename
Mashups used:
- interfaces
- genericentity
- genericsmf
- genericentity for entity_detail_id
SDP-43742 Under load the tasks status in Message Center Console stays In Progress when task is done
Under heavy concurrency, task status in MCC can show In Progress even though task has completed
Refactored the caching of task Ids to prevent a potential issue updating the task status.
SDP-43026 Passing editcheck cancels previous editcheck failure in EDS interface evaluation
Passing editcheck cancels previous editcheck failure in EDS interface evaluation
We adjusted the code regarding editchecks on EDS interfaces so that a failed editcheck persists as expected. Before this change if one editcheck failed, but the next editcheck passed, it would overwrite the failed editcheck. Now any order of failed editchecks will persist.
SDP-42662 Vault MVP Interfaces - New Metadata - warehousenav
Created warehousenav ontology
The Ontology for warehousenav interface was created.
Files added:
- estar/tpe/dynamic/metadata/core/ontology/warehouse/warehousenav.json
- estar/tpe/dynamic/metadata/core/ontology/warehouse/warehousenav-models.json
- estar/tpe/dynamic/metadata/core/ontology/warehouse/warehousenav.json
SDP-42640 Add genericsmf_asof data object definition
Data object genericsmf_asof added as core data object definition
Data object genericsmf_asof which has genericsmfhistory as its base model was added as core data object. This data object allows to execute adhoc extracts of data and cache data - if needed - under this data object.
Main purpose of this data object is to be stand-in in mashup definition in cases when GenericSMF data should be handled in different manners based on RTR parameters.
SDP-42390 Add Email Capability to Distribution Profiles
Enabled the use of email destination inside of Distribution Profiles
Enabled the definition of email targets inside of a distribution profile.Example:{"data_sinks": [{"sink": "email_sink","type": "EMAIL","cfg":
{"host": "eaglesmtp01.eagleinvsys.com","port": "25",
"mailTo": "abc.xyz@bnymellon.com,abc.xyz@eagleinvsys.com",
"subject": "Email from {env} was delivered by {profile_name}
","contentType": "text/plain",
"mailFrom": "test@test.com"}}] }
SDP-41658 usergroups filter is not applied to cached EDS data
Updated EDS to apply usergroups filter when main object is cached
Updated EDS to apply the usergroups filter when the main object is cached (e.g. genericentity) and the user has not specified disabledatasetcache = Y.