MC2 Release Notes - March 09 2021

Owned by Marius Navickas
Last updated: Jun 06, 2024 by Pamela Totas (Deactivated)
4 min read

SDP-34929 configuration for MC2 to work with RabbitMQ

Enabled RabbitMQ in MC2

It is now possible for MC2 to read from and write to RabbitMQ.
Example entry for RabbitMQ connection:
{
"ConnectionName": "qa.rabbitmq.connection",
"ComponentName": "rabbitmq",
"Parameters": {
"host": "inno-jenever01.eagleinvsys.com",
"port": 5672
}
}
Example entry for RabbitMQ SSL connection:
{
"ConnectionName": "qa.rabbitmq.ssl.connection",
"ComponentName": "rabbitmqssl",
"Parameters": {
"host": "inno-jenever01.eagleinvsys.com",
"port": 5671
},
"SSLTruststoreFile": "${eagle.path.to.root}/estar/tpe/dynamic/mc2/private/rabbitmq/trust-store.p12",
"SSLKeystoreFile" : "${eagle.path.to.root}/estar/tpe/dynamic/mc2/private/rabbitmq/key-store.p12",
"SSLVersion" : "TLSv1.2"
}

SDP-37753 Changes to MC2 Services, MC2 Redis Service and MC2 Kafka Service installation for integration with SMC

Integrated MC2 services with System Management Center

The MC2 services (MC2, REDIS, kafka) are now viewable in the System Management Center Environment/Services window.

SDP-43449 Fixed Parser

Added ability of parsing fixed format files

Parameters for fixed format are optional. Default values are:
"source_format_dialect": {
"headerlines": 1,
"footerlines": 1,
"lineterminator": "n",
"lines_per_row": 1,
"concat_start_pos": 0
}
Supported parameters:
"headerlines" - number of header lines.
"footerlines" - number of footer lines.
"lineterminator" - delimiter between lines.
"lines_per_row" - number of lines on which there is one logical row.
"concat_start_pos" - all lines is trimmed and start at the position specified in the parameter.
There are parameters that can be defined for each column.
Column parameters are optional. Default values are:
"vocabulary": { "COLUMN_NAME": {"startpos": 0, "colwidth": 0} }
Supported column parameters:
"startpos" - start position of column.
"colwidth" - column width
The startpos parameter can be omitted if the required column is located immediately after the end of the previous column.

SDP-44788 New Metadata - genericissueanalytic

Created genericissueanalytic ontology

The Ontology for genericissueanalytic resource was created.
Files added:
estar/tpe/dynamic/metadata/core/ontology/reference/genericissueanalytic.json estar/tpe/dynamic/metadata/core/ontology/reference/genericissueanalytic-models.json estar/tpe/dynamic/metadata/core/ontology/reference/genericissueanalytic_in.json
Extract method is supported for genericissueanalytic resource.
Available filters: effectivedate/fromeffectivedate/toeffectivedate entityfilter fromdate/todate sourcename securityalias processingsecuritytype securitytype investmenttype releasestatus primaryassetid ticker releasedate/fromreleasedate/toreleasedate heldonlyfilter securitysubtype

Mashups used: interfaces genericsmf

SDP-46509 No header/footer in extract if outputorderbyexpr and allowemptyfiles are enabled and source dataset is empty

Corrected EDS issue with allowemptyfiles = Y and an outputorderbyexpr

Corrected an EDS issue with no header or footer in extract when allowemptyfiles = Y and an outputorderbyexpr is specified and no data returned

SDP-46542 Implement into Ontology Canonical Model Output the DB precision and scale

Updated precision and scale values in resourceinfo and model

Added changes to show DB precision and scale values in resourceinfo. Added changes to show in model canonical precision and scale values defined in inventory file. Updated resource validator to check precision and scale.
Example:
{
"element_id": "bidSpreadFactor",
"path": "bidSpreadFactor",
"type": "number",
"type_dialect": { "precision": "26", "scale": "2" } ,
"source_type_dialect": { "precision": "38", "scale": "12" } ,
"label": "Bid Spread Factor element", "definition": ""
},

SDP-46743 Fix data encoding/escaping of special characters in pipe-delimited files

New options introduced for CSV OutputFormatDialect

New options have been introduced for CSV OutputFormatDialect:
"quoting": "QUOTE_ALL"
In this case all values and header will be wrapped in double quote.
Without this option specified only values with delimiter and double quote will be wrapped in double quote.
<taskParameter>
<name>OutputFormatDialect</name>
<value>
{"delimiter":"|","hasheader":"Y", "quoting": "QUOTE_ALL"}
</value>
</taskParameter>

SDP-47026 Implement TSR generation on EDS side

Updated EDS to generate TSR to match EagleML

Updated EDS to generate TSR to match format, name pattern, and delivery options of EagleML TSR, and to include error information.

SDP-47146 Add ability for Catalog API to return available methods

Added ability for Catalog API to return available methods

Added ability for Catalog API to return available methods including custom ontology.
Example of new result:
URL/eagle/v2/metadata/catalog
{
"resources": [
{ "id": "codecategory",
"url": "/eagle/v2/metadata/models/codecategory",
"methods": [ "extract" ],
"scope": "core" }
]
}

SDP-47398 EDS: additional date-time formats in ExtractNamePatternEx parameter

Added optional format codes to 5 keywords for the extractnamepatternex parameter in EDS.

Added optional format codes to 5 keywords for the extractnamepatternex parameter in EDS:

@EFFECTIVEDATE
@FROMEFFECTIVEDATE
@TOEFFECTIVEDATE
@DATETIMESTAMP
@DATESTAMP
All 5 are supported with Distribution Profile distributions.
The first 3 are available with default distribution.
See Date Formats in OutputFile Name for full details

SDP-47575 Research and document how to enable password on MC2 REDIS instance

Enabled optional configuration to require REDIS password

Enabled optional configuration to require REDIS password. See: Enabling REDIS Password

Note: the REDIS password configuration is preserved in the All Services deployment (not in the extractservice deployment).

SDP-47578 Change the MC2 installer to preserve the password configuration in REDIS config files if it is present

Update MC2 Installer to preserve REDIS password if enabled

Updated MC2 Installer to preserve REDIS password if the REDIS configuration file requires and specifies a password.
Note: the REDIS password configuration is preserved in the All Services deployment (not in the extractservice deployment).

SDP-47591 Old EDS codetranslation dataset has bad join definition

Changed join for the eds codetranslation object:
inner join PACE_MASTERDBO.CODE_VALUES FROM_CVAL on FROM_CODE.INSTANCE = FROM_CVAL.CODE_INST AND FROM_CODE.INSTANCE = MASTERTABLE.FROM_CODE -> inner join PACE_MASTERDBO.CODE_VALUES FROM_CVAL on FROM_CODE.INSTANCE = FROM_CVAL.CODE_INST AND FROM_CVAL.INSTANCE = MASTERTABLE.FROM_CODE_VALUE

SDP-47759 Review-Create IssueFXRate EDS Feed and configure to run from Model Office

Created issuefxrate EDS Resource

The Ontology for issuefxrate resource was created.
Files added:
estar/tpe/dynamic/metadata/core/ontology/reference/issuefxrate.json estar/tpe/dynamic/metadata/core/ontology/reference/issuefxrate-models.json
Extract method is supported for issueprice resource.
Available filters :
todate fromdate effectivedate fromeffectivedate toeffectivedate processingsecuritytype securitytype investmenttype releasestatus primaryassetid xrefid ticker securitysubtype securityalias sourcename heldonlyfilter
Mashups used: interfaces genericsmf

SDP-47760 Review - Create IssuePrice EDS Feed and configure to run from Model Office

The Ontology for issueprice resource was created.
Files added:
estar/tpe/dynamic/metadata/core/ontology/reference/issueprice.json estar/tpe/dynamic/metadata/core/ontology/reference/issueprice-models.json
Extract method is supported for issueprice resource.
Available filters :
sourcename fromdate todate effectivedate fromeffectivedate toeffectivedate securityalias processingsecuritytype securitytype investmenttype releasestatus primaryassetid xrefid ticker heldonlyfilter securitysubtype
Mashups used: interfaces genericsmf