Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

Version 1 Next »

A facility to define distribution profiles to distribute data to multiple destinations was added to Eagle Data Services.

The distribution profiles are JSON files stored in eagle/estar/tpe/dynamic/metadata/distributions/profilename_profile.json

The distribution profile can specify one or several destinations (sinks) to distribute the data. For example to distribute to two different SFTP destinations the following profile can be used:

Sample Distribution Profile
{
"data_sinks": [
       { 
       "sink": "sftp_sink",
       "type": "sftp",
       "credentials": {
           "user_id": "eagle",
           "password": "eagle1"
       },
       "cfg": {
           "host": "inno-mlperfapp01",
           "port": 22,
           "compression_codec": "gzip"
       }
     },
       { 
       "sink": "sftp_extracts",
       "type": "sftp",
       "credentials": {
           "user_id": "eagle",
           "password": "eagle1"
       },
       "cfg": {
           "host": "inno-mlperfapp01",
           "port": 22,
           "compression_codec": "gzip",
           "location": "extracts"
       }
     }
   ]
}


The distribution profile can be specified in the RTR either by name using the DistributionProfileName parameter or as an embedded distribution profile in the DistributionProfileJsonDefinition parameter as a JSON document.

For example if we have a eagle/estar/tpe/dynamic/metadata/distributions/client_destinations_profile.json we can refer it in the RTR by using DistributionProfileName=client_destinations parameter.


Encrypted Credentials

The credentials can be embedded in the distribution profile as shown in the example above, or the credentials can point to an encrypted configuration file.

For example:

"credentials": "clientcredentials/sftpcrendentials"

In this case the credentials are encrypted and retrieved from a file in dynamic/dataservices/configs/clientcredentials.json.

Each encrypted credentials file has a region designation, which is the Oracle SID of the environment. If the credentials file is copied to a different environment it will fail to work.

  • No labels