Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 16 Next »

In this page:

Required Parameters

The following table lists required parameters for the Control Message. You must define these parameters for each Control Message used to start the IWS workflow.

NOTE: Parameter names are case-sensitive.

Parameter

Example of Value

Comments

ActionType

EXTRACT

Constant value for all extracts.

StreamName

eagle_ml-2-0_default_cm_execwkfl

Constant value for all extracts.The name of the message stream to which data is sent from the Control Message stream. This stream runs the workflow.

WorkflowName

eagle_wrf_generic_extract


FeedType

SMF,FINANCIALEVENT

Defines the list of object to be extracted.
Should contain a comma-separated combination of values:

  • ENTITY – all ENTITY + RULE will be extracted
  • RULE – all RULE will be extracted
  • REFERENCE – all REFERENCE [ISSUER + SYSTEM + SECURITY] will be extracted
  • ISSUER – all ISSUER group only will be extracted
  • SYSTEM – all SYSTEM group only will be extracted
  • SECURITY – all SECURITY group only will be extracted
  • WAREHOUSE – all WAREHOUSE will be extracted
  • CORPORATEACTIONS – all CORPORATEACTIONS group will be extracted
  • SMF – all SMF will be extracted
  • ACCOUNTING – all ACCOUNTING group will be extracted
  • CORPORATEACTIONS – all CORPORATEACTIONS group will be extracted
  • FINANCIALEVENT – all FINANCIALEVENT will be extracted
  • CLIENT – all CLIENT will be extracted

You can also select only specific extract types from a group, such as: ENTITYLISTLISTEXTRACT, WRHSCLOSELOTEXTRACT, WRHSNAVEXTRACT
For a list of all object groups, see Extract Object Groups.

Compress

Y (default)
N
ZIP
GZIP
GZ
TAR (only UNIX)
TARGZ (only UNIX)
B64
TAR+B64 (only UNIX)
TARGZ+B64 (only UNIX)
GZIP+B64
GZ+B64
ZIP+B64

Y – Default method ZIP (.zip extension)
N – this setting disables compression for extracted files. Available only for FILE delivery method.
ZIP – files will be compressed as ZIP archive (.zip extension)
GZIP, GZ – files will be compressed altogether in one TAR archive, only after this it will be compressed as GZIP archive, because GZIP can work only with one file. (.tar.gz extension)
TAR – files will be compressed as TAR archive (.tar extension)
TARGZ – files will be compressed as TAR archive (.tar.gz extension)
B64– files will be compressed as ZIP archive and then encoded to base64 text
TAR+B64– files will be compressed as TAR archive and then encoded to base64 text
TARGZ+B64– files will be compressed as TARGZ archive and then encoded to base64 text
GZIP+B64 – files will be compressed as TARGZ archive and then encoded to base64 text
GZ+B64 – files will be compressed as TARGZ archive and then encoded to base64 text
ZIP+B64– files will be compressed as ZIP archive and then it will be compress to base64 text

Important notes:

  • If delivery method is JMS, all archive will be automatically base64-encoded
  • Generic Extract workflow and simple extract have several differences:
    1. Generic Extract cannot produce GZIP archives, because GZIP does not allow multiple files, so at first a TAR archive will be created and after this it will be compress as GZIP. Simple extract works fine with GZIP
    2. Default value for compress in Generic Extract is ZIP
    3. Default value for compress in simple extract is G ZIP
  • Compress parameter would affect the way you set Control Message Parameters#CustomExtractLocation parameter:
    1. Compress = Y or specific archivation format: the relative path for CustomExtractLocation is resolved relative to the extracts folder [...]/cmw/extracts/
    2. Compress = N: the relative path for CustomExtractLocation is resolved relative to the Packer stream folder [MessageCenter root directory]/estar/tpe/data/msgcenter/out/TAGVALUE/eagle_ml-2-0_default_cm_packer/

Archive Parameters

In the Control Message, archive parameters specify how results of the data extract are archived.

Note

Use forward slash (/) as folder delimiter.

The following table lists parameters for archiving the data extract results.

Parameter

Example of Value

Default Value

Comments

ExtractDeliveryMethodJMS,FTP,SFTP,FILEFILEChooses the way the extract to be delivered. This setting can affect the set of available parameters you can apply to your extract.
DeliveryStreamName
default value is taken from W_JMS_DEF_EXTRACT_STREAM global parameter in w_config_custom.incYou use this parameter is case of ExtractDeliveryMethod = JMS to use a custom stream

FromFolder

../../../cmw/extracts/

../../../cmw/extracts/

Folder containing files to be archived.

ToFolder

../../../cmw/extracts/

../../../cmw/extracts/

Folder in which the archives should be created.

Note

You must create this destination folder if it does not exist.

Especially useful when you use FTP/SFTP/JMS delivery method, but want to save archive locally in the path different from default

OutputZipFileName

test_new_name_002


Archive name. Only ZIP format compression is available.
If this parameter is specified, the archive name will be generated as "test_new_name_002" and this name will be hardcoded.

Note

  • If you run the extract twice with the same archive name (OutputZipFileName), files will be added to this archive.
  • This parameter has higher priority than "OutputZipNamePattern".
  • If OutputZipFileName and OutputZipNamePattern parameters are not specified, the name will be given as follows: "YYMMDD_HHMMSS"+ :CorrID:
    For example: 141209_113034_1KJ1JSQ8SVMMB15F5
  • This parameter has no sense in case you set Control Message Parameters#Compress to N.

OutputZipFileNamePattern

test_new_name_001


If this parameter is specified, the archive name will be generated as: OutputZipNamePattern:+ :CorrID: so each archive name will be unique.
For example:

test_new_name_001_1M6EX27SPOQZKMUN
test_new_name_001_8Y3ET42SPWEHMLG

Note

  • This parameter has a lower priority than "OutputZipFileName".
  • If OutputZipFileName and OutputZipNamePattern parameters are not specified, the name will be given according to the following pattern: "YYMMDD_HHMMSS"+ :CorrID:
    For example: 141209_113034_1KM1MSQ8SVMMB15F5
  • This parameter has no sense in case you set Control Message Parameters#Compress to N.

OutputZipFileNamePatternEx

tst_DateTimeStamp_my2_@feedtype.zip


Custom name pattern for the output file. List of valid expressions:

  • @CorrelationID 
  • @DateTimeStamp – date-time stamp  in format YYYYMMDDHHMISS
  • @DateTimeStampISO – date-time stamp in format YYYYMMDDTHHMISS (with 'T' separator between date and time values)
  • @DateStamp – date stamp in format YYYYMMDD
  • @RecordCount – count of extracted records
  • @EntityList – passed entity list filter
  • @EntityID  – passed entity id filter
  • @FeedType – passed feed type
  • @entityselectiontype – passed value of "entityselectiontype" task parameter
  • @entityselectiontype2 – passed value of "entityselectiontype2" task parameter
  • @entityselectionvalue – passed value of "entityselectionvalue" task parameter
  • @entityselectionvalue2 – passed value of "entityselectionvalue2" task parameter

    Note

    This parameter has no sense in case you set Control Message Parameters#Compress to N.

FilesAddToZip

*.XML

ProcessCorrId.*

Template files that will be added to the archive.

SendDataToFTP

Y or N

N

Before you use FTP/SFTP delivery, please make sure it is properly configured for your environment. See Configure Delivery Method Options page of EagleML and EJM wiki space.


If this parameter is set to Y, archive data will be loaded to FTP into the CMW\extracts folder. This is an equivalent of ExtractDeliveryMethod = FTP and means that result file will be sent to FTP, even if you choose another ExtractDeliveryMethod option.
You can also assign the default value in w_config_custom.inc:

<COL TAG="W_EAGLE_EXTRACTOR_ZIP_TO_FTP" EXPRESSION="'Y'" USERDESCRIPTION="Move resulting ZIP file to FTP Y or N"/>

or its obsolete variation (described to support backward compatibility):

 Click here to expand...
<COL TAG="EAGLE_EXTRACTOR_ZIP_TO_FTP" EXPRESSION="'1'" USERDESCRIPTION="Move result ZIP file to FTP yes(1) or no(0)"/>

ToFTPFolder is also supported, but this is an obsolete parameter, we highly recommend using CustomExtractLocation.

CustomExtractLocation

../../../cmw/custom_extract/

cmw/extracts/

Defines a custom folder on FTP/SFTP or on server into which the archive is loaded.
Custom extract location for Generic Extract can be specified in the following way:
:STREAMDIR: + CustomExtractLocation value].

Consequently, you cannot set an absolute path to a folder – and this behavior is different from the case of CustomExtractLocation for common extracts (you can set absolute folder paths there). 

Resulting path for defferent delivery methods is made up like this:

For Delivery Method = FILE:
ResultPath = :STREAMDIR: + CustomExtractLocation

For Delivery Method = FTP:
ResultPath = CustomExtractLocation

For Delivery Method = SFTP:
ResultPath = CustomExtractLocation

Note

To use this option for FTP, the ExtractDeliveryMethod parameter must be set to FTP or SendDataToFTP=Y.   

:STREAMDIR: = /envPath/estar/tpe/data/msgcenter/out/TAGVALUE/eagle_ml-2-0_default_cm_packer/
:STREAMDIR: = /envPath/estar/tpe/data/msgcenter/DIRECTION/TYPE/NAME (DIRECTION, TYPE, NAME can be different for different processes, but it is always 3 levels of storage.

CustomFTPFolder

CMW/custom_extract/

CMW/extracts/

Defines a custom FTP folder into which the archive is loaded.

Note

To use this option, the ExtractDeliveryMethod parameter must be set to FTP or SendDataToFTP=Y. 

ExtractSendNotification

1 or 0

1

The same logic is used for SendNotification, but this option can only be applied to extracts within the workflow.

Optional Parameters

Optional parameters include:

  • Pass to Extract parameters. These parameters are passed from the Control Message. If any of these parameters are not supported, they will not be passed along in the Control Messages for the individual extracts.
  • Supported by Extract parameters. These parameters are passed from the Control Messages for individual extracts.

The following table lists Pass to Extract parameters for the ENTITY extract.

NOTE: Parameter names are case-sensitive.

Subprocess name

Common

Date

Entity

Held

SMF

History Only

Source

Delta Based

Own

eagle_generic_extract_entity_and_rule_objects

+

+

+






entitytype, historyonlyflag

--eagle_generic_extract_entity_objects

+

+

+






entitytype, historyonlyflag

----extract_entity_manager

+

+

+







----extract_entity_manager_relationship

+

+

+






entitytype

----extract_entity_accounting_periods

+

+

+







----extract_entity_accounting_basis

+

+

+







---- extract_entity_analytics+++


+

----extract_entity_identifiers

+

+

+






entitytype

----extract_generic_entity

+

+

+






entitytype, historyonlyflag

----extract_entity_bank_relationship

+

+

+







----extract_entity_composite

+

+

+







----extract_entity_list

+

+

+







----extract_entities_in_composite

+

+

+







----extract_entities_in_list

+

+

+







----extract_gips_disclosures

+

+

+







----extract_gips_composite

+

+

+







----extract_entity_entitlement_list

+

+

+






usergroups

----extract_entity_share_class

+

+

+







----extract_entity_regulatory+
+





----extract_entity_security_lending+
+





--eagle_generic_extract_rule_objects

+

+

+







----extract_rule_contract_cash

+

+

+







----extract_rule_contractual_cash_settlement

+

+

+







----extract_rule_earn_thru

+

+

+







----extract_rule_basis_processing

+

+

+







----extract_rule_backup_withholding

+

+

+







----extract_rule_month_end_date

+

+

+







----extract_rule_primary_mixed_cost_basis

+

+

+







----extract_rule_amortization_schedule

+

+

+







----extract_rule_accounting

+

+

+







 
 
The following table lists Pass to Extract parameters for the REFERENCE extract.










eagle_generic_extract_reference_objects

+

+

+

+

+


+


selectiontype, selectionvalue, includeissuerxref, securitysubtype, xreftype, issueralias, issuerid, issuername, codecategory, codecategoryname, codevalue, codevaluename, fromcodecategory, fromcodecategoryname, CalendarBusinessName, CalendarType, DateTypeMacro, OnlyBusinessCalendar, ProcessCenterNamefromcodevalue, fromcodevaluename, fromsourcecode, tocodecategory, tocodecategoryname, tocodevalue, tocodevaluename, tosourcecode

--eagle_generic_extract_issuer_objects

+

+

+


+




includeissuerxref, issueralias, issuerid, issuername

--extract_issuer_entity_relationship

+

+

+


+




includeissuerxref, issueralias, issuerid, issuername

----extract_issuer_relationship

+

+







issueralias, issuerid, issuername

----extract_issuer_role

+

+



+




issueralias, issuerid, issuername

----extract_issuer_organization

+

+







issueralias, issuerid, issuername

----extract_issuer_rating

+

+







issueralias, issuerid, issuername

----extract_issuer

+

+







includeissuerxref, issueralias, issuerid, issuername

--eagle_generic_extract_security_related_objects

+

+


+

+


+


selectiontype, selectionvalue, securitysubtype

----extract_issue_fx_rate

+

+


+

+


+



----extract_issue_fx_pc_rate

+

+


+

+


+



----extract_issue_price

+

+


+

+


+



----extract_issue_exchange_price

+

+


+

+


+



----extract_smfidentifiers

+

+


+

+


+


securitysubtype, xreftype

----extract_schedule

+

+


+

+


+


securitysubtype

----extract_time_series

+

+


+

+


+



----extract_generic_issue_analytic

+

+


+

+


+



----extract_issue_variable_rate

+

+


+

+


+



----extract_rating

+

+


+

+


+


securitysubtype

----extract_cancelsmfidentifiers

+

+


+

+


+



----extract_forward_points

+

+


+

+


+



----extract_factor

+

+


+

+


+



----extract_securityoverrides

+

+



+


+



----extract_naic_designation

+

+


+

+


+



--eagle_generic_extract_system_objects

+

+





+


jobname, codecategory, codecategoryname, codevalue, codevaluename, fromcodecategory, fromcodecategoryname, fromcodevalue, fromcodevaluename, fromsourcecode, tocodecategory, tocodecategoryname, tocodevalue, tocodevaluename, tosourcecode, CalendarBusinessName, CalendarType, DateTypeMacro, OnlyBusinessCalendar, ProcessCenterName

----extract_code_translation

+

+







fromcodecategory, fromcodecategoryname, fromcodevalue, fromcodevaluename, fromsourcecode, tocodecategory, tocodecategoryname, tocodevalue, tocodevaluename, tosourcecode

----extract_code_category

+

+







codecategory, codecategoryname

----extract_code_value

+

+





+


codecategory, codecategoryname, codevalue, codevaluename

----extract_calendar

+

+








----extract_calendar_holiday

+

+







calendarname, calendardate, fromcalendardate, tocalendardate

----extract_calendar_business_dates

+

+







CalendarBusinessName, CalendarType, DateTypeMacro, OnlyBusinessCalendar, ProcessCenterName

----extract_error_definition

+

+








----extract_yield_curve

+

+








----extract_run_time_stamps

+

+







jobname

----extract_ledger_notes

+

+








--eagle_generic_extract_client_objects

+

+







selectiontype, selectionvalue

----extract_client

+

+







selectiontype, selectionvalue

----extract_client_communication

+

+







selectiontype, selectionvalue

 
 
The following table lists Pass to Extract parameters for the WAREHOUSE extract.










eagle_generic_extract_warehouse_objects

+

+

+


+


+


BatchMode, ExcludeCurrentDay, accountingdate, fromaccountingdate, frommonthendaccountingdate, GLBasic, monthendaccountingdate, toaccountingdate, tomonthendaccountingdate, fromvaluationdate, tovaluationdate, valuationdate

--extract_warehouse_cash_activity

+

+

+


+


+


BatchMode, ExcludeCurrentDay

--extract_warehouse_closelot

+

+

+


+


+


BatchMode, ExcludeCurrentDay

--extract_warehouse_openlot

+

+

+


+


+


BatchMode, ExcludeCurrentDay

--extract_warehouse_performance

+

+

+


+


+


BatchMode, ExcludeCurrentDay

--extract_warehouse_position

+

+

+


+


+


BatchMode, ExcludeCurrentDay

--extract_warehouse_trade

+

+

+


+


+


BatchMode, ExcludeCurrentDay

--extract_warehouse_nav

+

+

+


+


+


BatchMode, ExcludeCurrentDay

--extract_warehouse_custody_activity

+

+

+


+


+


BatchMode, ExcludeCurrentDay

--extract_warehouse_custody_balance

+

+

+


+


+


BatchMode, ExcludeCurrentDay

--extract_warehouse_pending_trade

+

+

+


+


+


BatchMode, ExcludeCurrentDay

--extract_warehouse_glbalances

+

+

+


+


+


BatchMode, ExcludeCurrentDay, accountingdate, fromaccountingdate, frommonthendaccountingdate, GLBasic, monthendaccountingdate, toaccountingdate, tomonthendaccountingdate

--extract_warehouse_nav_multi

+

+

+


+


+


BatchMode, ExcludeCurrentDay, fromvaluationdate, tovaluationdate, valuationdate

--extract_warehouse_broker_trade_quote

+

+

+


+


+


BatchMode, ExcludeCurrentDay

--extract_trade_quotes

+

+

+


+





--extract_warehouse_cash_flow_projection

+

+

+


+


+



--extract_warehouse_istarlot

+

+

+


+


+



--extract_warehouse_disclosure_status

+

+

+


+


+



 
 
The following table lists Pass to Extract parameters for the SMF, FINANCIALEVENT, CORPORATEACTIONS, and ACCOUNTING extracts and the extract archiving eagle_ml20_default_cm_packer message stream.










eagle_generic_extract_smf_objects

+

+


+

+


+


asofeffectivedate, securitysubtype

--extract_generic_smf

+

+


+

+


+


asofeffectivedate, securitysubtype

eagle_generic_extract_financial_event

+

+

+


+





--extract_financial_event

+

+

+


+





eagle_generic_extract_corp_actions_objects

+

+


+

+


+



--extract_generic_ca

+

+


+

+


+



--extract_voluntary_ca_election

+

+


+

+


+



eagle_generic_extract_accounting_objects

+

+

+

+

+


+



--extract_accounting_trade

+

+

+


+





--extract_accounting_gl_activity

+

+

+


+




FromGLAccount, GLBasic, GLClass, GLDateType, GLFromDate, GLSector, GLToDate, ToGLAccount

--extract_accounting_traded_cash_activity

+

+

+


+





--extract_accounting_cash_projections

+

+

+

+

+


+



--extract_accounting_accrued_income

+

+

+

+

+


+



--extract_accounting_subscribe_reedem

+

+

+

+

+


+



--extract_accounting_daily_mill_rate

+

+

+

+

+


+



--extract_accounting_unsettled_trans

+

+

+

+

+


+


reportType, reportDate, baseCurrency, accountingBasis, longShortIndicator, cashCategory, eventReason, cashEventType, shareClass, localCurrency, includeCorpAction, includeFwdTransactions

--extract_accounting_cash_journal

+

+

+

+

+


+


reportView, selectView, reportType

--extract_accounting_realized_gl

+

+

+

+

+


+


reportType, reportTypeSource, includeCancels, displayZero, displayUDF

--extract_accounting_settled_ca

+

+

+

+

+


+



--extract_accounting_cons_earned_income

+

+

+

+

+


+


reportType, reportPeriod, startDate, endDate, longShortIndicator, rollupIndicator, accountingBasis, baseCurrency, localCurrency, taxableIndicator, nraTaxCountry, governmentIndicator, processCenter, conversionStatus, urglRecpayIndicator

--extract_accounting_distribution_activity

+

+

+

+

+


+



--extract_accounting_custom_ledger_acct

+

+

+

+

+


+


Switch, ChartOfAccountName, GLAccount, CustomAccount, CustomAccountDescription, GLPostType

--extract_accounting_open_rec_pay_multi

+

+

+

+

+


+


issuename, settlecurrency, brokerName, brokerCode, targerEventId, eventType, tradeTicketNumber, category

--extract_accounting_unapproved_var_margin

+

+

+


+




accountingBasis, baseCurrency, longShortIndicator, reportType

--extract_accounting_trial_balance

+

+

+






selectView, ledgerProfile, reportType, generalLedgerCurrency, accountingBasis, showMEMOAccounts, showAssetLevelDetails, reportFilters, selectSector, sectorID, sector, selectShareClass, shareClass, detailedClassView, reportDate, monthEndDate, filterSecurityBy, issuename

--extract_accounting_intraday_nav

+

+

+







--extract_accounting_nav_for_nasdaq

+

+

+







--extract_accounting_fair_value_comparison

+

+

+







eagle_ml20_default_cm_packer









CustomExtractLocation, ExtractDeliveryMethod, FromFolder, ToFolder, OutputZipFileName, OutputZipFileNamePattern, SendDataToFTP

  • No labels