...
When you do a date range submission you will have two choices:·
Process Dates Concurrently—good for smaller submits; engines for multiple dates are launched without strict regard to date order. Engines “flood the cluster” to maximize process concurrency. If the system goes down during a concurrent built, you will likely need to restart your build from the beginning, as multiple days would probably be partially finished.
...
Process Dates Sequentially—good for larger submits; engines are launched for first date, and only after they complete are the next day’s engines launched, etc. This is a slower process, but it “throttles” processing so that other workflows are less likely to be adversely impacted. Also, if the system should go down during a sequential build, you will be able to resubmit starting with the day that was being processed when that happened.
You need to determine whether your job is small enough for a concurrent submission. Here is how:· Count Start by counting the number of engines required for your submit, as follows:o
For each model submitted, there
...
is one Model Manager engine for each group of N entities submitted, where N is the “No. of Funds per Model Manager” that you set in the Data Mart Configuration screen.
...
Within each model, there
...
is a number of OLAP report engines launched. To determine how many, find the Execution Logs of a previous build of that model and count the distinct types of OLAP you see. The basic logic
...
is:
...
One per type of OLAP report required (e.g., position, entity, security)
...
Times number of table extensions including the main extension
...
Times number of currencies built including BASE currency.
...
Times number of entities built divided by M, where M is “No. of Funds per Report” that you entered in the Data Mart Configuration screen.
...
For example, you have a group model that is built with entity, position and performance OLAP engines (factor of 3). You have one extension that also requires all 3 OLAP types (factor of 2). You are building for one base currency in addition to BASE (factor of 2). You are building 1000 entities, and “No. of Funds per Report” is 50 (factor of 20). Then you will have 3x2x2x20 = 240 engines launched to build that model.
...
Sum the engines launched all models that you are building in the same date range submission to get a grand total.
...
Divide your grand total by the number of concurrent PACE engines dedicated to OLAPs and Data Mart. If the result is 2 or fewer, you may use concurrent builds. If between 2 and 4, you can perform the entire build as one overall event but on a sequential rather than concurrent basis. If the result is 4 or more, you should use sequential builds phased over a period of time, completing one before launching the next. If you use sequential builds, you should try to schedule them off prime shift.
NOTE: There is a spreadsheet that you may use to enter a particular date range submission and see Eagle’s recommended best practice for using a Concurrent versus a Sequential build approach. This is attached to KB#11487.
You will notice that your date-range submit produces a Data Mart Manager Engine (top-level) log in Execution Logs for zero models. This log is normal—it is the log of the process that launches the several individual Data Mart Manager Engines of a date-range submit. In Data Steward the event associated with this log will appear as the parent event with each daily build as child events. The parent event will be marked as “Complete” almost immediately, as that event completes as soon as it spawns its child events. But if you drill into the parent event you will get a correct picture of the child event completion statuses.
Deleting Fields
Prior to release 9.0, if you delete a field from the user interface, the field will no longer be built, but its table column is not dropped.
Starting in 9.0 you are given the You have the option to drop the table column when you delete a field. But , but should you?·
No, if you want to keep past data for the field for reporting purposes, but not populate it going forward.
...
Yes, if you have no use for past data for the field in question.
Process Manager Keeps the Mart in Synch with the Warehouse
...
A proper build of the Data Mart Lot Level Details Table requires that the lot_number field [DC1] of the holdingdbo.lot_level_details table be unique within a given security_alias. This field should be the sequence number of the portfolio’s buy lot for the security in question. However, some accounting systems including STAR do not populate this field in the way that Data Mart’s underlying OLAP process requires.
If your lot_number field is not unique within each security, you may load rows to this table while populating the lot_number field with the value of the lot_level_position field, which is unique within the entire table. Knowledge Base Article #8005 discusses this.