| Table of Contents |
|---|
Streaming Translator
Icon of the streaming translatorIn this page:
| Table of Contents |
|---|
Streaming translators are used for big source files (100mb and 100Mb or more) and working work faster than classic translators. The main benefit is significantly smaller memory consumption. Streaming allows us not to load the whole file or even any sizeable chunk into memory.
...
| title | Note |
|---|
...
Streaming Translator
Icon of the streaming translator:
| Info |
|---|
Currently, streaming transformation works only for the next following data formats: -CSV to CSV -CSV to STAR -CSV to XML -STAR to CSV -STAR to STAR -STAR to XML. |
Here is an example of creating setting up a streaming translatorstranslator:
- Open IWS and create a simple solution with Task Translator:
- Create new transformation (for example CSV2STAR) in task translator:
Add simple data for both source and select “streaming” in the top of window:
Info title Note Oonce u select streaming translator? you cant change it. Also, if you did not select "streaming" in current phase you cant change it later
- Click “Apply”.
It’s “mapping” worksheet.
For the streaming translators we able to use “Preproc” worksheet (for example we can add a debug status for testing). Logic in “Preproc” tab is executed before the incoming file is processed:
Info title Note This worksheet already have the “Data Query” shape for handling incoming parameters. It’s created automatically
- Also, we can use “Postproc” feature on worksheet(for example add status shape for debugging). It is analogue of “Result Output translation” for regular translators:
...
For turn on python translator user need turn on the “Python Translation” checkbox:
