Description:
This step captures errors during the transformation of the CSV input file.
Inputs
- filePath – output file path
- variableOrGlobal - to hold the output file virtual path
Returns
- None
Usage:
Prerequisite:
The ETL process engine service should be running to execute the ETL definitions.
Example:
Let’s build and execute the “clsErrorCSVDef” example.
- Create a new ETL definition called “clsErrorCSVDef”
- Drag “clsInputCSV, clsTransformFormatStringToNumber, and clsOutputCSV” steps to the canvas.
- Connect the dots between the “clsInputCSV” step and other steps, as shown above.
- Define a variable or a global to store the file path.
- Click the "clsInputCSV" step to configure its "Required" properties. Provide a name for the step. Specify the path to the input CSV file on the application server. Select the ETL data schema from the drop-down list. Note: The schema's column names should match those in the input file. Click the Save button. Click here to learn about the ETL Data Schema Designer.
- Click the "clsInputCSV" step to view its "Optional" properties. Select the Show Schema button. The pop-up window displays the schema columns as shown below. This function helps to understand the schema at a glance.
- Click the "clsValidateData" step to configure its "Required" properties. Enter the step name. Click the button to configure the expressions, then click Save.
- Click the button to configure the expressions. The pop-up window opens for configuration. Click the Add Row (+) button to insert an empty row. Select the column and operator from the drop-down list. Enter the value to complete the expression. Select the And/Or operator to add multiple expressions. Click the Save button. You may add multiple expressions using the Add Row button.
- Click the "clsValidateData" step to configure its "Optional" properties. Click the button to configure column types. A pop-up window appears for configuration. Click the Add Row (+) button to insert an empty row. Select the column and the data type from the drop-down list. Click the Save button. You may insert multiple column types using the Add Row button.
- Click the "clsValidateData" step to configure its "Optional" properties. Click the button to configure the column with a regular expression. The configuration pop-up window appears. Click the Add Row (+) button to insert an empty row. Select the column from the drop-down list. Enter the regular expression. Click the Save button. Click the Add Row button to insert multiple columns.
- Click the "clsOutputCSV" step to configure its "Required" properties. Enter a name for the step. Enter the path to the output CSV file on the application server. Click the Save button.
- Click the "clsOutputCSV" step to configure its "Optional" properties. Provide a name for the step. Provide a variable or global reference to store the virtual output path. Click the Save button.
- Click the "clsErrorCSV" step to configure its "Required" properties. Enter a name for the step. Enter the path to the output CSV file on the application server. Click the Save button.
- Click the "clsErrorCSV" step to configure its "Optional" properties. Provide a name for the step. Provide a variable or global reference to store the virtual output path. Click the Save button.
- The “Logging” configuration is necessary for documentation and also measures workflow progress and percent complete. This is achieved by configuring the step state and percent fields individually, as shown below. Configure the “Logging” using the following properties.
- Save the ETL definition, create a new instance, execute it, and then render the ETL instance.
- Click the Items - Variables menu option to view the variable values. The variable or global contains the virtual path of the output file, as shown below.
Definition Sample:
You may download the sample definition(s) from the link provided and later import them (drag-and-drop) into your FlowWright ETL Process Definition (XML file) page.
Note: Please verify and complete the process steps for any missing configurations, such as file path references and database connections, after the import. Then, save the definition to confirm the changes.