Description:

This step captures the CSV input file transform errors.

Usage:

Prerequisite:

ETL process engine service should be running to execute the ETL definitions.

Example:

Let’s build and execute the “clsErrorCSVDef” example.          

• Create a new definition called “clsErrorCSVDef”
• Select the definition and click the “design” button
• Drag “clsInputCSV, clsTransformFormatStringToNumber, and clsOutputCSV” step from the toolbox
• Connect the dots between the Start and other steps as above
• Define variables/globals required for the execution
• Click the "clsInputCSV" step to configure its "Required" properties. Provide a name to the step. Provide the input CSV file path on the application server. Select the ETL data schema from the drop-down list. Note: The column names in the schema should match the columns in the input file. Click the Save button. 

• Click the "clsInputCSV" step to view its "Optional" properties. Select the Show Schema button. The popup window renders the schema columns as below. This function helps to understand the schema as a quick view.  

• Click the "clsValidateData" step to configure its "Required" properties. Provide the step name. Click the button to configure the expressions. Click the Save button. 

• Click the button to configure the expressions. The popup window is displayed for configuration. Click the Add Row (+) button to insert an empty row. Select the column and the operator from the drop-down list. Provide the value to complete the expression. Select the And/Or operator to add multiple expressions. Click the Save button. 

• Click the "clsValidateData" step to configure its "Optional" properties. Click the button to configure the column types. The popup window is displayed for configuration. Click the Add Row (+) button to insert an empty row. Select the column and the data type from the drop-down list. Click the Add Row button to insert multiple column types for validation. Click the Save button. 

• Click the "clsValidateData" step to configure its "Optional" properties. Click the button to configure the column using regular expression. The popup window is displayed for configuration. Click the Add Row (+) button to insert an empty row. Select the column from the drop-down list. Provide the regular expression. Click the Save button. Click the Add Row button to insert multiple columns. 

• Click the "clsOutputCSV" step to configure its "Required" properties. Provide a name to the step. Provide the output CSV file path on the application server. Click the Save button. 

• Click the "clsOutputCSV" step to configure its "Optional" properties. Provide a name to the step. Provide the variable/global reference to hold the virtual output path. Click the Save button. 

• Click the "clsErrorCSV" step to configure its "Required" properties. Provide a name to the step. Provide the output CSV file path on the application server. Click the Save button. 

• Click the "clsErrorCSV" step to configure its "Optional" properties. Provide a name to the step. Provide the variable/global reference to hold the virtual output path. Click the Save button. 

• The “Logging” setting configuration is necessary for documentation and also measures the workflow progress and the percent complete. This is achieved by configuring the step state and percent fields individually, as shown in the images below. Configure the “Logging” using the following properties.

• Save the ETL definition, create a new instance, and execute. Render the ETL instance. 

• Click the Items—Variables menu option to view the variable values. The variable/global holds the output file virtual path, as shown below. 

Definition Sample:

You may download the sample definition(s) from the link here and later import it (drag-drop) to your FlowWright ETL Process Definition (XML file) page.

NOTE: Please verify and complete the process steps for any missing configurations, such as file path references and database connections after import. Then, save the definition to confirm the changes.

Click here to download the sample file.