Description:

The Evaluate Expressions and Set Values step evaluates one or more configurable expressions and assigns the resulting values to specified ETL columns. The transformation is configured through the Configure Expression List property, where ETL designers define the expressions, destination columns, and assignment rules.

Many enterprise ETL processes require calculated or derived values that are not directly available in the source data. Examples include computing discounts, generating status values, concatenating fields, calculating totals, assigning business classifications, or creating derived attributes based on multiple source columns. This transformation allows these calculations to be performed declaratively without requiring custom scripting.

Each configured expression is evaluated during ETL execution, and the resulting value is assigned to the configured destination column. The calculated values immediately become available to downstream ETL components.

Unlike Transform Number Operations, which focuses on arithmetic operations on numeric values, this step evaluates configurable expressions that can combine values, apply conditional logic, and populate target columns based on business rules.

The Evaluate Expressions and Set Values step is typically placed after data import and validation but before filtering, aggregation, reporting, exporting, or database loading.

The step supports:

• Expression evaluation
• Computed column values
• Business rule implementation
• Derived field generation
• Multiple expression definitions
• Enterprise ETL workflows
• Integration with FlowWright ETL pipelines

Typical business uses include:

• Calculating order totals
• Deriving employee classifications
• Assigning workflow status values
• Computing business metrics
• Generating reporting attributes
• Preparing integration fields
• Enriching enterprise datasets

Usage:

The Evaluate Expressions and Set Values step is typically inserted after importing and validating source data but before grouping, filtering, reporting, exporting, or loading the transformed dataset into downstream systems.

During execution, each configured expression is evaluated using the current row values. The resulting value is written to the configured destination column, allowing subsequent ETL components to use the newly computed data.

A typical ETL pipeline might look like this:

Typical usage scenarios include:

• Calculating extended prices from quantity and unit price
• Generating customer priority levels
• Assigning approval status values
• Creating reporting categories
• Computing employee bonus amounts
• Deriving composite business attributes
• Preparing enriched datasets for analytics

The calculated values can then be validated, filtered, grouped, aggregated, exported, or loaded into downstream business systems.

Prerequisite:

The ETL process engine service should be running to execute the ETL definitions.

Example:

Let’s build and execute the “clsComputeExpressionSetValueDef” example.          

• Create a new ETL definition named “clsComputeExpressionSetValueDef” and open it in designer mode.  
• Drag “clsInputCSV, clsGroupByTransform, clsComputeExpressionSetValue, and clsOutputCSV” steps to the canvas.
• Connect the dots between the “clsInputCSV” step and other steps, as shown above.
• Define a variable or a global variable to store the virtual path. 
• Click the "clsInputCSV" step to configure its "Required" properties. Enter a name for the step. Enter the path to the input CSV file on the application server. Select the ETL data schema from the drop-down list. Note: The schema's column names should match those in the input file. Click the Save button. 

• Click the "clsInputCSV" step to view its "Optional" properties. Select the Show Schema button. The pop-up window displays the schema columns as shown below. This function helps to understand the schema at a glance.  

• Click the "clsGroupByTransform" step to configure its "Required" properties. Provide a name for the step. Click the button to configure the columns and operation type. Then click the Save button. 

• Click the "clsGroupByTransform" step to configure its "Required" properties. Click the button to configure the columns. A pop-up window will appear for configuration. Click the Add Row (+) button to insert an empty row. Select the column to group by. Click the Save button. You may add multiple columns for transformation by using the Add Row button. 

• Click the "clsGroupByTransform" step to configure its "Required" properties. Click the button to set the operation type. A pop-up window will appear for configuration. Click the Add Row (+) button to insert an empty row. Select the value column from the drop-down list. Select the operator type. Select the output column from the drop-down list. Click the Save button. You may add multiple columns for transformation by using the Add Row button. 

• Click the "clsComputeExpressionSetValue" step to configure its "Required" properties. Provide a name for the step. Click the button to configure the expression list, then click Save. 

• Click the "clsComputeExpressionSetValue" step to configure its "Required" properties. Provide a name for the step. Click the button to configure the expression list. A pop-up window will appear. Click the Add Row (+) button to insert an empty row. Enter the expression, the value expression (for match), and the else value expression (for no match) for evaluation. Click the Save button. You may add multiple transformation columns by using the Add Row button. 

• Click the "clsOutputCSV" step to configure its "Required" properties. Enter a name for the step. Enter the path to the output CSV file on the application server. Click the Save button. 

• Click the "clsOutputCSV" step to configure its "Optional" properties. Provide a name for the step. Provide a variable or global reference to store the virtual output path. Click the Save button. 

• The “Logging” configuration is necessary for documentation and also measures workflow progress and percent complete. This is achieved by configuring the step state and percent fields individually, as shown below. Configure the “Logging” using the following properties.

• Save the ETL definition, create a new ETL instance, and execute. Render the ETL instance. Click the ETL step to view its properties. The “clsComputeExpressionSetValue” step should transform the output file by evaluating the expression. The variable or global reference holds the output file’s virtual path, as shown below. 

Tips:

• Validate source data before evaluating expressions to reduce calculation errors.
• Use meaningful destination column names that clearly identify computed values.
• Keep complex business rules documented for easier maintenance and auditing.
• Preserve original source values if comparison or auditing is required.
• Test expressions using representative production datasets before deployment.
• Review expression logic whenever business rules or calculation formulas change.
• Combine this transform with Validate Data, Transform Number Operations, Transform Number Range, Filter with Conditions, Group By Transform, JSON Output File, or SQL Output components to create comprehensive enterprise ETL workflows.

Notes:

• Multiple expressions can be configured within a single transformation.
• Destination columns must exist within the ETL schema or be configured appropriately before execution.
• Expression evaluation is performed for each row processed by the ETL pipeline.
• Calculated values immediately become available to downstream ETL components.
• Invalid expressions or column mappings may result in ETL execution errors.
• This transform is commonly used for financial calculations, business rule automation, KPI generation, ERP migration, CRM integration, reporting, business intelligence, and enterprise data enrichment.
• Review downstream calculations whenever expression logic or business rules are modified.

Evaluate Expressions and Set Values vs. Transform Number Operations:

Although both transformations can produce calculated values, they are designed for different purposes:

As a general guideline:

• Use Evaluate Expressions and Set Values when calculated values are derived from configurable expressions or business rules.
• Use Transform Number Operations when straightforward arithmetic operations are required on numeric columns.

Definition Sample:

You may download the sample ETL definition(s) from the link provided and later import them into your FlowWright ETL Designer.

Note: Verify and complete any missing configuration after importing the sample, including:

• Expression definitions
• Destination column mappings
• Business rule logic
• ETL schema validation
• Downstream transformation mappings
• Destination component configuration
• Environment-specific settings

After verifying the configuration, save the ETL Definition before execution.

Click here to download the sample file.