For Each Block

TABLE OF CONTENTS

• In Brief
• Case Study
  • Workflow
    • Iteration parameters:
    • Output:
    • Advanced:
  • Output
  • Visualising the results
• Additional blocks to execute

In Brief

Function: The For Each block repeatedly executes a workflow, once for every record in the Iterations input, appending the results. Each execution will be customised by changing parameter values using fields in the Iterations data.

Typical Use Case: A company designed a workflow to prepare and analyse a dataset. The result of the workflow is used to power a dashboard highlighting key aspects of the analysis. The company would like to compare the analysis with other analyses of the same dataset where some options in blocks used in the workflow are changed. Without the For Each block, the company now needs to duplicate the workflow, change the options, run the analysis again and append the results to update the dashboard. With the For Each block, this can be automated.

Description: In the aforementioned example, it was described that the For Each block can automatically change options and re-run a workflow, or parts of it, and append the results of each iteration. This is done by utilising App or Project Parameters. After building a workflow, we need to identify block options for which we would like to execute the workflow with different values. Then, we create App or Project parameters with some meaningful initial values, and link the options to these parameters. Next, we create a tabular text input in which the fields correspond to the parameter names we just created. Each row is an iteration, and each cell in a row specifies the value we would like to give the parameter for this iteration.

Case Study

We are looking at Kaplan-Meier survival curves for patients of different hospitals. These curves visualise the probability that patients survive up to a certain time. The goal is to compare the curves of different hospitals to see in which patients have better chances of survival.

Workflow

The workflow is available here.

We are looking at a workflow which contains patient data in a Data Table Block. The data is filtered by a Record Filter Block which is configured to only let patients from one particular hospital pass. Next comes the Survival Analysis Custom Block which computes the Kaplan-Meier curves.

Initial Workflow

If we have a look at the Record Filter options, we see that it is currently configured to filter everything except patients from Saint Mary hospital.

Record Filter options

Without the For Each block, we would need to duplicate the Record Filter and Survival Analysis Block multiple times, once for each hospital. Using the For Each block, we automate these iterations.

We continue by parameterising the Record Filter with a Project Parameter. For this, we go to the 3-dots menu and select "Parameters".

3-dots menu

In the Parameters menu, we create a new Project Text Paramameter which we call "Hospital" and which we give the default value "Saint Mary".

Parameters menu

Now we can assign this parameter to the Record filter by clicking on the Parameter-link icon on the upper-right of the option and selecting "Hospital".

Parameterised Record Filter

It is now time to add a For Each Block and a Text Input Block to the workflow.

Final Workflow

The Text Input Block simply lists all the Hospitals over we wish to iterate. Notice that the Field name corresponds to the Parameter "Hospital" we previously created.

Text Input Block

Lastly, we have to configure the For Each Block. In this particular example, we don't actually need to change any options. However, the block allows us to configure various input and output options:

Iteration parameters:

Use all fields: If selected, all fields from the Iterations Input will be used to override parameters. If fields are defined which do not correspond to an App or Project Parameter, a warning is shown. When not selected, it is possible to choose which fields to use from a field selector.

Output:

Add field containing iteration number: If selected, a field with the given name is added to the output. If a field already exists with this name, a number is appended.

Include all Iteration input fields: If selected, all Iteration Input fields are included in the block output. If not, a subset can be chosen.

Include all Workflow input fields: If selected, all Workflow Input fields are included in the block output. If not, a subset can be chosen.

Advanced:

Some blocks produce a different output with different input data. For example, the Pivot Block adds new fields to its output data depending on the fields to pivot on from the input data. If such blocks exist in your workflow, it is possible each For Each iteration will produce a different output schema. If Require same schema is selected, and the schema changes from iteration to iteration, the execution is stopped. If Combine same schema is selected, the execution is not stopped, and all the different outputs are collected and combined. In this mode it operates the same way as the Append Block.

Output

The output of the block contains a new field with the default name "Iteration", or whatever was chosen in the options, as well as all, or a subset, of the Iteration input fields, followed by all, or a subset, of the Workflow input fields.

For Each outputs

Visualising the results

The results are visualised by creating a series of plots visualising the survival rates.



Additional blocks to execute

Every block connected in some way to the The For-Each workflow input will be executed during the loop execution. This unfortunately limits its use to blocks that have an output. Blocks without an output simply cannot subsequently be connected to a For-Each Block, which means that it would not be possible to utilise e.g. File Output blocks in a loop execution. For this reason we have added an option called "Additional blocks to execute". Here, you can define blocks that should also be executed (as well as their inputs) when the loop runs. These blocks, and also all blocks connected to them, will become part of the parameterisation defined in the iterations input.

The option can be found towards the bottom:

Clicking on the dropdown reveals all blocks that can become part of the loop execution. Adding blocks that are already part of the execution (e.g. because they are connected to the workflow input) has no effect. Cycles are not permitted: If the blocks are part of a workflow that contains other For-Each blocks, which reference in some way a workflow that leads back to this For-Each block, an error is shown to prevent infinite executions.