The Until activity in Azure Data Factory (ADF) is a control flow activity that allows you to repeatedly execute a set of activities until a specified condition is met. It is like a "do-while" loop in traditional programming languages. The Until activity is particularly useful for scenarios where you need to perform operations such as polling an external system for a status update, retrying operations until they succeed, or processing data in chunks until all data has been processed.

ADF- What is Until activity in ADF?

Definition:

This is an activity in Azure Data Factory, using which you can run a set of activities in a loop until a specific condition is met.

• This is like a Do While loop we write in programming.
• This activity guarantees that the loop will execute at least once and check the condition at the end.

ADF – Until activity implementation

Requirement:

• The requirement is quite simple and for explanatory purposes. It will obviously be more challenging scenarios when working on live projects.
• The variable value is 1 and we will run activity until the variable's value is 2.

Implementation:

• Create a variable with value = 1.



• Add an Until Activity.
  • Click on the activity.
  • In the Settings tab, there is a property called Expression.
  • Expression: Specify the condition that determines when the loop should stop. The condition must evaluate to a Boolean value (true or false). The loop continues until the expression evaluates to true.
    • Example: @equals(activity('CheckStatus').output.status, 'Completed')
  • Until the Expression output value is not true, the Activity within this Until will keep running.
    
• As you can see in the screenshot, the expression says that do this activity until the value of the Variable becomes 2.
• If we go within the Until activity, we have used a Set activity, which sets the variable to value 2 (Keeping it simple).
• Activities: Inside the Until activity, add one or more activities that should be executed in each iteration of the loop. This can include any control flow or data movement activities, such as a Web Activity to check the status of an external process or a Copy Activity to move data in chunks.
  
• Now, as the variable value becomes 2, it will not execute the next time, and it will just end the activity.
  
• Timeout: Optionally, you can specify a timeout for the Until activity. If the specified time elapses before the condition evaluates to true, the Until activity will fail.
• Settings:
• Batch Count: The number of iterations to run in parallel, if applicable.
• Delay: The time to wait between iterations, specified in seconds. This is useful for scenarios like polling, where you want to wait for a certain amount of time before checking a condition again.

Usage Scenarios

• Polling an External Process: One common use case for the Until activity is to poll an external process or service until a specific condition is met, such as waiting for a file to be available in a storage account or waiting for a long-running process to complete.
• Retry Mechanisms: The Until activity can be used to implement retry logic for operations that may fail due to transient errors. By enclosing the operation in an Until activity with a condition that checks for success, you can ensure the operation is retried until it succeeds, or a maximum number of attempts is reached.
• Processing Data in Chunks: In scenarios where you need to process large volumes of data in chunks, the Until activity can be used to iteratively process each chunk until all data has been processed. This is particularly useful when dealing with APIs that have rate limits or when processing large files that need to be split into smaller, more manageable pieces.