Advanced Flow Control
Master iteration, aggregation, and bundle context
Overview
Understanding how bundles flow through iterators and aggregators is essential for building complex automations. This guide covers the core concepts that power Serenities' data processing capabilities.
Iterators
Break arrays into individual bundles for processing one item at a time.
Aggregators
Collect bundles back into arrays or combined outputs.
Bundle Context
Track which iteration a bundle belongs to with scope stacks.
Bundle Limit
Control how many bundles flow through each iteration context.
Understanding Bundles
A bundle is a single unit of data flowing through your flow. When a node executes, it produces one or more bundles that are passed to downstream nodes.
Bundle Properties
{
data: { ... }, // The actual payload
bundleIndex: 0, // Position in current iteration (0-based)
bundleTotal: 10, // Total bundles in current iteration
iteratorId: "node_123", // Which iterator created this bundle
iteratorScopeStack: [...] // Nested context tracking
}Every bundle carries context about where it came from and which iteration it belongs to.
Loops & Iteration
Iterators transform arrays into individual bundles. Each array element becomes a separate bundle that flows through downstream nodes independently.
Basic Iteration
Input: [A, B, C, D, E]
↓
Array Iterator
↓
Output: Bundle 1 (A) → Bundle 2 (B) → Bundle 3 (C) → Bundle 4 (D) → Bundle 5 (E)Each bundle carries bundleIndex (0-4) and bundleTotal (5).
Iterator Scope Stack
When you nest iterators, each creates a new scope layer. The scope stack tracks all active iterator contexts from outermost to innermost.
i1 (outer) → i2 (inner) → Process Node
Scope Stack during i2 execution:
[
{ iteratorId: "i1", bundleIndex: 2, bundleTotal: 5 }, // Outer context
{ iteratorId: "i2", bundleIndex: 0, bundleTotal: 3 } // Inner context (current)
]The innermost iterator is always at the end of the stack. This is crucial for understanding how Bundle Limit and Aggregators work.
Loop Utility
The Loop utility creates bundles from a numeric range, useful for repeat operations:
Loop (start: 1, end: 5)
↓
Bundle 1 (value: 1) → Bundle 2 (value: 2) → ... → Bundle 5 (value: 5)Aggregators
Aggregators collect bundles back into a single output. They wait for all bundles from their source iterator before producing a result.
Array Aggregator
Iterator → Process → Array Aggregator Bundle 1 (processed A) ─┐ Bundle 2 (processed B) ─┼→ Array Aggregator → [processed A, B, C, D, E] Bundle 3 (processed C) ─┤ Bundle 4 (processed D) ─┤ Bundle 5 (processed E) ─┘
The aggregator knows how many bundles to expect from bundleTotal.
Aggregator Finalization
When an aggregator finalizes (receives all expected bundles), it pops its source iterator from the scope stack. This is critical for understanding context.
Before a2 finalization: Scope: [i1, i2] ← Both iterators active After a2 finalization: Scope: [i1] ← i2 removed, only i1 remains
Key insight: After an aggregator, subsequent nodes operate in the OUTER iterator's context, not the inner one.
Text Aggregator
Combines bundle values into a single string with a separator:
Iterator (names) → Text Aggregator (separator: ", ") Bundle 1 (Alice) ─┐ Bundle 2 (Bob) ─┼→ Text Aggregator → "Alice, Bob, Charlie" Bundle 3 (Charlie)┘
Nested Iteration
Complex data often requires nested loops. Understanding how scope stacks work is essential for correct data processing.
Two-Level Nesting
i1 (3 items) → i2 (4 items per) → Process → a2 Execution order: i1[0] → i2[0], i2[1], i2[2], i2[3] → a2 aggregates 4 items i1[1] → i2[0], i2[1], i2[2], i2[3] → a2 aggregates 4 items i1[2] → i2[0], i2[1], i2[2], i2[3] → a2 aggregates 4 items Total executions: 3 × 4 = 12 Total a2 outputs: 3 (one per i1 iteration)
Three-Level Nesting
i1 (2) → i2 (3) → i3 (4) → Process → a3 → a2 Scope during i3 processing: [i1, i2, i3] After a3: [i1, i2] After a2: [i1] Operations: 2 × 3 × 4 = 24 a3 outputs: 2 × 3 = 6 (one per i2 iteration) a2 outputs: 2 (one per i1 iteration)
Real-World Example: Orders with Items
HTTP (get orders) → i1 (orders) → i2 (order.items) → Enrich Item → a2 → Update Order Order 1: [Item A, Item B, Item C] Order 2: [Item X, Item Y] Flow: Order 1 → Enrich A, B, C → Aggregate → Update Order 1 with enriched items Order 2 → Enrich X, Y → Aggregate → Update Order 2 with enriched items
Bundle Limit & Context
Bundle Limit controls how many bundles pass through. Understanding which iterator it affects is crucial for correct behavior.
Immediate Parent Iterator Rule
Bundle Limit always affects the innermost iterator in the current scope stack. It never reaches past to affect outer iterators.
i1 (5) → i2 (6) → Bundle Limit (keep_first: 3) → a2 What happens: • Bundle Limit targets i2 (innermost) • Each i1 iteration: i2 limited to 3 bundles • i1 keeps all 5 iterations • Total operations: 5 × 3 = 15
Limiting Outer Iterator
To limit an outer iterator, place Bundle Limit after the inner aggregator:
i1 (5) → i2 (6) → a2 → Bundle Limit (keep_first: 3) What happens: • After a2, scope is [i1] only • Bundle Limit targets i1 (now innermost) • Only first 3 of i1's iterations continue • Total i1 iterations: 3 (not 5)
Per-Parent-Bundle State
Bundle Limit maintains separate state for each outer iteration combination:
i1 (3) → i2 (10) → Bundle Limit (keep_first: 2) → Process For i1[0]: i2 bundles 0,1 pass (2 bundles) For i1[1]: i2 bundles 0,1 pass (2 bundles) ← Counter resets! For i1[2]: i2 bundles 0,1 pass (2 bundles) Each outer iteration gets fresh Bundle Limit state.
Multiple Bundle Limits
You can chain multiple Bundle Limits. Each maintains independent state:
i1 (20) → Limit #1 (keep: 10) → IF → Limit #2 (keep: 3) → Process Flow: • Limit #1: Passes first 10 bundles • IF: May filter some (say 6 pass) • Limit #2: Passes first 3 of those 6 Result: 3 bundles processed
Post-IF counting: After IF/Bundle Filter, Bundle Limit counts independently. Only bundles that pass the condition are counted toward the limit.
Error Handling
Understanding how errors propagate through iterations is important for building robust flows.
Error in Iteration
Iterator (5 items) → Process (fails on item 3) Default behavior: • Items 1, 2: Processed successfully • Item 3: Error - execution stops • Items 4, 5: Never processed
Continue on Error
Enable "Continue on Error" on nodes to process remaining items:
Iterator (5 items) → Process [continue on error] → Aggregator With continue on error: • Items 1, 2: Processed successfully • Item 3: Error logged, continues • Items 4, 5: Processed successfully • Aggregator receives 4 successful results
Error Branches
Route errors to dedicated handling logic:
┌→ Success Path → Continue
Iterator → Process ─┤
└→ Error Path → Log Error → NotifyCommon Patterns
Pattern: Batch Processing
Process large datasets in controlled batches:
HTTP (1000 records) → Iterator → Bundle Limit (100) → Process → Aggregator
↑
Only first 100 items processed per runPattern: Transform & Collect
Transform each item and collect results:
Get Users → Iterator → Enrich User → Format → Array Aggregator → Save All
Pattern: Filter & Process
Filter items before heavy processing:
Iterator → IF (status = active) → Expensive API Call → Aggregator
↓
Inactive items skip the API callPattern: Nested Data Transformation
Process nested arrays maintaining structure:
Get Categories → i1 → i2 (products) → Transform → a2 → Update Category
↑ ↑
Outer loop Inner aggregator preserves
(categories) products per categoryBest Practices
1. Match iterators with aggregators: Every iterator should have a corresponding aggregator if you need to collect results. Unmatched iterators leave open scope contexts.
2. Limit early: Place Bundle Limit right after the iterator to avoid unnecessary processing of items that will be filtered out.
3. Use Break Iteration: When you find what you need, use Break Iteration to stop processing remaining bundles and save resources.
4. Test with small datasets: Use Bundle Limit during development to test with fewer items, then remove or increase the limit for production.
Watch out: Deep nesting (3+ levels) can be complex to debug. Consider breaking into separate flows connected via webhooks for maintainability.