Steps and Shaping Data
How items flow from step to step, how to write expressions and text templates, how to use data from earlier steps, and what each shaping step does: Set fields, Filter, Sort, Group and summarise, Formula, Date, Regular expression and the rest.
4 min read
On this page
Every step is handed a list of items and hands on a list of its own. A trigger hands on one item (the event, the row, the request); a reading step hands on what it read (every bank, a page of transactions); a shaping step changes, keeps or combines items; an acting step does something with each. This page is about the shaping steps and the expressions they take.
Once per item, or once for the list#
A step either runs once per item (a text template is filled in for each transaction) or once for the list (a sort orders the whole list). The library says which, and a step that can do either shows Change mode in its Setup. Inside a loop, "the list" is the one iteration's items.
Ports matter too. Most steps have one output, main. Some have a second: Filter hands the items it drops to rejected; Approval hands on approved and rejected; JSON and CSV send text they cannot read to an error port; a loop or branch can hand a failure on the same way. A wire from a second port is how a run carries on with the items that did not fit.
Data from earlier steps#
A step's Setup shows Data from earlier steps: every step visible from this one, nearest first, each with its fields and a sample value. Clicking a field puts it into the setting you last clicked, as a pill, or at the cursor in a formula. Only steps on the path to this one are visible: a step cannot read a step in another branch or another parallel path.
Expressions#
Any setting can be written as an expression instead of a fixed value: press the ƒx mark on the field. The language is the one table formulas use, with a few additions for automations. It is committed when you leave the field or press ⌘Enter (Ctrl+Enter); Esc puts the saved text back.
What an expression can read
| Name | What it is |
|---|---|
item | The item this step is handling (once per item), or the list (once for the list). |
items | The whole list handed to this step. |
index | The position of item in the list, from 0. |
node('name') | What an earlier step gave, by the name under its Notes → Advanced: node('read_balance').balance. |
trigger | What the trigger handed on. |
inputs | What the run was started with (by hand, by a call, or by an API). |
config | The automation's values from Settings: config.threshold. |
locals | Values a Remember values step set in this scope. |
run | About this run: its id and when it started. |
automation | About this automation. |
error | Inside a Try's handler: the failure being handled. |
current | Inside map, filter and the other list functions: the element being looked at. |
Paths and values. Read a field with a dot: item.account.name, trigger.data.bankId. Read a list's element with brackets: items[0]. Write a list as [1, 2, 3] and an object as { name: item.name, total: item.amount }. Text is in double quotes; numbers and true / false are bare.
Operators. = and != compare; <, <=, >, >= order; +, -, *, /, % and ^ do arithmetic; and, or and not combine conditions; a ? b : c chooses. not binds looser than a comparison, so not item.pending = true means not (item.pending = true).
Functions. The same library tables have, called as name(args) or with a dot: sum(items.amount) and items.map(current.amount).sum() are the same. Among them: if, ifs, switch, empty, coalesce; text — concat, lower, upper, trim, length, contains, find, left, right, substring, replace, replaceAll, split, regexMatch, regexExtract, regexReplace, format, toText, toNumber; numbers — abs, round, roundUp, roundDown, floor, ceil, mod, pow, sqrt, min, max; dates — today, dateAdd, dateSubtract, dateBetween, formatDate, parseDate, startOf, endOf, fiscalYear, period; lists — map, filter, some, every, findItem, first, last, at, slice, reverse, sort, unique, compact, flat, includes, join, count, countValues, sum, average, median, mode, range, stddev, variance, earliest, latest. There is no random(): use the Make a new id step, which records what it made so a replay sees the same one.
Text templates. Text between backticks is filled in: `Balance of ${item.account.name} is ${round(item.balance, 2)}`. Without backticks it is plain text and nothing is filled in. An object in a hole renders as JSON.
Examples
item.amount < 0 and not item.pendingitem.balance < config.threshold ? "low" : "ok"items.filter(current.category = "Fees").map(current.amount).sum()formatDate(dateAdd(today(), -7, "days"), "YYYY-MM-DD")Conditions as rows. A yes/no setting (a filter's condition, a branch's "when") can also be built as rows — Amount · is over · 100 — without writing the expression.
The shaping steps#
All of these are in the Shape data group. They are pure: they read and change items in memory and never touch anything outside the run.
| Step | What it does |
|---|---|
| Set fields | Adds or changes fields on each item from expressions or fixed values. Keep all adds to the item; keep only the set fields hands on just those. |
| Filter | Keeps the items for which the condition is true and sends the rest to rejected. Evaluated per item, with item and index. |
| Sort | Orders the items by one or more keys (up to eight), ascending or descending. Text sorts by code point, so it sorts the same every time. |
| Limit | Keeps the first or last n items. |
| Group and summarise | Groups items by up to four keys and works out a count, sum, minimum, maximum, average, first, last or list for each group — one item per group. |
| Formula | Works out one expression. Once per item (a new value for each) or once for the list (one value from all). |
| Text template | Builds text from a template, once per item: the words between backticks with ${ } holes. |
| Combine lists | Puts two lists together: one after the other, or side by side (pairs by position into { a, b }). Runs once both inputs have arrived. |
| JSON | Reads a text field as data, or turns a value into JSON text, per item. Text that cannot be read goes to the error port. |
| CSV | Reads spreadsheet-style text into items (one per row, with or without a header; comma, semicolon, tab or pipe), or writes the items out as CSV text. |
| Hash | Adds a SHA-256 fingerprint of a value to each item, for spotting repeats or changes. Not for passwords. |
| Date | Formats a date, adds to it, rounds it to the start of a minute, hour, day, week, month or year, or measures between two dates, per item, in the automation's time zone unless you say otherwise. |
| Regular expression | Pulls the first match (or every match) of a pattern out of a value into a field, per item. Patterns run in linear time, so a pattern cannot hang a run. |
| Repeat on dates | Makes one item per date of a schedule (every day, week, two weeks, month, three months, year or few days) from a start, for a number of dates or until an end date, keyed by name and date. |
| Find repeating payments | Finds the payments that repeat in a list of transactions: how often, the next date and the usual amount, one item per repeating payment. |
| Pay down debts | Plans paying off debts month by month (avalanche, snowball or your order) until each is zero: one item per debt per month, and a summary with each debt's payoff date. |
| Skip repeats | Keeps the first item of each key and drops the rest — during this run, or across every run of this automation (remembered in the automation's state). |
| Make a new id | Makes a new random id or secret token for later steps to use. |
| Remember values | Keeps values, by name, for the steps after it in the same scope (locals.name). Every expression is worked out before any value is set. |
| Label the run | Adds labels to the run so you can find it in history later. |
| Read a saved value | Reads a value an earlier run saved — the last date handled, a cursor. Saved on this step, for this automation or for the whole workspace. |
| Save a value | Saves a value for later runs to read. Workspace-wide state is shared by every automation and needs its own permission in the grant. |
Remember values, Label the run and the shaping steps spend no step of a run's budget: a run counts the calls, expression operations, bytes and credits it uses and stops with a budget error if it reaches a limit. Shaping is free; reading and acting are what count.
Finding a step#
The library's search box finds steps by what you type ("balance", "row", "notify", "csv"); Show more pages through the rest. The More → Settings panel also lists the kinds of step your workspace offers and how many of each. A step the workspace cannot run yet — an AI step in a workspace without a model, a mailbox step where mailboxes are not on — is not offered, and an imported automation that uses one is marked in To do for you to replace.
Ask AI steps
Where your workspace runs on BankSync's AI, the Ask AI group offers steps that classify, extract from, draft, summarise, validate or answer a question about your items, each spending your plan's credits. They are not offered in a workspace without a model.
Trust#
What a webhook sent, what a bank or table holds, what a saved value read back, and what an AI step wrote are untrusted: they may have been written by a stranger. A step that cannot be undone will not take its target (which table, which address, which automation) from untrusted data — the target must be a fixed value, a picker, a bound resource, a config value or the run's inputs. Shaping steps carry trust through: a field copied from an untrusted item stays untrusted.
Use this page with your AI assistant
Every BankSync doc is available as plain Markdown for agents and LLMs.