Skip to main content

Link reference

This section provides a description of different link types. Links should be defined in the workflows' links field, and actions should be defined in workflows/scripts' actions field. The main difference between them is that action handlers need to be triggered separately (for example, when a user clicks a button), while link handlers are triggered automatically whenever any of their inputs change.

Links and actions are created at runtime based on matching rules defined in Link Query Language (LQL). A link will not be created until the workflow state satisfies these matching rules. If changes in the current workflow break the link matching rules, the link will be destroyed.

Common​

Common options​

These options are available for all links and actions types.

id​

Should be unique on the same level.

type​

There are several types of links/actions:

  • data
  • validator
  • pipelineValidator
  • meta
  • nodemeta
  • funccall
  • pipeline
  • rule, see Rules
  • check, see Checks

from​

Describes link inputs via a Link Query Language. Can be either a LQL string or an array of zero or more LQL strings. All LQL queries should have at least one match, otherwise the link will not be created.

to​

Describes link outputs via a Link Query Language. Can be either a LQL string or an array of zero or more LQL strings. All LQL queries should have at least one match, otherwise the link will not be created.

handler​

A function that receives an object with a controller instance, whose type is based on the link type field. The handler function can be async, returning either a Promise<void> or an Observable<void>. No side effects made via controller methods are visible until the handler finishes. Controller must be the only way to read or write state for any handler. Also handlers must not internally mutate received states. If the same handler is triggered before a previous invocation is finished, the previous one will be gracefully canceled and no changes are applied. Handler is mandatory for all links and actions, except for data links, where a default handler can be used if none is provided.

not​

Optional

Can be either a LQL string or an array of zero or more LQL strings.

If any LQL query matches, the link will not be created. Queries are targeting script nodes or workflows. The (call) flag is accepted here as a clarity marker but has no effect — not queries are already node-level.

base​

Optional

Could be either a LQL string or an array of exactly 1 LQL string. Queries are targeting script nodes or workflows. Used in reference selectors.

dataFrameMutations​

Optional

By default internal script io mutations will not trigger links, however for DataFrames it is possible to configure such behavior. If true is set all link DataFrame inputs mutations will trigger the link handler. Also, it is possible to narrow down by matching input LQL names using an array of names.

nodePriority​

Optional

Used to specify link running order if multiple links have the same link output script node targets. Links with a higher priority run before those with a lower one. This is primarily used to order data links between script own inputs, since they don't have a well-defined order in the workflow.

runOnInit​

Optional

For data links and rules: runs the link once at init, after the init hook, even when nothing it reads has changed. See Init hook for when init happens and how its writes are handled.

params​

Optional

A static object attached to the link and readable from the handler with getParam. Lets one handler serve several links with different configuration.

Common controller methods​

These options are available for all controller types.

getAll​

getAll<T = any>(name: string): T[] | undefined;

Receives an input query name. Returns an array of one or more matched items. Throws for an optional query without a match, so check getMatchedInputs first.

getFirst​

getFirst<T = any>(name: string): T | undefined;

Receives an input query name. Returns the first matched item. Throws like getAll for an optional query without a match.

getMatchedInputs​

getMatchedInputs(): Readonly<Set<string>>;

Returns the set of from query names that produced at least one match for the current link instance. Iteration order follows a depth-first traversal of the workflow tree at the moment the link fires; when a query matches multiple nodes, the position of its first match determines its place in the order. Use this to detect which optional inputs are present before reading them with getFirst / getAll.

getMatchedOutputs​

getMatchedOutputs(): Readonly<Set<string>>;

Same as getMatchedInputs, but for the to query names. Use this to find out which optional outputs exist before writing to them.

getMatchedPositions​

type MatchedNodeInfo = {
  path: Readonly<NodePath>;
  position: number;
  ioName?: string;
};

getMatchedPositions(name: string): MatchedNodeInfo[];

Receives a from or to query name. Returns the positions of the nodes it matched, in the same order as getAll values. path is the node path relative to the node the link is defined on. position is the index of the matched node among its parent's children, or -1 when the query matched the link's own node (for example, a node meta output targeting the pipeline itself). Throws for query names without matches, so check getMatchedInputs / getMatchedOutputs first for optional queries.

Use this in node meta handlers to produce position-aware titles or descriptions, for example `Step ${controller.getMatchedPositions('in')[0].position + 1}`.

getBasePosition​

getBasePosition(): MatchedNodeInfo | undefined;

Returns the position of the matched base node for base-instantiated links, or undefined when the link has no base query. Each link instance created from a base match reports the position of its own base node.

getAdditionalParam​

getAdditionalParam(name: string): any | undefined;

Only used in actions, for passing additional parameters from validators.

getParam​

getParam(name: string): any | undefined;

Returns a value from the link params object.

hasCall​

hasCall(name: string): boolean;

Returns true iff name was declared in from with the (call) flag and matched at runtime. Outside FuncCall actions, (call) inputs must also be marked (optional) — the actual DG.FuncCall is not exposed; hasCall is the only way handlers see the result. Throws if name is not a declared (call) input.

Data flow​

Workflow links items with a type field data. Used to pass data between script IO. A script input should be a target of at most 1 data link output at the same time. Also script outputs should not be data links outputs. The controller has the following additional methods:

setAll​

setAll<T = any>(name: string, state: T, restriction?: RestrictionType): void;

Receives an output query name, state, and an optional restriction. Sets all matched outputs to the specified state with the restriction. For more details about restriction see Consistency.

getInputTemplates / getOutputTemplates​

getInputTemplates(): TemplateInfo[];
getOutputTemplates(): TemplateInfo[];

type TemplateInfo = {
  name: string | number;
  ios: {ioName: string, scriptIoId: string}[];
};

Enumerate the template slots produced by (template) wildcard selectors on each side, in spec order. Each entry pairs the link-io name with the underlying script IO id. Returns [] when no wildcard selectors are used.

propagateTemplatePair​

propagateTemplatePair(
  inputTemplate: string | number,
  outputTemplate: string | number,
  defaultRestrictions?: Record<string, RestrictionType> | RestrictionType,
): void;

Name-matches a single (input-template, output-template) pair and runs getFirst → setAll for each matched script IO id. Unmatched ids on either side are silently dropped. Throws if either name is not a known template. Used by the default handler for the template ↔ template case; available to custom handlers that want the same primitive.

clearRestriction​

clearRestriction(name: string): void;

Clears the consistency restriction on all matched outputs of name without changing their current value. After the handler completes the targeted inputs are no longer marked as "set by a link" and no longer report inconsistency — until another link writes them again. Calling clearRestriction on an output that has no active restriction is a no-op. If the handler also calls setAll(name, ...) for the same output, the later of the two calls wins.

See Consistency for the underlying model.

Default handler​

For data links it is possible to omit handler and use a default one.

Each top-level entry in from/to is one slot. The default handler pairs slots by index. For each pair:

  • bare ↔ bare: the first matched input is passed to the corresponding output, in declaration order. This is the original behavior.
  • template ↔ template: when both slots come from (template) wildcard selectors, the handler matches their IOs by script IO name (not position) and silently drops anything unique to one side.
  • template ↔ bare: throws — flip the order or split the link.

defaultRestrictions​

If a string is used, all outputs will use the same restriction. If a key-value object is used, outputs could have individual restrictions, the default restriction could be set as * key.

Consistency​

For data links a consistency mechanism is used. There are several restriction types restricting a user's ability to edit script inputs that were set by the data link.

  • disabled user is not allowed to edit input, consistency is tracked
  • restricted user is allowed to edit an input, however changes will make input inconsistent
  • info user is allowed to edit an input without make input inconsistent, however user changes are still tracked
  • none no consistency is tracked at all, always set values directly to script io

Consistency is tracked based on the value, stored separately from a script io.

Numbers, including the cells of a dataframe, count as equal when they differ by less than 0.0001. A script input can set its own tolerance with annotations: consistencyTolerance is absolute, consistencyRelTolerance is relative to the assigned value, and when either is set the default no longer applies:

//input: double k = 1e-6 {consistencyTolerance: 1e-9}
//input: double volume {consistencyRelTolerance: 1e-6}

Consistency is also applicable to changes propagated via data links. If a script was never run in the current workflow instance, data links just set data directly to scripts io. However, if a script was run, links will not overwrite script io data, but rather only trigger consistency changes. User has either to manually review, apply changes and rerun scripts, or do it via Update on a scope of workflow, nested workflow, or a script.

Inconsistent data mark

Validators​

IO validators​

Validators are used to validate inputs and have validator type. Multiple validators can target the same script io, validation results will be merged. There are several validation severity levels, also it is possible to show user executable actions alongside text messages.

By default, validators run without debounce and are executed in batches for better performance. To add a debounce delay (for example, for expensive validation logic that should not fire on every keystroke), set the debounce option.

Validators have the following additional configuration options:

actions​

Optional

Could be either a LQL string or an array of multiple LQL strings. Queries are targeting script nodes or workflows. Actions descriptors could be retrieved using getValidationAction controller method and later shown alongside validation results via setValidation controller method. It must receive ValidationResult as a data argument.

debounce​

Optional

Debounce time in milliseconds. When set to a positive value, the validator handler waits for the specified duration after the last input change before firing. This is useful for expensive validators (such as server-side checks) where firing on every keystroke is wasteful.

When omitted or set to 0, the validator runs immediately and is eligible for batched execution with other validators, which reduces per-validator scheduling overhead.

Validation controller has the following additional methods:

setValidation​

setValidation(name: string, validation?: ValidationResult | undefined): void;

Sets validation data to an output query with the specified name. There are several validations result types in ValidationResult:

  • errors blocking running a script
  • warnings non-blocking warnings
  • notifications non-blocking suggestions

Each of those fields could have an array of ValidationItem items with the following type:

export interface ActionItem {
  actionName: string;
  action: string;
  additionalParams?: Record<string, any>;
}

export interface Advice {
  description: string;
  actions?: ActionItem[];
}

export type ValidationItem = string | Advice;

A ValidationItem is either a plain string (a message with no follow-up) or an Advice (a message that comes with one or more user-runnable actions). Use the string form for simple errors/warnings; use Advice when the user can click a button to fix the problem or jump to the offending step. description is the text shown to the user, actionName is the label on the action button, action is the opaque action descriptor returned by getValidationAction, and additionalParams is a free-form bag of values forwarded to the action handler, where it is read with getAdditionalParam.

getValidationAction​

getValidationAction(id: string, actionId: string) : string | undefined;

Looks up an action that the validator wants to expose alongside its messages. id is the LQL query name used in the validator's actions field; actionId is the id of the matched action's configuration on the target node. Returns an opaque string descriptor that should be placed into the action field of an Advice item; returns undefined if no such action is wired up. The descriptor is consumed internally by the platform — your code only forwards it.

Not gated by showWhen/hideWhen: getValidationAction returns a descriptor regardless of whether the UI would currently render the action. Use isActionVisible if a validator needs to mirror the UI's decision.

isActionVisible​

isActionVisible(name: string, actionId: string): boolean;

Returns the same boolean that Compute2 uses to decide whether to render the action — true when the action has no showWhen/hideWhen or when its current evaluation matches; false when the action is hidden or when the name/actionId pair is not wired up. Pure query; does not affect execution.

For validation of the workflow structure itself (rather than a single script io) use pipeline validators — a reactive, multi-validator-per-node mechanism that can be triggered from any inputs or outputs.

Pipeline validators​

Pipeline validators attach a validation result to one or more workflow nodes rather than to a script io. They have type pipelineValidator. Unlike regular validators, they target workflow nodes — the link-defining workflow, a named descendant, or several at once via an array to. Like every other link type, each to entry is named, and the handler writes a result per name with setValidation(name, result), so it can validate a single matched node or several independently. Multiple pipeline validators on the same node combine their results, and the merged outcome is exposed on the workflow state as structureCheckResults.

Pipeline validators are reactive in two ways:

  • They subscribe to their from ports and re-fire (debounced) on input changes, the same way regular validators do.
  • They also re-fire after any tree structure update (step added, removed or moved), so they can react to topology changes their from ports might not capture.

Routing is expressed via the standard to query — see the Link query reference for the forms it accepts. A zero-segment path targets the link-defining workflow node (self); a segmented path targets a descendant.

The pipeline validator controller has the following methods:

setValidation (pipeline validator)​

setValidation(name: string, validation?: ValidationResult): void;

Sets (or clears) this validator's contribution for the to output named name, merged into the structureCheckResults of that target workflow node. name is the output's name in the to query (for example, to: 'check:first(inner)' is named check). Omit the validation to remove the contribution. To write the same result to every matched target, call setValidation once per name.

getOutline​

getOutline(): PipelineOutline;

Returns the current PipelineOutline of the workflow where this link is defined (the link's prefix node), including all nested workflows. This is independent of the to target — to says where the result is written; getOutline() describes the link's own scope. Use this when the validator needs to inspect the workflow topology, e.g. the number of nested steps or their configIds.

PipelineOutline is a structure-only snapshot of the workflow subtree: each node carries its id, configId, and type, and workflow nodes carry a recursive steps array. Runtime data (FuncCalls, validation state, mutable runtime metadata) is intentionally omitted so the outline is cheap to walk and stable across re-fires. The full type lives in @datagrok-libraries/compute-api; LSP will show the recursive shape.

debounce (pipeline validator)​

Same semantics as for regular validator links. When omitted, the validator runs without delay.

Metadata​

Metadata links have type meta. They write small chunks of metadata against script IO targets; the platform interprets the metadata to customize the UI that surrounds the input/output (which is rendered by viewer hooks). Meta controller has an additional method:

setViewMeta​

setViewMeta(name: string, meta: Record<string,any>): void;

Sets metadata to an output query with the specified name. Multiple links can have the same output target, provided they are using different keys.

UI metadata​

The platform understands the following well-known keys when set on a script input or output:

KeyEffect
hidden: trueHide the input from the script UI. If the input is a dataframe with one or more viewer: panels, those panels are hidden too. A hidden input is treated as optional regardless of its annotation: the default missing-value check and any validator link targeting it no longer block the run or show up in the UI, and the function is called with whatever value the input holds, possibly null.
items: string[]For choice inputs, replace the dropdown options. Example: { items: ['one', 'two', 'three'] }.
emptyChoice: booleanFor choice inputs, offer an empty option that clears the value, or remove it. Without the key, the input has an empty option when its annotation says nullable: true or optional: true. The key never changes the value itself.

Any other key/value pair is forwarded to the viewer hook unchanged, so custom viewer integrations can define their own meta keys.

Feature integrations (sensitivity analysis, optimization)​

The same meta link type wires script IOs into the sensitivity-analysis and optimization tools. Enabling these tool integrations can be done only for JavaScript functions using the following annotation:

//meta.features: {"sens-analysis": true, "fitting": true}

For sensitivity analysis scalar numerical inputs can have rangeSA meta item with the following properties:

default?: number;
min?: number;
max?: number;
step?: number;

This will set sensitivity analysis tool default input settings accordingly.

For optimization tool inputs settings could be set using rangeFitting meta item which has the same properties as the rangeSA. For outputs the targetFitting meta item is used, it has the following properties:

default?: any;
argumentCol?: string;

Description links are used to gather workflow metadata and display it on ui. They have type nodemeta. They have script io as inputs, but instead of targeting scripts in outputs, selectors will have the following workflow/script nodes targets:

  • title will change item name in the navigation tree, also used to set title when user saves a workflow
  • description will be used to fill description when user saves a workflow
  • tags will be used to fill tags when user saves a workflow. Note that tags here are completely unrelated to PipelineConfiguration ones. They tag a particular saved instance of a workflow.
  • body is rendered as markdown at the top of the workflow view, in place of the workflow description and above the action cards. Use this for inline notes, step instructions, or reactive status text. Works on workflows and action steps; script steps do not show it.

Node meta controller has an additional method:

setDescriptionItem(name: string, description: any): void;

Sets description to an output query with the specified name.

Rules​

A rule has type rule. It reads inputs through from, and while its formula when holds, applies its effects to the to targets: hiding and showing inputs, replacing dropdown items, adding messages, and writing values. It works like a set of meta, validator and data links without a handler:

{
  id: 'oralOnly',
  type: 'rule',
  from: 'route:model/route',
  to: 'k:model/ka',
  when: 'eq(route, "oral")',
  effects: ['show(k)'],
}

See Rules and checks for every effect, the formula language, sources and worked cases.

Checks​

A check has type check and validates one input with the options a function annotation declares, such as min, max, validator and column type, plus a choices list. The driver also applies the validation annotations of every step's inputs as checks, in every run. A choices annotation only fills the dropdown, and a column's table annotation only binds the column picker:

{id: 'weightRange', type: 'check', io: 'model/weight', check: {min: 1, max: 300}}

See Rules and checks for every option and for how a workflow uses annotation defaults, choices and lookup tables.

Actions​

Action options​

These options are available for all action types.

position​

Where to show action. Possible values are

  • buttons action is show in workflow/script ui as a button
  • menu action is show as menu item in the workflow/script view
  • globalmenu action is show as menu item in all nested workflow/script views
  • none action is not show on ui at all

friendlyName​

Optional

User-friendly action name to show on UI.

description​

Optional

Action description to show as a hover on UI.

Optional

Creates a separate menu with menuCategory name, if multiple actions has the same menuCategory, they will be grouped together. Applicable only for actions with position menu.

confirmationMessage​

Optional

A confirmation dialog with this text will be shown before running the action.

icon​

Optional

Font awesome icon to be shown as a part of the UI action button.

visibleOn​

Optional

Shows the action on a descendant step or pipeline instead of the declaring node. The value is the bare config.id of the target; the first DFS match wins. Not a link-spec path. If no descendant matches, the action is hidden. The action's from/to still resolve at the definition scope, so it can read from and write to anything the parent workflow can reach. Works with all action types (data, funccall, pipeline).

Because from/to resolve independently, a pipeline mutation action can read from a step inside its own target workflow. For example, an action with to: 'out1:targetPipeline' can also use from: 'in1:targetPipeline/step1/result' to read a child step's output and decide which step type to add. This works because from and to resolve independently, and an action runs only when the user triggers it, so the forward-only rule for data links does not apply to it.

Visibility​

showWhen and hideWhen make an action conditionally visible based on the current tree state. Both accept an LQL string or an array of them, with the same shape as the link-level from and not fields. They are re-evaluated on every tree mutation.

showWhen​

Optional

Positive gate. The action is visible only when every required entry matches at least one node. Entries flagged (optional) are ignored for the decision. An empty or absent showWhen imposes no positive gate.

hideWhen​

Optional

Negative gate. The action is hidden whenever any entry matches. An empty or absent hideWhen imposes no negative gate. hideWhen is checked first and overrides showWhen.

The (call) flag and the [id] selector form are not allowed in showWhen / hideWhen queries — only the same path/tag matchers used in from/not.

Visibility is a UI hint, not an execution gate. Compute2 honors it when rendering actions at positions buttons, menu, and globalmenu. RTD itself still runs hidden actions if invoked, and getValidationAction returns a descriptor regardless of visibility. Validators that need to mirror the UI's decision can call isActionVisible.

Data actions​

Triggered by a user and placed inside actions, otherwise the same as data links.

FuncCall actions​

Used to change the underlying script node FuncCall. Has a type funccall. FuncCall must be of the same function. Targets script IO as inputs and script nodes as outputs. Controller has additional methods:

setFuncCall​

setFuncCall(name: string, state: DG.FuncCall): void;

Sets FuncCall to an output query with the specified name.

Pipeline actions​

Used to replace dynamic workflow content. Has a type pipeline. Targets script io as inputs and dynamic workflows as outputs. To target a child dynamic workflow, use a path in to (e.g., to: 'out1:analyses'). To target the workflow where the action is defined (self-reference), use just the query name with no path (e.g., to: 'out1').

A read-only target, such as a saved run opened read-only, is skipped: its steps are not added, removed, moved or replaced, while the action's changes to writable targets still apply. Replacing a writable workflow that holds a read-only item is allowed.

Controller has additional methods:

setPipelineState​

setPipelineState(name: any, state: PipelineInstanceConfig): void;

Sets state to an output query with the specified name. PipelineInstanceConfig is a recursive type that should be compatible with the current workflow configuration. Each item should have a matching id, nested items are referenced in steps regardless of a workflow type. For script nodes it is possible to provide initialValues and inputRestrictions. A workflow item without steps gets the defaults from its configuration (steps or initialSteps); pass steps: [] for an empty workflow.

getSteps​

getSteps(name: string): StepHandle[];

Returns current steps of the target dynamic workflow as an array of StepHandle objects. Each handle has configId, position, and an internal _uuid.

addStep​

addStep(name: string, configId: string, position?: number): void;

Adds a step to the target dynamic workflow by configId (must match one of the stepTypes ids). Optional position inserts at the given index; omit to append.

removeStep​

removeStep(name: string, step: StepHandle): void;

Removes a step from the target dynamic workflow. Pass a handle obtained from getSteps.

moveStep​

moveStep(name: string, step: StepHandle, position: number): void;

Moves a step to a new position within the target dynamic workflow.

warning

A pipeline-action handler must not queue mutations on overlapping subtrees — for example, calling setPipelineState or granular ops on an outer pipeline together with granular ops on one of its descendants (reached via a separate to: port). The driver detects this and aborts the action. Combine the mutations into a single setPipelineState at the common ancestor, or operate only at the inner level.

Hooks​

Init hook​

Uses the onInit field instead of links. It is usually used to populate states, and otherwise works like a data link. This section is the one place that describes init, for the hook and for runOnInit links alike.

When init runs. Every workflow, including each nested one, runs its init when it is:

  • created;
  • loaded from history;
  • added to a running workflow, as a new step of a dynamic workflow or as a nested workflow loaded into it.

Each time, the workflow's init hook runs first, then its runOnInit links. The rules are the same for a top-level workflow and a nested one.

A duplicated step skips the init hook of every workflow it contains, because the copy already holds the values init produced. So does an instance config entry with skipOnInit: true. Links, runOnInit included, still run.

What init writes. A value written at init is a change like any other: data links that read it pass it on, and validators and meta links react to it. So a state filled by the init hook reaches script inputs through ordinary links, without runOnInit. Use runOnInit for a link that must compute its value at init even though nothing it reads changes, for example a default derived from another step.

When a workflow is created or loaded, a plain data link runs only when something it reads changes. Links that nothing at init touches keep the values that were loaded.

When a part is added to a running workflow, its links and the links from the rest of the workflow into it also run once, so the new part is filled from the values already in the workflow.

What a write does to a loaded value. Init and runOnInit links run again on every load, so their writes can meet values that were saved:

  • an input of a step that has not been run takes the new value;
  • an input of a step that has run keeps its value; when the link brings a different one, the input is marked inconsistent and can be reset to it, see Consistency.

An init hook that writes a value that differs on every load, such as the current time or the current user, therefore marks the steps that consume it as inconsistent after each load. Feed such values only into inputs where that is expected.

Return hook​

Uses the onReturn field instead of links and is triggered only once when a workflow is started programmatically and exited via the Confirm data button. Otherwise, the same as data links.

The return hook handler receives an IRuntimeReturnController with the following method in addition to the common controller methods:

  • controller.returnResult(data) — sends data back to the caller that started the workflow programmatically. The value is emitted through the result$ observable on the Driver.