feat: implement UDO by NicoLaval · Pull Request #539 · InseeFr/Trevas

NicoLaval · 2026-08-07T08:23:51Z

No description provided.

hadrienk

Few comments here and there. Given the scope, I'd question the LLM about how can we control the base implementation and validate the new UdoFunctionExpression and wiring until we know it works. Ie: unit tests on the model first with hard coded udo we replace later by dynamically resolved instances.

hadrienk · 2026-08-13T20:13:04Z

+**Source of truth:** engine-side `UdoDefinition` in `ENGINE_SCOPE` bindings.  
+**Dispatch hook:** trampoline `java.lang.reflect.Method` registered under the same name (not a second source of truth).
+
+No `vtl-model` DTO in P0.


Because the body is an ANTLR ExprContext (and we keep an engine handle for re-entry). That would pull the parser into vtl-model. Same as rulesets: engine-side artefact in bindings. Clarified in ffd5f0d.

hadrienk · 2026-08-13T20:13:34Z

+
+## Body representation
+
+**Parse subtree** (`ExprContext`). Define does not evaluate the body. Invoke re-enters `ExpressionVisitor` with a child map (params + outer bindings). Free vars resolve at **invoke time** ([08 §1](./08-open-questions.md)).


Names in the body that aren’t parameters (e.g. y in max_with_y). P0 looks them up at invoke time; params shadow outer names. Not a lexical closure — snapshot-at-define can come later. Clarified in ffd5f0d.

hadrienk · 2026-08-13T20:15:29Z

+| Action | Why |
+|--------|-----|
+| `bindings.put(name, udo)` | Source of truth for invoke |
+| `registerMethod(name, trampoline)` | Enables `FunctionExpression` / `Method.invoke` path |


Not sure. Might need to "resolve" the function id (like a variable) and then call it.

Registering the method with registerMethod goes around the bindings and make closure scoping moot.

You’re right. registerMethod was only there to reuse FunctionExpression / Method.invoke.

Now: define → bindings.put only (08be7cf). Call → resolve the id in the current bindings; if UdoDefinition, evaluate as a ResolvableExpression (02a258b, 8dcae61).

hadrienk · 2026-08-13T20:18:17Z

+
+```java
+String name = ctx.operatorID().getText();
+Object binding = engine.getBindings(ENGINE_SCOPE).get(name);


Might need to check what the scope mechanisms gives us here. Maybe create a scoped view of the bindings.

For closure binding

Yes. Call lookup uses the visitor map, not always ENGINE_SCOPE (8dcae61). P0 is still invoke-time free-var lookup, not a snapshot at define.

hadrienk · 2026-08-13T20:19:50Z

+5. If `returns` declared → assignability check (`integer` ⊆ `number` allowed).
+6. Return result.
+
+ThreadLocal CallSite is the P0 bridge (no per-UDO bytecode). Acceptable for the locked pattern; replace later only if needed.


Double check later feasibility

Dropped with the trampoline. UdoFunctionExpression.resolve evals the body with a child map (02a258b, 08be7cf).

hadrienk · 2026-08-13T20:25:15Z

+| scalar actual → opaque `dataset` formal | **reject** |
+| opaque `dataset` ↔ opaque `dataset` | allow (`instanceof Dataset` / `DatasetExpression`) |
+
+Do not invent a parallel assignability table for UDOs.


Probably the most important. Not sure about the value of this file. We have a type system, it will probably just work. Ask LLM about potential changes it foresees.

Agree, no parallel type system. 06 is now only which signature syntax we reject at define (component, constraints, …). Assignability should follow existing TypeChecking (ffd5f0d). Not recoding that in this pass.

hadrienk · 2026-08-13T20:25:48Z

I'd put that as the first step. Absolute requirement.

Needs more work. Maybe leverage the plan in prov? Tests written in vtl itself + unit testing at the engine level?

Yes. Hardcoded expression unit tests first (02a258b), plus the existing script-level UserDefinedOperatorTest. .vtl fixtures like the prov corpus can come later, not blocking this PR.

hadrienk · 2026-08-13T20:27:39Z

+| Tag | IDs |
+|-----|-----|
+| Doc | D1–D4 |
+| Métier | DS1–DS5 |


French word.

Fixed in ffd5f0d — “Métier” → “Recipes”.

hadrienk · 2026-08-13T20:29:33Z

+
+## 7. Component-level calls
+
+Commented grammar (`genericOperatorsComponent` / `callComponent`) suggests component-level UDO invoke is unfinished upstream. **Out of scope** until grammar is revived.


Make sure to implement so that it work when compExpr is removed from grammar.

UDO calls go through the normal expression path (visitCallDataset / ResolvableExpression), not callComponent (already commented out in Vtl.g4). 08 rephrased in ffd5f0d.

hadrienk · 2026-08-13T20:35:04Z

+ * Resolves call-site arguments (defaults / {@code _}) then returns a {@link UdoFunctionExpression}
+ * so evaluation goes through {@code FunctionExpression} → {@code Method.invoke}.
+ */
+public final class UdoInvokeExecutor {


Weird. With the resolvable expr this should be simpler. It should also implement the general expression contract.

Agree. The UDO is now a ResolvableExpression that binds formals and visits the body (02a258b). UdoInvokeExecutor is only argument wiring (defaults / _ need the raw parameter ctx). Trampoline removed in 08be7cf.

…tion

NicoLaval added 7 commits August 6, 2026 10:28

Add UDO spec

e3219c3

Init UDO tests

15c5b80

Format code

f984068

Refine UDO spec mentioning tests

8d9d508

Add first simple implementation to validate pattern

2df47d1

Format

a034fad

Update doc with chosen implementation pattern

181c3dd

hadrienk reviewed Aug 13, 2026

View reviewed changes

NicoLaval added 4 commits August 14, 2026 15:54

Retarget specs after review (bindings + ResolvableExpression)

ffd5f0d

Evaluate UDO as ResolvableExpression without Method.invoke

02a258b

drop trampoline registry and reject redefine with AlreadyDefinedExcep…

08be7cf

…tion

Resolve operator id from current expression bindings

8dcae61

NicoLaval requested a review from hadrienk August 14, 2026 14:10

Format code

c7bb9b5


		## Body representation

		Parse subtree (`ExprContext`). Define does not evaluate the body. Invoke re-enters `ExpressionVisitor` with a child map (params + outer bindings). Free vars resolve at invoke time ([08 §1](./08-open-questions.md)).


		## 7. Component-level calls

		Commented grammar (`genericOperatorsComponent` / `callComponent`) suggests component-level UDO invoke is unfinished upstream. Out of scope until grammar is revived.

Uh oh!

Conversation

NicoLaval commented Aug 7, 2026

Uh oh!

hadrienk left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

NicoLaval Aug 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

NicoLaval Aug 14, 2026 •

edited

Loading