Integrate resources into the registration API

[knoepfel]

Phlex supports serialized resource usage through the serial_node and serializer_node entities. These components are not integrated with the rest of the system and should become part of the registration interface.

[knoepfel]

Phase 1 Resource API

This document describes the current Phase 1 resource API. The original proposal included unlimited, single-token, and pooled resources. Pooled resources remain a reserved extension, but are not enabled in Phase 1.

Overview

A resource is a framework-owned object registered with a framework_graph. Algorithms declare a resource dependency in input_family(...); the framework then passes an access value to each invocation.

The resource class supplies domain state and behavior. It does not need to inherit from a Phlex type or implement a common interface.

Phase 1 supports two resource categories:

1. An unlimited-access resource has no token_type and may be used concurrently by any number of algorithm invocations.
2. A single-token resource has a token_type, but no tokens(), and permits one algorithm invocation at a time.

A declaration with both token_type and tokens() is recognized as a pooled resource form, but pooled resources are deferred and rejected by the Phase 1 concepts. They are not silently treated as single-token resources.

For bounded resources, the access value is also a concurrency token: it is held for the complete execution of the algorithm body. For unlimited resources, the access value grants access only and does not reserve capacity.

Unlimited-Access Resources

An unlimited-access resource is an ordinary C++ type. The resource author does not declare token_type or tokens().

class geometry {
public:
  explicit geometry(std::filesystem::path gdml_file);

  point position(detector_id id) const;
};

The graph owns one geometry object:

g.add_resource<geometry>("geometry.gdml");

An algorithm requesting the resource receives a pointer to that object. By default, the pointer is T const*, so the resource type must be safe for concurrent reads:

void locate_hit(hit const& input, geometry const* detector_geometry)
{
  auto const position = detector_geometry->position(input.detector());
  // Inspect or record the position.
}

m.observe("locate_hit", locate_hit, concurrency::unlimited)
  .input_family(selector, resource<geometry>{});

At the catalog boundary, resource_catalog::access_for<T>() returns gsl::not_null<T const*>. The node-level cache and algorithm-facing signature use the ordinary T const* type; the not_null wrapper expresses the non-null guarantee where the framework obtains the pointer without appearing in algorithm signatures or on the per-invocation path.

No TBB resource limiter is created for an unlimited-access resource. A node that uses only unlimited resources remains an ordinary function_node or multifunction_node, and its configured Phlex concurrency is unaffected.

The initial API defaults unlimited access to T const*. Mutable concurrent access requires an explicit future access-policy spelling; const expresses mutation rights, not resource concurrency policy.

Single-Token Resources

A single-token resource declares an algorithm-facing token type that the framework can construct from its owned resource object. Phase 1 supports either a pointer-compatible token or a copyable value token.

A pointer-compatible resource-object token can be T* or T const*:

class database {
public:
  using token_type = database const*;

  explicit database(connection_options options);
};

g.add_resource<database>(options);

void query(request const& request, database const* database_resource);

m.observe("query", query, concurrency::unlimited)
  .input_family(selector, resource<database>{});

The framework creates one limiter token from the address of its framework-owned database object and holds it while query executes. A pointer-compatible token refers to that framework-owned object.

A small resource may instead use its own type as the token type:

struct random_engine {
  using token_type = random_engine;

  explicit random_engine(unsigned int seed) : seed_{seed} {}

  unsigned int seed_;
};

g.add_resource<random_engine>(42);

void generate(random_engine engine_token);

In this form, the framework copies its registered random_engine into the one-token limiter. The algorithm receives the acquired value token, not the catalog-owned resource object. Mutating the algorithm's value parameter does not mutate the catalog-owned object. Value tokens are intended for small, copyable capability values, not shared mutable resource state.

The effective maximum concurrency for a node is the minimum of the operator's concurrency setting and the resource's one-token capacity. A distinct token type that cannot be produced from T*, or a value token type that is not exactly T and copy constructible, is not a Phase 1 resource declaration.

Pooled Resources

A pooled resource form declares token_type and returns its available tokens from tokens():

class database_pool {
public:
  using token_type = database_connection*;

  explicit database_pool(pool_options options);

  std::vector<token_type> tokens();
};

This syntax is reserved for the future pooled-resource extension. Although the form is identified as pooled_resource, it is excluded from the Phase 1 phase_1_resource concept. Registration and dependency APIs therefore reject it at compile time rather than constructing a one-token limiter or silently discarding the additional tokens.

Pooled resources will require an official supported oneTBB release with the range/container resource_limiter constructors described in oneTBB PR #2137. Until then, Phlex does not construct a limiter from tokens(), provide a runtime-sized pool, or emulate pooled scheduling with a custom semaphore.

Declaring Dependencies

All resource categories use the same registration-time marker:

resource<T>{}

Resources follow product selectors in input_family(...) and correspond, in order, to trailing algorithm parameters:

void process(input const&, geometry const*, database const*);

m.observe("process", process, concurrency::unlimited)
  .input_family(selector, resource<geometry>{}, resource<database>{});

The registration layer counts resource markers as callable inputs but uses only preceding product-selector parameters for product type deduction. It does not assume resource parameters are pointers, so value-token parameters are valid.

Resource types identify registered resources. A graph permits one registration of each concrete resource type, and a dependency list may name each resource type at most once; duplicate resource dependencies are rejected at compile time.

Framework Responsibilities

framework_graph owns registered resource objects and their limiter state. It must ensure that both outlive graph nodes, since bounded nodes retain references to their TBB resource limiters.

Registration must occur before an algorithm registration that references the resource. Referencing an unregistered resource is a registration error with a clear diagnostic.

For a node that uses multiple bounded resources, the framework acquires one token from each resource for the whole body invocation. Resource declaration order is therefore part of the observable scheduling behavior and remains consistent across algorithms.

Phase 2 and Future Extensions

Dynamically Loaded Resources

Phase 2 will support dynamically loaded resource registrations through the same plugin-discovery and loading mechanism used for modules, sources, and drivers. This permits a resource provider to construct and register framework-owned resources from workflow configuration without requiring the resource type or its construction details to be compiled into the application.

A resource plugin will use a registration macro with the following shape:

PHLEX_REGISTER_RESOURCES(r, config)
{
  r.add_resource<T>(/* construct from config */);
}

The registration opener receives the graph's resource-registration interface and the resource configuration. It is responsible for interpreting the configuration and registering each resource with add_resource<T>(...). The resource dynamically loadable libraries are loaded before any other modules are loaded.

The precise macro name, callback parameter types, configuration schema, and plugin ABI will follow the established conventions for dynamically loaded modules, sources, and drivers.

Pooled Resources

Phase 2 will enable the pooled-resource form described above. Each element returned by tokens() will represent one available concurrent access slot. A pool containing N tokens will bound a resource-dependent node to at most N concurrent invocations, subject to its configured operator concurrency.

Enabling pooled resources requires all of the following:

1. An official supported oneTBB release containing the resource_limiter iterator-range or container constructor.
2. A CMake feature probe for the required constructor.
3. Materialization of tokens() during registration, with token values retained for the graph lifetime when required by their type.
4. Construction of resource_limiter<T::token_type> from the complete token range or container.
5. Registration-time rejection of an empty token collection. An empty limiter can leave graph::wait_for_all() blocked.
6. Tests proving that N tokens allow no more than N concurrent invocations and that multiple token identities are exercised.

Until these conditions are met, pooled resources must remain rejected. Phlex must not silently reduce a pool to one token or emulate pooled scheduling with a Phlex-specific semaphore.

Distinct Single Tokens

A separate future extension may support a single token that is not derived from or equal to the registered resource type:

class database {
public:
  using token_type = database_connection*;

  token_type token();
};

The framework would use token() to construct the one-token limiter. This extension does not require runtime-sized oneTBB resource-limiter support and can be implemented independently of pooled resources.

Other Future Design Work

The following are not part of the Phase 1 API or the currently specified Phase 2 pooled-resource work:

• Mutable unlimited-resource access.
• Named resource instances.
• Multiple registrations of one concrete resource type.
• Duplicate resource dependencies.
• Provider-node resources.