kvcache-ai/Mooncake

Fork 0

mirror of https://github.com/kvcache-ai/Mooncake.git synced 2026-11-05 01:17:55 +00:00

[GH-ISSUE #2865] [RFC]: TENT Transfer Intent API v1 and Cross-Language Compatibility #743

New issue

Closed

opened 2026-10-29 21:18:59 +00:00 by bsakaguchi · 4 comments

bsakaguchi commented

2026-10-29 21:18:59 +00:00

Owner

Originally created by @catyans on GitHub (Jul 12, 2026).
Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/2865

Summary

This RFC proposes a stable TENT Transfer Intent API v1 and explicit semantic parity across the C++, C, pybind, and high-level Python interfaces.

TENT already has the core intent fields in its internal C++ Request:

priority
deadline_ns
policy_name
transport_hint
intent_type

However, these fields were added incrementally and are not exposed consistently:

C++ Request contains all fields.
pybind exposes the fields added by #2810 and earlier work.
the C tent_request_t currently contains priority and transport_hint, but not deadline_ns, policy_name, or intent_type.
the high-level declarative Python API is being developed separately in #2541.
upper-layer connectors do not yet share one versioned contract for defaults, unknown values, clock domain, authorization, and compatibility.

The RFC defines what an application is allowed to request and how TENT interprets that request. It does not define how TENT constructs an Execution Plan (#2863), how QoS is enforced (#2856), or how operators inspect the result (#2864).

Application / Store / SGLang / vLLM
                 |
                 v
        Transfer Intent API v1
                 |
       validation + normalization
                 |
       effective policy resolution
                 |
                 v
         Execution Plan (#2863)

The first implementation milestone should add a versioned C request ABI, shared validation/normalization, and cross-language compatibility tests. It must not change default runtime scheduling behavior.

Motivation

The TENT roadmap in #1058 calls for a clean declarative API where users express what data should move and the runtime selects the path. The existing fields are a strong foundation, but an API is not stable merely because fields exist in a C++ struct.

1. Cross-language behavior is incomplete

The current C API converts only the fields present in tent_request_t. A C caller cannot express the same intent as a C++ or pybind caller. This makes connector behavior depend on the language binding rather than on the request semantics.

2. The existing C struct is not safely extensible as an array ABI

tent_submit() accepts an array of fixed-size tent_request_t. Appending fields changes the array stride expected by a newly compiled library. A new library reading a request array produced by an old binary can step beyond the old element boundary. Zero-initialization alone does not solve binary layout/stride compatibility.

The current struct must remain supported. A new versioned request entry and submission API are required instead of silently extending the old array ABI.

3. Zero-initialization does not provide a safe priority default

The current C API requires zero-initialization. transport_hint=0 correctly means UNSPEC, but priority=0 currently means HIGH. A caller that only zero-initializes the struct therefore requests the highest priority unintentionally.

Transfer Intent API v1 should use UNSPEC=0 for every optional enum exposed through the new C ABI. The old API retains its existing behavior for compatibility.

4. Deadline clock semantics are not explicit enough

deadline_ns is an absolute steady_clock timestamp. It is meaningful when the caller and TENT runtime share the local monotonic clock domain, but it must not be serialized unchanged to another host or process boundary whose clock origin is not guaranteed to match.

Connectors need one documented rule for local submission and a separate rule for any future remote service API.

5. Requested policy is not authorization

An application-provided priority, policy name, or transport hint must not automatically grant access to reserved resources, a high-priority QP pool, or an otherwise forbidden backend. The API must distinguish requested intent from the effective authorized policy resolved by #2856.

6. Unknown values need predictable behavior

Rolling upgrades can produce new callers with old runtimes or old callers with new runtimes. Unknown intent enums, request versions, flags, and optional fields require explicit strict/compatibility behavior. Silent integer casting is not a stable contract.

Goals

Define stable semantics and defaults for Transfer Intent v1 fields.
Provide equivalent behavior across C++, C, pybind, and high-level Python.
Add a binary-safe, versioned C request submission API without breaking tent_submit().
Make every optional C enum safely zero-initializable with UNSPEC=0.
Define clock-domain rules for deadline_ns.
Define validation and unknown-value behavior.
Separate requested intent from effective authorized policy.
Define string ownership, lifetime, and size limits.
Add compile/link/runtime compatibility tests for old and new callers.
Provide a stable connector contract for Store, SGLang, vLLM, and other integrations.

Non-goals

Defining or implementing QoS authentication/RBAC.
Implementing bandwidth guarantees, policing, WDRR, or receiver-credit allocation.
Defining an Execution Plan or backend capability model; that belongs to #2863.
Defining CLI explain output; that belongs to #2864.
Replacing the high-level endpoint abstraction proposed in #2541.
Adding a network wire protocol for remote intent submission in v1.
Making Classic TE support every TENT intent field.
Changing the numeric values of existing internal C++ enums in the first PR.
Changing the behavior or layout of the legacy tent_request_t and tent_submit() API.

Transfer Intent v1 semantic model

A Transfer Intent is caller-provided metadata attached to one logical transfer request.

struct TransferIntentV1 {
    IntentType intent_type;
    RequestedPriority requested_priority;
    uint64_t deadline_ns;
    std::optional<std::string> policy_name;
    TransportType transport_hint;
    uint64_t flags;
};

The runtime normalizes and validates the request before policy resolution:

caller values
    -> ABI/version validation
    -> enum/string/deadline validation
    -> normalized requested intent
    -> authorization/effective policy (#2856)
    -> Execution Plan (#2863)

Field semantics

`intent_type`

Standard values:

INTENT_UNSPEC
FOREGROUND_GET
BACKGROUND_PREFETCH
MIGRATION
CHECKPOINT
WEIGHT_LOADING
STAGING_INTERNAL

Rules:

INTENT_UNSPEC preserves compatibility behavior.
one request has exactly one intent type in v1;
applications must not submit STAGING_INTERNAL; it is reserved for TENT-generated internal work unless an explicitly trusted internal API is used;
unknown numeric values fail validation in strict mode;
compatibility mode may map an unknown value to INTENT_UNSPEC, but must record the downgrade in explain/metrics;
enum numeric values become stable once published through the v1 C ABI.

Recommended upper-layer mapping:

Workload	Intent
latency-sensitive KV fetch	`FOREGROUND_GET`
speculative/background KV prefetch	`BACKGROUND_PREFETCH`
replica/KV live migration	`MIGRATION`
checkpoint transfer	`CHECKPOINT`
model/adapter weight loading	`WEIGHT_LOADING`
TENT-created staging hop	`STAGING_INTERNAL`

The table is a semantic default, not a requirement that every framework use the same internal operation names.

`requested_priority`

The new public C ABI should distinguish “unspecified” from “highest priority”:

TENT_PRIORITY_UNSPEC = 0
TENT_PRIORITY_HIGH   = 1
TENT_PRIORITY_MEDIUM = 2
TENT_PRIORITY_LOW    = 3

Internal C++ priority values may be converted without renumbering them initially.

Rules:

priority is a request, not authorization;
UNSPEC means derive the effective priority from the matched contract/default;
a caller cannot obtain HIGH, a reserved budget, a privileged QP pool, or wire QoS merely by setting HIGH;
an unauthorized priority is downgraded or rejected according to the effective QoS Contract mode;
invalid values fail validation; they must not be clamped silently;
the legacy tent_request_t.priority=0 remains HIGH for compatibility, but the legacy API should be documented as unable to express UNSPEC.

`deadline_ns`

Rules for local API v1:

0 means no deadline;
otherwise it is an absolute timestamp in the monotonic clock domain used by the local TENT runtime;
C callers use clock_gettime(CLOCK_MONOTONIC, ...) or a provided helper;
Python callers use time.monotonic_ns() or a provided helper;
wall-clock/Unix timestamps are invalid;
an already expired deadline is preserved as expired and handled by the configured Admission/degradation policy; it is not silently extended;
overflow when computing now + budget must fail validation or saturate through a documented helper;
the deadline applies to the user-visible logical request, not independently to every derived slice/stage.

A raw absolute monotonic timestamp must not cross a host boundary. If a future remote TENT service API transports intent, it should send a remaining deadline budget plus measurement/receive timestamps and convert it into the receiver's local clock domain. That wire protocol is outside v1.

Suggested helpers:

uint64_t tent_monotonic_now_ns(void);
int tent_deadline_after_ns(uint64_t budget_ns, uint64_t *deadline_ns_out);

`policy_name`

Rules:

NULL means no explicitly requested named policy;
an empty string is invalid rather than a second spelling of unset;
the string is UTF-8 or a documented restricted ASCII identifier;
length is bounded;
the callee copies the value before the submit function returns;
a named policy remains subject to tenant/caller authorization;
an unknown or unauthorized policy follows strict/compatibility behavior from the QoS Contract;
logs/diagnostic bundles redact or pseudonymize policy names according to #2864.

`transport_hint`

The semantics should remain consistent with #2339:

TRANSPORT_UNSPEC follows policy selection;
a concrete hint pins the first attempt only within the matched policy's authorized transport list;
a hint is not permission to bypass the policy allowlist;
if the hinted transport is unavailable or unauthorized, validation/selection fails rather than silently using a different first transport;
bounded failover may move to later authorized candidates after the hinted transport fails;
unknown transport values fail validation.

`flags`

v1 should reserve a bounded flags field for request-semantic modifiers that can be safely ignored only when explicitly documented.

Rules:

all unknown required-semantics bits fail validation;
optional advisory bits must live in a separately documented range if introduced;
v1 may require flags=0 until the first flag is standardized;
flags must not become an unstructured replacement for versioned fields.

Requested versus effective intent

The API exposes the caller's request. TENT derives the effective policy:

requested intent
    intent_type = foreground_get
    requested_priority = high
    policy_name = latency-critical
    transport_hint = rdma

effective policy
    intent_type = foreground_get
    priority = medium
    matched_policy = tenant-a.foreground_get
    authorized_transports = [rdma, tcp]
    qp_pool = foreground
    degraded_actions = [fallback_transport, local_recompute]

The normalized requested intent and effective policy should both be observable through #2864/#2863 explain output. They must not be conflated in public metrics labels with unbounded cardinality.

Proposed binary-safe C ABI

The legacy API remains unchanged:

int tent_submit(tent_engine_t engine,
                tent_batch_id_t batch_id,
                tent_request_t *entries,
                size_t count);

Transfer Intent v1 should add a new immutable versioned entry rather than append fields to tent_request_t.

Illustrative layout:

#define TENT_REQUEST_SCHEMA_V1 1u

struct tent_request_v1 {
    uint32_t struct_size;
    uint32_t schema_version;

    int32_t opcode;
    void *source;
    tent_segment_id_t target_id;
    uint64_t target_offset;
    uint64_t length;

    int32_t intent_type;
    int32_t requested_priority;
    uint64_t deadline_ns;
    const char *policy_name;
    int32_t transport_hint;
    uint32_t reserved0;
    uint64_t flags;
};

typedef struct tent_request_v1 tent_request_v1_t;

Because requests are submitted as an array, the runtime must know the caller's element stride. One possible generic ABI is:

int tent_submit_requests(
    tent_engine_t engine,
    tent_batch_id_t batch_id,
    const void *entries,
    size_t count,
    size_t entry_stride,
    uint32_t schema_version);

The header can provide a type-safe inline wrapper:

static inline int tent_submit_v1(
    tent_engine_t engine,
    tent_batch_id_t batch_id,
    const tent_request_v1_t *entries,
    size_t count) {
    return tent_submit_requests(engine, batch_id, entries, count,
                                sizeof(tent_request_v1_t),
                                TENT_REQUEST_SCHEMA_V1);
}

This design allows the runtime to advance by the caller-provided stride and validate struct_size instead of assuming the library's compile-time struct size. A future v2 can use a new schema/version without changing legacy tent_submit().

Alternative ABI designs are welcome, but they must pass the binary compatibility cases below. Simply appending fields to the existing array element is not sufficient.

C ABI validation

For every entry:

entry_stride is at least the minimum size for the declared schema;
struct_size is within entry_stride and the supported bounds;
required fields are present before reading them;
missing optional tail fields use documented defaults;
extra tail bytes are ignored only for a compatible schema version;
reserved fields and unknown required flags are zero;
strings are non-dangling for the duration of the call and copied before return;
count × stride arithmetic is overflow checked;
no field is read through an unaligned pointer.

ABI discovery

Suggested query functions:

uint32_t tent_get_api_version(void);
int tent_get_request_schema_info(uint32_t schema_version,
                                 size_t *minimum_size,
                                 size_t *known_size);

The exact names can change, but callers need a supported way to distinguish “symbol unavailable”, “schema unsupported”, and “request invalid”.

C++ API

C++ Request remains the internal native representation initially. Add shared normalization/validation helpers so C++, C, and Python do not implement separate semantics.

Possible API:

Status normalizeTransferIntent(const Request &request,
                               NormalizedTransferIntent &out,
                               ValidationMode mode);

All submit paths, including notification variants, must call the same normalization logic.

The RFC does not require immediately renaming Request::priority to requested_priority, but documentation and explain output must state that it is a requested value before contract resolution.

Python APIs

pybind

expose typed IntentType, RequestedPriority, and TransportType enums;
prefer keyword arguments for optional intent fields;
preserve current defaults;
reject unknown raw integers rather than accepting arbitrary casts;
provide deadline_after_ns() or accept a duration helper that converts locally;
expose the same validation errors as C++ with actionable messages.

High-level declarative Python API

#2541 defines source/destination endpoint ergonomics. This RFC should not replace that work. The high-level API should forward an optional intent object or equivalent keyword fields into the same normalized model:

transfer(
    src,
    dst,
    intent_type=IntentType.FOREGROUND_GET,
    requested_priority=Priority.UNSPEC,
    deadline_ns=time.monotonic_ns() + budget_ns,
    policy_name=None,
    transport_hint=TransportType.UNSPEC,
)

Python must not invent different fallback or authorization semantics.

Connector contract

Connectors should map their business operation to an intent before submitting to TENT.

Recommended behavior:

compute the remaining TENT transfer deadline from the upper-layer SLO budget in the submitting process;
select one standard intent_type;
normally leave priority and transport unspecified so policy controls them;
use policy_name only when configured/authorized;
use transport_hint for explicit testing or a well-defined backend requirement, not as the default routing mechanism;
preserve the intent for retries unless the upper layer explicitly changes its degradation decision;
record both requested and effective policy IDs in bounded trace attributes.

The first implementation milestone does not require modifying SGLang or vLLM. Connector PRs should follow after the API and compatibility tests stabilize.

Strict and compatibility modes

Suggested behavior:

Condition	Strict mode	Compatibility mode
unknown request schema	reject	reject or use legacy API explicitly; never guess layout
unknown `intent_type`	reject	map to `INTENT_UNSPEC` and record downgrade
unknown priority	reject	map to `UNSPEC` and record downgrade
unknown transport	reject	reject; silently changing an explicit pin is unsafe
unknown/unauthorized policy	reject	fall back to contract default and record downgrade
unsupported optional tail field	reject if required	ignore only when documented advisory
expired deadline	preserve expired state	same

Compatibility mode must never reinterpret bytes from an unknown struct layout.

Error reporting

The new API should distinguish at least:

unsupported ABI/schema version;
invalid struct size/stride;
invalid enum/flags;
invalid or unauthorized policy;
deadline clock/range error;
unsupported field in the current build;
runtime submit failure after intent validation.

Returning only -1 makes connector fallback unsafe. The first PR may keep the existing return type if necessary, but should provide a thread-safe status-detail query or a new result structure before connector adoption. This should align with, not preempt, any future general Transfer Outcome/Error Contract.

Compatibility and rollout

Existing callers

tent_request_t layout is unchanged.
tent_submit() and tent_submit_notif() behavior is unchanged.
zero-initialized legacy priority continues to mean HIGH.
legacy callers cannot express all v1 fields and are documented accordingly.

New callers

use tent_request_v1_t and tent_submit_v1();
all optional enum defaults are zero/UNSPEC;
unknown request schemas fail before any work is submitted;
the runtime copies variable-length intent metadata before returning.

Runtime behavior

the new fields initially normalize into the existing C++ Request;
INTENT_UNSPEC, no policy, no deadline, and no transport hint preserve existing behavior;
no scheduling/QoS policy becomes enabled merely because the new API exists;
enforcement remains controlled by existing opt-in policies and later PRs.

Proposed PR sequence

PR1: Semantic contract and shared normalization

Add stable public enum definitions for v1.
Add normalized intent/validation helpers.
Add strict/compatibility validation tests.
Document legacy versus v1 defaults.
No C ABI or runtime selection change yet.

PR2: Versioned C ABI

Add tent_request_v1_t, schema discovery, stride-safe submission, and notification equivalent.
Preserve legacy symbols/layout.
Add old/new binary compatibility fixtures.

PR3: pybind and high-level Python parity

Route all Python fields through shared normalization.
Add typed enums and deadline helpers.
Coordinate with #2541 rather than creating a second high-level transfer API.

PR4: Explain and observability

Expose normalized requested intent and effective policy through #2864/#2863.
Add bounded trace attributes and downgrade counters.
Avoid unbounded policy/tenant labels in metrics.

Later connector PRs

Mooncake Store intent mapping;
SGLang foreground/prefetch/migration mapping;
vLLM connector mapping;
other framework integrations;
remote-service deadline-budget protocol if a standalone TENT service is introduced.

Validation plan

Semantic tests

every field default and explicit value;
unknown enums and flags in strict/compatibility mode;
policy NULL/empty/unknown/unauthorized behavior;
transport hint authorization;
past, present, overflowed, and zero deadlines;
STAGING_INTERNAL rejected for untrusted public submission;
requested versus effective priority distinction;
notification and non-notification submit parity.

C ABI compatibility matrix

old client binary + new library using legacy API;
new client + new library using v1;
new client detecting a library without the v1 symbol/schema;
minimum-size v1 entry;
larger compatible entry with ignored tail bytes;
too-small/too-large/misaligned entry;
count × stride overflow;
mixed invalid entry inside a batch: all-or-nothing intent validation;
32-bit/64-bit layout checks where supported;
C compiler and C++ compiler header compatibility;
AddressSanitizer/UBSan/fuzzing for struct parsing and strings.

Cross-language parity

For a shared table of request vectors, C++, C, pybind, and high-level Python must produce the same normalized intent or the same validation category.

The test corpus should include:

all standard intent types;
every priority and transport hint;
absent and named policies;
no deadline and tight/expired deadlines;
unknown values;
legacy compatibility defaults.

Real-cluster validation

On two H20/RoCE nodes:

submit the same foreground/background/migration intent through C++ and Python;
verify identical effective policy and Execution Plan explain output;
verify transport_hint remains inside the policy allowlist;
verify deadlines use local monotonic time and distinguish tight/loose requests;
verify no-intent v1 requests match the legacy transfer result and throughput;
verify data integrity for Direct and forced Staged paths;
verify unsupported/unauthorized fields fail before transport submission.

The API itself should add no per-slice work. Measure submit throughput and plan/validation time; transfer throughput/P99 should remain unchanged for equivalent normalized requests.

Expected benefits and claims boundary

This RFC should provide:

one stable vocabulary for upper-layer transfer semantics;
binary-safe evolution of the C request API;
identical intent interpretation across languages;
safe zero-initialization for new C callers;
explicit deadline clock semantics;
predictable rolling-upgrade behavior;
a clean boundary between caller request, policy authorization, and path planning.

It does not claim better SLO attainment, fairness, or throughput by itself. Those depend on QoS enforcement, Admission, receiver credits, and Execution Plan integration. The initial value is interface correctness and compatibility.

Relationship to existing work

#1058: implements the declarative Transfer Intent API roadmap direction.
#2810: reuse the standard IntentType vocabulary already merged into C++/pybind.
#2618/#2519: preserve deadline_ns and deadline-aware Admission semantics; clarify the public clock contract.
#2295: preserve explicit policy_name semantics.
#2339: preserve policy-authorized transport_hint semantics.
#2541: complement the high-level Python endpoint API; do not replace it.
#2847: intent-to-policy binding consumes the normalized intent.
#2856/#2857: effective QoS Contract authorizes and resolves requested intent.
#2849: receiver-credit allocation may later consume effective intent/QoS class; this RFC does not define credits.
#2863: Execution Plan consumes normalized intent and effective policy.
#2864: Operator CLI explains requested/effective intent and validation downgrades.

Open questions

Do maintainers agree that the existing tent_request_t array ABI should remain frozen and a new stride-aware v1 submission API should be added?
Is struct_size + entry_stride + schema_version the preferred C ABI, or should each immutable request version use a distinct exported submission function forever?
Should public v1 priority use a new zero-safe UNSPEC=0, HIGH=1, MEDIUM=2, LOW=3 enum while converting to existing internal values?
Should compatibility mode downgrade unknown intent/policy values, or should v1 always fail closed and leave compatibility only to the legacy API?
Should v1 expose only absolute local deadline_ns, or also a relative deadline_budget_ns helper/field?
Is STAGING_INTERNAL allowed only through a private/internal API?
What maximum length and character set should policy_name use?
Should detailed validation errors use a new result structure or a thread-safe last-error query in the first C ABI PR?
Which connector should be the first end-to-end adopter after the API stabilizes: Mooncake Store, SGLang, or vLLM?

Before submitting a new issue...

Searched existing issues and PRs for Transfer Intent API, intent type, deadline, policy name, transport hint, C API, Python API, and cross-language compatibility.
Reviewed the current C++, C, pybind, and high-level Python request surfaces and related roadmap/RFC discussions.

Originally created by @catyans on GitHub (Jul 12, 2026). Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/2865 ## Summary This RFC proposes a stable TENT Transfer Intent API v1 and explicit semantic parity across the C++, C, pybind, and high-level Python interfaces. TENT already has the core intent fields in its internal C++ `Request`: - `priority` - `deadline_ns` - `policy_name` - `transport_hint` - `intent_type` However, these fields were added incrementally and are not exposed consistently: - C++ `Request` contains all fields. - pybind exposes the fields added by #2810 and earlier work. - the C `tent_request_t` currently contains `priority` and `transport_hint`, but not `deadline_ns`, `policy_name`, or `intent_type`. - the high-level declarative Python API is being developed separately in #2541. - upper-layer connectors do not yet share one versioned contract for defaults, unknown values, clock domain, authorization, and compatibility. The RFC defines what an application is allowed to request and how TENT interprets that request. It does not define how TENT constructs an Execution Plan (#2863), how QoS is enforced (#2856), or how operators inspect the result (#2864). ```text Application / Store / SGLang / vLLM | v Transfer Intent API v1 | validation + normalization | effective policy resolution | v Execution Plan (#2863) ``` The first implementation milestone should add a versioned C request ABI, shared validation/normalization, and cross-language compatibility tests. It must not change default runtime scheduling behavior. ## Motivation The TENT roadmap in #1058 calls for a clean declarative API where users express what data should move and the runtime selects the path. The existing fields are a strong foundation, but an API is not stable merely because fields exist in a C++ struct. ### 1. Cross-language behavior is incomplete The current C API converts only the fields present in `tent_request_t`. A C caller cannot express the same intent as a C++ or pybind caller. This makes connector behavior depend on the language binding rather than on the request semantics. ### 2. The existing C struct is not safely extensible as an array ABI `tent_submit()` accepts an array of fixed-size `tent_request_t`. Appending fields changes the array stride expected by a newly compiled library. A new library reading a request array produced by an old binary can step beyond the old element boundary. Zero-initialization alone does not solve binary layout/stride compatibility. The current struct must remain supported. A new versioned request entry and submission API are required instead of silently extending the old array ABI. ### 3. Zero-initialization does not provide a safe priority default The current C API requires zero-initialization. `transport_hint=0` correctly means `UNSPEC`, but `priority=0` currently means `HIGH`. A caller that only zero-initializes the struct therefore requests the highest priority unintentionally. Transfer Intent API v1 should use `UNSPEC=0` for every optional enum exposed through the new C ABI. The old API retains its existing behavior for compatibility. ### 4. Deadline clock semantics are not explicit enough `deadline_ns` is an absolute `steady_clock` timestamp. It is meaningful when the caller and TENT runtime share the local monotonic clock domain, but it must not be serialized unchanged to another host or process boundary whose clock origin is not guaranteed to match. Connectors need one documented rule for local submission and a separate rule for any future remote service API. ### 5. Requested policy is not authorization An application-provided priority, policy name, or transport hint must not automatically grant access to reserved resources, a high-priority QP pool, or an otherwise forbidden backend. The API must distinguish requested intent from the effective authorized policy resolved by #2856. ### 6. Unknown values need predictable behavior Rolling upgrades can produce new callers with old runtimes or old callers with new runtimes. Unknown intent enums, request versions, flags, and optional fields require explicit strict/compatibility behavior. Silent integer casting is not a stable contract. ## Goals 1. Define stable semantics and defaults for Transfer Intent v1 fields. 2. Provide equivalent behavior across C++, C, pybind, and high-level Python. 3. Add a binary-safe, versioned C request submission API without breaking `tent_submit()`. 4. Make every optional C enum safely zero-initializable with `UNSPEC=0`. 5. Define clock-domain rules for `deadline_ns`. 6. Define validation and unknown-value behavior. 7. Separate requested intent from effective authorized policy. 8. Define string ownership, lifetime, and size limits. 9. Add compile/link/runtime compatibility tests for old and new callers. 10. Provide a stable connector contract for Store, SGLang, vLLM, and other integrations. ## Non-goals - Defining or implementing QoS authentication/RBAC. - Implementing bandwidth guarantees, policing, WDRR, or receiver-credit allocation. - Defining an Execution Plan or backend capability model; that belongs to #2863. - Defining CLI explain output; that belongs to #2864. - Replacing the high-level endpoint abstraction proposed in #2541. - Adding a network wire protocol for remote intent submission in v1. - Making Classic TE support every TENT intent field. - Changing the numeric values of existing internal C++ enums in the first PR. - Changing the behavior or layout of the legacy `tent_request_t` and `tent_submit()` API. ## Transfer Intent v1 semantic model A Transfer Intent is caller-provided metadata attached to one logical transfer request. ```cpp struct TransferIntentV1 { IntentType intent_type; RequestedPriority requested_priority; uint64_t deadline_ns; std::optional<std::string> policy_name; TransportType transport_hint; uint64_t flags; }; ``` The runtime normalizes and validates the request before policy resolution: ```text caller values -> ABI/version validation -> enum/string/deadline validation -> normalized requested intent -> authorization/effective policy (#2856) -> Execution Plan (#2863) ``` ## Field semantics ### `intent_type` Standard values: ```text INTENT_UNSPEC FOREGROUND_GET BACKGROUND_PREFETCH MIGRATION CHECKPOINT WEIGHT_LOADING STAGING_INTERNAL ``` Rules: - `INTENT_UNSPEC` preserves compatibility behavior. - one request has exactly one intent type in v1; - applications must not submit `STAGING_INTERNAL`; it is reserved for TENT-generated internal work unless an explicitly trusted internal API is used; - unknown numeric values fail validation in strict mode; - compatibility mode may map an unknown value to `INTENT_UNSPEC`, but must record the downgrade in explain/metrics; - enum numeric values become stable once published through the v1 C ABI. Recommended upper-layer mapping: | Workload | Intent | |---|---| | latency-sensitive KV fetch | `FOREGROUND_GET` | | speculative/background KV prefetch | `BACKGROUND_PREFETCH` | | replica/KV live migration | `MIGRATION` | | checkpoint transfer | `CHECKPOINT` | | model/adapter weight loading | `WEIGHT_LOADING` | | TENT-created staging hop | `STAGING_INTERNAL` | The table is a semantic default, not a requirement that every framework use the same internal operation names. ### `requested_priority` The new public C ABI should distinguish “unspecified” from “highest priority”: ```text TENT_PRIORITY_UNSPEC = 0 TENT_PRIORITY_HIGH = 1 TENT_PRIORITY_MEDIUM = 2 TENT_PRIORITY_LOW = 3 ``` Internal C++ priority values may be converted without renumbering them initially. Rules: - priority is a request, not authorization; - `UNSPEC` means derive the effective priority from the matched contract/default; - a caller cannot obtain HIGH, a reserved budget, a privileged QP pool, or wire QoS merely by setting HIGH; - an unauthorized priority is downgraded or rejected according to the effective QoS Contract mode; - invalid values fail validation; they must not be clamped silently; - the legacy `tent_request_t.priority=0` remains HIGH for compatibility, but the legacy API should be documented as unable to express UNSPEC. ### `deadline_ns` Rules for local API v1: - `0` means no deadline; - otherwise it is an absolute timestamp in the monotonic clock domain used by the local TENT runtime; - C callers use `clock_gettime(CLOCK_MONOTONIC, ...)` or a provided helper; - Python callers use `time.monotonic_ns()` or a provided helper; - wall-clock/Unix timestamps are invalid; - an already expired deadline is preserved as expired and handled by the configured Admission/degradation policy; it is not silently extended; - overflow when computing `now + budget` must fail validation or saturate through a documented helper; - the deadline applies to the user-visible logical request, not independently to every derived slice/stage. A raw absolute monotonic timestamp must not cross a host boundary. If a future remote TENT service API transports intent, it should send a remaining deadline budget plus measurement/receive timestamps and convert it into the receiver's local clock domain. That wire protocol is outside v1. Suggested helpers: ```c uint64_t tent_monotonic_now_ns(void); int tent_deadline_after_ns(uint64_t budget_ns, uint64_t *deadline_ns_out); ``` ### `policy_name` Rules: - `NULL` means no explicitly requested named policy; - an empty string is invalid rather than a second spelling of unset; - the string is UTF-8 or a documented restricted ASCII identifier; - length is bounded; - the callee copies the value before the submit function returns; - a named policy remains subject to tenant/caller authorization; - an unknown or unauthorized policy follows strict/compatibility behavior from the QoS Contract; - logs/diagnostic bundles redact or pseudonymize policy names according to #2864. ### `transport_hint` The semantics should remain consistent with #2339: - `TRANSPORT_UNSPEC` follows policy selection; - a concrete hint pins the first attempt only within the matched policy's authorized transport list; - a hint is not permission to bypass the policy allowlist; - if the hinted transport is unavailable or unauthorized, validation/selection fails rather than silently using a different first transport; - bounded failover may move to later authorized candidates after the hinted transport fails; - unknown transport values fail validation. ### `flags` v1 should reserve a bounded flags field for request-semantic modifiers that can be safely ignored only when explicitly documented. Rules: - all unknown required-semantics bits fail validation; - optional advisory bits must live in a separately documented range if introduced; - v1 may require `flags=0` until the first flag is standardized; - flags must not become an unstructured replacement for versioned fields. ## Requested versus effective intent The API exposes the caller's request. TENT derives the effective policy: ```text requested intent intent_type = foreground_get requested_priority = high policy_name = latency-critical transport_hint = rdma effective policy intent_type = foreground_get priority = medium matched_policy = tenant-a.foreground_get authorized_transports = [rdma, tcp] qp_pool = foreground degraded_actions = [fallback_transport, local_recompute] ``` The normalized requested intent and effective policy should both be observable through #2864/#2863 explain output. They must not be conflated in public metrics labels with unbounded cardinality. ## Proposed binary-safe C ABI The legacy API remains unchanged: ```c int tent_submit(tent_engine_t engine, tent_batch_id_t batch_id, tent_request_t *entries, size_t count); ``` Transfer Intent v1 should add a new immutable versioned entry rather than append fields to `tent_request_t`. Illustrative layout: ```c #define TENT_REQUEST_SCHEMA_V1 1u struct tent_request_v1 { uint32_t struct_size; uint32_t schema_version; int32_t opcode; void *source; tent_segment_id_t target_id; uint64_t target_offset; uint64_t length; int32_t intent_type; int32_t requested_priority; uint64_t deadline_ns; const char *policy_name; int32_t transport_hint; uint32_t reserved0; uint64_t flags; }; typedef struct tent_request_v1 tent_request_v1_t; ``` Because requests are submitted as an array, the runtime must know the caller's element stride. One possible generic ABI is: ```c int tent_submit_requests( tent_engine_t engine, tent_batch_id_t batch_id, const void *entries, size_t count, size_t entry_stride, uint32_t schema_version); ``` The header can provide a type-safe inline wrapper: ```c static inline int tent_submit_v1( tent_engine_t engine, tent_batch_id_t batch_id, const tent_request_v1_t *entries, size_t count) { return tent_submit_requests(engine, batch_id, entries, count, sizeof(tent_request_v1_t), TENT_REQUEST_SCHEMA_V1); } ``` This design allows the runtime to advance by the caller-provided stride and validate `struct_size` instead of assuming the library's compile-time struct size. A future v2 can use a new schema/version without changing legacy `tent_submit()`. Alternative ABI designs are welcome, but they must pass the binary compatibility cases below. Simply appending fields to the existing array element is not sufficient. ### C ABI validation For every entry: - `entry_stride` is at least the minimum size for the declared schema; - `struct_size` is within `entry_stride` and the supported bounds; - required fields are present before reading them; - missing optional tail fields use documented defaults; - extra tail bytes are ignored only for a compatible schema version; - reserved fields and unknown required flags are zero; - strings are non-dangling for the duration of the call and copied before return; - count × stride arithmetic is overflow checked; - no field is read through an unaligned pointer. ### ABI discovery Suggested query functions: ```c uint32_t tent_get_api_version(void); int tent_get_request_schema_info(uint32_t schema_version, size_t *minimum_size, size_t *known_size); ``` The exact names can change, but callers need a supported way to distinguish “symbol unavailable”, “schema unsupported”, and “request invalid”. ## C++ API C++ `Request` remains the internal native representation initially. Add shared normalization/validation helpers so C++, C, and Python do not implement separate semantics. Possible API: ```cpp Status normalizeTransferIntent(const Request &request, NormalizedTransferIntent &out, ValidationMode mode); ``` All submit paths, including notification variants, must call the same normalization logic. The RFC does not require immediately renaming `Request::priority` to `requested_priority`, but documentation and explain output must state that it is a requested value before contract resolution. ## Python APIs ### pybind - expose typed `IntentType`, `RequestedPriority`, and `TransportType` enums; - prefer keyword arguments for optional intent fields; - preserve current defaults; - reject unknown raw integers rather than accepting arbitrary casts; - provide `deadline_after_ns()` or accept a duration helper that converts locally; - expose the same validation errors as C++ with actionable messages. ### High-level declarative Python API #2541 defines source/destination endpoint ergonomics. This RFC should not replace that work. The high-level API should forward an optional intent object or equivalent keyword fields into the same normalized model: ```python transfer( src, dst, intent_type=IntentType.FOREGROUND_GET, requested_priority=Priority.UNSPEC, deadline_ns=time.monotonic_ns() + budget_ns, policy_name=None, transport_hint=TransportType.UNSPEC, ) ``` Python must not invent different fallback or authorization semantics. ## Connector contract Connectors should map their business operation to an intent before submitting to TENT. Recommended behavior: - compute the remaining TENT transfer deadline from the upper-layer SLO budget in the submitting process; - select one standard `intent_type`; - normally leave priority and transport unspecified so policy controls them; - use `policy_name` only when configured/authorized; - use `transport_hint` for explicit testing or a well-defined backend requirement, not as the default routing mechanism; - preserve the intent for retries unless the upper layer explicitly changes its degradation decision; - record both requested and effective policy IDs in bounded trace attributes. The first implementation milestone does not require modifying SGLang or vLLM. Connector PRs should follow after the API and compatibility tests stabilize. ## Strict and compatibility modes Suggested behavior: | Condition | Strict mode | Compatibility mode | |---|---|---| | unknown request schema | reject | reject or use legacy API explicitly; never guess layout | | unknown `intent_type` | reject | map to `INTENT_UNSPEC` and record downgrade | | unknown priority | reject | map to `UNSPEC` and record downgrade | | unknown transport | reject | reject; silently changing an explicit pin is unsafe | | unknown/unauthorized policy | reject | fall back to contract default and record downgrade | | unsupported optional tail field | reject if required | ignore only when documented advisory | | expired deadline | preserve expired state | same | Compatibility mode must never reinterpret bytes from an unknown struct layout. ## Error reporting The new API should distinguish at least: - unsupported ABI/schema version; - invalid struct size/stride; - invalid enum/flags; - invalid or unauthorized policy; - deadline clock/range error; - unsupported field in the current build; - runtime submit failure after intent validation. Returning only `-1` makes connector fallback unsafe. The first PR may keep the existing return type if necessary, but should provide a thread-safe status-detail query or a new result structure before connector adoption. This should align with, not preempt, any future general Transfer Outcome/Error Contract. ## Compatibility and rollout ### Existing callers - `tent_request_t` layout is unchanged. - `tent_submit()` and `tent_submit_notif()` behavior is unchanged. - zero-initialized legacy priority continues to mean HIGH. - legacy callers cannot express all v1 fields and are documented accordingly. ### New callers - use `tent_request_v1_t` and `tent_submit_v1()`; - all optional enum defaults are zero/UNSPEC; - unknown request schemas fail before any work is submitted; - the runtime copies variable-length intent metadata before returning. ### Runtime behavior - the new fields initially normalize into the existing C++ `Request`; - `INTENT_UNSPEC`, no policy, no deadline, and no transport hint preserve existing behavior; - no scheduling/QoS policy becomes enabled merely because the new API exists; - enforcement remains controlled by existing opt-in policies and later PRs. ## Proposed PR sequence ### PR1: Semantic contract and shared normalization - Add stable public enum definitions for v1. - Add normalized intent/validation helpers. - Add strict/compatibility validation tests. - Document legacy versus v1 defaults. - No C ABI or runtime selection change yet. ### PR2: Versioned C ABI - Add `tent_request_v1_t`, schema discovery, stride-safe submission, and notification equivalent. - Preserve legacy symbols/layout. - Add old/new binary compatibility fixtures. ### PR3: pybind and high-level Python parity - Route all Python fields through shared normalization. - Add typed enums and deadline helpers. - Coordinate with #2541 rather than creating a second high-level transfer API. ### PR4: Explain and observability - Expose normalized requested intent and effective policy through #2864/#2863. - Add bounded trace attributes and downgrade counters. - Avoid unbounded policy/tenant labels in metrics. ### Later connector PRs - Mooncake Store intent mapping; - SGLang foreground/prefetch/migration mapping; - vLLM connector mapping; - other framework integrations; - remote-service deadline-budget protocol if a standalone TENT service is introduced. ## Validation plan ### Semantic tests - every field default and explicit value; - unknown enums and flags in strict/compatibility mode; - policy NULL/empty/unknown/unauthorized behavior; - transport hint authorization; - past, present, overflowed, and zero deadlines; - `STAGING_INTERNAL` rejected for untrusted public submission; - requested versus effective priority distinction; - notification and non-notification submit parity. ### C ABI compatibility matrix - old client binary + new library using legacy API; - new client + new library using v1; - new client detecting a library without the v1 symbol/schema; - minimum-size v1 entry; - larger compatible entry with ignored tail bytes; - too-small/too-large/misaligned entry; - count × stride overflow; - mixed invalid entry inside a batch: all-or-nothing intent validation; - 32-bit/64-bit layout checks where supported; - C compiler and C++ compiler header compatibility; - AddressSanitizer/UBSan/fuzzing for struct parsing and strings. ### Cross-language parity For a shared table of request vectors, C++, C, pybind, and high-level Python must produce the same normalized intent or the same validation category. The test corpus should include: - all standard intent types; - every priority and transport hint; - absent and named policies; - no deadline and tight/expired deadlines; - unknown values; - legacy compatibility defaults. ### Real-cluster validation On two H20/RoCE nodes: - submit the same foreground/background/migration intent through C++ and Python; - verify identical effective policy and Execution Plan explain output; - verify `transport_hint` remains inside the policy allowlist; - verify deadlines use local monotonic time and distinguish tight/loose requests; - verify no-intent v1 requests match the legacy transfer result and throughput; - verify data integrity for Direct and forced Staged paths; - verify unsupported/unauthorized fields fail before transport submission. The API itself should add no per-slice work. Measure submit throughput and plan/validation time; transfer throughput/P99 should remain unchanged for equivalent normalized requests. ## Expected benefits and claims boundary This RFC should provide: - one stable vocabulary for upper-layer transfer semantics; - binary-safe evolution of the C request API; - identical intent interpretation across languages; - safe zero-initialization for new C callers; - explicit deadline clock semantics; - predictable rolling-upgrade behavior; - a clean boundary between caller request, policy authorization, and path planning. It does not claim better SLO attainment, fairness, or throughput by itself. Those depend on QoS enforcement, Admission, receiver credits, and Execution Plan integration. The initial value is interface correctness and compatibility. ## Relationship to existing work - #1058: implements the declarative Transfer Intent API roadmap direction. - #2810: reuse the standard `IntentType` vocabulary already merged into C++/pybind. - #2618/#2519: preserve `deadline_ns` and deadline-aware Admission semantics; clarify the public clock contract. - #2295: preserve explicit `policy_name` semantics. - #2339: preserve policy-authorized `transport_hint` semantics. - #2541: complement the high-level Python endpoint API; do not replace it. - #2847: intent-to-policy binding consumes the normalized intent. - #2856/#2857: effective QoS Contract authorizes and resolves requested intent. - #2849: receiver-credit allocation may later consume effective intent/QoS class; this RFC does not define credits. - #2863: Execution Plan consumes normalized intent and effective policy. - #2864: Operator CLI explains requested/effective intent and validation downgrades. ## Open questions 1. Do maintainers agree that the existing `tent_request_t` array ABI should remain frozen and a new stride-aware v1 submission API should be added? 2. Is `struct_size + entry_stride + schema_version` the preferred C ABI, or should each immutable request version use a distinct exported submission function forever? 3. Should public v1 priority use a new zero-safe `UNSPEC=0, HIGH=1, MEDIUM=2, LOW=3` enum while converting to existing internal values? 4. Should compatibility mode downgrade unknown intent/policy values, or should v1 always fail closed and leave compatibility only to the legacy API? 5. Should v1 expose only absolute local `deadline_ns`, or also a relative `deadline_budget_ns` helper/field? 6. Is `STAGING_INTERNAL` allowed only through a private/internal API? 7. What maximum length and character set should `policy_name` use? 8. Should detailed validation errors use a new result structure or a thread-safe last-error query in the first C ABI PR? 9. Which connector should be the first end-to-end adopter after the API stabilizes: Mooncake Store, SGLang, or vLLM? ### Before submitting a new issue... - [x] Searched existing issues and PRs for Transfer Intent API, intent type, deadline, policy name, transport hint, C API, Python API, and cross-language compatibility. - [x] Reviewed the current C++, C, pybind, and high-level Python request surfaces and related roadmap/RFC discussions.

bsakaguchi

2026-10-29 21:18:59 +00:00

closed this issue
added the
RFC

stale

auto-closed
labels

bsakaguchi commented

2026-10-29 21:18:59 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Jul 12, 2026):

Thanks for opening this issue, @catyans!

Field	Value
Issue	#2865
GitHub user ID	`18214026`
Reporter	@catyans

A maintainer will triage this when possible. To help us respond faster, please include:

Mooncake version or commit SHA
Environment (OS, CUDA/driver, RDMA stack if relevant)
Steps to reproduce and expected vs. actual behavior

Useful links: Documentation · Contributing guide

This message was posted automatically by the issue bot.

@github-actions[bot] commented on GitHub (Jul 12, 2026): Thanks for opening this issue, @catyans! | Field | Value | |-------|-------| | **Issue** | #2865 | | **GitHub user ID** | `18214026` | | **Reporter** | @catyans | A maintainer will triage this when possible. To help us respond faster, please include: - Mooncake version or commit SHA - Environment (OS, CUDA/driver, RDMA stack if relevant) - Steps to reproduce and expected vs. actual behavior Useful links: [Documentation](https://kvcache-ai.github.io/Mooncake/) · [Contributing guide](https://github.com/kvcache-ai/Mooncake/blob/main/CONTRIBUTING.md) > This message was posted automatically by the issue bot.

bsakaguchi commented

2026-10-29 21:19:00 +00:00

Author

Owner

@catyans commented on GitHub (Jul 12, 2026):

Small implementation-backed feasibility experiment on 2026-07-13 using the intent-policy binding branch at 65ee26f8.

Environment: Lingjun H20 lingjun-099, Linux 5.10.134, 192-vCPU Xeon Platinum 8575C host, GCC 13.3, Release, USE_TENT=ON, USE_CUDA=OFF.

Results:

tent_transport_selector_test + tent_intent_type_test: 2/2 passed;
repeated 200 rounds: 400/400 test executions passed, 21.12 s total;
covered canonical intent parsing, invalid/unknown handling, default behavior, and deterministic intent-to-policy matching.

Conclusion: a normalized intent_type is already viable as a stable selection input without embedding application vocabulary in transport backends. This is evidence for the normalization/policy-binding seam, not yet for the full RFC: C/Python field parity, unknown-value forward compatibility, requested/effective provenance serialization and connector round trips still need dedicated ABI/compatibility tests before API v1 can be called complete.

@catyans commented on GitHub (Jul 12, 2026): Small implementation-backed feasibility experiment on 2026-07-13 using the intent-policy binding branch at `65ee26f8`. Environment: Lingjun H20 `lingjun-099`, Linux 5.10.134, 192-vCPU Xeon Platinum 8575C host, GCC 13.3, Release, `USE_TENT=ON`, `USE_CUDA=OFF`. Results: - `tent_transport_selector_test` + `tent_intent_type_test`: **2/2 passed**; - repeated 200 rounds: **400/400 test executions passed**, 21.12 s total; - covered canonical intent parsing, invalid/unknown handling, default behavior, and deterministic intent-to-policy matching. Conclusion: a normalized `intent_type` is already viable as a stable selection input without embedding application vocabulary in transport backends. This is evidence for the normalization/policy-binding seam, not yet for the full RFC: C/Python field parity, unknown-value forward compatibility, requested/effective provenance serialization and connector round trips still need dedicated ABI/compatibility tests before API v1 can be called complete.

bsakaguchi commented

2026-10-29 21:19:00 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Oct 11, 2026):

This issue has had no activity for 90 days and will be closed in 7 days if there is no further activity. Please comment or react if it should stay open.

@github-actions[bot] commented on GitHub (Oct 11, 2026): This issue has had no activity for 90 days and will be closed in 7 days if there is no further activity. Please comment or react if it should stay open.

bsakaguchi commented

2026-10-29 21:19:00 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Oct 19, 2026):

Closing due to 3 months of inactivity. If this is still relevant, please comment and we can reopen.

@github-actions[bot] commented on GitHub (Oct 19, 2026): Closing due to 3 months of inactivity. If this is still relevant, please comment and we can reopen.

bsakaguchi referenced this issue

2026-10-29 21:30:21 +00:00

[PR #743] [CLOSED] Coro rpc transfer 0812 #1788