agent-desktop-ffi

Direct C-ABI access to PlatformAdapter operations. Build the cdylib with the
release-ffi profile:

cargo build --profile release-ffi -p agent-desktop-ffi

The output is target/release-ffi/libagent_desktop_ffi.dylib (.so on Linux,
agent_desktop_ffi.dll on Windows) plus a committed C header at
crates/ffi/include/agent_desktop.h. The default release profile uses
panic = "abort": a Rust panic inside an extern "C" function would SIGABRT
the host. release-ffi keeps panic = "unwind", which the panic traps need.

Four reference topics, loaded as needed:

• ownership.md: who allocates and who frees every

pointer the FFI hands back.

• error-handling.md: errno-style last-error

contract, error codes, enum validation, panic boundary.

• threading.md: host-thread contract, mutation

ordering across threads and processes, adapter-bound native handles,
AXIsProcessTrusted inheritance.

• build-and-link.md: prebuilt archives, ABI

handshake, struct size validation, C and Python examples.

Observe-act workflow (canonical path)

ad_init(AD_ABI_VERSION_MAJOR)                    // verify header ↔ dylib match
adapter = ad_adapter_create_with_session("s1")   // or ad_adapter_create()
rc = ad_snapshot(adapter, "Finder", 0, 10, false, false, &json_out)
// parse json_out: locate snapshot-qualified refs in data.tree
ad_free_string(json_out)
// build action:
AdAction act = {0}; act.kind = AD_ACTION_KIND_CLICK;
rc = ad_execute_by_ref(adapter, "@s8f3k2p9:e5", &act, 0, &result_out)
ad_free_string(result_out)
ad_adapter_destroy(adapter)

ad_snapshot returns the CLI envelope {version, ok, command, data}.
data.tree carries snapshot-qualified refs for interactive elements. Read
data.complete before trusting a tree: false means the observation budget ran
out and the tree is partial, not an error. ad_snapshot always targets the
focused window of app; window targeting, skeleton mode and --root drill-down
are not exposed over the ABI. A bare @eN ref returns
AD_RESULT_ERR_INVALID_ARGS.

Core constraints

• ABI handshake. Call ad_init(AD_ABI_VERSION_MAJOR) once after loading the

library. A mismatch returns AD_RESULT_ERR_INVALID_ARGS: abort. ad_abi_version()
returns the library's major for display. The major increments on a removed
symbol or a layout change; new ad_* symbols and error codes are additive.
Pin the exact libagent_desktop_ffi version you link against.

• Session adapters. ad_adapter_create_with_session("session-id") scopes

refmap persistence to a session namespace, like CLI --session <id>. IDs are 1
to 64 ASCII alphanumeric, -, or _ characters. A null ID behaves like
ad_adapter_create(). An invalid ID returns null (read ad_last_error_*).

• Ref actions. ad_execute_by_ref is the primary action entrypoint. It loads

the RefStore, resolves the ref strictly (STALE_REF when the snapshot has no
such ref or the element changed, AMBIGUOUS_TARGET on several candidates), runs
the live actionability preflight (visible, stable, enabled, supported action,
policy, editable), then dispatches. A disabled or unsupported target fails
before any platform call; on AD_RESULT_ERR_ACTION_FAILED the check report is
JSON in ad_last_error_details(). It auto-waits 5000 ms before the preflight;
ad_execute_by_ref_timeout takes timeout_ms (-1 = 5000 ms default, 0 =
single-shot). PressKey defaults to focus_fallback; every other action,
including TypeText, defaults to headless. Pass AD_POLICY_KIND_FOCUS_FALLBACK
(1) to allow focus without cursor movement, or AD_POLICY_KIND_HEADED (2) to allow
physical cursor and keyboard delivery. ad_execute_ref_action_exact_with_policy
runs the same preflight from an AdExactRefEntry.

• Generation-safe direct APIs. Every direct targeting function takes

process-generation evidence: AdExactRefEntry, AdExactWindowInfo,
ad_list_windows_exact, ad_launch_app_exact, and the other *_exact
symbols. AdRefEntry, AdWindowInfo, and AdSurfaceInfo exist
only as fields embedded in the exact structs. ad_list_surfaces_exact keeps the
surface id. From AD_EXACT_SURFACE_INFO_VERSION 2, AdExactSurfaceInfo.unclassified
names the surface kinds its probe could not read (for example sheet,menu), or
is null. A missing sheet or menu next to a listed kind is unknown, not
absent. Only the Windows adapter fills it; macOS and Linux return null.

• Native handles. ad_resolve_element_exact and ad_find_exact return an

AdNativeHandle. ad_execute_action (headless, no preflight) and
ad_execute_action_with_policy act on it directly, bypassing the ref-action
pipeline. Release every handle with ad_free_handle(adapter, &handle) on the
same adapter and thread, before ad_adapter_destroy, on every platform. It
zeroes handle.ptr, so a second call is a no-op.

• Action results. AdActionResult.steps mirrors the CLI steps array for

activation-chain actions. Each AdActionStep has label and outcome strings,
mechanism (AD_STEP_MECHANISM_SEMANTIC_API = 1 or
AD_STEP_MECHANISM_PHYSICAL_SYNTHETIC = 2, valid only when has_mechanism),
and verified (valid only when has_verified). AdActionResult.disposition
carries the same delivery/retry pair as ad_last_error_delivery_semantics.
Release the result with ad_free_action_result(&out).

• Display discovery. Call ad_list_displays before using

AdScreenshotTarget.screen_index. List order is the screenshot index order.
Each AdDisplayInfo carries a stable display ID, bounds, primary flag, and scale.
Release the list with ad_display_list_free.

• Errors. A failing call sets thread-local last-error state; read it with the

ad_last_error_* accessors before the next failing call. ad_last_error_details()
returns structured JSON (the actionability report on ACTION_FAILED, candidate
summaries on AMBIGUOUS_TARGET) that can contain element names, values, and
window titles from the user's screen: keep it out of shared logs.
ad_last_error_delivery_semantics reports whether the failed action was
delivered and whether a retry is safe. Invalid enum discriminants return
AD_RESULT_ERR_INVALID_ARGS. See error-handling.md.

• Threading. Entrypoints may be called from any host thread. Native handles

stay on the adapter and thread that created them; pass qualified refs across
threads. Desktop mutations take the same cross-process interaction lease as the
CLI; reads take no lease and carry finite deadlines. See
threading.md for the read/read, read/mutation, and
mutation/mutation matrix.

• Tracing. File tracing (same JSONL contract as CLI --trace; segments carry

event, ts_ms, seq, and redacted fields) is active only when the session has
a manifest with trace: on, created by session start (or an equivalent manifest
on disk) before the adapter is created. ad_adapter_create_with_session alone
writes nothing. Each OS process appends to one segment,
~/.agent-desktop/sessions/<id>/trace/<pid>-<procTs>.jsonl, and a long-lived host
reuses it. ad_trace_show (JSON envelope) and ad_trace_export (self-contained
HTML file) read the adapter session's merged timeline; limit -1 = CLI default.

• Log callback. ad_set_log_callback(cb) registers one process-wide callback

for diagnostics regardless of session manifest. It receives, as JSON, the
tracing events emitted inside later ad_* calls on the same thread; the FFI
never installs or replaces the host process's global tracing subscriber. cb
receives an int32_t level (1=ERROR ... 5=TRACE) and a const char *msg valid
only for the duration of the call. Pass NULL to unregister. Sensitive field
values (password, token, text, ...) are replaced with {"redacted":true}. The
callback must not unwind across the C ABI: catch C++ exceptions and Rust panics
inside it, because a violation may abort the host. It may fire from several host
threads at once and briefly after a NULL unregister: keep the callback and
any data it captures valid for the process lifetime.

• Wait. ad_wait(adapter, args, &out) runs the CLI wait command. Set

mode.pause (AdOptionalU64, ms), mode.element, mode.window (waits for a
window with that title), mode.text, or one of mode.surfaces.menu,
.menu_closed, .notification; add the predicate group (predicate,
value, action, count) and scope.timeout_ms / scope.app. The CLI event
wait (--event, --window-id) is not exposed. Zero-initialize AdWaitArgs and
validate the struct size against AD_WAIT_ARGS_SIZE / ad_wait_args_size()
before calling. The output is a {version, ok, command, data} envelope freed with
ad_free_string. ad_wait blocks the calling thread up to scope.timeout_ms ms. A
concurrent ad_adapter_destroy does not invalidate an in-flight wait.

• Text input privacy. On macOS, focus-fallback or headed text insertion may

briefly use the clipboard for non-ASCII text. For sensitive text, prefer
AD_ACTION_KIND_SET_VALUE with AD_POLICY_KIND_HEADLESS when the target
supports settable values.

• ad_get_tree_exact vs ad_snapshot. ad_get_tree_exact returns a raw flat

breadth-first tree: no refs, no refmap persistence, no JSON envelope (free it
with ad_free_tree). Use it for custom traversal or UI inspection. For
observe-act agents that drive actions via ad_execute_by_ref, start with
ad_snapshot.