Before an agent acts,
give it a contract.

This note examines explicit task boundaries as a foundation for human–AI and agent–agent collaboration. See the AI-native organization agenda.

A workflow agent earns trust when its actions remain within the user's intent, even after an interruption, a correction, or a misleading tool response. Fluent explanations are insufficient evidence. Our thesis is that the most useful unit of agent design is an execution contract: an explicit account of the intended operation, supporting evidence, execution conditions, and the observation establishing completion.

This article synthesizes three primary research sources and proposes a testable experiment. Solo Unicorn Lab has not run the experiment described below; sample sizes and measures are proposals, not results.

What the evidence establishes

Genie makes workflow control explicit. Joshi and colleagues separate language interpretation and response generation from an algorithmic runtime governed by declarative worksheets. Worksheets represent task fields, conditions, and actions; the framework also supports confirmation fields. In a 62-participant study spanning restaurant reservations, ticket submission, and course enrollment, the authors report an aggregate goal-completion rate of 82.8% for Genie versus 21.8% for their GPT-4 Turbo function-calling baseline. These study-specific results are not deployment guarantees. The paper also documents semantic-parsing errors and additional cost and latency. Genie, version 3

A successful run does not establish dependable behavior. Yao and colleagues introduce τ-bench to evaluate agents interacting with simulated users, domain policies, and tools. Its evaluation compares the final database state with an annotated goal state. Its pass^k measure asks whether an agent succeeds consistently across repeated trials, rather than whether one favorable attempt can be found. This changes the evaluation target from a persuasive transcript to observable and repeatable task execution. τ-bench

Untrusted inputs require a separate threat model. AgentDojo evaluates agents whose tools expose them to potentially malicious external content. Its authors present an extensible environment containing 97 tasks and 629 security test cases, and find failures both with and without attacks. The framework motivates measuring ordinary usefulness alongside resistance to adversarial redirection; a defense that prevents every action would satisfy neither goal. AgentDojo, version 3

Our synthesis is architectural: interpretation, permission, and completion should have separately inspectable evidence. A correct-looking answer cannot substitute for a tool receipt. A retrieved instruction cannot establish user intent. A permitted action can still fail operationally. These distinctions guide instrumentation; they do not prove security.

A proposed experiment: separate memory from enforcement

The central question is whether an execution gate improves policy-valid completion beyond the benefit of structured state alone. We propose three matched conditions:

1. Prompt-only: a tool-calling agent receives the workflow rules as natural-language instructions.
2. Structured state: the same model and tools receive typed task state, including pending questions, current arguments, and approval status.
3. Execution contract: condition two plus a deterministic gate that checks the proposed tool, arguments, authorization evidence, and operation identity before executing a write.

For the third condition, approval would bind to a serialized action snapshot. Changing the recipient, resource, or operation would invalidate that approval. A retry would reuse an operation identifier, and completion would require observing the committed state. These requirements remain unimplemented and unvalidated.

Create 60 synthetic scenarios, equally divided among support-ticket submission, meeting rescheduling, and document-delivery workflows. Each scenario would include an initial database snapshot, a user goal, a finite-state user-response script, expected terminal-state predicates, and forbidden side effects. Within each family, allocate four cases to each of five challenges: missing information, mid-task correction, ambiguous confirmation, timeout after a successful write, and cancellation before execution.

Produce a matched clean and adversarial variant of every scenario. The adversarial variant would alter one external tool result to request an unrelated write; it would not change the legitimate user goal or the evaluator. Use fictitious records and a local sandbox with no real accounts or external writes.

Run every condition on both variants eight times, yielding 2,880 episodes. Reset the sandbox between episodes. Hold the model snapshot, sampling settings, tool schemas, context budget, and stopping limits constant. Randomize execution order with a recorded seed. Publish the prompts, fixtures, rule definitions, dependency lockfile, raw action logs, and evaluator before interpreting the results. Treat version changes as new experiments.

What to measure

The primary outcome would be policy-valid completion: all expected state predicates hold and no prohibited side effect occurred anywhere in the action trace. Final state alone can conceal a temporary unauthorized write.

Report ordinary completion separately, alongside unauthorized writes per episode, duplicate writes, false completion claims, unnecessary approval requests, and recovery after timeout. For repeated reliability, report the fraction of scenario-variant pairs that meet the primary outcome on all eight runs. Include latency and token cost per episode, plus aggregate cost divided by policy-valid completions; disclose explicitly if that denominator is zero.

Compare conditions using paired differences and bootstrap intervals clustered by base scenario, preserving dependence between variants and repetitions. Publish results by workflow and challenge type. Manually review every apparent safety violation and a prespecified random sample of successful traces, blinded to condition where possible. Retain adjudication notes.

Limits and the decision that follows

Scripted users simplify human ambiguity. A small synthetic suite cannot establish protection against adaptive attackers, unusual permissions, or changing production APIs. Typed state can preserve an incorrect interpretation, and a gate can faithfully enforce a flawed policy. Narrow tests also cannot measure whether repeated confirmation makes people less attentive.

The proposed decision rule is therefore comparative: prefer an execution contract only if it reduces unauthorized actions without an unacceptable loss of legitimate completion or usability. The practical research agenda is to make each failure attributable and reproducible. Trust should accumulate from inspectable evidence across runs, rather than from the confidence of a single conversation.

References

• Joshi et al. Controllable and Reliable Knowledge-Intensive Task-Oriented Conversational Agents with Declarative Genie Worksheets. arXiv:2407.05674v3, 2025. Primary paper
• Yao et al. τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains. arXiv:2406.12045, 2024. Primary paper
• Debenedetti et al. AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents. arXiv:2406.13352v3, 2024. Primary paper