Engineering Agentic Software Systems
From AI Assistance to Bounded Autonomy, Governed Execution, and Operational Trust
The defining engineering question is no longer whether an agent can act. It is whether the system can constrain, observe, verify, interrupt, and account for that action.
Executive Summary¶
Software agents are moving from conversational assistance to consequential action. They can inspect repositories, plan multi-step work, invoke tools, modify files, run tests, create pull requests, coordinate specialized agents, modernize applications, and increasingly participate in operational workflows.
This shift changes the unit of engineering. A model response can be reviewed as content; an agent action must be governed as behavior. The relevant system is larger than the model. It includes prompts, context, memory, identities, tools, permissions, policies, environments, orchestration logic, evaluations, telemetry, human decision rights, and recovery mechanisms.
WP-004 argues that agentic systems require a disciplined engineering model built around bounded autonomy. Authority should be explicit, least-privileged, task-scoped, time-bounded, observable, reversible where feasible, and connected to accountable human ownership. Context should be treated as controlled configuration; tools as privileged interfaces; memory as governed state; agent-to-agent communication as a trust boundary; evaluations as behavioral evidence; and runtime intervention as a first-class architectural capability.
The paper introduces an authority gradient from assist to coordinate, execute, and operate. Each level creates a progressively greater burden for identity, authority control, provenance, evaluation, observability, human oversight, containment, and recovery.
ETIS operationalizes these ideas through repository-centered engineering, evidence-centered engineering, explicit authority boundaries, design-time and runtime governance, continuous verification, and operational stewardship. The objective is not to eliminate autonomy. It is to earn it progressively, expanding delegated authority only when the surrounding controls and evidence are strong enough for the consequences.
Why Read This Paper?¶
WP-004 provides the core ETIS treatment of agentic software systems. It is especially useful for organizations moving from AI assistance toward multi-step execution, tool use, multi-agent coordination, or live operational autonomy.
After reading it, you should be able to:
- distinguish assistance from delegated execution and operational autonomy;
- classify agentic use through an authority gradient;
- identify the major control surfaces of an agentic system;
- explain why context is part of the control plane;
- design tool, identity, permission, and task boundaries;
- identify new trust boundaries created by multi-agent systems and memory;
- define behavioral evaluations for agentic systems;
- explain why oversight requires intervention capability;
- describe runtime governance and AgentOps;
- outline an operating model for progressively earning autonomy.
Key Topics¶
Intended Audience¶
What the Paper Examines¶
- The transition from assistance to agency.
- The authority gradient: assist, coordinate, execute, and operate.
- The agentic system as a sociotechnical system larger than the model.
- Intent, context, authority, execution, assurance, and accountability as control surfaces.
- Context engineering as a core control discipline.
- Tools, identities, permissions, and least-privilege authority boundaries.
- Multi-agent trust boundaries, correlated failure, and cascading action.
- Memory as governed state rather than conversation history.
- Behavioral evaluation, human oversight, and runtime intervention.
- Runtime governance, AgentOps, organizational operating models, and standards direction.
Relationship to ETIS¶
Related Publications¶
- WP-001 — Engineering Trustworthy Software in the AI Era
- WP-002 — Repository-Centered Engineering
- WP-003 — Engineering Evidence
- WP-006 — Engineering Governance
- WP-009 — Context Engineering
- WP-010 — Engineering Digital Colleagues
- WP-011 — Engineering Trust
- EB-003 — Preparing Engineering Organizations for Agentic Development
- EB-004 — Building an AI Engineering Platform
Citation
IEEE
W. T. O’Connell, “Engineering Agentic Software Systems: From AI Assistance to Bounded Autonomy, Governed Execution, and Operational Trust,” ETIS White Paper Series, WP-004, ver. 1.0, July 2026.APA 7th Edition
O’Connell, W. T. (2026). Engineering agentic software systems: From AI assistance to bounded autonomy, governed execution, and operational trust (WP-004, Version 1.0). Engineering Trustworthy Intelligent Systems.Chicago
O’Connell, William T. “Engineering Agentic Software Systems: From AI Assistance to Bounded Autonomy, Governed Execution, and Operational Trust.” ETIS White Paper Series, WP-004, version 1.0. July 2026.BibTeX
@techreport{oconnell2026agenticsystems,
author = {William T. O'Connell},
title = {Engineering Agentic Software Systems: From AI Assistance to Bounded Autonomy, Governed Execution, and Operational Trust},
institution = {Engineering Trustworthy Intelligent Systems},
type = {ETIS White Paper},
number = {WP-004},
year = {2026},
month = {July},
note = {Version 1.0},
url = {https://etisframework.org/publications/white-papers/wp-004/}
}Version History
| Version | Date | Status | Notes |
|---|---|---|---|
| 1.0 | July 2026 | Current | Initial publication. |