docs(rfc): propose SDK consumption entrypoints and file transfer#1590
docs(rfc): propose SDK consumption entrypoints and file transfer#1590zanetworker wants to merge 2 commits into
Conversation
Add RFC 0006 proposing official Python and TypeScript SDKs for programmatic sandbox consumption by agent platforms and frameworks. The RFC covers: - Three sandbox modes and which the SDK serves (Mode 2 and 3) - Extending the Python SDK with OIDC auth, provider management, streaming watch, and file transfer - Streaming UploadFile/DownloadFile gRPC RPCs for the gateway - A new TypeScript SDK for OpenClaw and Node.js frameworks - Five implementation phases with dependency analysis - Integration examples for Anthropic Managed Agents and OpenAI Agents SDK Includes architecture diagrams for the three modes, file transfer sequence flow, phase dependencies, and Anthropic worker end-to-end. Signed-off-by: Adel Zaalouk <azaalouk@redhat.com>
zanetworker
requested review from
a team,
derekwaynecarr,
maxamillion and
mrunalp
as code owners
| frameworks. Add streaming UploadFile/DownloadFile gRPC RPCs to the | ||
| gateway so SDK consumers can move files in and out of sandboxes | ||
| without shelling out to the CLI. Support OIDC authentication in both | ||
| SDKs so cross-namespace K8s deployments work without copying mTLS |
There was a problem hiding this comment.
can you clarify what is meant by cross namespace deployments?
There was a problem hiding this comment.
Good call. "Cross-namespace" is K8s jargon that does not belong here. What I meant: the current Python SDK only supports mTLS, which requires distributing TLS client certificates to every consumer. In a K8s deployment that means copying Secrets across namespaces, but the same friction applies outside K8s — any consumer on a different machine needs those certs.
OIDC removes that distribution problem regardless of deployment model. I will rewrite this section to frame it as "SDK consumers need bearer-token auth so they can connect to any OIDC-enabled gateway without distributing client certificates."
|
|
||
| | Method | RPC | Why | | ||
| |--------|-----|-----| | ||
| | OIDC auth | gRPC metadata interceptor | mTLS-only locks SDK to single namespace. Every K8s production deployment needs cross-namespace auth. | |
There was a problem hiding this comment.
I don’t understand what oidc has to do with k8s, agree the sdk needs to work with a server that is oidc auth
There was a problem hiding this comment.
Agreed — OIDC has nothing to do with K8s specifically. The SDK needs to work with any gateway that has OIDC auth enabled, whether that is on K8s, bare metal, or a managed service. I conflated "the most common deployment where this matters" with "the reason it matters." Will decouple the two in the next revision.
|
Positioning relative to RFC 0005 (#1617) and the Python SDK OIDC PR (#1621) Three SDK efforts are in flight under #1044. Here is how they relate:
RFC 0005 and RFC 0006 are complementary, not competing. RFC 0005 delivers the shared Rust core and TypeScript binding — the "how" for the TS half. This RFC frames the "why" and "what" across both languages: which consumption patterns platforms need, what RPC gaps block adoption (file transfer), and how the SDK surface maps to real platform integrations. Concretely, this RFC covers areas that RFC 0005 explicitly defers or does not address:
The intended reading order: this RFC for the overall SDK strategy and surface area, RFC 0005 for the shared core architecture and TS binding implementation. Will update the RFC text to reference RFC 0005 directly and to decouple OIDC from K8s framing per @derekwaynecarr feedback. |
- Decouple OIDC motivation from K8s framing per @derekwaynecarr feedback - Reframe as certificate distribution problem, not namespace problem - Add relationship section positioning RFC 0006 relative to RFC 0005 - Add links to RFC 0005 (NVIDIA#1617), Python OIDC PR (NVIDIA#1621), roadmap (NVIDIA#1044)
| **Mode 1: Sandbox the entire agent.** The agent process runs inside | ||
| the sandbox. Interface: CLI. No SDK needed. | ||
|
|
||
| **Mode 2: Platform-managed sandbox.** The platform (Anthropic, OpenAI) | ||
| owns the agent loop. A separate worker on your infrastructure embeds | ||
| the SDK and creates sandboxes. Brain and worker are physically | ||
| separate systems. Mode 2 is a spectrum: |
There was a problem hiding this comment.
I've heard this commonly described as: Agent in a Sandbox (mode 1) and Sandbox as a Tool (mode 2).
Reference: https://www.langchain.com/blog/the-two-patterns-by-which-agents-connect-sandboxes.
OpenShell has always intended to support both. If that analogy is similar to mode 1 and 2, I would suggest framing this RFC around those two terms. This is how I think we should document usage as well.
|
|
||
| ### Three sandbox modes | ||
|
|
||
| The SDK serves Mode 2 and Mode 3. Mode 1 stays CLI-driven. |
There was a problem hiding this comment.
nit: I think we still want to programmatically launch agents in a sandbox from the SDK. CLI and SDK should generally have API parity outside of things like interactive shell access. For example, I think we also want the policy prover in our SDK.
| - **Provider CRUD in the SDK.** Providers are created by the platform | ||
| engineer via CLI. SDK consumers attach existing providers, not | ||
| create new ones. |
There was a problem hiding this comment.
Any reason not to include this? I would expect to have api parity between the CLI and SDK.
| **Mode 3: Framework sandbox extension.** The developer's process owns | ||
| the agent loop. Harness + SDK live in one process. Logical separation. | ||
| Examples: OpenAI Agents SDK, OpenClaw, LangChain. |
There was a problem hiding this comment.
I'm not understanding how this mode is different. How does this relate to OpenShell? It seems most similar to Mode 1, Agent in a Sandbox, but perhaps I'm missing some nuance.
Can you elaborate with more concrete examples?
| Areas this RFC covers that RFC 0005 explicitly defers: | ||
|
|
||
| - File transfer RPCs (UploadFile/DownloadFile) | ||
| - Python SDK surface expansion (provider, watch, policy, services) | ||
| - Platform consumption patterns (Anthropic, OpenAI, OpenClaw, CI/CD) | ||
| - Python-on-shared-core migration path |
There was a problem hiding this comment.
This RFC seems to also detail some of the "how" (see implementation plan). I think we should try and close both RFCs at the same time since they inform each other.
My recommendation is we build a rust core (#1617) so we're not re-implementing common workflows across all languages for features such as OIDC, file transfer, image building, etc. SDKs are thin wrappers over the rust core.
This RFC can serve as
- guidance for how the SDK should be used, eg Agents in a Sandbox and Sandbox as a Tool.
- guidance on what goes into the SDK vs CLI vs gRPC service routes.
| 4. **Provider CRUD in SDK.** This RFC scopes the SDK to attach/detach | ||
| (bind existing providers). Should full provider CRUD | ||
| (create/update/delete) be in scope for platforms that want fully | ||
| programmatic provider lifecycle? |
There was a problem hiding this comment.
yes, i think we want to strive for api parity as a principle.
| **TypeScript SDK maintenance.** A second SDK doubles the maintenance | ||
| surface. Mitigation: both SDKs are thin wrappers around generated | ||
| proto stubs. The Python SDK is ~600 lines. The TypeScript SDK would | ||
| be similar. The proto files are the source of truth; SDK updates are | ||
| mechanical when protos change. |
| | Anthropic worker | Create sandboxes, download skills, run tool calls, retrieve artifacts | No OIDC auth, no file transfer RPC | | ||
| | OpenAI Agents SDK adapter | Implement SandboxClient: materialize Manifest, exec, snapshot | No file transfer RPC (session.write() for LocalDir has no clean implementation) | | ||
| | OpenClaw plugin | Create sandboxes, sync workspace, exec commands | No TypeScript SDK (plugins are TS-only), currently shells out to CLI 5+ times per command | | ||
| | Multi-tenant platform | Per-tenant sandboxes with policies and credentials | No OIDC auth, no provider attach/detach in SDK | |
There was a problem hiding this comment.
OIDC auth in OpenShell's current form != multi-tenancy right? For example, if you and I both authenticate against a gateway and create sandboxes, we can see each other's work.
| - "Issue #1044 (SDK roadmap)" | ||
| --- | ||
|
|
||
| # RFC 0006 - SDK Consumption Entrypoints and File Transfer |
There was a problem hiding this comment.
nit: sdk design principles and streaming file transfers seem like two distinct things. is there a reason these are linked together? can we decouple the efforts?
| | `expose_service()` / service CRUD | ExposeService, GetService, ListServices, DeleteService | Sandbox-hosted HTTP services | | ||
| | `get_logs()` | GetSandboxLogs | One-shot log retrieval for debugging | | ||
|
|
||
| ### 2. Add streaming file transfer RPCs to the gateway |
There was a problem hiding this comment.
I'm supportive of this feature, but I think we'll want to do some more design work around the feature. Streaming file uploads can be hard!
3 participants
Related: #1044
Summary
RFC 0006 proposing official Python and TypeScript SDKs that make OpenShell consumable as programmable infrastructure for agent platforms and frameworks, plus streaming file transfer RPCs.
Why: Agent platforms (Anthropic Managed Agents, OpenAI Agents SDK, OpenClaw, Cloudflare) all need a secure execution layer. OpenShell has the enforcement (Landlock, seccomp, L4/L7 policy, credential injection, OCSF audit) and the API (54 gRPC RPCs). But only 8 RPCs are wrapped in the Python SDK, there's no TypeScript SDK, no file transfer RPC, and no OIDC auth in the SDK. Every integration must shell out to the CLI binary or build a custom gRPC client.
What this RFC proposes:
Three sandbox modes covered:
Five implementation phases with a dependency analysis showing Phase 1 (OIDC + providers) and Phase 2 (file transfer) can run in parallel.
Related Issues
python/openshell/sandbox.pyChanges
rfc/0006-sdk-and-file-transfer/README.md- Full RFC documentrfc/0006-sdk-and-file-transfer/sdk-modes.png- Three sandbox modes architecture diagramrfc/0006-sdk-and-file-transfer/sdk-phase-deps.{mmd,png}- Phase dependency diagramrfc/0006-sdk-and-file-transfer/sdk-file-transfer.png- UploadFile/DownloadFile sequence diagramrfc/0006-sdk-and-file-transfer/sdk-anthropic-worker.png- Anthropic Mode 2 end-to-end sequenceTesting
mise run pre-commitpassesChecklist
rfc/0000-templatestructure