Agentic
Agentic AI & multi-agent systems
LLM agents that plan and call tools — the hard parts are tool safety, isolation, and cost control.
LangGraphCrewAIRay ServevLLMMCP
Architecture
An orchestrator runs agent loops that call an inference backend and a set of tools. Tools are sandboxed and scoped per tenant; actions are logged and gated.
Sizing
GPU demand is the inference backend; agents multiply requests, so batch and cache aggressively (prefix caching).
Networking
Standard; isolate tool egress and enforce per-tenant network policy.
Storage
Object storage for artifacts/traces; a state store for agent memory.
Scaling
Scale the inference backend on token throughput; rate-limit tool calls to bound cost.
Monitoring
Tokens/sec, tool-call rate and latency, cost per task, guardrail triggers, error rate.
starting point
apiVersion: apps/v1
kind: Deployment
metadata:
name: agent-orchestrator
spec:
replicas: 2
template:
spec:
containers:
- name: orchestrator
image: your-registry/agents:latest
env:
- { name: LLM_ENDPOINT, value: "http://llm/v1" }
- { name: TOOL_ALLOWLIST, value: "search,retrieve" }