Building How AI Runs in Prod | Principal AI Architect & Forward Deployed Engineer | AAIF & CNCF Ambassador | Microsoft MVP | 4x Author & International Speaker
New Jersey
- Tweets 10,865
- Following 729
- Followers 9,217
- Likes 4,549
Pinned Tweet
An SRE Agent needs 4 things:
1. A proper prompt/system message. This will give the direction for the Agent to take.
2. Agent Skills that are designed to turn your Agent into an SRE and observability expert.
3. A good LLM
4. Targets
Number 4 is where the "magic happens".
When you put together a solid Agent for a particular use case, the target can be anything in its speciality. For example, in an SRE Agent, the target could be anything from a cloud provider to a Kubernetes cluster.
#agenticai #ai
3
24
165
10,278
A fundamental need for sandboxes and isolation?
🔐 Security.
But there's another crucial need...
Scale.
Without proper scale and assignment of Agents in terms of where they're running, we're back to "one Agent, one Pod" methodology, which isn't something that many organizations want to bet on.
Hardware resources and ultimately managing said resources is going to end up being a nightmare for organizations that want all teams across the company to utilize AI in a safe and governed way.
I break down how to properly scale Agents on Agent Substrate in this article.
cloudnativedeepdive.com/many…
#agenticai #ai #kubernetes
1
213
PUMPED to be speaking at @AllThingsOpen today! 🎉
My session will be on how to build production-grade Agents.
I’ll be starting off with the infrastructure/low-level primitive needed to run an Agent in a secure, scalable, and efficient way using platforms you already have (k8s + Agent Substrate).
I’ll then dive into the DevEx of how your Agents will run (@kagent_dev)
#agenticai #ai #kubernetes
3
1
16
556
It's absolutely WILD to me how much better traction/engagement/convos I have on X in comparison to LinkedIn.
LinkedIn I have 40K followers
X I have 10K
And the engagement on X is so much better.
Anyone else seeing that?
2
8
547
Running Agents at "scale" doesn't necessarily mean "I have a massive amount of Agents running".
It also means "how can Agents move from one place to another?".
Agents across multiple Pods
Agents across multiple Workers
Agents across multiple clusters
And ensuring that memory/session/context won't be lost.
#agenticai #ai #kubernetes
4
1
16
1,009
And here's a blog that goes into depth on this topic: solo.io/blog/which-agent-san…
1
104
Cool new project - Microsoft LiteBox.
It's like Microsoft's version of gVisor.
The cool thing is, unlike gVisor, it can run on both Linux and Windows.
Going to get it running and deploy an Agent in it. Maybe @agentgateway as the Gateway to call out to an LLM?
github.com/microsoft/litebox
#agenticai #ai #kubernetes
1
12
85
5,108
"One Agent, One Pod"
That's what the industry is generally used to when running agentic workloads in a Kubernetes cluster.
With Agent Substrate (starting with v0.5.0), you can run multiple Actors (where Agents run) in one Worker (Pod). This lets us scale out for better efficiency.
No more having to be tied to the "one Agent, one Pod" architecture and design.
By default, a Worker has a capacity of 1,000 Actors.
The key to doing this successfully, of course, is ensuring that each Worker has enough memory/CPU to run them (much like any other workload in a server that needs to run).
#agenticai #ai #kubernetes
9
5
51
3,093
So honestly? Not bad at all. I definitely didn't feel like I was on a "cheaper LLM" in any way. It was solid.
I can definitely see, 100%, why the Muse Models are gaining the traction they're gaining.
#agenticai #ai #kubernetes
Interesting thing to keep in mind - because I'm on the $15.00 per month plan, I'm using Muse Wonder, not Muse Wonderful. It's the equivalent of roughly GPT-6 Astra territory and a notch below Sonnet 5.5.
Not bad for $15.00 bucks.
319
Interesting thing to keep in mind - because I'm on the $15.00 per month plan, I'm using Muse Wonder, not Muse Wonderful. It's the equivalent of roughly GPT-6 Astra territory and a notch below Sonnet 5.5.
Not bad for $15.00 bucks.
First test didn't go great, but to be fair, Grok also messed this up.
You can, in fact, enable the PodCertificate APIs on GKE v1.36 because GKE allows you to modify the APIs on the Control Plane.
You can't, however, do this on AKS or EKS. You NEED to use v1.37 on those managed k8s environments.
620
Adding more visual components to Agentic Field Guide!
🚀 Agentic AI The Hard Way and the Agentic Field Guide have been updated!
Updates on the labs include all things for agentic sandboxing:
✅ Updated installation and configuration of Substrate and kagent
✅ Agent, Harness, and AgentTemplate objects updated
✅ Breakdown and architecture diagram for Agent Substrate components
✅ Hermes + Agent Substrate
✅ Egress for Substrate
✅ Multi-session identity
✅ microVM in Agent Substrate
✅ ReBAC with OpenFGA
✅ Autoscaling Substrate
✅ Observability in Substrate
and a few other goodies!
Check it out on the git repo here: github.com/AdminTurnedDevOps…
You can also see it and go through the labs with more developer experience on agenticfieldguide.ai/
#agenticai #ai #kubernetes
288
First test didn't go great, but to be fair, Grok also messed this up.
You can, in fact, enable the PodCertificate APIs on GKE v1.36 because GKE allows you to modify the APIs on the Control Plane.
You can't, however, do this on AKS or EKS. You NEED to use v1.37 on those managed k8s environments.
Testing out @Muse today. Subscribed to the $15.00 per month plan (want to test the usage against @grok $30.00 per month plan).
Tasks that I'm going to try:
1. Interact with my GKE cluster
2. Install Agent Substrate (cluster is on v1.36, Muse will need to be able to figure out that the PodCertificate API needs to be enabled)
3. Understand the Substrate codebase
#agenticai #ai #kubernetes
590
Massive attack via @opencode where everything from adding an API key to authenticate to your LLM or even adding a provider can result in your keys being stolen.
If you're going through a gateway, like @agentgateway, however, your keys are protected on the backend.
You aren't going straight to the provider via your API key or connected provider in opencode.
Instead, you're going through a gateway/proxy that you can lock down, secure, govern, and observe for things like this.
Egress is critical, and having your line of communication (an AI Gateway) sit between your Agent and the destination (e.g., an LLM provider) helps significantly with attacks like this.
#agenticai #ai #kubernetes
14
8
58
12,953
🚀 Agentic AI The Hard Way and the Agentic Field Guide have been updated!
Updates on the labs include all things for agentic sandboxing:
✅ Updated installation and configuration of Substrate and kagent
✅ Agent, Harness, and AgentTemplate objects updated
✅ Breakdown and architecture diagram for Agent Substrate components
✅ Hermes + Agent Substrate
✅ Egress for Substrate
✅ Multi-session identity
✅ microVM in Agent Substrate
✅ ReBAC with OpenFGA
✅ Autoscaling Substrate
✅ Observability in Substrate
and a few other goodies!
Check it out on the git repo here: github.com/AdminTurnedDevOps…
You can also see it and go through the labs with more developer experience on agenticfieldguide.ai/
#agenticai #ai #kubernetes
1
4
658
Testing out @Muse today. Subscribed to the $15.00 per month plan (want to test the usage against @grok $30.00 per month plan).
Tasks that I'm going to try:
1. Interact with my GKE cluster
2. Install Agent Substrate (cluster is on v1.36, Muse will need to be able to figure out that the PodCertificate API needs to be enabled)
3. Understand the Substrate codebase
#agenticai #ai #kubernetes
5
5
944
Absolutely excellent presentation about inference from @discoposse
piped.video/qLLqt78grMI?t=433
#agenticai #ai #kubernetes
1
3
462