True foundry research companion
satya - 5/31/2026, 8:41:23 PM
what is it
- started in 2021
- As an MLops tool in cloud and onprem
- Pivoted to LLM Ops (LLM gateway, deploy, finetune, rag, prompt lib)
- Further pivot to MCP and Agentic gateways and governance in the cloud and on prem
satya - 5/31/2026, 8:43:05 PM
how does True Foundry compares to MLflow
how does True Foundry compares to MLflow
satya - 5/31/2026, 8:53:25 PM
MLFlow and True Foundry
- MLflow
- ****************
- MLflow is opensource and widely used for running and evaluating models during training
- Also expanded into AI space such as prompt effectiveness, evals, and cost control of LLMs
- Free but managing and hosting is on enterprises
- ---------------
- True Foundry
- ***********************
- Managed
- Commercial only
- LLM and Agentic shift seem strong
- What an enterprise has been implementing internally over the last couple of years using LangChain ecosystem and libraries and utilities are now offered by True Foundry
- ----------------------
- Caution
- ************************
- QUestion to ask is the licensing cost
- Vendor lock in
- Probably good for the bottom 60% of the LLM capabilites
- However if one is to go to agentic core or similar cloud based platform these may be out of the box
- The "agent run time" is likely to be managed and controlled by frontier companies
- So it is not clear how it will integrate into that 40% space
satya - 5/31/2026, 8:57:47 PM
So questions to ask are from the above
- Cost factors
- vendor lock
- How does it compare to native aws and azure offerings
- How can it help in "operationalizing" ai in enterprises, even at mid and lower levels
- What would an enterprise want in its "operational ai platform"?
satya - 5/31/2026, 9:06:16 PM
Another detailed question
- If one is to use say Bedrock, what is the overlap of this functionality?
- what will one be missing from that that True Foundry still provides?
- Can you list the "absolute" requirements of an ai operational platform that one would want?
satya - 5/31/2026, 9:47:09 PM
Key features interested in licensing
- AI Gateway: Routes LLM calls, enforces rate limits and quotas, meters usage per team, applies semantic caching and guardrails
- MCP Gateway: Central registry for all MCP servers with per-server RBAC, OAuth 2.0, and environment grouping (dev/staging/prod)
- Agent Gateway: Governs multi-agent workflows, traces agent-to-tool calls, enforces agent-level access policies, supports human-in-the-loop approvals
- Prompt Management: Versioned prompt templates stored centrally and shared across teams
- Control Plane UI: Single dashboard for administering all three gateways
satya - 5/31/2026, 9:52:54 PM
Hyper scaler like aws vs True Foundry
- First of all very similar features in hyper scalers
- Both offer agent registries, mcp registries, rbac, administration etc.
- True Foundry may be more focused, at a higher cost
satya - 5/31/2026, 10:11:17 PM
Ok, true competitors
- Only hyper scalers honestly
- There are many LLM gateways but none for MCP and Agentic layers along with rbac and control planes
- Likely the later will be done by Frontier labs likely
satya - 5/31/2026, 10:13:30 PM
Summary
- if adopts aws agentcore for example, it has most of the functionality. so just use it
- Or use frontier agentic run times: Copilot, agentcore, Google, etc
- Space is still evolving