A guidance language for controlling large language models.
About Guidance
Guidance is a Python library from Microsoft for steering language models. Instead of asking a model nicely for a certain format and hoping, a developer writes a program that mixes ordinary Python control flow with calls to generate text, and constrains the output so that it must match a pattern.
Constraints can be a regular expression, a JSON schema, a grammar or a choice between options. Programs can loop and branch, and a model can stop when it calls a tool, wait for the result and carry on. It works with several backends, including Transformers, llama.cpp and OpenAI, and the strongest controls apply when it can integrate with the model's inference, as it can with local models.
Microsoft Research describes the library as enforcing constraints token by token, and claims that this guarantees structure and can reduce latency and cost, though those figures come from the project itself. Guidance is aimed at developers who need dependable structured output and are willing to write a little more code to get it.
Key features
- Python control flow mixed with model generation
- Constraints by regular expression, JSON schema, grammar or choice
- Loops and branches inside generation
- Tool calls that pause and resume generation
- Backends including Transformers, llama.cpp and OpenAI
- Open source from Microsoft Research
Links
More from Microsoft
A programming framework for agentic AI
UFO³: Weaving the Digital Agent Galaxy
Playwright MCP server
Microsoft Foundry Full-stack AI platform with agent capabilities.
Low-code agent builder. Integrates with M365, Dynamics, Power Platform.
JARVIS, a system to connect LLMs with ML community. Paper: https://arxiv.org/pdf/2303.17580.pdf
A framework for building, orchestrating and deploying AI agents and multi-agent workflows with support for Python and .NET.
Integrate cutting-edge LLM technology quickly and easily into your apps
TypeChat is a library that makes it easy to build natural language interfaces using types.
Microsoft's AI assistant across Windows and Office.
All content and agents from Microsoft
Frequently asked questions about Guidance
- What is Guidance?
- A guidance language for controlling large language models. It is listed on botshub under agent frameworks and sdks.
- Who makes Guidance?
- Guidance is made by Microsoft.
- Is Guidance free or open source?
- Guidance is tagged as open source on botshub. Check the official site for current terms.
- What can Guidance do?
- Key features include python control flow mixed with model generation, constraints by regular expression, JSON schema, grammar or choice, loops and branches inside generation, tool calls that pause and resume generation and backends including Transformers, llama.cpp and OpenAI.
- What are alternatives to Guidance?
- Other agent frameworks and sdks on botshub include LazyMind, LLM Agents, DSPY and Strands Agents. See all alternatives to Guidance.
