Skip to content
botshub
Visit Guidance

A guidance language for controlling large language models.

About Guidance

Guidance is a Python library from Microsoft for steering language models. Instead of asking a model nicely for a certain format and hoping, a developer writes a program that mixes ordinary Python control flow with calls to generate text, and constrains the output so that it must match a pattern.

Constraints can be a regular expression, a JSON schema, a grammar or a choice between options. Programs can loop and branch, and a model can stop when it calls a tool, wait for the result and carry on. It works with several backends, including Transformers, llama.cpp and OpenAI, and the strongest controls apply when it can integrate with the model's inference, as it can with local models.

Microsoft Research describes the library as enforcing constraints token by token, and claims that this guarantees structure and can reduce latency and cost, though those figures come from the project itself. Guidance is aimed at developers who need dependable structured output and are willing to write a little more code to get it.

Key features

  • Python control flow mixed with model generation
  • Constraints by regular expression, JSON schema, grammar or choice
  • Loops and branches inside generation
  • Tool calls that pause and resume generation
  • Backends including Transformers, llama.cpp and OpenAI
  • Open source from Microsoft Research

More from Microsoft

All content and agents from Microsoft

Frequently asked questions about Guidance

What is Guidance?
A guidance language for controlling large language models. It is listed on botshub under agent frameworks and sdks.
Who makes Guidance?
Guidance is made by Microsoft.
Is Guidance free or open source?
Guidance is tagged as open source on botshub. Check the official site for current terms.
What can Guidance do?
Key features include python control flow mixed with model generation, constraints by regular expression, JSON schema, grammar or choice, loops and branches inside generation, tool calls that pause and resume generation and backends including Transformers, llama.cpp and OpenAI.
What are alternatives to Guidance?
Other agent frameworks and sdks on botshub include LazyMind, LLM Agents, DSPY and Strands Agents. See all alternatives to Guidance.
Report this listing
Is this your product? Claim this listing