DSPy

Signatures are typed declarations of a task. You define inputs and outputs as Python class fields, not prompt strings. For example, a classification task looks like this:

Modules control how a signature executes. dspy.Predict produces a direct completion. dspy.ChainOfThought adds step-by-step reasoning. dspy.ReAct adds tools and a reasoning loop. You swap strategies by changing the module, not the signature.

Optimizers (formerly called teleprompters) search for the prompts and few-shot examples that produce the best outputs against your chosen metric. You call optimizer.compile() with your program, training data, and evaluation metric, and DSPy does the search. This process can include prompt rewriting, example selection, and multi-stage pipeline optimization.

You use DSPy by writing Python. You connect it to a model through the dspy.LM object, which supports OpenAI, Anthropic, local models via Hugging Face Transformers, or any OpenAI-compatible endpoint. DSPy itself does not serve models. You point it at a model endpoint that someone else serves, or run a local model alongside it.

DSPy does not manage inference performance directly. It is not a serving engine like vLLM or TensorRT-LLM. It does not handle batching, KV cache management, quantization, or GPU memory allocation. Those concerns belong to the underlying model provider or inference engine you connect to DSPy.

What DSPy does affect is pipeline efficiency. Because DSPy can search for shorter, more effective prompts and fewer few-shot examples, it can reduce token usage per call. That translates to lower latency and cost on whatever engine you are using underneath.

DSPy itself runs on CPU. It is pure Python. The hardware requirements for DSPy are minimal: any machine that can run Python can run DSPy. The GPU requirements come from the models it calls. If you are running a local model, you need whatever hardware that model requires. DSPy works with any inference engine that exposes a Python API, including vLLM, Ollama, llama.cpp, and Hugging Face Transformers.

For practitioners running DSPy in production, the typical pattern is to run a model server like vLLM on a GPU node and point DSPy at its OpenAI-compatible endpoint. DSPy handles the orchestration and optimization; the inference engine handles the throughput and latency.

Pipelines as code. Task definitions are modules with typed inputs and outputs. This makes pipelines testable, versionable, and reviewable through normal code practices. No more prompt templates hidden in config files.

Automatic optimization. DSPy optimizers search for prompts and few-shot examples that maximize a user-defined metric. This replaces hours of manual prompt tuning with a compile step that runs automatically. The underlying research, published at Stanford and arXiv, shows that DSPy-optimized pipelines consistently outperform hand-tuned prompts across multiple benchmarks.

Model agnostic. The same pipeline definition works across OpenAI, Anthropic, local models, and custom endpoints. Switching models requires changing one line in the LM configuration, then re-optimizing if desired.

Structured output. DSPy signatures enforce structured outputs with typed fields. You get back Python objects, not raw text. This is critical for building reliable agents and multi-step pipelines where downstream steps depend on specific output formats.

Streaming support. DSPy supports streaming from models that expose it, useful for chat interfaces and real-time applications.

Improving agent quality. Teams building autonomous agents use DSPy to optimize the prompts behind each agent step. Instead of tuning prompts by hand until the agent behaves correctly, you define a metric and let DSPy search for the prompts that maximize it.

Reproducible pipelines. In regulated environments or team settings, treating prompt engineering as code that can be tested and versioned is a significant advantage. DSPy enables this by keeping all prompt logic in Python rather than in external templates.

Switching models cleanly. When a new model version ships, or when you want to migrate from GPT-4 to a local model, DSPy lets you re-optimize your pipeline against the new model without rewriting your task definitions.

Not a fit for: Raw inference serving, real-time latency-sensitive applications where the optimization overhead is prohibitive, or teams that want a turnkey model server.

Install with pip:

The smallest meaningful path to using DSPy is:

1. Configure a language model: lm = dspy.LM("openai/gpt-4o-mini")
2. Define a signature: a class with InputField and OutputField annotations
3. Create a module: classify = dspy.Predict(MySignature)
4. Call it: result = classify(input="your text")

To optimize, you need:

• A training set of example inputs and outputs
• A metric function that scores outputs
• An optimizer, e.g., dspy.teleprompt.BootstrapFewShot

Documentation is at dspy.ai. The GitHub repository at github.com/stanfordnlp/dspy contains examples and the active Discord community.

DSPy vs. LangChain. Both let you compose LLM pipelines as code. LangChain focuses on integrations and pre-built components. DSPy focuses on optimization. If you need to connect to 20 different vector stores and document loaders, LangChain wins. If you need to systematically improve the quality of your pipeline outputs against a measurable metric, DSPy wins. They are complementary; some teams use LangChain for integrations and DSPy for optimizing the prompts within those integrations.

DSPy vs. manual prompt engineering. This is not a comparison against another framework. It is a comparison against the status quo. Manual prompt engineering works for simple tasks with stable models. As pipelines grow in complexity or models change, manual tuning becomes unsustainable. DSPy replaces guesswork with search. The cost is learning the programming model and setting up evaluation metrics. The benefit is reproducible, optimizable, measurable pipelines.

• Pipelines as code

  Define tasks as modules with clear inputs and outputs, not raw prompt strings.

• Automatic optimization

  Optimizers search for the prompts and few-shot examples that score best.

• Model-agnostic

  Run the same pipeline across different models and providers.