Python: ADR: Unifying Context Management with ContextMiddleware (Python)

[eavanvalkenburg]

Summary

This ADR proposes unifying \ContextProvider, \ChatMessageStore, and \AgentThread\ into a single \ContextMiddleware\ concept for the Python SDK.

Problem

Currently, developers doing 'Context Engineering' must understand multiple abstractions:

• \ContextProvider\ - Injects instructions, messages, and tools
• \ChatMessageStore\ - Stores and retrieves conversation history
• \AgentThread\ - Coordinates between them

Proposed Solution

A unified \ContextMiddleware\ using the onion/wrapper pattern (like existing \AgentMiddleware):

\\python
class RAGMiddleware(ContextMiddleware):
async def process(self, context: SessionContext, next) -> None:
# Pre-processing: add context
docs = await self.retrieve_documents(context.input_messages[-1].text)
context.add_messages(self.source_id, [ChatMessage.system(f'Context: {docs}')])

    await next(context)

    # Post-processing: store/audit
    await self.store_interaction(context.input_messages, context.response_messages)

agent = ChatAgent(
chat_client=client,
context_middleware=[
InMemoryStorageMiddleware('memory'),
RAGMiddleware('rag'),
]
)
\\

Key Decisions

Decision	Rationale
Onion/wrapper pattern	Familiar from existing middleware, natural pre/post processing
Agent owns config, Session owns pipeline	Enables per-session factories
Mandatory \source_id\	Attribution in \context_messages\ dict
Default \InMemoryStorageMiddleware\	Zero-config conversation history
Single \StorageContextMiddleware\ class	Configure for memory/audit/evaluation
Clean break (no shims)	We're in preview

Migration Impact

Current	New
\ContextProvider\	\ContextMiddleware\
\ChatMessageStore\	\StorageContextMiddleware\
\AgentThread\	\AgentSession\

See the full ADR in \docs/decisions/00XX-python-context-middleware.md\ for detailed design, code examples, and implementation workplan.

Related Issues

This ADR addresses the following issues:

• Python: sessions: Rename AgentThread to AgentSession #3587 - Rename AgentThread to AgentSession
• Python: sessions: Add get_new_session, get_session_by_id methods #3588 - Add get_existing_thread, get_new_session, get_session_by_id methods
• Python: sessions: Move serialize method into the agent #3589 - Move serialize method into the agent
• Python: sessions: Design orthogonal ChatMessageStore for service vs local threads #3590 - Design orthogonal ChatMessageStore for service vs local threads
• Python: sessions: Rename ChatMessageStore to ChatHistoryProvider #3601 - Rename ChatMessageStore to ChatHistoryProvider

[Copilot]

Pull request overview

Adds a new ADR proposing a unified ContextMiddleware abstraction for Python to replace ContextProvider, ChatMessageStore, and AgentThread, using an onion/wrapper middleware pipeline pattern.

Changes:

• Introduces a proposed ContextMiddleware + SessionContext + pipeline design for composable context engineering.
• Describes a StorageContextMiddleware approach for loading/storing conversation history and optional auditing.
• Outlines migration impact and a phased implementation/testing plan.

[Copilot]

This ADR is using the placeholder number “00XX” in the filename/path. Per docs/decisions/README.md (step 1), ADRs should be named with the next sequential number (currently 0015-…). Please rename the file accordingly and update any references to the old filename.

[Copilot]

The SessionContext docstring refers to add_context_messages(), but the class defines add_messages() as the API for adding context messages. Please update the docstring to match the actual method name to avoid confusion.

[Copilot]

In the code sample, ContextMiddlewareFactory / ContextMiddlewareConfig reference ContextMiddleware before the ContextMiddleware class is defined. As written, this would raise a NameError at runtime unless you use from __future__ import annotations or quote the type / move these aliases below the class definition.

[Copilot]

The example in the ContextMiddleware docstring has incorrect indentation/structure: the “POST-PROCESSING” block appears nested under the factory function, not inside process(), which makes the example invalid and hard to follow. Please fix the snippet structure so the post-processing example is shown within process() after await next(context).

[Copilot]

The ContextMiddleware.process() docstring references context.history_messages, but SessionContext does not define a history_messages attribute (history appears to be stored under context_messages keyed by source_id). Please update the docstring to reference the correct API/field so implementers know where to read loaded history from.

[Copilot]

In the AgentSession sample, _ensure_default_storage() uses len(self._context_pipeline) and calls self._context_pipeline.prepend(...), but the provided ContextMiddlewarePipeline sample does not define __len__ or prepend. Please either add these APIs to the pipeline sample or adjust the sample logic (e.g., check self._context_pipeline is None / expose middleware length) to keep the ADR’s code consistent.

[Copilot]

In the ChatAgent sample, context_middleware is typed as Sequence[ContextMiddleware], but earlier in the ADR the configuration type is ContextMiddlewareConfig = ContextMiddleware | ContextMiddlewareFactory and ContextMiddlewarePipeline.from_config() expects configs (instances or factories). Please update the sample signature/type to Sequence[ContextMiddlewareConfig] (or similar) so it matches the proposed API.

[westey-m]

Can you modify response_messages here? If you do, do those modifications get returned to the callers, both middleware higher in the stack and the user?

[eavanvalkenburg]

I think the intent will be (and i'll clarify) that this is not the place to alter the responses, just read them and do something, the AgentMiddleware can be used for that purpose.

[westey-m]

If the user only wanted to use the user input to do a rag search, rather than that plus the chat history, how do they filter to only user input?

[eavanvalkenburg]

In the setup of each middleware there are controls over which messages they want to use, and since all messages from other context providers are stored separately that is fully configurable

[westey-m]

Is there an example of that?

[westey-m]

If the user only supplies regular context middleware and no storage context middleware, does the user get no chat history storage?

[eavanvalkenburg]

So we can go two routes here, both only for when there is no service_session_id and store==False:

1. If the pipeline is present but does not have StorageMiddleware -> Add InMemoryStorage
2. If there is NO middleware(pipeline) -> Add pipeline with InMemoryStorage

The first might be easier for getting started but the second is clearer, because if you do already have a pipeline with a middleware the order suddenly matters, so we should clarify that in that case the user should do this themselves because we do not want to make assumptions about ordering. This will make sure that the simple case always works, but once people start adding their own middlewares, they should know what they want to do and thus I think approach two is the way to do.

[westey-m]

I agree that 1 is problematic for the same reasons as you listed.

so we should clarify that in that case the user should do this themselves

At the same time, I think this ^ is a pit of failure, since just adding RAG middleware to a functional agent would make that agent break, and the user has to go and read the documentation to figure out why.

[victordibia]

Thanks for this ADR! The unified ContextMiddleware pattern is a nice improvement for context organization and attribution.

One question: would this design support implementing context compaction strategies?

For example, a common need is letting agents run for arbitrary long periods by automatically compacting/replacing the message history when a max context budget is hit. E.g., agent runs calls 10s or 100s of tool calls + model call in succession and between those calls may compact context between.

Currently this is challenging to do today because:

• ChatMessageStore.list_messages() is only called once at the start of agent.run(), not during the tool loop
• ChatMiddleware operates on a copy of messages, so modifications don't persist across tool loop iterations

see some related notes here.

With the new ContextMiddleware, would it be possible to:

Have middleware run during tool loop iterations (not just at the invocation boundary)?
Allow a compaction strategy to truly replace the context mid-execution?

If this is out of scope for this ADR, it might be worth noting as a future consideration—or confirming whether the new architecture would make such a feature easier to add later (or there is some other recommended parttern for this).

@dmytrostruk @alliscode @markwallace-microsoft

[eavanvalkenburg]

@victordibia thanks for the reply, I added a discussion section for this topic, I'm leaning for a CompactionStrategy abstraction that can be used both for and after storage and for function call loop (with no requirement to use the same one in different places as they serve different purposes), but check the discussion in the latest update here.