fix: provider_data_var context leak (backport #5227)#5250
Open
jaideepr97 wants to merge 1 commit intollamastack:release-0.6.xfrom
Open
fix: provider_data_var context leak (backport #5227)#5250jaideepr97 wants to merge 1 commit intollamastack:release-0.6.xfrom
jaideepr97 wants to merge 1 commit intollamastack:release-0.6.xfrom
Conversation
Backport of commit 9b86ce8 from main to release-0.6.x. PROVIDER_DATA_VAR — the ContextVar that carries authenticated user identity — leaks through asyncio.create_task copy semantics into long-lived background workers. When a background worker is spawned, it permanently inherits the spawning request's PROVIDER_DATA_VAR, causing all subsequent DB writes to be stamped with the wrong user's identity. This introduces a unified RequestContext in core/task.py that captures both OTel trace context and PROVIDER_DATA_VAR together. Background workers in InferenceStore and OpenAIResponsesImpl now capture context at enqueue time and re-activate it per work-item, ensuring each operation runs under the correct user identity and trace. Adapted for release-0.6.x directory structure (meta_reference paths instead of builtin). Signed-off-by: Jaideep Rao <jrao@redhat.com> Made-with: Cursor
jaideepr97
requested review from
ashwinb,
bbrowning,
cdoern,
ehhuang,
franciscojavierarceo,
leseb,
mattf and
raghotham
as code owners
meta-cla
bot
added
the
CLA Signed
Contributor
|
LGTM, can replace #5228 as well |
Collaborator
|
@jaideepr97 so this includes two PR in a single backport? if so, then this is wrong, we need 2 distincts commits (so 2 PRs) thanks |
Contributor
Author
if we need 2 separate commits so the git history matches what is in main then @iamemilio please fix #5228 |
Contributor
|
I'll try to get to this later this week or next week in that case. My hands are pretty full right now and this is just a backport, so hopefully thats ok |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This backport was generated and pushed by claude
Backport of #5227 to
release-0.6.x.PROVIDER_DATA_VARcontextvar leak through long-lived background worker tasks (asyncio.create_taskcopies all contextvars at creation time, so workers permanently inherit the spawning request's auth identity)RequestContextincore/task.pythat captures both OTel trace context andPROVIDER_DATA_VARtogether, with helpers to capture at enqueue time and re-activate per work-itemInferenceStoreandOpenAIResponsesImplbackground workers to use the new context propagation patternAdapted for
release-0.6.xdirectory structure (meta_referencepaths instead ofbuiltin).Closes #5221
Test plan
tests/unit/core/test_task.py(10 tests): VerifiesRequestContextcapture/activate semantics, detached task isolation for both OTel andPROVIDER_DATA_VAR, caller context restoration, queue-based propagation patterns, and cross-contamination prevention.tests/unit/utils/inference/test_provider_data_leak.py(1 test): Reproduces theInferenceStorewrite queue leak end-to-end — two users store completions through the async queue, then verifies each user can only see their own completions viaAuthorizedSqlStoreaccess policies.tests/unit/providers/agents/meta_reference/test_responses_background.py(6 new tests):TestResponsesOtelContextPropagation(3 tests): Verifies OTel trace attribution through the responses background worker.TestResponsesProviderDataPropagation(3 tests): Verifies user identity propagation through the responses background worker.All 447 unit tests in affected directories pass. All pre-commit hooks pass.
Made with Cursor