Skip to content

[SPARK-55448][CONNECT] Fix query events loss when session closes during query execution#54225

Open
biruktesf-db wants to merge 1 commit intoapache:masterfrom
biruktesf-db:closed-session-events
Open

[SPARK-55448][CONNECT] Fix query events loss when session closes during query execution#54225
biruktesf-db wants to merge 1 commit intoapache:masterfrom
biruktesf-db:closed-session-events

Conversation

@biruktesf-db
Copy link
Contributor

@biruktesf-db biruktesf-db commented Feb 9, 2026

What changes were proposed in this pull request?

Allow Spark Connect to send query events after the session is closed.

Why are the changes needed?

A race condition exists between Spark Connect session closure and execution cleanup. When a session closes while an execution is still running:

  1. SessionHolder.close() interrupts active executions via removeAllExecutionsForSession(), then transitions the session status to Closed via eventManager.postClosed()
  2. The interrupted execution thread's finally block calls ExecuteHolder.cleanup(), which attempts to post terminal events (postCanceled, postClosed) through ExecuteEventsManager
  3. ExecuteEventsManager.assertStatus() requires sessionStatus == SessionStatus.Started, but the session is now Closed by the original thread handling SessionHolder.close().
  4. An IllegalStateException is thrown, silently swallowing the terminal events.
  5. Any downstream listeners never receive the Canceled/Closed events for the operation, leaving it appearing as perpetually running

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Added unit tests.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude Code v2.1.37

@hvanhovell
Copy link
Contributor

@biruktesf-db can you update the PR?
@terana can you also take a look here?

@terana
Copy link
Contributor

terana commented Feb 27, 2026

LGTM (with a small adjustment of the exception message).
I didn't find any code downstream which would be affected by relaxing this invariant check. Adding a logic in SessionEventsManager to wait for all operations closure would be an overcomplication.

sessionHolder.eventManager.status != SessionStatus.Closed) {
throw new IllegalStateException(s"""
sessionId: $sessionId with status $sessionStatus
is not Started for event $eventStatus
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: also need to update the comment?

@biruktesf-db biruktesf-db force-pushed the closed-session-events branch from 81d0d31 to 796d8f7 Compare March 18, 2026 10:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants