Downgrade SQL Server AOAG failover logs from error to warn#35868
Open
Downgrade SQL Server AOAG failover logs from error to warn#35868
Conversation
During SQL Server Always On Availability Group failovers, two conditions can occur that are expected and recoverable but were previously logged at error level, triggering unnecessary Sentry alerts: 1. Uninitialized LSN in CDC stream - when a capture instance temporarily has no LSN after failover 2. Upstream LSN going backwards - when the new primary reports a lower max LSN than previously seen Both cases are already handled gracefully (skip and continue), so downgrade from error/soft_panic_or_log to warn level. Fixes MaterializeInc/database-issues#9986 https://claude.ai/code/session_015rRQ5puqtBufPEQ5RoUtjt
Contributor
|
Thanks for opening this PR! Here are a few tips to help make the review process smooth for everyone. PR title guidelines
Pre-merge checklist
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
tracing::error!("found uninitialized LSN!")totracing::warn!insrc/sql-server-util/src/cdc.rsmz_ore::soft_panic_or_log!("upstream SQL Server went backwards in time...")withtracing::warn!insrc/storage/src/source/sql_server/progress.rsDuring SQL Server Always On Availability Group (AOAG) failovers, these two conditions can occur and are expected/recoverable, but were previously logged at error level — triggering unnecessary Sentry alerts. Both code paths already handle the conditions gracefully (skip and continue).
Fixes MaterializeInc/database-issues#9986
Test plan
cargo check -p mz-sql-server-util -p mz-storagecompiles cleanlyhttps://claude.ai/code/session_015rRQ5puqtBufPEQ5RoUtjt