Skip to content

Session Scalability tests for console#35529

Open
leedqin wants to merge 2 commits intoMaterializeInc:mainfrom
leedqin:console-playwright-tests
Open

Session Scalability tests for console#35529
leedqin wants to merge 2 commits intoMaterializeInc:mainfrom
leedqin:console-playwright-tests

Conversation

@leedqin
Copy link
Copy Markdown
Contributor

@leedqin leedqin commented Mar 17, 2026

Motivation

This PR creates a test suite to understand how different console pages behave in a loaded environment. We can leverage materialize's testing tools like workload capture to create a loaded workload replay environment.

What these tests measure

  • Query count and latency per page — which queries fire, how often they poll, and their server/queue time breakdown
  • Time to content — how long from navigation until meaningful content is visible
  • Multi-tab degradation — how latency and queue times increase with 6 concurrent tabs

What these tests don't measure

  • Scale with object count — all tests run against one fixed workload (prod_analytics). They don't answer "what happens with 5000 objects?"
  • Server breaking point — tab stress uses 6 browser tabs, not 100+ concurrent users. Finding the actual server limit requires load testing tools (k6)
  • Real user experience — these are synthetic lab tests in a controlled environment, not production monitoring

I am keeping track of findings in the markdown file running against production analytics workload capture and will keep updating that as we make improvements to tests.

Future scale testing directions

Object scale | How does page load change with 10 vs 1000 objects?

  • Workload replay with --extra-objects multiplier. I will work on expanding this workload capture to support these tests and update findings
  • User scale: At what concurrency does environmentd degrade? Planning to use grafana k6 replaying each page's SQL queries. Will have that in a followup PR

@github-actions
Copy link
Copy Markdown
Contributor

Thanks for opening this PR! Here are a few tips to help make the review process smooth for everyone.

PR title guidelines

  • Use imperative mood: "Fix X" not "Fixed X" or "Fixes X"
  • Be specific: "Fix panic in catalog sync when controller restarts" not "Fix bug" or "Update catalog code"
  • Prefix with area if helpful: compute: , storage: , adapter: , sql:

Pre-merge checklist

  • The PR title is descriptive and will make sense in the git log.
  • This PR has adequate test coverage / QA involvement has been duly considered. (trigger-ci for additional test/nightly runs)
  • If this PR includes major user-facing behavior changes, I have pinged the relevant PM to schedule a changelog post.
  • This PR has an associated up-to-date design doc, is a design doc (template), or is sufficiently small to not require a design.
  • If this PR evolves an existing $T ⇔ Proto$T mapping (possibly in a backwards-incompatible way), then it is tagged with a T-proto label.
  • If this PR will require changes to cloud orchestration or tests, there is a companion cloud PR to account for those changes that is tagged with the release-blocker label (example).

@jubrad jubrad added A-CONSOLE Area: Console labels Apr 3, 2026
These are some playwright tests that help determine the session scalability of different console pages
- query-audit.spec.ts: per-page query baselines (cluster detail, cluster
  list, sources list, sinks list, object explorer)
- page-load.spec.ts: cold load timing per page and cluster list-to-detail
  navigation flow
- tab-stress.spec.ts: progressive 6-tab stress and mixed page multi-tab
- helpers.ts: shared query observation, SQL labeling, timing reports,
  concurrency analysis, and auth helpers

Added SQL query label dictionary mapping each console query to a
human-readable name, grouped by page and verified against compiled SQL
snapshots. Added baseline results from prod_analytics workload replay
and updated guide-testing.md with per-page run commands.
@leedqin leedqin changed the title session scalability tests for cluster details Session Scalability tests for console Apr 4, 2026
@leedqin leedqin marked this pull request as ready for review April 4, 2026 00:16
@leedqin leedqin requested a review from a team as a code owner April 4, 2026 00:16
@leedqin leedqin requested review from SangJunBak and removed request for a team April 4, 2026 00:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-CONSOLE Area: Console

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants