Skip to content

feat: add graceful shutdown for k8s deployments#365

Merged
Taure merged 5 commits intomasterfrom
feat/graceful-shutdown
Mar 19, 2026
Merged

feat: add graceful shutdown for k8s deployments#365
Taure merged 5 commits intomasterfrom
feat/graceful-shutdown

Conversation

@Taure
Copy link
Collaborator

@Taure Taure commented Mar 10, 2026

Summary

  • Add nova:graceful_shutdown/0 public API that handles the full shutdown sequence: delay → suspend listener → drain connections → stop listener
  • Called automatically via prep_stop/1 during application shutdown (SIGTERM)
  • Configurable via sys.config:
    • shutdown_delay — ms to wait before suspending (lets k8s remove pod from endpoints, default 0)
    • shutdown_drain_timeout — max ms to wait for in-flight requests to complete (default 15000)
  • Logs each shutdown phase with structured metadata

Test plan

  • Compiles cleanly
  • Release builds successfully
  • Heartbeat endpoint responds 200
  • Graceful shutdown logs all phases (delay → suspend → drain → stop)
  • Container stops cleanly on SIGTERM

🤖 Generated with Claude Code

Taure and others added 2 commits March 10, 2026 22:31
Add nova:graceful_shutdown/0 that suspends the ranch listener, drains
active connections with a configurable timeout, then stops the listener.
Called automatically via prep_stop/1 during application shutdown.

Configurable via sys.config:
- shutdown_delay: ms to wait before suspending (default 0)
- shutdown_drain_timeout: max ms to wait for connections to drain (default 15000)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
ranch:info/1 returns a map in ranch 2.x, not a proplist.
Match on map keys directly to satisfy dialyzer.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Copy link

@kivra-hennys kivra-hennys left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.

Copy link
Contributor

@burbas burbas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should not this behavior be implemented as a callback? Then it will automatically be invoked when calling application:stop(nova) or when the beam machine is shutting down.

@Taure
Copy link
Collaborator Author

Taure commented Mar 18, 2026

I can look into it

@Taure
Copy link
Collaborator Author

Taure commented Mar 19, 2026

But looking at it prep_stop is a callback that is called if application is closed. https://www.erlang.org/doc/apps/kernel/application.html#c:prep_stop/1 so during prep_stop we will do some things before the stop?

@burbas
Copy link
Contributor

burbas commented Mar 19, 2026

But looking at it prep_stop is a callback that is called if application is closed. https://www.erlang.org/doc/apps/kernel/application.html#c:prep_stop/1 so during prep_stop we will do some things before the stop?

You are right. This needs to go into the prep_stop/1 callback since that will be invoked when the application is about to be stopped. stop/1 is called once the application actually is closed, so that won't work in this case.

@burbas
Copy link
Contributor

burbas commented Mar 19, 2026

So my suggestion is that we move the graceful_shutdown into the application behavior. I don't see any value in having it exported in nova-module

Only called from prep_stop/1, no reason to export from nova.erl.
@Taure Taure force-pushed the feat/graceful-shutdown branch from 2d5f481 to b4ea5ee Compare March 19, 2026 08:13
@Taure Taure merged commit 7e8b89e into master Mar 19, 2026
14 checks passed
@Taure Taure deleted the feat/graceful-shutdown branch March 19, 2026 15:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants