storage: add splitfdstream for efficient layer transfer with reflink support by giuseppe · Pull Request #651 · containers/container-libs

giuseppe · 2026-02-10T12:32:04Z

Add a new splitfdstream package that enables efficient container layer transfer between storage instances by separating tar metadata from file content. File content is passed as file descriptors.

mtrmac

6% of the code base for an extremely niche use case :/

I’m afraid I can’t spend time on this now; and anyway some other work on pkg/archive needs to happen first, and expect the rebase to be somewhat non-trivial.

giuseppe · 2026-02-10T15:55:34Z

6% of the code base for an extremely niche use case :/

I’m afraid I can’t spend time on this now; and anyway some other work on pkg/archive needs to happen first, and expect the rebase to be somewhat non-trivial.

what other work would block this feature?

I don't love either the amount of extra code, but it is almost all in new APIs/packages that are experimental and not leaking too much to the rest of the library.

We went around this problem for a long time, and in the end this seems like an ~ok solution to not expose too much of the containers storage internals augmenting the tar format that is already plugged everywhere.

giuseppe · 2026-02-10T17:43:19Z

to add more context: the goal of this feature is to be able to copy a layer, similarly to what we would do with Diff, but having the possibility to deduplicate files using reflinks. We need two separate copies because OS files can't be kept on the containers storage for different reasons, a podman system reset -f should not break the system, and OS files need to be relabeled.

There are other interesting use cases enabled with a RPC like facilitate usage from different languages (Rust in our case), inject into a build container to grab R/O access to layers/images, but this is out of scope now.

I realize complexity is not trivial just to get access to the raw files, but what alternatives do we have?

From the store API PoV we could just extend it to return a map $LAYER_FILE -> $PATH_TO_THE_FILE, but how would we use it from Rust?

Any suggestions?

mtrmac · 2026-02-10T18:24:55Z

(Just to avoid a misunderstanding, I didn’t really read the PR. I have, at this point, ~no opinion on RPC vs. Go API.)

cgwalters · 2026-02-10T19:18:18Z

storage/pkg/archive/archive.go

+	return &tarReaderIterator{
+		tr:     tr,
+		trBuf:  trBuf,
+		buffer: make([]byte, 1<<20),


Seems like 1<<20 should be in a const somewhere with a rationale?

cgwalters · 2026-02-10T19:19:02Z

storage/pkg/chrootarchive/splitfdstream_linux.go

+
+// splitFDStreamSocketDescriptor is the fd for the Unix socket used to
+// receive file descriptors via SCM_RIGHTS in the re-exec child.
+const splitFDStreamSocketDescriptor = 5


Hardcoding this seems a bit fragile, can't we pass it as an env var or a CLI arg?

cgwalters · 2026-02-10T19:20:11Z

storage/pkg/chrootarchive/splitfdstream_linux.go

+// FDs are streamed to the child process one-at-a-time over a Unix socket
+// using SCM_RIGHTS, avoiding EMFILE from inheriting too many FDs at exec.


It doesn't have to be truly one at a time, see also https://github.com/cgwalters/jsonrpc-fdpass?tab=readme-ov-file#41-fd-batching

In my initial PoC work here I handled this by just sending the splitfdstream inline data itself as a fd (could be O_TMPFILE or memfd or pipe), and then sending all the associated fds over jsonrpc-fdpass which automatically handles buffering.

Ensure we processing just 1 recvmsg() at a time (only ~200 fds on Linux) seems really reasonable and unlikely to get anywhere close to modern fd limits.

So if we do choose jsonrpc-fdpass, I think we also need to standardize this "splitfdstream serialization".

cgwalters · 2026-02-10T19:30:29Z

storage/pkg/fileutils/reflink_linux.go

+		return err
+	}
+
+	// Try copy_file_range - kernel-level copy, more efficient than userspace


Surprising we weren't doing this before.

cgwalters · 2026-02-10T19:33:57Z

storage/pkg/splitfdstream/fdpass.go

+
+// ReadLine reads bytes until a newline is encountered, using recvmsg.
+// Any FDs received are closed since line-reading doesn't expect them.
+func (p *FDPasser) ReadLine() ([]byte, error) {


Why would one want this?

cgwalters · 2026-02-10T19:35:02Z

storage/pkg/splitfdstream/protocol.go

+
+// JSON-RPC 2.0 protocol constants
+const (
+	JSONRPCVersion = "2.0"


BTW see also https://github.com/cgwalters/jsonrpc-fdpass-go

giuseppe · 2026-02-11T16:13:53Z

still PoC, so you can avoid a review as it is not yet ready but we can use to analyze the complexity.

Moved to use https://github.com/cgwalters/jsonrpc-fdpass-go

Now the code size is almost 50% than the last revision

packit-as-a-service · 2026-02-11T16:46:06Z

Packit jobs failed. @containers/packit-build please check.

TomSweeneyRedHat · 2026-03-18T21:51:02Z

@giuseppe looks like a rebase may be needed here.

giuseppe · 2026-03-23T07:57:10Z

@kolyshkin are these changes to the tar package a problem for the work you are doing there?

giuseppe · 2026-03-27T10:47:39Z

@kolyshkin can you PTAL?

cgwalters · 2026-04-06T15:54:48Z

storage/drivers/overlay/overlay_splitfdstream.go

+	}
+	defer tarStream.Close()
+
+	// Write splitfdstream data directly to a memfd to avoid buffering in memory.


This type of comment I find a bit distracting because it's almost circular - a memfd is memory too...how about just:

A memfd is a convenient way to buffer the metadata

Or maybe just no comment at all. Dunno.

Here and in other places like

// Set up ID mappings idMappings := options.IDMappings

The comment basically adds nothing.

cgwalters · 2026-04-06T15:57:05Z

storage/drivers/overlay/overlay_splitfdstream.go

+	return streamFile, fds, nil
+}
+
+// convertTarToSplitFDStream converts a tar stream to a splitfdstream by parsing


This obviously works but it seems pretty inefficient to do - i.e. to read the entire contents of large file into memory and just discard it.

I guess we're doing this because the current API is "give a tarball" and this just builds on that without changing anything more fundamental?

cgwalters · 2026-04-06T15:57:29Z

storage/drivers/overlay/overlay_splitfdstream.go

+				if buf == nil {
+					buf = make([]byte, archive.CopyBufferSize)
+				}
+				// File not found in diff directory (e.g., naiveDiff was used),


When would the file not be found?

cgwalters · 2026-04-06T15:58:47Z

storage/drivers/overlay/overlay_splitfdstream.go

+				// Write a single prefix for the total size, then stream
+				// data in chunks.  The reader expects exactly one prefix
+				// per file entry.
+				if err := writer.WriteInlinePrefix(header.Size); err != nil {


Would perhaps be more elegant if writer.WriteInline returned a sub-io.Write impl then we could just do io.Copy to that?

cgwalters · 2026-04-06T16:06:28Z

common/pkg/json-proxy/handler.go

 		rb, err = h.GetLayerInfoPiped(ctx, req.Args)
 	case "FinishPipe":
 		rb, err = h.FinishPipe(ctx, req.Args)
+	case "GetSplitFDStreamSocket":


It seems like this may grow into a general replacement for the bespoke proxy API right? How about calling it

OpenJsonRPCFdPass ?

cgwalters · 2026-04-06T16:09:30Z

storage/pkg/splitfdstream/server_linux.go

+	"runtime"
+	"sync"
+
+	fdpass "github.com/cgwalters/jsonrpc-fdpass-go"


This can use https://github.com/bootc-dev/jsonrpc-fdpass-go now

bootc-dev/jsonrpc-fdpass-go#7

cgwalters · 2026-04-06T16:11:08Z

storage/pkg/splitfdstream/server_linux.go

+	// The library limits to MaxFDsPerMessage per sendmsg, so remaining
+	// FDs are sent as follow-up "fds" notifications.


Hmm the jsonrpc-fdpass protocol specifies to send more fds as whitespace but it's possible the Go implementation isn't doing this right

bootc-dev/jsonrpc-fdpass-go#4

cgwalters · 2026-04-06T16:11:32Z

storage/pkg/splitfdstream/types.go

+// findManifest finds the image manifest from BigData keys.
+// It looks for manifest-* keys containing an image manifest (has "config" field),
+// filtering out manifest lists/indexes.
+func findManifest(store Store, imageID string) ([]byte, error) {


Surely there's a common API for this?

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>

cgwalters · 2026-04-08T13:59:26Z

I think this is overall OK as is, however - just to cross-reference from composefs/composefs-rs#228 (comment) - I think we should get a bit more ambitious and try adding a full new proxy API based on jsonrpc-fdpass instead. There's a lot of warts and bugs we could fix at the same time (like the bad idea of auto-converting v2s2 to OCI and other things from the historical PRs there).

That would then make the architecture much clearer: There's a jsonrpc-fdpass API for containers-storage:. We rework https://github.com/bootc-dev/containers-image-proxy-rs/ to support this (it'd be a relatively thin wrapper on top of jsonrpc-fdpass, just with stronger Rust types basically) and then it becomes very straightforward to plug in all of the existing places.

Just to also xref again I think this API has the potential to solve problems like containers/skopeo#658 (comment) too. But of course, that's where the choice of IPC becomes a big deal.

giuseppe · 2026-04-08T14:17:17Z

I think this is overall OK as is, however - just to cross-reference from composefs/composefs-rs#228 (comment) - I think we should get a bit more ambitious and try adding a full new proxy API based on jsonrpc-fdpass instead. There's a lot of warts and bugs we could fix at the same time (like the bad idea of auto-converting v2s2 to OCI and other things from the historical PRs there).

isn't that something we can do incrementally? I've tried to keep this as small as possible (e.g. dropped write support) to facilitate merging it, but I think we should get that back and let the copy code handle this as a shortcut for more efficient copies to different stores

cgwalters

Assisted-by: OpenCode (Claude Opus 4)

Comments prefixed with "AI:" are unedited AI output.

Comments on lines outside the diff:

storage/store.go:3829
AI: Important: Shutdown() cleans up the graph driver (line 3883: s.graphDriver.Cleanup()) but never calls s.jsonRPCServer.Stop(). Any goroutines spawned by HandleConnection may still be running and holding references to the now-cleaned-up driver, which could cause panics or undefined behavior.

Shutdown should stop the JSON-RPC server and wait for connections to drain before cleaning up the driver.

cgwalters · 2026-04-08T19:47:28Z

storage/pkg/splitfdstream/server_linux.go

+	}
+
+	resp := fdpass.NewResponse(result, req.ID)
+	if err := sendRetry(sender, &fdpass.MessageWithFds{


AI: Important: After sendRetry succeeds, the server-side copies of streamFile and all fileFDs in allFDs are never closed. The sendmsg syscall duplicates file descriptors into the kernel's SCM_RIGHTS, so the sender's copies remain open. The Sender.Send() method in the transport library does not close them either (it only calls f.Fd() for the raw int).

This leaks 1 + N file descriptors per successful GetSplitFDStream call for the lifetime of the connection. On the error path (line 280-281) they're also not closed.

Suggested fix: add cleanup after send:

for _, f := range allFDs { f.Close() }

Or, perhaps we take ownership of them in the jsonrpc-fdpass-go side?

cgwalters · 2026-04-08T19:47:28Z

storage/pkg/splitfdstream/server_linux.go

+	}
+}
+
+func (s *JSONRPCServer) handleRequest(sender *fdpass.Sender, req *fdpass.Request, fds []*os.File) {


AI: Important: The fds parameter (file descriptors received from the client message) is accepted but never used and never closed in any code path. If a client sends file descriptors with any request, they will be leaked. Even if current methods don't expect incoming FDs, the server should close them defensively:

defer func() { for _, f := range fds { f.Close() } }()

cgwalters · 2026-04-08T19:47:28Z

storage/pkg/splitfdstream/server_linux.go

+)
+
+// sendRetry retries sender.Send on EAGAIN (non-blocking socket buffer full).
+func sendRetry(sender *fdpass.Sender, msg *fdpass.MessageWithFds) error {


AI: Important: This is an unbounded busy-loop with only runtime.Gosched() as backoff. If the socket buffer stays full (e.g., the peer is slow or blocked), this will spin a CPU core indefinitely with no backpressure, no maximum retry count, and no context/deadline support.

At minimum this should use exponential backoff (e.g., time.Sleep starting at 1ms, doubling up to some cap), or ideally poll/select on the fd for writability. A context parameter would also allow callers to set deadlines.

Wait sorry and perhaps I'm missing something but why do we need this API at all? I think we should just be able to use the jsonrpc-fdpass-go library at a high level without doing any socket stuff directly, no?

cgwalters · 2026-04-08T19:47:28Z

storage/pkg/splitfdstream/server_linux.go

+		return nil, nil, fmt.Errorf("failed to create client connection: %w", err)
+	}
+
+	serverConn, err := net.FileConn(serverFile)


AI: FD leak on error: if net.FileConn(serverFile) fails at this line, clientFile is not closed — only clientConn (which wraps a dup'd copy) and serverFile are cleaned up. The original fd held by clientFile leaks.

Fix: defer clientFile.Close() and defer serverFile.Close() immediately after creating them at lines 497-498. The redundant Close() calls at 514-515 are then just harmless no-ops.

cgwalters · 2026-04-08T19:47:28Z

storage/store.go

+	// Add to the WaitGroup before spawning the goroutine to avoid
+	// a race with Stop() -> connections.Wait().
+	s.jsonRPCServer.TrackConnection()
+	go s.jsonRPCServer.HandleConnection(serverConn)


AI: Important: SplitFDStreamSocket() acquires the graph lock via startUsingGraphDriver() but releases it at function return via defer s.stopUsingGraphDriver(). The spawned goroutine running HandleConnection will call s.driver.GetSplitFDStream() on the stored driver reference after the lock has been released.

This means the handler goroutine accesses the graph driver without the graph lock held. If another goroutine calls startUsingGraphDriver() which detects a modification and creates a new driver instance, the handler may be using a stale/replaced driver. The handler goroutine needs its own locking strategy (e.g., calling startUsingGraphDriver/stopUsingGraphDriver around each request).

cgwalters · 2026-04-08T19:47:28Z

storage/drivers/overlay/splitfdstream_test.go

+	"go.podman.io/storage/pkg/splitfdstream"
+)
+
+func TestGetSplitFDStreamStub(t *testing.T) {


AI: The test coverage here is very thin — only two trivial error paths (nil options, non-existent layer) and an interface assertion. There are no tests for the happy path, composefs xattr resolution, inline fallback, path traversal rejection, FD ordering, empty layers, or zero-length regular files.

Given the complexity and security sensitivity of this code (path traversal checks, FD lifecycle, tar reconstruction), more substantial test coverage would significantly increase confidence. An integration test with a real layer diff would be especially valuable.

cgwalters · 2026-04-08T19:47:28Z

storage/pkg/splitfdstream/types.go

+// InlineWriter writes a negative prefix for size bytes and returns an
+// io.Writer that passes data straight through to the underlying stream.
+// This lets callers use io.Copy instead of manual chunked loops.
+func (w *SplitFDStreamWriter) InlineWriter(size int64) (io.Writer, error) {


AI: This writes a prefix declaring exactly size bytes will follow, then returns the raw underlying io.Writer with no enforcement. If a caller writes more or fewer bytes than size, the stream is silently corrupted — the reader will misinterpret subsequent data.

Current callers do use LimitReader, so this works in practice, but the API is a footgun for future callers. Consider either wrapping the returned writer in a LimitedWriter that enforces the contract, or at least adding a prominent doc comment warning.

cgwalters · 2026-04-08T19:47:28Z

storage/pkg/splitfdstream/types.go

+	// Walk the layer chain using store.LayerParent
+	var layerIDs []string
+	layerID := topLayerID
+	for layerID != "" {


AI: The layer chain walk silently swallows errors from store.LayerParent (line 92 breaks on error). If the store returns a transient error mid-chain, the caller gets a truncated layer list with no error indication. This could lead to silent data loss during transfer — the receiving side would only get a subset of layers.

Also, this produces a leaf-first (reverse) ordering, while the manifest fallback (line 100-103) produces root-first. The consumer will see inconsistent ordering depending on which code path was taken.

cgwalters · 2026-04-08T19:47:28Z

storage/drivers/overlay/overlay_splitfdstream.go

+	}
+
+	// Verify it's still a regular file
+	var fdStat unix.Stat_t


AI: After opening the file and verifying it is regular, the code does not verify that the file size matches header.Size. If the file on disk has been modified since the diff was computed (TOCTOU race or partial write), the consumer would get an FD reference but the actual file size wouldn't match what the tar header says, potentially causing data corruption on the receiving end.

Adding a size check (if fdStat.Size != header.Size) and falling back to inline would make this more robust.

cgwalters · 2026-04-08T19:47:28Z

storage/pkg/archive/archive.go

+// TarEntryIterator abstracts iteration over tar entries.
+// Standard implementation wraps tar.Reader; splitfdstream provides
+// entries from its chunk-based format with reflink support.
+type TarEntryIterator interface {


AI: (low) The doc comment on WriteContentTo (line 1102) says "Only called for TypeReg entries with Size > 0", but looking at the actual call site in UnpackFromIterator (line 1261-1263), writeContent is constructed unconditionally for every entry type, and inside extractTarFileEntry the guard is writeContent != nil (not "TypeReg with Size > 0"). So WriteContentTo will be called for zero-size regular files too. The doc comment should be corrected, or the code should add the Size > 0 guard.

cgwalters · 2026-04-08T20:11:28Z

isn't that something we can do incrementally? I've tried to keep this as small as possible (e.g. dropped write support) to facilitate merging it, but I think we should get that back and let the copy code handle this as a shortcut for more efficient copies to different stores

Yes, there's obviously virtues to being incremental. Though a different but related problem is that as is it's harder to get real test coverage as we have to create a different synthetic client using this API. But if it was plumbed through the proxy more directly, it'd be way easier to test e2e.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>

Extend the store with splitfdstream capabilities exposed via a UNIX socket for JSON-RPC communication. Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>

Implement the SplitFDStreamDriver interface for the overlay driver, enabling efficient layer operations with reflink support. Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>

Add a new json-proxy method that returns a Unix socket file descriptor to the client. The client then speaks the jsonrpc-fdpass protocol directly over that socket for splitfdstream operations, bypassing the json-proxy for bulk data transfer. Bump protocolVersion to "0.2.9". Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>

github-actions bot added the storage Related to "storage" package label Feb 10, 2026

This was referenced Feb 10, 2026

[WIP] composefs-splitfdstream: new crate composefs/composefs-rs#228

Draft

reflink by default (if possible) when copying between containers-storage: instances #144

Open

giuseppe force-pushed the splitfdstream branch 2 times, most recently from 4cca2a3 to 3b6f591 Compare February 10, 2026 14:59

mtrmac reviewed Feb 10, 2026

View reviewed changes

giuseppe force-pushed the splitfdstream branch from 3b6f591 to 26cd4b6 Compare February 10, 2026 16:01

cgwalters reviewed Feb 10, 2026

View reviewed changes

giuseppe force-pushed the splitfdstream branch 2 times, most recently from 23ed544 to 8d74f84 Compare February 11, 2026 16:07

giuseppe force-pushed the splitfdstream branch from 8d74f84 to 51a6c01 Compare February 11, 2026 16:25

giuseppe force-pushed the splitfdstream branch 2 times, most recently from 1b087bb to bc6c5ff Compare February 12, 2026 09:15

giuseppe mentioned this pull request Feb 12, 2026

Add an undocumented general "run with RPC service" containers/buildah#6675

Merged

giuseppe force-pushed the splitfdstream branch 7 times, most recently from a0876b6 to d77fbc4 Compare February 17, 2026 12:44

giuseppe force-pushed the splitfdstream branch from d77fbc4 to 6d605ac Compare February 19, 2026 10:12

giuseppe changed the title ~~[WIP] storage: add splitfdstream for efficient layer transfer with reflink support~~ storage: add splitfdstream for efficient layer transfer with reflink support Feb 19, 2026

giuseppe force-pushed the splitfdstream branch from 6d605ac to 0ebb56c Compare February 19, 2026 14:49

giuseppe marked this pull request as ready for review February 19, 2026 14:58

giuseppe force-pushed the splitfdstream branch from 42fcc4e to d9f9694 Compare March 20, 2026 17:57

github-actions bot added common Related to "common" package image Related to "image" package labels Mar 20, 2026

giuseppe force-pushed the splitfdstream branch 2 times, most recently from 88bf912 to a7806a8 Compare March 23, 2026 07:39

cgwalters reviewed Apr 6, 2026

View reviewed changes

giuseppe added 2 commits April 7, 2026 08:53

pkg/fileutils: use CopyFileRange if possible

bf9df7d

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>

storage, chunked: new function GenerateDumpFromTarHeaders

d5428aa

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>

giuseppe force-pushed the splitfdstream branch from a7806a8 to b304a2e Compare April 7, 2026 11:05

storage/vendor: add github.com/bootc-dev/jsonrpc-fdpass-go

8141b89

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>

giuseppe force-pushed the splitfdstream branch 3 times, most recently from 7af6099 to daa8688 Compare April 7, 2026 13:33

cgwalters reviewed Apr 8, 2026

View reviewed changes

giuseppe and others added 9 commits April 9, 2026 12:17

storage/pkg/splitfdstream: new package

d50db34

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>

storage/store: add new APIs to resolve image ID

da928ef

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>

storage: add SplitFDStreamStore interface

873cd11

Extend the store with splitfdstream capabilities exposed via a UNIX socket for JSON-RPC communication. Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>

storage/archive,chrootarchive: add support for splitfdstream

a1b3bf8

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>

storage, overlay: use openat2 instead of using procfs

02cedfc

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>

composefs: do not leak open file

0abddc9

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>

storage/overlay: implement SplitFDStreamDriver

4177122

Implement the SplitFDStreamDriver interface for the overlay driver, enabling efficient layer operations with reflink support. Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>

image, storage: expose the underlying SplitFDStreamStore

68e58d6

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>

giuseppe force-pushed the splitfdstream branch from daa8688 to 604910e Compare April 9, 2026 10:17

		// FDs are streamed to the child process one-at-a-time over a Unix socket
		// using SCM_RIGHTS, avoiding EMFILE from inheriting too many FDs at exec.

		// The library limits to MaxFDsPerMessage per sendmsg, so remaining
		// FDs are sent as follow-up "fds" notifications.

Conversation

giuseppe commented Feb 10, 2026

Uh oh!

mtrmac left a comment

Choose a reason for hiding this comment

Uh oh!

giuseppe commented Feb 10, 2026

Uh oh!

giuseppe commented Feb 10, 2026

Uh oh!

mtrmac commented Feb 10, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

giuseppe commented Feb 11, 2026

Uh oh!

packit-as-a-service bot commented Feb 11, 2026

Uh oh!

TomSweeneyRedHat commented Mar 18, 2026

Uh oh!

giuseppe commented Mar 23, 2026

Uh oh!

giuseppe commented Mar 27, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cgwalters commented Apr 8, 2026

Uh oh!

giuseppe commented Apr 8, 2026

Uh oh!

cgwalters left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment