-
Notifications
You must be signed in to change notification settings - Fork 3
Open
Labels
enhancementNew feature or requestNew feature or request
Description
The process of grabbing chunks and converting to Float16 can be a bit slow for very large or poorly chunked data.
Since both Zarr and Netcdf4 support parallel I/O, it should be possible to distribute this process into webworkers.
Right now the main thread calls copyChunk2Array() which writes the chunk values to the main data-array.
I think I can create the main data-array with a SharedArrayBuffer
If I understand this article correctly
https://dev.to/rigalpatel001/high-performance-javascript-simplified-web-workers-sharedarraybuffer-and-atomics-3ig1
I can then call copyChunk2Array() in a webworker while passing the shared buffer to it and the worker will mutate the main array.
This should dramatically speed up datafetching.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request