Skip to main content
MANIFOLD
Will anyone suggest a better way than I currently know about for a browser to store large amounts of data on the client?
6
Ṁ90Ṁ117
resolved Nov 9
Resolved
NO

The best way to store client-side data is usually localstorage, but I believe it's limited to 5MB (or somewhere around there) on most browsers. For larger amounts, indexedDB and the filesystem API are limited only by the client's total storage capacity, but they're quite slow when retrieving large quantities of data all at once. Loading a ~30MB file via either method generally takes 3-10 seconds on my laptop, even with nothing else running that could conflict with them.

Is there a better option, or some way to optimize them to be faster?

(I've tried gzipping the data to be smaller, but the extra time taken to zip/dezip the file seems to just about cancel out any loading gains. I've also thought of splitting the data up half and half between indexedDB and the filesystem API, or splitting it up into smaller chunks within each and performing multiple calls, but have not yet tested either option. (Those won't count to resolve this to YES unless someone provides a particularly clever way to do it that I hadn't thought of.))

Market context
Get
Ṁ1,000
to start trading!

🏅 Top traders

#TraderTotal profit
1Ṁ28
2Ṁ22
Sort by:

With the caveat that I have not used wasm/webassembly at all, and so don't know anything about applicability/performance:

https://emscripten.org/docs/api_reference/Filesystem-API.html

@CamillePerrin Interesting! This may indeed be faster, but I think I'm going to have to resolve this NO, as learning a whole new language is not worth it for this use-case, and therefore this is not overall "better" than my current alternatives.