I am developing a web-based interface for a hardware sensor that produces approx. 250kB/s of raw data (125 kS/s, 16 bit per sample). The web application is designed to visualize (using Canvas) and store (using IndexedDB) this data in real-time. I am having performance issues with indexedDB storage.
This application is designed to run for days or even weeks and should reliably store large amounts of data (tens to the low hundreds of MB)
Because write commits seem to be the general performance issue, I have rewritten my application to only store a big chunk of data every 5 seconds as a non-sparse integer array object. This kind of works, but I still get very choppy visualization performance, high CPU and high memory usage. The exact storage code:
//dataDB = indexedDB database opened in another function
//slice = data to be stored
//sessionID = object store index
//this function is called about once every 5 seconds
//with 700 000 values in the slice array
//slice is a multidimensional array
function storeFastData(slice, sessionID){
var s = dataDB.transaction(["fastData"],"readwrite").objectStore("fastData");
var fdreq = s.get(sessionID);
fdreq.onsuccess = function(e){
var d = fdreq.result;
for(i = 0; i < slice.length; i++){
d.data[i][1] = slice[i][1];
}
s.put(d);
}
}
Concretely:
- Is IndexedDB the right choice for this application?
- Am I being an idiot in the way I implemented this? This is the first IndexedDB-based project I am doing
- I have read that using WebWorkers may at least fix the stuttering issues as it can run on another thread. Would this solve my performance problems?
I am willing to use new (draft) functionality, but requiring user interaction for storage beyond 5MB (e.g. using Quota Management API) every time the application is opened, is quite bothersome and if at all possible I would like to avoid this.
I do not use jquery. This cannot be written as a native application (it has to run inside a browser).
