Consider a database scheme like this:
CREATE TABLE log (
observation_point_id INTEGER PRIMARY KEY NOT NULL,
datetime TEXT NOT NULL,
value REAL NOT NULL
)
which contains 'observations' of some value; say for example a temperature measurement. The observation device (i.e., thermometer :) ) samples the temperature every 5 seconds and this gets logged to the database.
There are multiple thermometers, each of which is identified (for the purposes of this simplified example) by an 'observation_point'.
Now, let's assume that the precision of my thermometer is one degree; then I will have many observations that are redundant. Let's say I log x degrees at 9h00m00s, then it's quite likely it will still be x degrees at 9h00m05s, 9h00m10s etc. So I only need to store the value and time at which I first measured this temperature, and at which I last measured it.
I can check on every insert if the value immediately preceding it is redundant, and then delete that. But that's quite expensive, especially considering that there are many loggers to write to my database, and the frequency of logging is higher than 5 seconds in my real use case.
So my idea is to run a 'cleanup' every, say 1 minute, that will delete all values between extremes e1 and e2 where the interval [e1,e2] is defined as each series of subsequent values v1, v2, ..., vn where v1 = v2 = ... = vn. 'Subsequent' here meaning when ordered by 'datetime'.
My question: is there a way to express this in an SQL query? Is there another way to approach this?
(my baseline is to do a 'select order by', then loop over all results). I can't do anything 'before' my values hit the database (i.e., cache values until I get the next measurement and only write value if that measurement is different), because I might also get observations at a much lower frequency than once every few seconds, and I cannot afford to lose observations. (now that I'm typing this, maybe I could 'cache' values in a separate database table, but I think I'm straying too far from my real question now).
Aucun commentaire:
Enregistrer un commentaire