samedi 9 janvier 2016

Acclerate SQLite

I'm currently learning SQLite (called by Python).

According to my previous question (Reorganising Data in SQLLIte) I want to store multible time series (Training data) in my database. I have defined the following fields:

CREATE TABLE VARLIST (VarID INTEGER PRIMARY KEY AUTOINCREMENT, name TEXT UNIQUE NOT NULL)    
CREATE TABLE DATAPOINTS (DataID INTEGER PRIMARY KEY, timeID INTEGER, VarID INTEGER, value REAL)
CREATE TABLE TIMESTAMPS (timeID INTEGER PRIMARY KEY AUTOINCREMENT, TRAININGS_ID INT, TRAINING_TIME_SECONDS FLOAT)

VARLIST has 8 entries, TIMESTAMPS 1e5 entries and DATAPOINTS around 5e6.

When I now want to extract data for a given TrainingsID and VarID I try it like:

SELECT             
(SELECT TIMESTAMPS.TRAINING_TIME_SECONDS FROM TIMESTAMPS WHERE t.timeID = timeID) AS TRAINING_TIME_SECONDS,
(SELECT value FROM DATAPOINTS WHERE DATAPOINTS.timeID = t.timeID and DATAPOINTS.VarID=2) as value
FROM (SELECT timeID   FROM TIMESTAMPS WHERE TRAININGS_ID=96) as t;

This basically works.

But there are two Problems:

(1) Minor problem: If there is a timeID where no data for the requested VarID' is availabe I get an line with the valueNone`. I would prefer this line to be skipped.

(2) Big Problem: The search is increadibly slow (approx 5 minutes using http://ift.tt/1onWUjW).

How do I best improve the performance? Are there better ways to formulate the SELECT command, or should I modify the database structure itself?

Aucun commentaire:

Enregistrer un commentaire