mardi 30 juin 2015

Python code running too slow (SQLITE)

I have located a piece of code that runs quite slow (in my opinion) and would liek to know what you guys think. The code in question is as follows and is supposed to:

  • Query a database and get 2 fields, a field and its value
  • Populate the object dictionary with their values

The code is:

query = "SELECT Field, Value FROM metrics " \
        "WHERE Status NOT LIKE '%ERROR%' AND Symbol LIKE '{0}'".format(self.symbol)
query = self.db.run(query, True)
if query is not None:
    for each in query:
        self.metrics[each[0].lower()] = each[1]

The query is run using a db class I created that is very simple:

def run(self, query, onerrorkeeprunning=False):
    # Run query provided and return result
    try:
        con = lite.connect(self.db)

    cur = con.cursor()
    cur.execute(query)
    con.commit()

    runsql = cur.fetchall()
    data = []
    for rows in runsql:
        line = []
        for element in rows:
            line.append(element)
        data.append(line)
    return data

except lite.Error, e:
    if onerrorkeeprunning is True:
            if con:
                con.close()
            return
    else:
        print 'Error %s:' % e.args[0]
        sys.exit(1)

finally:

    if con:
        con.close()

I know there are tons of ways of writting this code and I was trying to keep things simple but for 24 fields this takes 0.03s so if I have 1,000 elements that is 30s and I find it a little too long!

Any help will be much appreciated.

Aucun commentaire:

Enregistrer un commentaire