mardi 2 février 2016

Convert python dictionary of lists to SQLite database

I have a python dictionary that is currently pickled using cPickle, but is too large (~8Gb on disk) to load in at once. Since I don't necessarily need the whole dictionary at once, I'd like to use SQLite or another database system to store and access this data.

The current dictionary format is {id: (list)}, where the id is always an integer and the list is always 165 floats. The order of the floats is important, so each of them should be stored in a column.

Only I will be accessing this data, and only using python. How can I most easily write this into a database and then retrieve the list from the id?

Aucun commentaire:

Enregistrer un commentaire