dimanche 21 juin 2015

How lightweight is SQLite3?

I'm currently planning to write a downloader in Python 3.

It keeps a queue (kinda huge, maybe more than 300000 entries) in a database (SQLite3 in this case) and lets worker threads in a thread pool fetch requests from the queue when notified.

On initialization or the cache being emptied, the downloader fetches the oldest requests (about 256) to the cache so that worker threads can immediately pull works from the cache queue.

The downloader also has a management interface which shows pending and/or completed requests so that it can allow the user to manage requests.

My question is, is SQLite3 lightweight enough or has an internal cache, so that implementing a cache is unnecessary?

Aucun commentaire:

Enregistrer un commentaire