mercredi 26 août 2015

Database for concurrent writes of multiple processes

I am in the process of choosing the best suited database for the following requirements:

  • Good performance for a DB size up to a few Gb (~2Gb)
  • Interface for Java und C++
  • high stability/maturity
  • will have active developement for more than 5 years
  • UTF-8 datastore

The data in the database will be HTTP requests, so structuring of the data is preferred.

Scenario: In my case, we will have many concurrent processes (e.g. 50) which want to write to the database, but only one process which will read the DB.

So far, I decided to go with sqlite3. Now I'm worried about the multiprocess writes, because of the read/write lock each time a process wants to insert, he needs to lock the entire file, which might slow down the other processes (waiting for write lock) (Source). Furthermore it is not possible to have the database in main memory, because the filesystem mechanisms are used for locking (Source).

Do you have any experience with this? Is any other DB (e.g. Berkley DB) better suited for this kind of use case?

Aucun commentaire:

Enregistrer un commentaire