lundi 23 mars 2015

SQLite alternate SELECT followed by insert for every record in a set of 1 miilion records

I am making an application in which data is transferred to a database which consist of 2 tables, and for the main table i need to check whether the record i am trying to insert already exists or not. If the record is found , its Id is used for storing along with the record to store in the second table If the record is not found in the first table, a new entry is made and the id for the new record is stored along with data in second table


I am having issue while executing select before insert everytime to get the existing ID if the record exists.



  1. If i Index the column to be searched , it decreases the speed of my insertion exponentially. after a hundred thousand records.

  2. If i do not index the column to be searched it still takes a hell lot of time.

  3. Incase for direct insertion without select , the time taken to insert all records were 10 secs - 12 secs.

  4. So i need help with how can i insert the speed of my task


So far i have refered these links , but this just helped me to increase the speed of inserts but slows down when i use select to Improve INSERT-per-second performance of SQLite?


I thaught of using INSERT OR IGNORE too check if a record exists, and if doesnot it will insert it to database, but even in that case to get Id of the record i need to do a select


Aucun commentaire:

Enregistrer un commentaire