I have tables in SQLite database. I use C# and System.Data.SQLite DLL to access the database. My test shows that the following query is very slow:
string sql = "select column1, column2 from table_1 where column1=1 and column2=2"
SQLiteCommand mycommand = new SQLiteCommand(cnn);
mycommand.CommandText = sql;
SQLiteDataReader reader = mycommand.ExecuteReader();
tb.Load(reader);
reader.Close();
But if I first load the whole table into .NET DataTable like below, and then use DataTable.Select(), it's is much faster:
string sql = @"SELECT * FROM '" + data_table_name + "'" + ";\n";
SQLiteCommand mycommand = new SQLiteCommand(cnn);
mycommand.CommandText = sql;
SQLiteDataReader reader = mycommand.ExecuteReader();
tb.Load(reader);
reader.Close();
DataRow[] rows = tb.Select("column1=1 AND column2=2");
The difference is even more significant if have multiple queries after loading the whole table.
This is essentially cache the whole table in .NET. I tried different kind of tables with different sizes and primary keys, all behave the same.
For now, I know we would use such optimization, but wondering if there are other workaround.
Also, I figured out that if the index of a table is string rather than integer, as long as the string is short (tested up to 10 characters string), the direct SQL query is actually faster without caching the table. However, after caching the table, using integer index is faster than string index.
Anybody can explain the behavior?
Aucun commentaire:
Enregistrer un commentaire