I am developing a graphical analysis application in c# and wpf, for that I have a written custom parser to process csv and txt files and I'll keep those processed data in-memory to use it later.
Right now I am using Data Table to keep all the data in-memory, However I have serious concern about memory usage of Data Table, a input data of 150 MB will become 1 GB after storing it into Data Table.
Obviously I cannot mange my data with Data Table, so I have created my own custom Data Table class and filled the data to it. But I don't find any significant improvement in a memory usage. Consider all my data will be in a string.
I have tried Object data base like velocity db and perst, both were some or other reason I cannot use it in my application.
So, lastly I thought I can keep the process data in Sqlite database. However I suspect using SQL query will definitely slow down the application performance.
Give me some insight to handled the millions of record in an c# application.
Aucun commentaire:
Enregistrer un commentaire