I have two tables: trainSearcStream and SearchInfo. My code is supposed to merge the two tables based on a common column named SearchID. the problem is that the code runs for sometime and then terminates without reporting any error in my eclipse console. the number or rows in each tables is: trainSearchStream|392,356,948 and SearchInfo|112,159,462. the size of my database (which include other tables) is 40GB.
import pandas as pd
import sqlite3
# Create connection.
con = sqlite3.connect(path + 'database.sqlite')
#read into a data frame
df = pd.read_sql_query("SELECT * FROM trainSearchStream, SearchInfo WHERE trainSearchStream.SearchID = SearchInfo.SearchID;", con)
#save to file
df.to_csv(path + 'traindata.csv',index=False,encoding='utf-8')
Aucun commentaire:
Enregistrer un commentaire