lundi 28 mars 2016

Write Blob into AdWords Editor sqlite database with Pandas to_sql

I try to write directly into the sqlite database of a tool called AdWords Editor (this makes it working much faster).

import sqlite3
import pandas as pd
import sqlalchemy

connection = sqlalchemy.create_engine('http://sqlite/path\\to\database\\ape_9822240157.db')
kw = pd.read_sql("SELECT * from Keyword", connection)

After importing I add some more rows to the DataFrame and re-import with:

new_kw.to_sql('Keyword', connection, if_exists='append', index=False)

Works perfectly fine.

Except for some columns. These contain only URLs but have a Blob datatype according to sqlite browser:

Screenshot

Checking on one of the rows of such a column returns a buffer:

kw['finalUrls_base'].iloc[0]

<read-write buffer ptr 0x000000012C155D78, size 96 at 0x000000012C155D40>

The representation of one of these entries in ipython notebook looks like:

kw[['localId','finalUrls_base']].loc[3591]

finalUrls_base    [, , , , , , , :, , h, , t, , t, , p, , s, , :, , /, , /, , w, , w, , w, , ., , e, , x, , a, , m, , p, , l, , e, , ., , c, , o, , m, , /, , a, , b, , c, , d, , e]
Name: 3591, dtype: object

I like to create this also for my new data that gets appended to the database. So I tried running this on my new entries in that column:

kw['finalUrls_base'] = kw['finalUrls_base'].apply(lambda x: buffer(x))

But this does not work. The database does not accept such values and leaves the field empty:

Screenshot 2

How can I re-create this kind of Blob data to put it in the existing database?

Aucun commentaire:

Enregistrer un commentaire