vendredi 24 avril 2015

SQLite reads with multiprocessing, not starting the imap

I've tried everything I could think of, but I can not figure out why the following multiprocessing code will not start the loop:

import sqlite3, itertools

# Dummy table
conn = sqlite3.connect(":memory:")
conn.execute('CREATE TABLE numbers (num integer)')
conn.executemany("INSERT INTO numbers VALUES (?)",
                 ((x,) for x in range(5)))
conn.commit()

cmd_search = "SELECT * FROM numbers"
cursor = conn.execute(cmd_search)

def nothing(x): return x

import multiprocessing
P = multiprocessing.Pool()
#ITR  = P.imap(nothing,cursor)           # parallel version
ITR  = itertools.imap(nothing, cursor)   # serial   version

for x in ITR: print x

When I run it with the "serial" version (using itertools.imap) I get the expected output of (0,) (1,) (2,) (3,) (4,). Using the multiprocessing.imap version, I get nothing and the loop exits silently. I clearly has to do with the sqlite cursor, switching to cursor=range(5) works.

Why won't multiprocessing work here?

Aucun commentaire:

Enregistrer un commentaire