samedi 19 septembre 2015

python spider not selecting start_uls from databse

I'm trying to populate start_urls from SQLite database for spider

from priceComp.items import PricecompItem
from scrapy.spiders import CrawlSpider
import sqlite3 as lite_db
con = None



class ProductdataSpider(CrawlSpider):
    name = "productData"
    allowed_domains = ["pricematch.pk/"]
    start_urls  = []



    def start_requests(self):
        self.con = lite_db.connect('price_comp.db')
        self.curr = self.con.cursor()
        self.curr.execute("SELECT p_url FROM site_url ")
        rows = self.curr.fetchall()
        for row in rows:
            self.start_urls.append(row[0])
        self.con.close()

But it's giving this exception

exceptions.TypeError: 'NoneType' object is not iterable

I know it means "rows" is none. But I already have 4/5 values in database which means I think my code is not selecting values from database, is there something wrong with select query or something else please help me Thanks

Aucun commentaire:

Enregistrer un commentaire