mercredi 13 janvier 2016

Django Query too slow because of for()

Suppose context['cat'] is a list which can have several values ['X', 'Y', 'Z'] and 'category' field can also have several values: ['X'] or ['X,Y'].

This code lets me obtain all the objects that I want. The problem is that I have 4 million records. When I tried this code on 100k records there was no delay, but with 4M records there is a 20secs delay.

paper_list = model1.objects.filter(reduce(operator.or_, (Q(category__icontains=x) for x in context['cat'])))

I have been told that the solution may be to use ManyToMany relations on the models but that is something that I don't know how to use even though I read the documentation ManyToMany.

Is there a way of making this possible without that structure? I wanted to use something like the LIKE expression on MySQL, but 'icontains' isn't giving me what I want unless I use a for(), which has delays with so many records.

In case it is of any help, this is the model I loaded all the records onto, and I don't have more records:

class model1(models.Model): 
   author= models.CharField(max_length=200, blank=True)
   title= models.CharField(max_length=200, blank=True)
   sum= models.CharField(max_length=200, blank=True)
   key= models.CharField(max_length=200, blank=True)
   iso= models.CharField(max_length=200, blank=True)
   extra= models.CharField(max_length=200, blank=True)
   url= models.CharField(max_length=200, blank=True)
   category= models.CharField(db_index=True, max_length=199)

Could someone point me in the right direction?

Aucun commentaire:

Enregistrer un commentaire