Queries based on rowid are slow in SQLite

I'm using SQLite through DBD::SQLite in perl to query large text databases (~10GB, ~10 million records). The regular FTS MATCH searches work fine (they usually run under a second), but searches based on ROWID are atrociously slow and hog massive amounts of memory. I'm trying to retrieve a couple of adjacent rows like so:

my $rowrange_bottom = 20;
my $rowrange_top = 40;
my $q_c = $dbh->prepare( "SELECT * FROM tmdata WHERE (ROWID BETWEEN $rowrange_bottom AND $rowrange_top)" );
# my $q_c = $dbh->prepare( "SELECT $hitcols FROM tmdata LIMIT 21 OFFSET $rowrange_bottom" ); # tried this too, it isn't any better
$q_c->execute();

The execute takes several minutes and uses ~600 MB of memory. Now, http://ift.tt/1eYjsVw writes that:

3.8.1: FTS4 queries are better able to make use of docid<$limit constraints to limit the amount of I/O required

I was using 3.7.x so I updated DBD::SQLite and that got me up to SQLite 3.8.6, but I see no significant improvement. Could it be that I need to reimport my databases to get improved performance on docid/rowid? I wouldn't expect so. Maybe there is a better way to write the query?