By table you mean index, right ?
What matters most is the size of the index, and most index data structures (see btrees) work so that very few disk seeks are needed, even if the index is large.
Perhaps if you're using a really slow spinning disk and have no indexes, or the results are distributed evenly across the entire dataset (i.e lots of random, non sequential access).
It's just 300ms is really slow, even with a largeish dataset that doesn't fit into memory. Perhaps you hit a corner case in some way that destroyed performance in sqlite, but I'd be surprised if those results were representative.