kitcn 0.33 stops long filters from reading the whole table
2 hours ago
kitcn 0.33 cuts read cost. Queries that filter on a long list of values, or pair an in list and a limit with row-level security, used to read far more documents than they returned. Now they stay on the index. Two paging orders change too, and both migrations come at the end.
Ask about this presentation
Chapters
Show transcriptHide transcript
kitcn 0.33 cuts read cost. Queries that filter on a long list of values, or pair an in list and a limit with row-level security, used to read far more documents than they returned. Now they stay on the index. Two paging orders change too, and both migrations come at the end.
Take a findMany with an in list and a limit of three. With row-level security on, or beside a filter like contains, it read every row it matched. That was 500 documents on a 500-row table. Now it reads 6, at any table size, with the same rows in the same order.
The limit is also counted against rows the caller can see. Put 80 rows an RLS policy hides in front of the matches. The three visible rows come after them. A limit of three still returns all three, and reads 86 documents instead of 200.
A filter with more than 64 values could fall back to reading the full table. Now indexed in, notIn, ne and same-field OR filters stay bounded at any length. Up to 64 ranges are merged into one stream. A wider list is read one range after another, so its pages come back in index order.
The select pipeline gets the same treatment. A where on a union source is compiled against your indexes. And a flatMap stage's where rides an index that extends the relation's foreign key, so only posts with more than ten likes are read. A callback where now runs once, instead of twice.
Aggregates get cheaper to maintain. Inside one statement, kitcn reuses bucket reads and folds shared updates. A 40-row key migration used to make 80 bucket writes. Now it makes two, plus two extrema writes and the forty membership rows. Your hooks still see the statement's own writes.
On the React side, a server-rendered page could lose its signed-in cRPC results while hydrating, when Convex confirmed the token the client already held. Now that confirmation keeps them. Signing out, a rejected token or a different user still resets them.
Now the two breaking changes. Paging a filter with more than 64 values follows index order, grouped by value, rather than creation order. For newest first, add an orderBy on createdAt and keep maxScan, which that sort still needs past 64 values.
Second, flatMap. When your schema has an index that extends the relation's foreign key, a stage where reads through it. Children then arrive in that index's order, ahead of creation time. Page cursors already out for those queries don't carry over.
So, bump kitcn to 0.33.0. First check any filter over 64 values you page newest first, and any flatMap where you page with live cursors. One limit: a where that filters through a relation keeps its old read cost. That's kitcn 0.33, where long filters stop reading the whole table.