mirror of
https://github.com/neuml/txtai.git
synced 2026-10-12 02:19:52 +00:00
Sanjay Santhanam
796c5bb0a8
The token consumer loops in Expression.bracket(), Expression.function() and
Expression.similar() read the next token with next(iterator, (None, None)) but
never checked for the end-of-stream sentinel. When a clause was left
unterminated (e.g. "select [a from txtai", "select func(a from txtai" or
"select * from txtai where similar('abc'"), the iterator was exhausted and the
code indexed tokens[None], raising a cryptic
TypeError: list indices must be integers or slices, not NoneType
instead of the SQLError that the database layer documents and callers catch for
invalid SQL. The parse step runs before the query is handed to the database, so
this TypeError escaped the SQLError wrapping in Database.execute().
Add Expression.advance(), a small helper that wraps next() and raises SQLError
when the token stream is exhausted mid-clause, and route the three delimited
consumer loops through it. Auto-closing the clause instead was rejected because
it would silently reinterpret malformed input (e.g. "select [a" would become the
valid column "a"), masking the user error. This mirrors the existing None-guard
convention already used in Expression.compound().
Behavior on valid queries is unchanged (advance() only diverges from the old
code path when the iterator is empty). Adds testUnterminated covering
unterminated bracket, function and similar clauses.