|
I propose that we throw a ParseException - it seems very straightforward to do - any comments / concerns / objections?
I think the idea of a parse exception is good. The issue is relatively subtle. A user can relatively easily generate a query that ignores terms. Doing that silently is just asking for people to spend a lot of time trying to understand why they aren't getting something they expect.
The question is whether it's feasible to detect that an invalid query has been entered. The problem is of course having to ever enumerate all documents. But with multilevel queries (sums of products of sums, etc.), it's not immediately obvious from the parse tree whether that is being attempted or not. a(b + !c) is okay but (a+!d)(b + !c) isn't. Basically you can't do a term-by-term analysis of a multilevel query and decide. It's a global property of the query. It one reduces the multilevel query to a two level sum-of-products form (i.e., distribute all AND operations over all OR operaitons), you can inspect the result. If any term does not contain a positive term, the query is invalid. But going from multilevel to two level can generate an exponential number of terms. Maybe not an issue in IR, but it makes me nervous. I would think that somewhere between query parsing and search execution, the invalidity of a query would have to pop out but I haven't delved into this path enough to know where that might happen. If no one's looking at this (generating an exception somewhere) and no one knows of any reason it's not possible, I'll look ... On a related topic, and maybe a naive question, has anyone considered keeping a posting that contains all documents? Is it really that hard to do in Lucene today? That would remove this restriction. Perhaps the issue then is that the combination of the number of valid use cases combined with the number of times it would cause people trouble because they were using it invalidly means it's not a good idea? I can come up with some potentially interesting queries, especially when you combine with other features like facets. "Tell me the facet breakdown of all documents that don't contain the word microsoft ..." In many places, !microsoft is not such a terribly big set and can be computed with about the same effort that microsoft can. (Is that true?) Any chance this may be addressed in 2.0?
For Dejan Nenov - please use this new email address:
d [at] panaton [dot] com After 10 years dejannenov@jollyobject.com is being overwhelmed by spam. – Can we rewrite the query (A OR NOT B) to NOT(NOT(A) AND B) to solve this issue?
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
 |
|
Bug
Open
Major
TERM1 OR NOT TERM2 does not perform as expected
The QueryParser (QP) is not really designed to handle this case and there is some misunderstanding as to what theuse of multiple boolean operators in a single clause.
I don't think the QP is designed to handle two boolean operators between 2 terms. Thus, the above query really doesn't make sense, but the QP doesn't really prevent it either. Logically, A OR NOT B is equivalent to just searching for A. However, the QP, from what I can tell, only looks at the last operator before the B term and builds it clause based on that, thus building the query A NOT B, which then yields the results as above.
I am not sure if there is something to fix here other than the documentation. I suppose we could throw a Parse Exception if we detected 2 or more boolean operators between terms, but I am not that familiar with JavaCC, so I am not 100% sure.