Patch, I think it's close.

I started from Areek's patch on LUCENE-5350, to add context to InputIterator and LookupResult, and then fixed AnalyzingInfixSuggester to index all contexts as StringField, and add MUST clauses to the BooleanQuery at lookup time. I also store the contexts using SSDV field, and return them for each LookupResult.

New patch, fixing one issue I hit when I was folding this into http://jirasearch.mikemccandless.com, which was the contexts should all be OR'd together not AND'd (ie, if a suggestion has any of the contexts, it's accepted).

I think it's ready.

I have a patch with contexts() returning BytesRefIterator rather than Set<BytesRef>, along with implementation for the contexts in all the InputIter implementation (except HighFreqInputIter and FileInputIter). Do you think It would make sense to have the context() return a BytesRefIterator instead?

Patch with context() and hasContext() implementations of various InputIterator:

• Added contexts() [returns BytesRefIterator] and hasContexts() APIs in InputIterator
• Implementation of contexts() in DocumentDictionaryIterator, DocumentValueSourceIterator, BufferedInputInterator, SortedInputIteraor and UnSortedInputIterator
• Added tests for context() API in DocumentDictionaryIterator, DocumentDictionaryIterator and SortedInputIterator
• Disable all suggesters from accepting InputIterator with contexts

NOTE: this patch does not add context to AnalyzingInfixSuggester and co!

This patch merges the previous patch and the original patch together. It adds support to contexts() in various InputIterator implementation, along with using the contexts in AnalyzingInfixSuggester (original patch by Michael). The patch changes InputIterator's context from Set<BytesRef> to BytesRefIterator. The lookup() API still uses the Set<BytesRef>. Thoughts?

Hmm, I think I prefer the simpler Set<BytesRef>? And e.g. one problem with BytesRefIterator is you can only iterate it once (we'd sort of need a BytesRefIterable I guess), which might be a hassle for some suggesters?

That sounds good. I will change the InputIterator contexts() api to return Set<BytesRef> then. Will get a patch up tomorrow.

I will change the InputIterator contexts() api to return Set<BytesRef> then. Will get a patch up tomorrow.

Thanks Areek!

Updated Patch (LUCENE-5528-1.patch):

• uses Set<BytesRef> to represent contexts in InputInterator

Thanks Areek, new patch looks great! It's nice that Document/ExpressionDictionary, and the wrapping InputIterators (Sorted/Unsorted) handle contexts as well. I think it's close.

I noticed a lot of added but unused "import import org.apache.lucene.util.BytesRefIterator;" lines.

Also, AnalyzingInfixSuggester calls InputIterator.contexts() regardless of whether hasContexts return true or false; I think this is actually OK, but can you update InputIterator's javadocs to state that this is allowed, and contexts() is expected to return null even if hasContexts() had returned false?

Finally just a small whitespace issue: instead of:

    if(iterator.hasContexts()) {

Can you do this:

    if (iterator.hasContexts()) {

(i.e. just add space between "if" and "("). Thanks!

BTW I pushed this patch to the jirasearch "production" site (http://jirasearch.mikemccandless.com ) and it seems to be working very well!

If you drill down on a specific project (Lucene, Solr, Tika) then the suggestions you see are only for that project. This also happens if you type the project name, e.g. "lucene word" shows only lucene issues/users containing word*, and "tika word" shows only tika issues/users with word*

Nice to eat your own dog food ...

Updated patch (LUCENE-5528-1.patch):

• removed unused imports (my bad!)
• updated InputIteration.context() docs to record that it can return null
• minor formatting fixes (as suggested)

Thanks Michael for the review!
That is awesome, I played around with the jirasearch to see the functionality for myself.

Thanks Areek, this looks great! I'll commit soon.

Commit 1580510 from Michael McCandless in branch 'dev/trunk'
[ https://svn.apache.org/r1580510 ]

LUCENE-5528: add contexts to AnalyzingInfixSuggester

Commit 1580517 from Michael McCandless in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1580517 ]

LUCENE-5528: add contexts to AnalyzingInfixSuggester

Commit 1580518 from Michael McCandless in branch 'dev/trunk'
[ https://svn.apache.org/r1580518 ]

LUCENE-5528: remove generics

Close issue after release of 4.8.0

Hello Michael McCandless
For the AnalazingInfixSuggester, does it make sens to have a new override for the method lookup() with an additional param allContextsRequired , similar to allTermsRequired

 lookup(CharSequence key, Set<BytesRef> contexts, int num, boolean allTermsRequired, boolean doHighlight, boolean allContextsRequired)

which will make it possible to have an "AND" behaviour for the contexts when you have many contexts instead of the default "OR"?

This means that line
https://github.com/apache/lucene-solr/blob/trunk/lucene/suggest/src/java/org/apache/lucene/search/suggest/analyzing/AnalyzingInfixSuggester.java#L486
will now use BooleanClause.Occur.SHOULD or BooleanClause.Occur.MUST depending on the value of the new param allContextsRequired ...

We currently have a use-case for this.

Thanks.

Arcadius.

Another alternative may be to have something like Map<BytesRef,Boolean> instead of Set<BytesRef> where the boolean indicates whether the entry is a SHOULD or a MUST

Hi Arcadius Ahouansou, I think it makes sense to give more control over how context is applied; could you open a new issue for this?

Hi Michael McCandless.
The new issue is LUCENE-6050
Thanks.