A simple Java client for updating and searching

• Key: SOLR-20
• Type: New Feature
• Status: Resolved
• Resolution: Fixed
• Priority: Minor
• Assignee: Ryan McKinley
• Reporter: Darren Erik Vengroff
• Component/s: clients - java
• Fix Version/s: 1.3

Description

I wrote a simple little client class that can connect to a Solr server and issue add, delete, commit and optimize commands using Java methods. I'm posting here for review and comments as suggested by Yonik.

Comments

[Darren Erik Vengroff]

Previous version didn't properly escape the query in the deleteByQuery() case.

[Erik Hatcher]

This looks quite good and well documented! Thanks for this contribution. The only issue my current project would have with this is the Map which prevents multiple fields of the same name from being added. I use a lot of multi-valued fields.

[Erik Hatcher]

One idea for dealing with multivalued fields is to check the type of the object in the Map and if is an array or Collection then iterate over it rather than just doing .toString() on it. Would that logic work for your use as well?

Iterating over an array or Collection of values is a great suggestion. I'll change it and resubmit when I have some time later today.

[Darren Erik Vengroff]

Here is the latest, incorporating Erik's suggestion about supporting multi-valued fields.

BTW, is there any way to delete the older attached versions of this file from JIRA? There's no real need for them to be there any more.

[Darren Erik Vengroff]

New exception type for reporting server-side exceptions.

[Darren Erik Vengroff]

New client code that uses the new exception.

[Darren Erik Vengroff]

Here's the latest. There is an abstract base class that handles client connection and request/response and two subclasses. One is as before, with java APIs, and the other is for cases where you have an XML document you want to transform and send to the server.

[Yonik Seeley]

Great! Now we need to figure out where it lives, and how to work out the dependencies (a solr-util.jar that a client could use, or perhaps just pull the needed class or two directly into the solr-client.jar)

[Philip Jacob]

delete() in the DocumentManagerClient ought to be doing this:

<delete><id>1234</id></delete>

It's currently doing this:

<delete><query>1234</query></delete>

[Darren Erik Vengroff]

Good catch Philip. For the benefit of future downloaders, here's a complete zip file with everything including this fix.

[Darren Erik Vengroff]

Here is the latest version of the client code, in the form of solr-client-source.jar. The big difference here is that there are now two clients, DocumentManagerClient for adding, inserting, and updating, and SearchClient for searching. They share the same underlying communication mechanism, which consists of a low-level mechanism for doing queries and parsing responses by reading from an InputStream (see ResponseParser) and a slightly higher level mechanism that handles some of the XML for you (see XmlResponseParser).

I've been building this as a seperate project with a Maven2 dependency on Solr, but if the source is dropped into the Solr source tree at the appropriate place I suspect it will just compile and work. There are no other outside dependencies.

I have some unit tests as well, but for the moment they are too tied in to my environment to be useful to the broader community. I will correct this and submit them.

[Darren Erik Vengroff]

Please ignore that last attachment. It contains an earlier version of the code than I intended, and has a couple of serious bugs. I'm sure that what I'm running now is a lot better, but I'm going to iterate a little more and build some more complete tests before I submit it again. Unless anyone out there is dieing to be on the bleeding edge of this.

[Yonik Seeley]

Ping...
any chance you could make your latest version available?

[Fuad Efendi]

I'm dieing... which files should I download?

[David Halsted]

it is a little confusing – any chance one of the attachments could be designated as the right one? Thanks!

[Hoss Man]

I don't know that there is a specific "right" one at the moment ... Darren's last comment suggests that he has a better version but it's not quite ready for submission.

FYI: if you click the "All" link at teh top left of the comment listing, you can see where in the flow of time each of the attachemnts was added – from there it's pretty easy to tell that while solr-client-sources.jar is the most recent attachment, it's the one Darren said should be ignored...

at this moment solr-client-java-2.zip.zip seems to be the most recent "good" version.

[Otis Gospodnetic]

SOLR-20 and SOLR-30 seem to be lingering in JIRA.
Is the plan to merge them, and get the code into Solr?

[Fuad Efendi]

SOLR-20: add, delete, commit, optimize
SOLR-30: search

So, should be merged.

HttpClient seems to be right choice (easily configurable; 'follow redirects', 'buffer size', etc.).

Possible improvements:

• make it independent on implementation (for instance, use interface and HttpClient-based implementation)
• XML-based external configuration file
• no need for JDOM...

Probably, we need separate src/client folder for source files (sources do not depend on SOLR)

[Ryan McKinley]

I've attached the client code i have written that merges SOLR-20 & SOLR-30. It can add, delete, commit, optimize and search.

The main interface is SolrClient.

It uses the XPP parser to parse the search results.

If there is interist, i can clean it up some more...

[Fuad Efendi]

My previous comment is visible to jira-users only, sorry.
Code submitted by Ryan looks great!

[Ryan McKinley]

I just posted a new version of a java client. This moves things to proper org.apache... packages and adds the waitFlush, waitSearcher suggested by Fuad.

If people are interested, i think this should sit next to /client/ruby in: /client/java/solrj/

there is a build.xml file that will generate a solr-client.jar file.

As a taste, this is how you perform a search:
<code>
SolrClient client = new SolrClientImpl( new URL("http://localhost:8983/solr/") );

SolrQuery query = new SolrQuery();
query.setQuery( "video" );

QueryResults results = client.query( query );

for( ResultDoc doc : results.getDocs() ) { System.out.println( "["+doc.getId()+"] "+ doc.getField( "name" ) ); }
</code>

Also, if there is interest, i can post an example webapp using this client library to search and explore a solr repository.

[Ryan McKinley]

added APL to zip

[Hoss Man]

(NOTE: revised summary since this issue has moved beyond just updating)

I finally had a chance to look this over, here's a few comments in
no particular order...

1) i like the name solrj, i think this code should definitely live in client/java/solrj so that there is the potential for other java client code that is independent (if nothing else, i suspect something like SOLR-86 might be handy) ... we should probably put solrj in the package name as well.

2) i wouldn't worry about having a special package for the exceptions ... they've got exception in their name, no ones going to be confused.

3) I'm really not fond of "ParamNames.java" being a copy of the constants in "SolrParams.java", or XML.java being copied, or the xpp jar being duplicated ... it seems like we should just pull in those (compiled) classes at build time ... but that would require that the whole Solr tree be checked out, and there seems to be interest in making it possible to "svn checkout client/lang/impl" and build that in isolation ... perhaps we could use svn:externals to pull in specific utility classes and jars from other places in the tree? (although based on what I've read today, branching for releases would be hard since all of the svn:external props would have to be updated).

what do people think in general about how the client code can/should/shouldn't depend on the core server code?

4) one thing we should really try to support in a client is executing query requests against non-standard request handlers ... handlers that might take in request params that we can't even imagine. The SolrQuery class has explicit setters for many of the params that the built in request handlers support, but there is no easy way for people to build other queries. I think it might make sense if SolrQuery was an interface that just defined the methods needed by the SolrClient – probably just getQueryString(). Then their can be a SimpleSolrQuery that has all of the setters in the current SolrQuery class, possibly using a general baseclass with an impl of getQueryString that uses some SolrParams...

public class AbstractSolrQuery implements SolrQuery {
protected abstract SolrParams getSolrParams();
public String getQueryString() { ... your current code, looping over getSolrParams() ... }
}

5) what is the purpose of SolrClientStub ?

6) what is the purpose of SolrDocumentable being an empty interface? ... it seems like you could replace SolrDocumentable, SolrDocument, and SolrDocumented with something like this...

public interface SolrDocument {
public Map<String,Object> getSolrDocumentFields();
}
public abstract class SolrDocumented implements SolrDocument {
protected abstract SolrDocument getSolrDocument();
public Map<String,Object> getSolrDocumentFields() { return getSolrDocument().getSolrDocumentFields() }
}

Then you wouldn't need that instanceof code in SolrClientImpl

Note that we should probably support field and document boosts as well, but field boosts don't really need to be specified in the Map since they apply to the whole field and not the individual values, so we could just add...

public int getDocumentBoost();
public Map<String,Integer> getFieldBoosts()

...to SolrDocument.

7) The ResultsParser and QueryResults classes seem to suffer the same limitation that i was mentioning about the SolrQuery class – they assume a very specific response structure (only one doc list, an optional facet block, an optional highlighting block, an optional debug block) ... I think since the ResultsParser already understands the all of the various tags that are used, it should be easy to do this as long as the QueryResult object becomes a more general container that any named data can be shoved into (just like SolrQueryResponse is on the server side) ... then a "SimpleQueryResults" class could be written that had the convenience methods that make sense when using StandardRequestHandler or DisMaxRequestHandler.

8) There was a comment in SOLR-30 regarding the issue of that code only parsing the XML response ... i think it's completely practical to focus on client code which currently supports only the XmlResponseWriter output – especially with the solrj ResultsParser class currently having a single public method...

public QueryResults process( Reader reader ) throws SolrClientException, SolrServerException, XmlPullParserException, IOException

...i think if we removed XmlPullParserException from that list of exceptions (it could always be wrapped in a SolrClientException, or a new SolrClientParseException) we have a really simple API where other ResultParser classes could be written to handle JSON or what not down the road just by adding a simple setResultParser to SolrClient.

[J.J. Larrea]

Regarding Hoss' point #3, perhaps it's time to reorganize into something like

/solr/server/...
/solr/client/...
/solr/webapp/,,, (or /solr/server/webapp)
/solr/shared/...

"To build client XXX check out /solr/client or just /solr/client/java/XXX and /solr/shared"

Shared would include external constants and exceptions.

[Ryan McKinley]

I have dramatically reworked the client code to fit with the pluggable ContentStream model in SOLR-104. This version makes it easy to customize/extend the request/response behavior. Once it stabilizes and is better tested, I'll upload a zip, but for now, you can preview it:

http://svn.lapnap.net/solr/solrj/

Major changes:

• it is based on commons-httpclient-3.0.1.jar
• I'm using wt=JSON rather then XML. (It maps to a hash easier)
• I moved some of the common classes to o.a.s.util. Hopefully the core classes will be refactored to make it easier to share some classes
• handles multiple ContentStreams using multi-part form upload
• Got rid of the SolrDocumentable/SolrDocumented distinction. – now there is only SolrDocument()
• You can define and automatically build a solr document with annotations
• Includes a first draft for a HibernateEventListener. When stuff is added/updated/deleted, it gets sent to solr. (Note, this class should probable not be in the main client as the hibernate prerequisite libraries are substantial - I've included them because its what i need to have working soon) When this is more stable, it will be something similar to a Compass Hibernate3GpsDevice (http://www.opensymphony.com/compass/versions/1.1RC1/html/gps-hibernate.html)

The key interfaces are:

public interface SolrClient
{
public abstract SolrResponse process( final SolrRequest req );
}

public interface SolrRequest
{
public String getMethod();
public String getHandlerPath();
public RequestParams getParams();
public Collection<ContentStream> getContentStreams();
public SolrResponse parseResponseBody(InputStream in);
public SolrResponse execute(SolrClient solr);
}

• - - - - - -
  Here is some sample usage:

SolrClient client = new CommonsHttpSolrClient(
new URL("http://localhost:8983/solr/") );

// Set up a simple query
SolrQuery query = new SolrQuery();
query.setQuery( "solr" );
query.addFacetField( "cat" );
query.setFacetLimit( 15 );
query.setQuery( "video" );
query.setShowDebugInfo( true );

QueryResponse rsp = query.execute( client );
for( ResultDoc doc : rsp.getDocs() ) {
System.out.println( doc.get( "name" ) );
System.out.println( doc.getScore() );
System.out.println( doc.getExplain() );
}

SimpleSolrDoc doc = new SimpleSolrDoc();
doc.setField( "id", "xxx" );
doc.setField( "price", 12.34f );
doc.setField( "cat", new String[] { "aaa", "bbb", "ccc" } );
new AddDocuments( doc ).execute( client );
new CommitIndex().execute( client );

• - - - - - - - - - -

This also includes a utility to make solr documents from annotations. Given the class:

@SolrSearchable( boost=2.0 )
public class Example
{
@SolrSearchable
public String getName()

{ return "hello" }

@SolrSearchable( name="cat", boost=3 )
public String getSomeOtherName()

{ return "there" }

}

The DocumentBuilder can automatically make:

<doc boost="2.0">
<field name="name">hello</field>
<field name="cat" boost="3">there</field>
</doc>

• - - - - - - - - - -

There are a few parts of the API i think are awkward, I'd love any feedback / review you may have.

thanks
ryan

[Yonik Seeley]

> * it is based on commons-httpclient-3.0.1.jar
Cool, +1

> * I'm using wt=JSON rather then XML. (It maps to a hash easier)
Heh... I quickly checked out the code, but didn't see where you were parsing the code, or where the JSONObject class referenced is.

Anyway, if you want the best JSON parser on the planet, check out
http://www.nabble.com/Apache-Lab-proposal%3A-noggit-tf2701405.html#a7532843
http://svn.apache.org/repos/asf/labs/noggit/

I haven't had a chance to do the writing side, or the "create full object graph" part, but the parser is screaming fast.

> * handles multiple ContentStreams using multi-part form upload
Will a client need to do that? I had thought a browser would be the only one using multi-part

> * You can define and automatically build a solr document with annotations
Sounds cool

> * Includes a first draft for a HibernateEventListener.
Sounds very cool... it should go in a separate contrib eventually.

[rubdabadub]

Hi:

I was really hoping that this patch will make it to trunk soon. I been using it without any problem. I was wondering if there are any specifics that are left for SOLR-20 to make it to trunk? I would be more then happy to help out anyway I can. I need SOLR-20 for Nutch-Solr integration so it would be nice if it was included.

Thankful for your kind attention to SOLR-20.

[Erik Hatcher]

I want this on trunk also. I'll be reviewing it and testing it out this afternoon and committing if all is fine.

[Erik Hatcher]

My bad... it was SOLR-86 that interested me, and I've just committed it. I can't take on these patches just yet.

[Ryan McKinley]

I don't think this one is quite ready to go, but anyone interested can see an updated versions at:

http://solrstuff.org/svn/solrj/
http://solrstuff.org/svn/solrj-hibernate/

I have extracted the hibernate specific stuff into its own project so solrj is a bit more manageable.

• - - - -

If you are looking for a stable, simple client "solr-client.zip" is still your best bet.

[Ryan McKinley]

to do SOLR-20 properly, it should use shared libraries rather then redefine things like constants and common utility classes.

[Frederic Hennequin]

Hello, we have been testing the solr-client and think we have found a small bug :

the xml parsers on the query-side is not setup to use "UTF-8" encoding
this resulted in weird characters being returned by the solr-client...
for example : "é" came out like "©" ... not really what we would like...

we fixed it by setting the input stream for the xmlparser to "UTF8" which gave us this code in ResultsParser.java :
[code]
public QueryResults process( InputStream reader ) throws SolrClientException, SolrServerException, XmlPullParserException, IOException
{
QueryResults res = new QueryResults();

try {
XmlPullParser xpp = null;
try { xpp = factory.newPullParser(); xpp.setInput(reader,"UTF-8"); xpp.nextTag(); }
.....
[/code]

notice we changed the argument for this method to InputStream instead of the reader so we could add "UTF-8" to the stream.
by doing this we had to change the reader in SolrClientImpl.java to an inputstream :
[code]
....
InputStream inputStream = urlc.getInputStream();
try { QueryResults res = parser.process( inputStream ); res.setSolrURL( qurl ); res.setQuery( query ); return res; }
....
[/code]

in our opinion this was a major bug (since all solr-xml is encoded in utf-8) and we guess somebody just forgot to put it in...

yay, now we can all start using freaky characters without the client actually freaking out enjoy

[Thierry Collogne]

I found that there was no way of adding highlight paramters to an SolrQuery, so I made some modifications to SolrQuery.java and ParamNames.java to allow highlighting

Changes to SolrQuery

First add a new variable

private HighlightParams _highlight = new HighlightParams();

Than I defined a new innerclass (a bit like FacetParams)

public static class HighlightParams {
public List<String> field = new ArrayList<String>();
public int snippets = 3;
public int fragsize = 100;

public String simple_pre = "<b>";
public String simple_post = "</b>";

public boolean isEnabled()

{ return field.size() > 0; }

}

Than add the following methods

public void addHighlightField( String f )

{ _highlight.field.add( f ); }

public void setHighlightSnippets( int snippets )

{ _highlight.snippets = snippets; }

public void setHighlightFragSize( int fragsize )

{ _highlight.fragsize = fragsize; }

// ATTENTION : only simple tags. No quotes like in <span class="tag">
public void setHighlightSurroundingTags( String pre, String post )

{ _highlight.simple_pre = pre; _highlight.simple_post = post; }

Add the following code to getQueryString method (for example right before return builder.toString()

if( _highlight.isEnabled() ) {
builder.append( '&' ).append( ParamNames.HIGHLIGHT ).append( "=true" );
builder.append( '&' ).append( ParamNames.HIGHLIGHT_FIELDS ).append( "=" );
for( String f : _highlight.field ) { builder.append( f ).append( ',' ); }
builder.append( '&' ).append( ParamNames.HIGHLIGHT_SNIPPETS ).append( "=" ).append( _highlight.snippets );
builder.append( '&' ).append( ParamNames.HIGHLIGHT_FRAGSIZE ).append( "=" ).append( _highlight.fragsize );

builder.append( '&' ).append( ParamNames.HIGHLIGHT_SIMPLE_PRE ).append( "=" ).append( _highlight.simple_pre );
builder.append( '&' ).append( ParamNames.HIGHLIGHT_SIMPLE_POST ).append( "=" ).append( _highlight.simple_post );
}

Changes to ParamNames.java

Add/modify the following variables

/** wether to highlight */
public static final String HIGHLIGHT = "hl";
/** fields to highlight */
public static final String HIGHLIGHT_FIELDS = "hl.fl";
/** maximum highlight fragments to return */
public static final String HIGHLIGHT_SNIPPETS = "hl.snippets";
/** override default highlight fragsize */
public static final String HIGHLIGHT_FRAGSIZE = "hl.fragsize";
/** override default pre highlight */
public static final String HIGHLIGHT_SIMPLE_PRE = "hl.simple.pre";
/** override default post highlight */
public static final String HIGHLIGHT_SIMPLE_POST = "hl.simple.post";

This can be used as follwed
SolrClient client = new SolrClientImpl( new URL("http://localhost:8080/solr/") );
// required
query.addHighlightField("title");
query.addHighlightField("content");
// following is optional
query.setHighlightFragSize(100);
query.setHighlightSnippets(3);
query.setHighlightSurroundingTags("<i>","</i>");

I think that is all. If I forgot something, post it here. One remark. The setHighlightSurroundingTags method can only take simple tags,
no tags containing quotes or such.

Greetz.

[Chen Lei]

org.apache.solr.client.impl.ResultsParser.java
org.apache.solr.client.QueryFacet.java (a copy of FieldFacet)
org.apache.solr.client.QueryResults.java
org.apache.solr.client.SolrQuery.java

I changed these four files, add some functions for facet query.
Add a facet query is similar as field query.
Existing methods, new methods:
SolrQuery.addFacetField() SolrQuery.addFacetQuery
QueryResults.getFacets() QueryResults.getQueryFacets()
QueryResults.getLimitingFacets QueryResults.getLimitingFacetsForQueryFacet()

Some code here..
SolrQuery query = new SolrQuery();
// ...
query.addFacetQuery("cat:music card");
query.addFacetQuery("video");

SolrClient client = ..
// ...
QueryResults results = client.query( query );

for (QueryFacet qf : results.getQueryFacets()) {
System.out.println("query facet: "+qf.getName() +" "+qf.getValueCount());
for (org.apache.solr.client.QueryFacet.Count c : qf.getValues() ) { System.out.println(" "+c.getName()+": "+c.getCount()); }
}

[carlos orrego]

I am using the client library nad have no issues with adding docs to solr. But when i want to make a simple test query i keep getting this error:

org.apache.solr.client.exception.SolrClientException: unknown type: status

Solr returns and xml with a statis tag in the header wich the client does not know how to handle?

am i right? is this the case?

[Will Johnson]

is there an updated package or anyone working on such a thing at the moment? the solr-client.zip at the top of the thread works like a charm but seems to be very outdated and the bits on the svn://solrstuff.org site have some rather serious bugs.

i'm happy to do all the leg work of packaging things, fixing bugs, submitting a patch, etc but i wanted to make sure i'm not about to walk right behind someone else. also, if anyone has any ideas for the best starting point i'm happy to take suggestions.

• will

[Ryan McKinley]

For now, I'd still recommend using solr-client.zip

The code on solrstuff.org is in the middle of a big overhaul (hopefully stable by the end of this week) – but it will rely on some changes to solr that are not likely to make it into solr-1.2.

I'll post another message here when that settles down.

thanks
ryan

[Ryan McKinley]

For anyone interested, I've finished a major overhaul of the client at:
http://solrstuff.org/svn/solrj/
It is a dramatically different architecture then before. Essentially it reads each response into a NamedList and each response type knows what the contents mean.

After solr1.2, I'll work on getting something like this into the official apache distribution.

This client source duplicates many classes that will eventually be extracted into an independent solr-utils.jar (SOLR-135). Everything that is not in "client"
http://solrstuff.org/svn/solrj/src/org/apache/solr/ will be in this .jar

I am using this in production code – but i don't suggest that it is production ready just yet.

[Will Johnson]

the trunk version at http://solrstuff.org/svn/solrj/ seems to be missing a dependency and a copy of SolrParams. ant returns....

compile:
[javac] Compiling 40 source files to C:\data\workspace\solrj\bin
[javac] C:\data\workspace\solrj\src\org\apache\solr\client\solrj\impl\XMLResponseParser.java:10: package javax.xml.stream does not exist
[javac] import javax.xml.stream.XMLInputFactory;

....

[javac] C:\data\workspace\solrj\src\org\apache\solr\client\solrj\query\SolrQuery.java:10: cannot find symbol
[javac] symbol : class SolrParams
[javac] location: package org.apache.solr.util
[javac] import org.apache.solr.util.SolrParams;

[Ryan McKinley]

aaah, It was compiling with java 6.

I just added stax-api-1.0.jar and cleaned up some imports, it should run on java 5 now.

[Will Johnson]

the new api's work great, thanks! what's the plan for this going forward? id' like to start doing some work on this as it's rather critical to my current project and an are i've dealt with a lot in the past. assuming it's not getting dumped into org.apache.* land any time soon are you accepting patches to this code? if so i have some modifications to the api's that i think will make them easier to use (such as a method to set FacetParams on SolrQuery) and i'll even flush out the SolrServerTest for fun.

also, i noticed that all the methods on SolrServer throw undeclared SolrExceptions which extends RuntimeException when things so south. should those throw some other sort of non-ignorable exception like a new SolrServerException? while it made coding/compiling easier to leave out all the usually required try's and catches it made running/debugging much less enjoyable.

• will

[Ryan McKinley]

great! Any feedback/help would be wonderful.

I hope it is not too long before this can enter solr trunk, but it will first need two solr1.3 additions SOLR-193 and SOLR-135, until then I can apply any patches necessary.

Re RuntimeException vs SolrServerException, I'm not sure the best choice. Earlier versions had a client exception and server exception, but in practice those got lumped together (in my case) anyway. I ended up just using SolrException because it is there.

[Ben Incani]

The SolrClientImpl does not implement the following optional attributes for "add" as documented in http://wiki.apache.org/solr/UpdateXmlMessages

allowDups = "true" | "false" — default is "false"
overwritePending = "true" | "false" — default is negation of allowDups
overwriteCommitted = "true"|"false" — default is negation of allowDups

Attached is patch for SolrClientImpl.java which implements allowDups.

• • • SolrClientImpl.java.patch ***
      48a49,55
      >
      > /**
      > * Optional attributes for "add"
      > */
      > protected boolean allowDups;
      > protected boolean overwritePending; // TODO: not implemented
      > protected boolean overwriteCommitted; // TODO: not implemented
      86a94,97
      > public SolrClientImpl(URL baseURL) throws Exception { > this(baseURL, false); > }
      >
      91a103
      > * @param allowDups allow duplicates to be added to the index
      95c107
      < public SolrClientImpl(URL baseURL) throws Exception
      —
      > public SolrClientImpl(URL baseURL, boolean allowDups) throws Exception
      103c115
      <
      —
      > this.allowDups = allowDups;
      243c255,260
      < writer.write("<add>");
      —
      > StringBuffer strAdd = new StringBuffer("<add ");
      > if (allowDups == true) { > strAdd.append("allowDups=\"true\""); > }
      > strAdd.append(">");
      > writer.write(strAdd.toString());
    • SolrClientImpl.java.patch ***

[Ryan McKinley]

Hi Ben-

Thanks for the patch. Will Johnson added the options to the java client on:
http://solrstuff.org/svn/solrj/

This implementation is now quite stable (thanks to lots of help from Will!) and I hope will be integrated into solr trunk shortly after 1.2 (this week?!)

About the allowDups/overwritePending/overwriteCommited options... what part of them do you use? In SOLR-60, there is talk of replacing the three options with a (simpler) overwrite=true/false – maybe we should change the client api to:

UpdateResponse add( SolrDocument doc, boolean overwrite ) throws SolrServerException;

In anticipation of this change?

[Walter Ferrara]

[I'm new to solrj, so everything I'm writing can be useless]

While trying to execute range query, using this query:
Text:Hello +Date:[1895011 TO 18971128]
[jdk 1.6/netbeans 5.5/solr1.2/solrj revision 125]

I kept getting IllegalArgumentException:
[...]
Caused by: java.lang.IllegalArgumentException: Invalid uri 'http://localhost:8983/solr/select?q=Text:Hello+%2BDate:[1895011+TO+18971128]&fl=ID,Date,score&rows=800&wt=xml&version=2.2': Invalid query
at org.apache.commons.httpclient.HttpMethodBase.<init>(HttpMethodBase.java:222)
at org.apache.commons.httpclient.methods.GetMethod.<init>(GetMethod.java:89)
at org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:108)
[...]
It seems related to un-escaped '[' and ']'.

the solution was to patch StrUtils.java, in order to encode square brackets.
If it is of any use, I'm attaching the patch
– Walter

====
StrUtils.java.patch

204,207d203
< case '[': dest.append("%5B"); break; // to allow range query, 20070706 (w)
< case ']': dest.append("%5D"); break;
< case '{': dest.append("%7B"); break; < case '}': dest.append("%7D"); break;

[Ryan McKinley]

Hi Walter-

I just updated http://solrstuff.org/svn/solrj/ to use:

URLEncoder.encode( val, "UTF-8" )

rather then:

StrUtils.partialURLEncodeVal( val )

Give it a try and let me know if you have problems... (i have done date range queries successfully with resin/jetty, but netbeans must be different!)

[Yonik Seeley]

FYI partialURLEncodeVal was meant for readable, yet unambiguous logging... hence a minimum of escaping is done (but enough to easily paste into a browser and let it do the rest of the escaping when you sumbit).

[Walter Ferrara]

Latest rev works perfectly thanks.
I've been making some time test with this client (only searching), and overall results show high times: this maybe due to my minimal knowledge on solr, but solr seems fast, is data-receiving/parsing on client that seems slow. Even when solr report 0ms (due to cache it I presume) it still take 200ms to get results (QueryResponse .getElapsedTime()). I'm using this code:

SolrQuery query = new SolrQuery(queryString);
CommonsHttpSolrServer server = new CommonsHttpSolrServer("http://localhost:8983/solr/");
QueryResponse response = server.query(query);
SolrDocumentList list = response.getResults();

The slowdown seems to be in client.executeMethod(method) (CommonsHttpSolrServer) Any way to speed up (assuming I'm not totally wrong on how to use this client...)? Reusing same http connection for multiple queries? Playing with MultiThreadedHttpConnectionManager helped a bit, but doesn't seems the solution

[Ryan McKinley]

I don't know if you are on solr-dev, Yonik noted that the QTime does not include the time to write the response, only the query time. To get an accurate number for how long the whole query takes, check your app server logs
http://www.nabble.com/Re%3A-A-simple-Java-client-for-updating-and-searching-tf3890950.html

To get a quick response from solr, try rows=0 or a 404 path. (Of course, the speed will depend on you network connection speed between client-server)

[Ryan McKinley]

I'm integrating SOLR-20 with trunk now...

The basic stuff is no problem. I'm struggling with the best way to:

• included shared libraries (commons-io, etc)
• structure build.xml
• add tests that launch jetty
• package solrj (and dependencies)

I think the best approach is to commit my best effort and then have you
all review/fix/modify. Building a useful patch is unrealistic.

ryan

[Michael Young]

We are planning to replace our custom Lucene implementation with Solr in the next release of Liferay. This Java client would be extremely useful to us and we would like to see it in the next stable release. When do you anticipate this, or at least an alpha version?

[Ryan McKinley]

solr 1.2 was released ~1 week ago so the next official stable release is at least a few months out.

The solrj client is quite stable (I won't say that too strongly until more people are using it) and will be included in solr nightly builds. While I don't recommend using the solr server nightly builds, the client should be ok.

[Ryan McKinley]

Added to trunk... any new problems should get their own issue.

[Hoss Man]

This bug was modified as part of a bulk update using the criteria...

• Marked "Resolved" and "Fixed"
• Had no "Fix Version" versions
• Was listed in the CHANGES.txt for 1.3 as of today 2008-03-15

The Fix Version for all 29 issues found was set to 1.3, email notification was suppressed to prevent excessive email.

For a list of all the issues modified, search jira comments for this (hopefully) unique string: batch20070315hossman1