Solr
  1. Solr
  2. SOLR-5003

Add option to add rowid/line number to CSV Update Handler

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.4, 5.0
    • Component/s: None
    • Labels:
      None

      Description

      In some cases of exporting from a DB to CSV, the only "unique id" you have is the rowid. This issue is to add an optional (off by default) rowid field to the document which simply contains the line number of the row.

      1. SOLR-5003.patch
        4 kB
        Grant Ingersoll

        Activity

        Hide
        Grant Ingersoll added a comment -

        Pretty straightforward

        Show
        Grant Ingersoll added a comment - Pretty straightforward
        Hide
        Yonik Seeley added a comment -
        +    //validate our row id
        +    if (rowId != null && rowId.equals("") == false){
        +      SchemaField sf = schema.getFieldOrNull(rowId);
        +      if(sf == null)
        +        throw new SolrException( SolrException.ErrorCode.BAD_REQUEST,"Invalid field name for rowId:'"+ rowId +"'");
        +    }
        

        In general, we should let downstream handle this type of stuff so things like schemaless will work.

        Show
        Yonik Seeley added a comment - + //validate our row id + if (rowId != null && rowId.equals("") == false ){ + SchemaField sf = schema.getFieldOrNull(rowId); + if (sf == null ) + throw new SolrException( SolrException.ErrorCode.BAD_REQUEST, "Invalid field name for rowId:'" + rowId + "'" ); + } In general, we should let downstream handle this type of stuff so things like schemaless will work.
        Hide
        ASF subversion and git services added a comment -

        Commit 1500046 from Grant Ingersoll
        [ https://svn.apache.org/r1500046 ]

        SOLR-5003: add rowid (line number) option to CSV Loader

        Show
        ASF subversion and git services added a comment - Commit 1500046 from Grant Ingersoll [ https://svn.apache.org/r1500046 ] SOLR-5003 : add rowid (line number) option to CSV Loader
        Hide
        ASF subversion and git services added a comment -
        Show
        ASF subversion and git services added a comment - Commit 1500049 from Grant Ingersoll [ https://svn.apache.org/r1500049 ] SOLR-5003 : merge
        Hide
        Grant Ingersoll added a comment -

        Going to add a rowid_offset as well, in case people want to use this in connection with more than one file

        Show
        Grant Ingersoll added a comment - Going to add a rowid_offset as well, in case people want to use this in connection with more than one file
        Hide
        ASF subversion and git services added a comment -

        Commit 1500097 from Grant Ingersoll
        [ https://svn.apache.org/r1500097 ]

        SOLR-5003: add rowidOffset as well

        Show
        ASF subversion and git services added a comment - Commit 1500097 from Grant Ingersoll [ https://svn.apache.org/r1500097 ] SOLR-5003 : add rowidOffset as well
        Hide
        ASF subversion and git services added a comment -

        Commit 1500098 from Grant Ingersoll
        [ https://svn.apache.org/r1500098 ]

        SOLR-5003: merge to 4x

        Show
        ASF subversion and git services added a comment - Commit 1500098 from Grant Ingersoll [ https://svn.apache.org/r1500098 ] SOLR-5003 : merge to 4x
        Hide
        Steve Rowe added a comment -

        Bulk close resolved 4.4 issues

        Show
        Steve Rowe added a comment - Bulk close resolved 4.4 issues

          People

          • Assignee:
            Grant Ingersoll
            Reporter:
            Grant Ingersoll
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development