Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-10500

custom json loading is wrong when there are multiple toplevel docs w/ child docs

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 6.5
    • Fix Version/s: 6.6, 7.0
    • Component/s: update
    • Security Level: Public (Default Security Level. Issues are Public)
    • Labels:
      None

      Description

      Sending two parent docs with nested children in custom JSON leads to hijacking neighbor's children. As a workaround, every parent can be sent separately.
      Let me better describe the problem:

      curl 'http://localhost:8983/solr/my_collection/update/json/docs?split=/|/orgs'\
          -H 'Content-type:application/json' -d '[{
        "name": "Joe Smith", "phone": 876876687, "orgs": [
          {  "name": "Microsoft", "city": "Seattle", "zip": 98052 },
          {   "name": "Apple", "city": "Cupertino", "zip": 95014 }
        ]
       },
      {
        "name": "Frank Miller", "phone": 1234556, "orgs": [
          {  "name": "Oracle", "city": "Redwood", "zip": 98052 },
          {   "name": "HTC", "city": "San Francisco", "zip": 95014 }
        ]
       }
      ] '
      

      produces docs with accumulating children across parents

      {
        "name":"Joe Smith", "phone":876876687,
        "_childDocuments_":[
          { "name":"Microsoft", "city":"Seattle","zip":98052},
          {"name":"Apple","city":"Cupertino","zip":95014}]},
      {
        "name":"Frank Miller", "phone":1234556,
        "_childDocuments_":[
          { "name":"Microsoft", "city":"Seattle","zip":98052},
          {"name":"Apple","city":"Cupertino","zip":95014},
            {  "name": "Oracle", "city": "Redwood", "zip": 98052 },
          {   "name": "HTC", "city": "San Francisco", "zip": 95014 }]}
      
      1. 0001-SOLR-10500-fixed-json-hierarchy-read.patch
        4 kB
        Alexey Suprun
      2. SOLR-10500.patch
        7 kB
        Noble Paul
      3. SOLR-10500.patch
        8 kB
        Mikhail Khludnev
      4. SOLR-10500-test.patch
        2 kB
        Mikhail Khludnev

        Issue Links

          Activity

          Hide
          mkhludnev Mikhail Khludnev added a comment -

          Attaching test extension to reproduce SOLR-10500-test.patch. Failure is exactly the same what I've seen during a manual test today.

          java.lang.AssertionError: expected:<2> but was:<4>
          	at 
          org.apache.solr.handler.JsonLoaderTest.testJsonDocFormat(JsonLoaderTest.java:446)
          

          the line 446 is

                  assertEquals(2, p.addCommands.get(i).solrDoc.getChildDocuments().size());
          

          both parent are expected to have 2 children, but one of them got two.

          Show
          mkhludnev Mikhail Khludnev added a comment - Attaching test extension to reproduce SOLR-10500-test.patch . Failure is exactly the same what I've seen during a manual test today. java.lang.AssertionError: expected:<2> but was:<4> at org.apache.solr.handler.JsonLoaderTest.testJsonDocFormat(JsonLoaderTest.java:446) the line 446 is assertEquals(2, p.addCommands.get(i).solrDoc.getChildDocuments().size()); both parent are expected to have 2 children, but one of them got two.
          Hide
          alexey_suprun@epam.com Alexey Suprun added a comment -

          Fixed the issue. The solution attached in 0001-SOLR-10500-fixed-json-hierarchy-read.patch

          Show
          alexey_suprun@epam.com Alexey Suprun added a comment - Fixed the issue. The solution attached in 0001- SOLR-10500 -fixed-json-hierarchy-read.patch
          Hide
          mkhludnev Mikhail Khludnev added a comment -

          Alexey Suprun, thanks for the patch. I'll add asserts and commit it this week.
          Noble Paul, are you ok with the fix?

          Show
          mkhludnev Mikhail Khludnev added a comment - Alexey Suprun , thanks for the patch. I'll add asserts and commit it this week. Noble Paul , are you ok with the fix?
          Hide
          noble.paul Noble Paul added a comment -

          I'll be able to get to this on Monday only

          Show
          noble.paul Noble Paul added a comment - I'll be able to get to this on Monday only
          Hide
          mkhludnev Mikhail Khludnev added a comment -

          SOLR-10500.patch also asserts nested values and encompasses Alexey Suprun's fix.

          Show
          mkhludnev Mikhail Khludnev added a comment - SOLR-10500.patch also asserts nested values and encompasses Alexey Suprun 's fix.
          Hide
          noble.paul Noble Paul added a comment -

          Don't commit this fix. This creates more objects than it should. We can't create new maps . I'll get to this is a couple of days

          Show
          noble.paul Noble Paul added a comment - Don't commit this fix. This creates more objects than it should. We can't create new maps . I'll get to this is a couple of days
          Hide
          noble.paul Noble Paul added a comment -

          fixed

          Show
          noble.paul Noble Paul added a comment - fixed
          Hide
          mkhludnev Mikhail Khludnev added a comment -

          I like it. Do you like me to commit or take care yourself?

          Show
          mkhludnev Mikhail Khludnev added a comment - I like it. Do you like me to commit or take care yourself?
          Hide
          noble.paul Noble Paul added a comment -

          Go ahead and commit it

          Show
          noble.paul Noble Paul added a comment - Go ahead and commit it
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 61d02a47e205bbf6c18058853b93466e6cfbd51b in lucene-solr's branch refs/heads/master from Mikhail Khludnev
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=61d02a4 ]

          SOLR-10500: fix many parents with nested children per /update/json/docs request

          Show
          jira-bot ASF subversion and git services added a comment - Commit 61d02a47e205bbf6c18058853b93466e6cfbd51b in lucene-solr's branch refs/heads/master from Mikhail Khludnev [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=61d02a4 ] SOLR-10500 : fix many parents with nested children per /update/json/docs request
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 1791f6924a063867e04b53fc85bfbef0f26af2c7 in lucene-solr's branch refs/heads/branch_6x from Mikhail Khludnev
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=1791f69 ]

          SOLR-10500: fix many parents with nested children per /update/json/docs request

          Show
          jira-bot ASF subversion and git services added a comment - Commit 1791f6924a063867e04b53fc85bfbef0f26af2c7 in lucene-solr's branch refs/heads/branch_6x from Mikhail Khludnev [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=1791f69 ] SOLR-10500 : fix many parents with nested children per /update/json/docs request

            People

            • Assignee:
              Unassigned
              Reporter:
              mkhludnev Mikhail Khludnev
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development