Solr
  1. Solr
  2. SOLR-2668

DIH - multithreaded does not rollback on errors from EntityProcessor

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 3.3
    • Fix Version/s: None
    • Labels:
      None

      Description

      If the EntityProcessor of a subentity throws an Exception in its init() Method, DocBuilder logs "Import completed successfully" and calls commit instead of calling rollback.

      1. SOLR-2668.patch
        7 kB
        Shalin Shekhar Mangar
      2. SOLR-2668.patch
        6 kB
        Shalin Shekhar Mangar
      3. SOLR-2668.patch
        5 kB
        Frank Wesemann

        Activity

        Hide
        Frank Wesemann added a comment -

        patch for the unittest to clarify the problem

        Show
        Frank Wesemann added a comment - patch for the unittest to clarify the problem
        Hide
        Shalin Shekhar Mangar added a comment -

        Thanks for the test Frank. There are more problems. In single thread mode, an error causes a rollback but in threaded mode, a commit is called.

        Show
        Shalin Shekhar Mangar added a comment - Thanks for the test Frank. There are more problems. In single thread mode, an error causes a rollback but in threaded mode, a commit is called.
        Hide
        Shalin Shekhar Mangar added a comment -

        I was looking at this problem again today. The onError attributes are not used to deal with exceptions from EntityProcessor.init() method. They are only used for reading rows, applying transformers and inserting documents into Solr.

        The real problem was that in multi-threaded mode, the exceptions from EntityProcessor.init were being eaten up so a commit was called instead of rolling back the changes. I've fixed that to re-throw the exception up the hierarchy.

        Show
        Shalin Shekhar Mangar added a comment - I was looking at this problem again today. The onError attributes are not used to deal with exceptions from EntityProcessor.init() method. They are only used for reading rows, applying transformers and inserting documents into Solr. The real problem was that in multi-threaded mode, the exceptions from EntityProcessor.init were being eaten up so a commit was called instead of rolling back the changes. I've fixed that to re-throw the exception up the hierarchy.
        Hide
        Shalin Shekhar Mangar added a comment -

        Fixed a bug in the tests. The index was not being emptied in tearDown so even after rollback, the index had 4 documents.

        Just to show that multi-threaded does honor the onError attribute, I've added a test which uses a transformer which always fails.

        The older two tests are used to demonstrate that both multi-threaded and serial mode show the same characteristic for a failing entity processor.

        Show
        Shalin Shekhar Mangar added a comment - Fixed a bug in the tests. The index was not being emptied in tearDown so even after rollback, the index had 4 documents. Just to show that multi-threaded does honor the onError attribute, I've added a test which uses a transformer which always fails. The older two tests are used to demonstrate that both multi-threaded and serial mode show the same characteristic for a failing entity processor.
        Hide
        Shalin Shekhar Mangar added a comment -

        Committed revision 1161505 on trunk and 1161510 on branch_3x.

        Show
        Shalin Shekhar Mangar added a comment - Committed revision 1161505 on trunk and 1161510 on branch_3x.

          People

          • Assignee:
            Shalin Shekhar Mangar
            Reporter:
            Frank Wesemann
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development