Solr
  1. Solr
  2. SOLR-2851

Allow Data Import Handler to Archive Files After Processing

    Details

      Description

      It would be great if, after processing flat files, DIH could place them in an archive folder so that other system processes could compress them, delete them, etc. We needed this feature, so I extended FileDataSource and decorated the InputStreamReader such that when the close method is called, the file is moved out of the way. This concrete implementation did exactly what I needed. However, a more correct textbook approach would involve refactoring FileListEntityProcessor such that it could take a configurable list of post-processing action policy/command classes so that the user could provide custom behavior that suits nearly any scenario (in a Chain of Responsibility style pattern). Nonetheless, I wanted to provide the suggestion, a basic one-off concrete implementation of archiving and start the discussion for where things might head in the future.

        Activity

        Hide
        Josh Harness added a comment -

        Custom file data source that archives files when the close method on the input stream reader is called. Java package name is still specific to my company and needs to be modified to reflect the apache structure. Also, tests are needed.

        Show
        Josh Harness added a comment - Custom file data source that archives files when the close method on the input stream reader is called. Java package name is still specific to my company and needs to be modified to reflect the apache structure. Also, tests are needed.

          People

          • Assignee:
            Unassigned
            Reporter:
            Josh Harness
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:

              Development