Uploaded image for project: 'Apache Storm'
  1. Apache Storm
  2. STORM-2358

Update storm hdfs spout to remove specific implementation handlings

Attach filesAttach ScreenshotAdd voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

      Description

      I was looking at storm hdfs spout code in 1.x branch, I found below
      improvements can be made in below code.

      1. Make org.apache.storm.hdfs.spout.AbstractFileReader as public so
      that it can be used in generics.

      2. org.apache.storm.hdfs.spout.HdfsSpout requires readerType as
      String. It will be great to have class<? extends AbstractFileReader>
      readerType; So we will not use Class.forName at multiple places also it
      will help in below point.

      3. HdfsSpout also needs to provide outFields which are declared as
      constants in each reader(e.g.SequenceFileReader). We can have abstract
      API AbstractFileReader in which return them to user to make it generic.

        Attachments

        1. AbstractFileReader.java
          2 kB
          Sachin Pasalkar
        2. FileReader.java
          1 kB
          Sachin Pasalkar
        3. HDFSSpout.java
          25 kB
          Sachin Pasalkar
        4. SequenceFileReader.java
          7 kB
          Sachin Pasalkar
        5. TextFileReader.java
          5 kB
          Sachin Pasalkar

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              Sachin Sachin Pasalkar

              Dates

              • Created:
                Updated:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 4h 50m
                4h 50m

                  Issue deployment