Uploaded image for project: 'Apache Storm'
  1. Apache Storm
  2. STORM-2358

Update storm hdfs spout to remove specific implementation handlings

    XMLWordPrintableJSON

    Details

      Description

      I was looking at storm hdfs spout code in 1.x branch, I found below
      improvements can be made in below code.

      1. Make org.apache.storm.hdfs.spout.AbstractFileReader as public so
      that it can be used in generics.

      2. org.apache.storm.hdfs.spout.HdfsSpout requires readerType as
      String. It will be great to have class<? extends AbstractFileReader>
      readerType; So we will not use Class.forName at multiple places also it
      will help in below point.

      3. HdfsSpout also needs to provide outFields which are declared as
      constants in each reader(e.g.SequenceFileReader). We can have abstract
      API AbstractFileReader in which return them to user to make it generic.

        Attachments

        1. SequenceFileReader.java
          7 kB
          Sachin Pasalkar
        2. TextFileReader.java
          5 kB
          Sachin Pasalkar
        3. AbstractFileReader.java
          2 kB
          Sachin Pasalkar
        4. FileReader.java
          1 kB
          Sachin Pasalkar
        5. HDFSSpout.java
          25 kB
          Sachin Pasalkar

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                Sachin Sachin Pasalkar
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 4h 50m
                  4h 50m