Uploaded image for project: 'Apache Storm'
  1. Apache Storm
  2. STORM-2358

Update storm hdfs spout to remove specific implementation handlings

    XMLWordPrintableJSON

Details

    Description

      I was looking at storm hdfs spout code in 1.x branch, I found below
      improvements can be made in below code.

      1. Make org.apache.storm.hdfs.spout.AbstractFileReader as public so
      that it can be used in generics.

      2. org.apache.storm.hdfs.spout.HdfsSpout requires readerType as
      String. It will be great to have class<? extends AbstractFileReader>
      readerType; So we will not use Class.forName at multiple places also it
      will help in below point.

      3. HdfsSpout also needs to provide outFields which are declared as
      constants in each reader(e.g.SequenceFileReader). We can have abstract
      API AbstractFileReader in which return them to user to make it generic.

      Attachments

        1. AbstractFileReader.java
          2 kB
          Sachin Pasalkar
        2. FileReader.java
          1 kB
          Sachin Pasalkar
        3. HDFSSpout.java
          25 kB
          Sachin Pasalkar
        4. SequenceFileReader.java
          7 kB
          Sachin Pasalkar
        5. TextFileReader.java
          5 kB
          Sachin Pasalkar

        Issue Links

          Activity

            People

              Unassigned Unassigned
              Sachin Sachin Pasalkar
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 4h 50m
                  4h 50m