Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-8707 Implement an async pure c++ HDFS client
  3. HDFS-10679

libhdfs++: Implement parallel find with wildcards tool

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Resolved
    • None
    • None
    • hdfs-client
    • None

    Description

      The find tool will issue the GetListing namenode operation on a given directory, and filter the results using posix globbing library.

      If the recursive option is selected, for each returned entry that is a directory the tool will issue another asynchronous call GetListing and repeat the result processing in a recursive fashion.

      One implementation issue that needs to be addressed is the way how results are returned back to the user: we can either buffer the results and return them to the user in bulk, or we can return results continuously as they arrive. While buffering would be an easier solution, returning results as they arrive would be more beneficial to the user in terms of performance, since the result processing can start as soon as the first results arrive without any delay. In order to do that we need the user to use a loop to process arriving results, and we need to send a special message back to the user when the search is over.

      Attachments

        1. HDFS-10679.HDFS-8707.013.patch
          35 kB
          Anatoli Shein
        2. HDFS-10679.HDFS-8707.012.patch
          34 kB
          Anatoli Shein
        3. HDFS-10679.HDFS-8707.011.patch
          34 kB
          Anatoli Shein
        4. HDFS-10679.HDFS-8707.010.patch
          33 kB
          Anatoli Shein
        5. HDFS-10679.HDFS-8707.009.patch
          28 kB
          Anatoli Shein
        6. HDFS-10679.HDFS-8707.008.patch
          28 kB
          Anatoli Shein
        7. HDFS-10679.HDFS-8707.007.patch
          28 kB
          Anatoli Shein
        8. HDFS-10679.HDFS-8707.006.patch
          28 kB
          Anatoli Shein
        9. HDFS-10679.HDFS-8707.005.patch
          34 kB
          Anatoli Shein
        10. HDFS-10679.HDFS-8707.004.patch
          32 kB
          Anatoli Shein
        11. HDFS-10679.HDFS-8707.003.patch
          27 kB
          Anatoli Shein
        12. HDFS-10679.HDFS-8707.002.patch
          27 kB
          Anatoli Shein
        13. HDFS-10679.HDFS-8707.001.patch
          27 kB
          Anatoli Shein
        14. HDFS-10679.HDFS-8707.000.patch
          20 kB
          Anatoli Shein

        Activity

          People

            anatoli.shein Anatoli Shein
            anatoli.shein Anatoli Shein
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: