Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • HADOOP-13345
    • fs/s3
    • None

    Description

      I think we need to rework the S3GuardTool options. A couple of problems I've observed in the patches I've done on top of that and seeing other developers trying it out:

      • We should probably wrap the current commands in an S3Guard-specific command, since 'init', 'destroy', etc. don't touch the buckets at all.
      • Convert to whole-word options, as the single-letter options are already getting overloaded. Some patches I've submitted have added functionality where the obvious flag is already in use (e.g. -r for region, and read throughput, -m for minutes, and metadatastore uri). I may do this early as part of HADOOP-14090.
      • We have some options that must be in the config in some cases, and can be in the command in other cases. But I've seen someone try to specify the table name in the config and leave out the -m option, with no luck. Also, since commands hard-code table auto-creation, you might have configured table auto-creation, try to import to a non-existent table, and it tells you table auto-creation is off.

      We need a more consistent policy for how things should get configured that addresses these problems and future-proofs the command a bit more.

      Attachments

        1. HADOOP-14094-HADOOP-13345.007.patch
          26 kB
          Sean Mackrory
        2. HADOOP-14094-HADOOP-13345.006.patch
          26 kB
          Sean Mackrory
        3. HADOOP-14094-HADOOP-13345.005.patch
          26 kB
          Sean Mackrory
        4. HADOOP-14094-HADOOP-13345.004.patch
          26 kB
          Sean Mackrory
        5. HADOOP-14094-HADOOP-13345.003.patch
          25 kB
          Steve Loughran
        6. HADOOP-14094-HADOOP-13345.003.patch
          25 kB
          Sean Mackrory
        7. HADOOP-14094-HADOOP-13345.002.patch
          23 kB
          Sean Mackrory
        8. HADOOP-14094-HADOOP-13345.001.patch
          20 kB
          Sean Mackrory

        Issue Links

          Activity

            People

              mackrorysd Sean Mackrory
              mackrorysd Sean Mackrory
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: