Uploaded image for project: 'Parquet'
  1. Parquet
  2. PARQUET-1948

TransCompressionCommand Inoperable

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Blocker
    • Resolution: Unresolved
    • Affects Version/s: 1.11.1
    • Fix Version/s: None
    • Component/s: parquet-mr
    • Labels:
    • Environment:

      I am using parquet-tools 1.11.1 on a Mac machine running Catalina, and my parquet-tools jar was downloaded from Maven Central. 

      Description

      TransCompressionCommand in parquet-tools is intended to allow translation of compression types in parquet files.  We are intending to use this functionality to debug a corrupted file, but this command fails to run at the moment entirely. 

      Running the following command (on the uncorrupted file):

      java -jar ./parquet-tools-1.11.1.jar trans-compression ~/Downloads/part-00048-69f65188-94b5-4772-8906-5c78989240b5_00048.c000.snappy.parquet

      This results in 

       

      Unknown command: trans-compression

       

      I believe this is due to the Registry class silently catching any errors to initialize which subsequently is misinterpreted as an unknown command.

      We need to: 

      1. Write a test for the TransCompressionCommand to figure out why it's showing up as unknown command
      2. Probably expand these tests to cover all the other commands

       

      This will then unblock our debugging work on the suspect file. 

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              vanhooser Shelby Vanhooser
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: