Uploaded image for project: 'Parquet'
  1. Parquet
  2. PARQUET-241

ParquetInputFormat.getFooters() should return in the same order as what listStatus() returns

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.6.0
    • 1.9.0, 1.8.2
    • None
    • None

    Description

      Because of how the footer cache is implemented, getFooters() returns the footers in a different order than what listStatus() returns.

      When I provided url "hdfs://.../part-00001.parquet,hdfs://.../part-00002.parquet,hdfs://.../part-00003.parquet", ParquetInputFormat.getSplits(), which internally calls getFooters(), returned the splits in a wrong order.

      Attachments

        Issue Links

          Activity

            People

              mkim Mingyu Kim
              mkim Mingyu Kim
              Votes:
              2 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: