Uploaded image for project: 'Parquet'
  1. Parquet
  2. PARQUET-2017

Handle special values of floating point statistics

Attach filesAttach ScreenshotAdd voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: parquet-mr
    • Labels:
      None

      Description

      Based on PARQUET-1251 we have implemented the suggested workaround but it is not complete in all situations.

      • We handle the special floating point values at row-group level in parquet-mr but only for the read path. For the write path we still write these values.
      • For column indexes we handle these values but only for the write path and not for the read path.

      We should implement the workaround for both read and write paths for all cases so we not only handle potentially invalid values but also don't write them to the file.

        Attachments

          Activity

            People

            • Assignee:
              gszadovszky Gabor Szadovszky
              Reporter:
              gszadovszky Gabor Szadovszky

              Dates

              • Created:
                Updated:

                Issue deployment