Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-7719

[Python][Dataset] Table equality check occasionally fails

    XMLWordPrintableJSON

    Details

      Description

      Failing build https://dev.azure.com/ursa-labs/crossbow/_build/results?buildId=5843&view=logs&j=0da5d1d9-276d-5173-c4c4-9d4d4ed14fdb&t=6c939d89-0d1a-51f2-8b30-091a7a82e98c&l=265

          def _check_dataset_from_path(path, table, **kwargs):
              import pathlib
          
              # pathlib object
              assert isinstance(path, pathlib.Path)
              dataset = ds.dataset(ds.source(path, **kwargs))
              assert dataset.schema.equals(table.schema, check_metadata=False)
              result = dataset.to_table()
              assert result.replace_schema_metadata().equals(table)
          
              # string path
              dataset = ds.dataset(ds.source(str(path), **kwargs))
              assert dataset.schema.equals(table.schema, check_metadata=False)
              result = dataset.to_table()
      >       assert result.replace_schema_metadata().equals(table)
      E       AssertionError: assert False
      E        +  where False = <built-in method equals of pyarrow.lib.Table object at 0x7fecacf0cab0>(pyarrow.Table\na: int64\nb: double)
      E        +    where <built-in method equals of pyarrow.lib.Table object at 0x7fecacf0cab0> = pyarrow.Table\na: int64\nb: double.equals
      E        +      where pyarrow.Table\na: int64\nb: double = <built-in method replace_schema_metadata of pyarrow.lib.Table object at 0x7fecacf0c990>()
      E        +        where <built-in method replace_schema_metadata of pyarrow.lib.Table object at 0x7fecacf0c990> = pyarrow.Table\na: int64\nb: double\nmetadata\n--------\nOrderedDict([(b'ARROW:schema',\n              b'/////7AAAAAQAAAAAAAK...AAQAAAAAAAB'\n              b'AiQAAAAUAAAABAAAAAAAAAAIAAwACAAHAAgAAAAAAAABQAAAAAEAAABhAAAA'\n              b'AAAAAA==')]).replace_schema_metadata
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                kszucs Krisztian Szucs
                Reporter:
                kszucs Krisztian Szucs
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 0.5h
                  0.5h