Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-10505

[Python] Improve validation of cython arguments

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 2.0.0
    • None
    • Python
    • None

    Description

      Typing parameters in cython functions (for example

      def validate(self, Schema schema not None):
      

      )
      is intuitive but provides uninformative error messages when the type doesn't match expectations. See FileSystemDataset.from_paths (ARROW-8290) for an instance of manual validation to make a high traffic function more friendly.

              for arg, class_, name in [
                  (schema, Schema, 'schema'),
                  (format, FileFormat, 'format'),
                  (filesystem, FileSystem, 'filesystem'),
                  (root_partition, Expression, 'root_partition')
              ]:
                  if not isinstance(arg, class_):
                      raise TypeError(
                          "Argument '{0}' has incorrect type (expected {1}, "
                          "got {2})".format(name, class_.__name__, type(arg))
                      )
      

      It seems we could do better by applying a validating decorator:

      @param_types(schema=Schema, format=Format, filesystem=FileSystem,
                   root_partition=(None,Expression))
      def from_paths(paths, schema, format,
                     filesystem, partitions, root_partition):
      

      This seems compact enough to add to any or all def functions, improving error messages and hiding validation boilerplate.

      Attachments

        Activity

          People

            Unassigned Unassigned
            bkietz Ben Kietzman
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated: