Uploaded image for project: 'Crunch'
  1. Crunch
  2. CRUNCH-650

Support case classes and tuples without reflection or codegen using shapeless

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Scrunch
    • Labels:
      None

      Description

      Scrunch support for case classes is pretty buggy, only supports a subset of scala versions, and fails at runtime instead of compile time if something goes wrong. Additionally, the support of tuple types via code generation is awkward and unnecessary.

      This patch should hopefully solve both of these with a shapeless (https://github.com/milessabin/shapeless) based solution which should be much simpler and more reliable, by defining a mechanism for deriving PTypeH instances for HLists where PTypeH instances for each type in the list is known.

      We have used it in a few production cases at SC for a while, and the unit and integration test suite passes, but I didn't write any new tests specifically for this purpose. Hopefully they should be covered by the existing tests using case classes and tuples.

        Activity

        Hide
        jwills Josh Wills added a comment -

        Hey David Whiting is super-nice, thanks. My concern with shapeless has always been the extra JAR dependency, esp. since folks use it so heavily downstream of the project, and I wasn't sure how good the backwards compatibility story was there (i.e., I would like to avoid another Guava nightmare.) What do you think?

        Show
        jwills Josh Wills added a comment - Hey David Whiting is super-nice, thanks. My concern with shapeless has always been the extra JAR dependency, esp. since folks use it so heavily downstream of the project, and I wasn't sure how good the backwards compatibility story was there (i.e., I would like to avoid another Guava nightmare.) What do you think?
        Hide
        davw David Whiting added a comment -

        Are you worried about transient deps or shapeless itself? There's only one transient dependency on macro-compat, but shapeless itself could be awkward wrt Scala version numbering, as we don't build different versions of Scrunch for different versions of Scala. If we can somehow fix that, and do _2.10, _2.11 and _2.12 builds of scrunch then I think this is a safe addition, but without that perhaps not.

        Show
        davw David Whiting added a comment - Are you worried about transient deps or shapeless itself? There's only one transient dependency on macro-compat, but shapeless itself could be awkward wrt Scala version numbering, as we don't build different versions of Scrunch for different versions of Scala. If we can somehow fix that, and do _2.10, _2.11 and _2.12 builds of scrunch then I think this is a safe addition, but without that perhaps not.
        Hide
        jwills Josh Wills added a comment -

        Mostly shapeless itself, but I'm not super-familiar with it's use across the different versions of Scala. Is there a "safe" version that we can depend on across versions, or for a specific version? I do not question that the current approach is janky af and needs to be replaced.

        Show
        jwills Josh Wills added a comment - Mostly shapeless itself, but I'm not super-familiar with it's use across the different versions of Scala. Is there a "safe" version that we can depend on across versions, or for a specific version? I do not question that the current approach is janky af and needs to be replaced.

          People

          • Assignee:
            davw David Whiting
            Reporter:
            davw David Whiting
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:

              Development