Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-25398

Minor bugs from comparing unrelated types

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 2.3.1
    • 2.4.0
    • Mesos, Spark Core, YARN
    • None

    Description

      I noticed a potential issue from Scala inspections, like this clause in LiveEntity.scala around line 586:

       (!acc.metadata.isDefined ||
        acc.metadata.get != Some(AccumulatorContext.SQL_ACCUM_IDENTIFIER))

      The issue is that acc.metadata is Option[String], so can't equal Some[String]. This just meant to be:

       acc.metadata != Some(AccumulatorContext.SQL_ACCUM_IDENTIFIER)

      This may or may not actually cause a bug, but seems worth fixing. And then there are a number of other ones like this, mostly in tests, that might likewise mask real assertion problems.

      Many are, interestingly, flagging items like this on a Seq[String]:

      .filter(_.getFoo.equals("foo"))

      It complains that Any => Any is compared to String. Either it's wrong, or somehow, this is parsed as (_.getFoo).equals("foo")). In any event, easy enough to write this more clearly as:

      .filter(_.getFoo == "foo")

      And so on.

      Attachments

        Activity

          People

            srowen Sean R. Owen
            srowen Sean R. Owen
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: