Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 2.4.0
    • Back End, General
    • None

    Description

      See the linked review comments for more details. The jist of it is that when an %ES; is in a delimiter, the associated DFA is not getting compiled correctly, and it actually will match a literal %ES;. There is code in DelimiterParser.scala that inefficiently looks for ES when a match isn't found, and accepts it as an empty match. Instead, the DFA scanner should be returning the %ES; as a successful match in the parse results, and then DelimiterParser.scala can use the same logic to determine which DFA matched. This would allow us to get rid of the hasES functions and be more efficient.

      Note that some things may be tricky because when scanning for a delimiter, we have to ignore the %ES;, since they'll match anything. We may need new iterators that ignore these.

      Attachments

        Activity

          slawrence Steve Lawrence added a comment -

          This is a fairly major change, only relates to parsing. The only negatives of not fixing this is 1) performance could be improved and 2) this could match a literal %ES; in the data. Since this release is focued on unparsing and it's very unlikely for data to have a literal %ES; in it, we'll punt this to the next release.

          slawrence Steve Lawrence added a comment - This is a fairly major change, only relates to parsing. The only negatives of not fixing this is 1) performance could be improved and 2) this could match a literal %ES; in the data. Since this release is focued on unparsing and it's very unlikely for data to have a literal %ES; in it, we'll punt this to the next release.
          mbeckerle Mike Beckerle added a comment -

          See related (or dup?) DFDL-1477

          mbeckerle Mike Beckerle added a comment - See related (or dup?) DFDL-1477
          slawrence Steve Lawrence added a comment -

          User reports that this also affects unparsing. An element with initiator="%ES;" is unparsing as a literal "%ES;"

          slawrence Steve Lawrence added a comment - User reports that this also affects unparsing. An element with initiator="%ES;" is unparsing as a literal "%ES;"
          mbeckerle Mike Beckerle added a comment -

          Note that DFDL Spec is being changed. %ES; is allowed as the only delimiter but only when lengthKind is NOT 'delimited', so we're not scanning for it (since there is nothing to scan for).

          mbeckerle Mike Beckerle added a comment - Note that DFDL Spec is being changed. %ES; is allowed as the only delimiter but only when lengthKind is NOT 'delimited', so we're not scanning for it (since there is nothing to scan for).
          slawrence Steve Lawrence added a comment -

          Fixed in commit 48ad264405690e15788fca74c14ed75b2fd952b4

          slawrence Steve Lawrence added a comment - Fixed in commit 48ad264405690e15788fca74c14ed75b2fd952b4
          dfthompson Dave Thompson added a comment -

          Closed due to no activity and no re-occurrence. If the issue re-occurs the ticket can be reopened or addressed in a new one.

          dfthompson Dave Thompson added a comment - Closed due to no activity and no re-occurrence. If the issue re-occurs the ticket can be reopened or addressed in a new one.

          People

            slawrence Steve Lawrence
            slawrence Steve Lawrence
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: