Uploaded image for project: 'Apache Hop (incubating)'
  1. Apache Hop (incubating)
  2. HOP-2266

Beam generating unbounded synthetic data too fast

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Normal
    • Resolution: Fixed
    • Affects Version/s: 0.50
    • Fix Version/s: 0.50
    • Component/s: Beam
    • Labels:
      None

      Description

      The "Generate Rows" transforms in Hop is translated to Beam using the 

      SyntheticUnboundedSource from SyntheticIO.

      // The processing time delay is a random value between 0 and intervalMs so we need to double this to get the same speed on average.
      //
      options.nextProcessingTimeDelay( intervalMs * 2 );

       

      Apparently this is no longer the case.  We should investigate and at the very least document the discrepancies.
      Worst case scenario we file a bug report against the Beam IO.

       

        Attachments

        1. HOP-2266.zip
          5 kB
          Matt Casters

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              mcasters Matt Casters
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: