Uploaded image for project: 'Apache Gobblin'
  1. Apache Gobblin
  2. GOBBLIN-2113

Process HB messages with empty FlowExecutionId str

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • gobblin-service
    • None

    Description

      After changing flowExecutionID from a Str to a long in a previous PR, we encounter NumberFormatException in the DagActionStoreChangeMonitor when processing HB events. This ends up killing the HighLevelConsumer queues for the hosts that receive the HB events in their partition. 

      {{Encountered exception while processing record so stopping queue processing. Record: LiKafka10ConsumerRecord(consumerRecord=ConsumerRecord(topic = ds_mysql_makto-db-152_prod_SHARED_GOBBLIN_DAG_ACTION_STORE_20221208211255, partition = 0, leaderEpoch = null, offset = 905733, NoTimestampType = -1, serialized key size = -1, serialized value size = -1, headers = RecordHeaders(headers = [], isReadOnly = false), key = , value = {"changeEventIdentifier":

      {"key": "", "txId": "", "produceTimestampMillis": 1721078158347, "operationType": "HEARTBEAT"}

      , "flowGroup": "", "flowName": "", "flowExecutionId": "", "jobName": "", "dagAction": null})) Exception: java.lang.NumberFormatException: For input string: ""}}

      Attachments

        Activity

          People

            abti Abhishek Tiwari
            umustafi Urmi Mustafi
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 1h 20m
                1h 20m