Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-4745

KafkaLZ4BlockOutputStream.java incorrectly finishes the last frame

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: 0.10.1.1
    • Fix Version/s: 0.10.1.1
    • Component/s: compression
    • Labels:
      None

      Description

      There is a scenario where by the delegated OutputStream does not call flush before close there will be missing data in the stream. The reason for this is the stream is actually marked close before it is actually flushed.

      The end mark is written before the flush, also the writeEndMark was finishing the stream so its redundant in this context to mark it finished. In my fork the 'finished=true' was removed from the 'writeEndMark' method.

          @Override
          public void close() throws IOException {
              if (!finished) {
                  writeEndMark();
                  flush();
                  finished = true;
              }
              if (out != null) {
                  out.close();
                  out = null;
              }
          }
      

      should be

          @Override
          public void close() throws IOException {
              if (!finished) {
                  // finish any pending data
                  writeBlock();
                  // write out the end mark
                  writeEndMark();
                  // mark the stream as finished
                  finished = true;
              }
              if (out != null) {
                  out.close();
                  out = null;
              }
          }
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                william.droste@arris.com Will Droste
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: