Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-6167

Timestamp on streams directory contains a colon, which is an illegal character

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 1.0.0
    • Fix Version/s: 1.0.1, 1.1.0
    • Component/s: streams
    • Labels:
    • Environment:
      AK 1.0.0
      Kubernetes
      CoreOS
      JDK 1.8
      Windows

      Description

      Problem:

      Development on Windows, which is not fully supported, however still a bug that should be corrected.

      It looks like a timestamp was added to the streams directory using a colon as separator. I believe this is an illegal character and potentially the cause for the exception below.

      Error Stack:

      2017-11-02 16:06:41 ERROR [StreamDeduplicatorAcceptanceTest1-a3ae0ac6-a024-4006-bcb1-01ff0f433f6e-StreamThread-1] org.apache.kafka.streams.processor.internals.AssignedTasks:301 - stream-thread [StreamDeduplicatorAcceptanceTest1-a3ae0ac6-a024-4006-bcb1-01ff0f433f6e-StreamThread-1] Failed to process stream task 0_0 due to the following error:
      org.apache.kafka.streams.errors.StreamsException: Exception caught in process. taskId=0_0, processor=KSTREAM-SOURCE-0000000000, topic=input-a_1, partition=0, offset=0
      at org.apache.kafka.streams.processor.internals.StreamTask.process(StreamTask.java:232) ~[kafka-streams-1.0.0.jar:?]
      at org.apache.kafka.streams.processor.internals.AssignedTasks.process(AssignedTasks.java:403) [kafka-streams-1.0.0.jar:?]
      at org.apache.kafka.streams.processor.internals.TaskManager.process(TaskManager.java:317) [kafka-streams-1.0.0.jar:?]
      at org.apache.kafka.streams.processor.internals.StreamThread.processAndMaybeCommit(StreamThread.java:942) [kafka-streams-1.0.0.jar:?]
      at org.apache.kafka.streams.processor.internals.StreamThread.runOnce(StreamThread.java:822) [kafka-streams-1.0.0.jar:?]
      at org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:774) [kafka-streams-1.0.0.jar:?]
      at org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:744) [kafka-streams-1.0.0.jar:?]
      Caused by: org.apache.kafka.streams.errors.ProcessorStateException: Error opening store KSTREAM-JOINTHIS-0000000004-store:1509624000000 at location C:\Users\ADRIAN~1.MCC\AppData\Local\Temp\kafka3548813472740086814\StreamDeduplicatorAcceptanceTest1\0_0\KSTREAM-JOINTHIS-0000000004-store\KSTREAM-JOINTHIS-0000000004-store:1509624000000
      at org.apache.kafka.streams.state.internals.RocksDBStore.openDB(RocksDBStore.java:204) ~[kafka-streams-1.0.0.jar:?]
      at org.apache.kafka.streams.state.internals.RocksDBStore.openDB(RocksDBStore.java:174) ~[kafka-streams-1.0.0.jar:?]
      at org.apache.kafka.streams.state.internals.Segment.openDB(Segment.java:40) ~[kafka-streams-1.0.0.jar:?]
      at org.apache.kafka.streams.state.internals.Segments.getOrCreateSegment(Segments.java:89) ~[kafka-streams-1.0.0.jar:?]
      at org.apache.kafka.streams.state.internals.RocksDBSegmentedBytesStore.put(RocksDBSegmentedBytesStore.java:81) ~[kafka-streams-1.0.0.jar:?]
      at org.apache.kafka.streams.state.internals.RocksDBWindowStore$RocksDBWindowBytesStore.put(RocksDBWindowStore.java:43) ~[kafka-streams-1.0.0.jar:?]
      at org.apache.kafka.streams.state.internals.RocksDBWindowStore$RocksDBWindowBytesStore.put(RocksDBWindowStore.java:34) ~[kafka-streams-1.0.0.jar:?]
      at org.apache.kafka.streams.state.internals.ChangeLoggingWindowBytesStore.put(ChangeLoggingWindowBytesStore.java:67) ~[kafka-streams-1.0.0.jar:?]
      at org.apache.kafka.streams.state.internals.ChangeLoggingWindowBytesStore.put(ChangeLoggingWindowBytesStore.java:33) ~[kafka-streams-1.0.0.jar:?]
      at org.apache.kafka.streams.state.internals.MeteredWindowStore.put(MeteredWindowStore.java:96) ~[kafka-streams-1.0.0.jar:?]
      at org.apache.kafka.streams.state.internals.MeteredWindowStore.put(MeteredWindowStore.java:89) ~[kafka-streams-1.0.0.jar:?]
      at org.apache.kafka.streams.kstream.internals.KStreamJoinWindow$KStreamJoinWindowProcessor.process(KStreamJoinWindow.java:64) ~[kafka-streams-1.0.0.jar:?]
      at org.apache.kafka.streams.processor.internals.ProcessorNode$1.run(ProcessorNode.java:46) ~[kafka-streams-1.0.0.jar:?]
      at org.apache.kafka.streams.processor.internals.StreamsMetricsImpl.measureLatencyNs(StreamsMetricsImpl.java:208) ~[kafka-streams-1.0.0.jar:?]
      at org.apache.kafka.streams.processor.internals.ProcessorNode.process(ProcessorNode.java:124) ~[kafka-streams-1.0.0.jar:?]
      at org.apache.kafka.streams.processor.internals.ProcessorContextImpl.forward(ProcessorContextImpl.java:85) ~[kafka-streams-1.0.0.jar:?]
      at org.apache.kafka.streams.processor.internals.SourceNode.process(SourceNode.java:80) ~[kafka-streams-1.0.0.jar:?]
      at org.apache.kafka.streams.processor.internals.StreamTask.process(StreamTask.java:216) ~[kafka-streams-1.0.0.jar:?]
      ... 6 more
      Caused by: org.rocksdb.RocksDBException: Failed to create dir: C:\Users\ADRIAN~1.MCC\AppData\Local\Temp\kafka3548813472740086814\StreamDeduplicatorAcceptanceTest1\0_0\KSTREAM-JOINTHIS-0000000004-store\KSTREAM-JOINTHIS-0000000004-store:1509624000000: Invalid argument
      at org.rocksdb.RocksDB.open(Native Method) ~[rocksdbjni-5.7.3.jar:?]
      at org.rocksdb.RocksDB.open(RocksDB.java:231) ~[rocksdbjni-5.7.3.jar:?]
      at org.apache.kafka.streams.state.internals.RocksDBStore.openDB(RocksDBStore.java:197) ~[kafka-streams-1.0.0.jar:?]
      at org.apache.kafka.streams.state.internals.RocksDBStore.openDB(RocksDBStore.java:174) ~[kafka-streams-1.0.0.jar:?]
      at org.apache.kafka.streams.state.internals.Segment.openDB(Segment.java:40) ~[kafka-streams-1.0.0.jar:?]
      at org.apache.kafka.streams.state.internals.Segments.getOrCreateSegment(Segments.java:89) ~[kafka-streams-1.0.0.jar:?]
      at org.apache.kafka.streams.state.internals.RocksDBSegmentedBytesStore.put(RocksDBSegmentedBytesStore.java:81) ~[kafka-streams-1.0.0.jar:?]
      at org.apache.kafka.streams.state.internals.RocksDBWindowStore$RocksDBWindowBytesStore.put(RocksDBWindowStore.java:43) ~[kafka-streams-1.0.0.jar:?]
      at org.apache.kafka.streams.state.internals.RocksDBWindowStore$RocksDBWindowBytesStore.put(RocksDBWindowStore.java:34) ~[kafka-streams-1.0.0.jar:?]
      at org.apache.kafka.streams.state.internals.ChangeLoggingWindowBytesStore.put(ChangeLoggingWindowBytesStore.java:67) ~[kafka-streams-1.0.0.jar:?]
      at org.apache.kafka.streams.state.internals.ChangeLoggingWindowBytesStore.put(ChangeLoggingWindowBytesStore.java:33) ~[kafka-streams-1.0.0.jar:?]
      at org.apache.kafka.streams.state.internals.MeteredWindowStore.put(MeteredWindowStore.java:96) ~[kafka-streams-1.0.0.jar:?]
      at org.apache.kafka.streams.state.internals.MeteredWindowStore.put(MeteredWindowStore.java:89) ~[kafka-streams-1.0.0.jar:?]
      at org.apache.kafka.streams.kstream.internals.KStreamJoinWindow$KStreamJoinWindowProcessor.process(KStreamJoinWindow.java:64) ~[kafka-streams-1.0.0.jar:?]
      at org.apache.kafka.streams.processor.internals.ProcessorNode$1.run(ProcessorNode.java:46) ~[kafka-streams-1.0.0.jar:?]
      at org.apache.kafka.streams.processor.internals.StreamsMetricsImpl.measureLatencyNs(StreamsMetricsImpl.java:208) ~[kafka-streams-1.0.0.jar:?]
      at org.apache.kafka.streams.processor.internals.ProcessorNode.process(ProcessorNode.java:124) ~[kafka-streams-1.0.0.jar:?]
      at org.apache.kafka.streams.processor.internals.ProcessorContextImpl.forward(ProcessorContextImpl.java:85) ~[kafka-streams-1.0.0.jar:?]
      at org.apache.kafka.streams.processor.internals.SourceNode.process(SourceNode.java:80) ~[kafka-streams-1.0.0.jar:?]
      at org.apache.kafka.streams.processor.internals.StreamTask.process(StreamTask.java:216) ~[kafka-streams-1.0.0.jar:?]
      ... 6 more

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                mjsax Matthias J. Sax
                Reporter:
                Juma Justin Manchester
              • Votes:
                3 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: