[STORM-1758] Distributed log search doesn't work in dockerized environment

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Won't Fix
• Affects Version/s: 1.0.0
• Component/s: storm-ui
• Assignee: Unassigned
• Reporter: Elisey Zanko
• Created: 03/May/16 14:15
• Updated: 02/Jul/24 09:47
• Resolved: 02/Jul/24 09:47

Description

I run the Storm using the following docker-compose.yml

version: '2'

services:
    zookeeper:
        image: jplock/zookeeper:3.4.8
        restart: always

    nimbus:
        image: 31z4/storm:1.0.0
        command: nimbus -c storm.log.dir="/logs" -c storm.zookeeper.servers="[\"zookeeper\"]" -c nimbus.host="nimbus"
        depends_on:
            - zookeeper
        restart: always
        ports:
            - 6627:6627
        volumes:
            - logs:/logs

    supervisor:
        image: 31z4/storm:1.0.0
        command: supervisor -c storm.log.dir="/logs" -c storm.zookeeper.servers="[\"zookeeper\"]" -c nimbus.host="nimbus"
        depends_on:
            - nimbus
        restart: always
        volumes:
            - logs:/logs

    logviewer:
        image: 31z4/storm:1.0.0
        command: logviewer -c storm.log.dir="/logs"
        restart: always
        ports:
            - 8000:8000
        volumes:
            - logs:/logs

    ui:
        image: 31z4/storm:1.0.0
        command: ui -c storm.log.dir="/logs" -c nimbus.host="nimbus"
        depends_on:
            - nimbus
            - logviewer
        restart: always
        ports:
            - 8080:8080
        volumes:
            - logs:/log

volumes:
    logs: {}

And distributed log search doesn't work because the Storm UI tries to access the logviewer by supervisor's container id as a host.

Here is the list of running containers

$ docker ps
7ae118eef55c        31z4/storm:1.0.0         "bin/storm ui -c stor"   5 minutes ago       Up 5 minutes               0.0.0.0:8080->8080/tcp         stormdocker_ui_1
5a9101dc2510        31z4/storm:1.0.0         "bin/storm supervisor"   5 minutes ago       Up 5 minutes                                              stormdocker_supervisor_1
4d954096cf18        31z4/storm:1.0.0         "bin/storm nimbus -c "   5 minutes ago       Up 5 minutes               0.0.0.0:6627->6627/tcp         stormdocker_nimbus_1
070080342c4f        31z4/storm:1.0.0         "bin/storm logviewer "   5 minutes ago       Up 5 minutes               0.0.0.0:8000->8000/tcp         stormdocker_logviewer_1
8650786a13cc        jplock/zookeeper:3.4.8   "/opt/zookeeper/bin/z"   5 minutes ago       Up 5 minutes               2181/tcp, 2888/tcp, 3888/tcp   stormdocker_zookeeper_1

And here is what the Storm UI requests

curl 'http://5a9101dc2510:8000/search/topology-1-1462284216%2F6701%2Fworker.log?search-string=split&num-matches=1' -H 'Accept: application/json, text/javascript, */*; q=0.01' -H 'Referer: http://192.168.99.100:8080/search_result.html?search=split&id=topology-1-1462284216&count=1' -H 'Origin: http://192.168.99.100:8080' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.86 Safari/537.36' --compressed

I guess adding an ability to explicitly specify the logviewer host in the storm.yaml would solve the issue.

Issue Links

• is part of: STORM-1759 Viewing logs from the Storm UI doesn't work in dockerized environment
• relates to: STORM-1342 support multiple logviewers per host for container-isolated worker logs