Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-7947

Run impalad minicluster processes inside docker containers

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: Impala 3.3.0
    • Component/s: Infrastructure
    • Labels:

      Description

      I'm looking on some docker-related tasks, e.g. IMPALA-7941 and it'd be cool to be able to start up a minicluster with each impala process inside its own container. This is distinct from test-with-docker which runs the whole minicluster inside a container and is much closer to how containerised services are deployed in production.

      I'm imagining this would be a start-impala-cluster.py flag.

        Attachments

        1.
        Create docker container for impalad/statestored/catalogd Sub-task Resolved Tim Armstrong
        2.
        Figure out how to handle localtime in docker minicluster containers Sub-task Resolved Tim Armstrong
        3.
        Extend start-impala-cluster.py to start and stop daemon docker containers Sub-task Resolved Tim Armstrong
        4.
        Get start-impala-cluster.py to start up a usable minicluster Sub-task Resolved Tim Armstrong
        5.
        Support loading data into a dockerised minicluster Sub-task Resolved Tim Armstrong
        6.
        Create coordinator and executor containers Sub-task Resolved Tim Armstrong
        7.
        Sort out what to do with bin/start-*d.sh functionality Sub-task Resolved Tim Armstrong
        8.
        Run end-to-end tests against docker in minicluster Sub-task Resolved Tim Armstrong
        9.
        Clean up config files in docker containers Sub-task Resolved Tim Armstrong
        10.
        Improve experience for configuring JVM heap size in docker container. Sub-task Resolved Tim Armstrong
        11.
        Pick better default flags in containers Sub-task Resolved Tim Armstrong
        12.
        Automate setup of docker bridge network for dockerised minicluster Sub-task Resolved Tim Armstrong
        13.
        make -j $IMPALA_BUILD_THREADS docker_images fails with OSError: [Errno 17] File exists: '/home/ubuntu/Impala/docker/build_context' Sub-task Resolved Tim Armstrong
        14.
        Run container as non-root user Sub-task Resolved Tim Armstrong
        15.
        Re-enable large_string tests disabled for JVM OOM Sub-task Resolved Tim Armstrong
        16.
        Enable data cache by default for end-to-end containerised tests Sub-task Resolved Tim Armstrong
        17.
        Collect logs from docker containers in tests Sub-task Resolved Tim Armstrong
        18.
        Generate list of docker images generated by the build Sub-task Resolved Tim Armstrong
        19.
        Expose HS2 HTTP port in containers Sub-task Resolved Tim Armstrong
        20.
        Docker build should not depend on test config Sub-task Resolved Tim Armstrong
        21.
        container entrypoint should exec binary Sub-task Resolved Tim Armstrong
        22.
        --rpc_use_loopback may not work for runtime filter RPCs Sub-task Resolved Tim Armstrong
        23.
        Cmake dependencies for container build do not work correctly Sub-task Resolved Tim Armstrong
        24.
        Script to automate pushing images to docker repo, e.g. dockerhub Sub-task Resolved Tim Armstrong
        25.
        Re-enable catalog v2 in containers Sub-task Resolved Vihang Karajgaonkar

          Activity

            People

            • Assignee:
              tarmstrong Tim Armstrong
              Reporter:
              tarmstrong Tim Armstrong
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: