Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-7947

Run impalad minicluster processes inside docker containers

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • Impala 3.3.0
    • Infrastructure

    Description

      I'm looking on some docker-related tasks, e.g. IMPALA-7941 and it'd be cool to be able to start up a minicluster with each impala process inside its own container. This is distinct from test-with-docker which runs the whole minicluster inside a container and is much closer to how containerised services are deployed in production.

      I'm imagining this would be a start-impala-cluster.py flag.

      Attachments

        1.
        		 Create docker container for impalad/statestored/catalogd Sub-task Resolved Tim Armstrong
        2.
        		 Figure out how to handle localtime in docker minicluster containers Sub-task Resolved Tim Armstrong
        3.
        		 Extend start-impala-cluster.py to start and stop daemon docker containers Sub-task Resolved Tim Armstrong
        4.
        		 Get start-impala-cluster.py to start up a usable minicluster Sub-task Resolved Tim Armstrong
        5.
        		 Support loading data into a dockerised minicluster Sub-task Resolved Tim Armstrong
        6.
        		 Create coordinator and executor containers Sub-task Resolved Tim Armstrong
        7.
        		 Sort out what to do with bin/start-*d.sh functionality Sub-task Resolved Tim Armstrong
        8.
        		 Run end-to-end tests against docker in minicluster Sub-task Resolved Tim Armstrong
        9.
        		 Clean up config files in docker containers Sub-task Resolved Tim Armstrong
        10.
        		 Improve experience for configuring JVM heap size in docker container. Sub-task Resolved Tim Armstrong
        11.
        		 Pick better default flags in containers Sub-task Resolved Tim Armstrong
        12.
        		 Automate setup of docker bridge network for dockerised minicluster Sub-task Resolved Tim Armstrong
        13.
        		 make -j $IMPALA_BUILD_THREADS docker_images fails with OSError: [Errno 17] File exists: '/home/ubuntu/Impala/docker/build_context' Sub-task Resolved Tim Armstrong
        14.
        		 Run container as non-root user Sub-task Resolved Tim Armstrong
        15.
        		 Re-enable large_string tests disabled for JVM OOM Sub-task Resolved Tim Armstrong
        16.
        		 Enable data cache by default for end-to-end containerised tests Sub-task Resolved Tim Armstrong
        17.
        		 Collect logs from docker containers in tests Sub-task Resolved Tim Armstrong
        18.
        		 Generate list of docker images generated by the build Sub-task Resolved Tim Armstrong
        19.
        		 Expose HS2 HTTP port in containers Sub-task Resolved Tim Armstrong
        20.
        		 Docker build should not depend on test config Sub-task Resolved Tim Armstrong
        21.
        		 container entrypoint should exec binary Sub-task Resolved Tim Armstrong
        22.
        		 --rpc_use_loopback may not work for runtime filter RPCs Sub-task Resolved Tim Armstrong
        23.
        		 Cmake dependencies for container build do not work correctly Sub-task Resolved Tim Armstrong
        24.
        		 Script to automate pushing images to docker repo, e.g. dockerhub Sub-task Resolved Tim Armstrong
        25.
        		 Re-enable catalog v2 in containers Sub-task Resolved Vihang Karajgaonkar

        Activity

          People

            tarmstrong Tim Armstrong
            tarmstrong Tim Armstrong
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: