Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-7947 Run impalad minicluster processes inside docker containers
  3. IMPALA-8534

Enable data cache by default for end-to-end containerised tests

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: Impala 3.3.0
    • Component/s: None
    • Labels:
      None

      Description

      Following on from IMPALA-8121, I don't think we can enable the data cache by default, since it depends on what volumes are available to the container at runtime. But we should definitely enable it for tests.

      Michael Ho said

      When I tested with the data cache enabled in a mini-cluster with 3 node using the default scale of workload, I ran with 500 MB with 1 partition by running

      start-impala-cluster.py --data_cache_dir=/tmp --data_cache_size=500MB

      You can also a pre-existing directory as the startup flag of Impala like

      --data_cache=/tmp/data-cache-0:500MB

      start-impala-cluster.py already mounts some host directories into the container, so we could either do the same for the data cache, or just depend on the container root filesystem (which is likely to be slow, unfortunately).

        Attachments

          Activity

            People

            • Assignee:
              tarmstrong Tim Armstrong
              Reporter:
              tarmstrong Tim Armstrong
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: