Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-7926

long-lived daemons for query fragment execution, I/O and caching

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.0.0
    • Component/s: None
    • Labels:
    • Target Version/s:
    • Release Note:
      Hide
      LLAP is the new hybrid execution model that enables efficiencies across queries, such as caching of columnar data, JIT-friendly operator pipelines, and reduced overhead for multiple queries (including concurrent queries), as well as new performance features like asynchronous I/O, pre-fetching and multi-threaded processing. The hybrid model consists of a long-lived service interacting with on-demand elastic containers serving as a tightly integrated DAG-based framework for query execution.

      The first version of LLAP is being shipped in Hive 2.0 release. The component has been extensively exercised on test and live clusters, and tested, but is expected to have rough edges in this initial release.
      The current limitations are: supported with Tez only; does not support ACID tables; the I/O elevator and cache only support ORC format and vectorized execution.
      Show
      LLAP is the new hybrid execution model that enables efficiencies across queries, such as caching of columnar data, JIT-friendly operator pipelines, and reduced overhead for multiple queries (including concurrent queries), as well as new performance features like asynchronous I/O, pre-fetching and multi-threaded processing. The hybrid model consists of a long-lived service interacting with on-demand elastic containers serving as a tightly integrated DAG-based framework for query execution. The first version of LLAP is being shipped in Hive 2.0 release. The component has been extensively exercised on test and live clusters, and tested, but is expected to have rough edges in this initial release. The current limitations are: supported with Tez only; does not support ACID tables; the I/O elevator and cache only support ORC format and vectorized execution.

      Description

      We are proposing a new execution model for Hive that is a combination of existing process-based tasks and long-lived daemons running on worker nodes. These nodes can take care of efficient I/O, caching and query fragment execution, while heavy lifting like most joins, ordering, etc. can be handled by tasks.
      The proposed model is not a 2-system solution for small and large queries; neither it is a separate execution engine like MR or Tez. It can be used by any Hive execution engine, if support is added; in future even external products (e.g. Pig) can use it.

      The document with high-level design we are proposing will be attached shortly.

        Attachments

        1. LLAPdesigndocument.pdf
          242 kB
          Sergey Shelukhin
        1.
        LLAP: refactor prototype code to salvage some parts Sub-task Closed Sergey Shelukhin
        2.
        LLAP: introduce low-level cache for ORC Sub-task Closed Sergey Shelukhin
        3.
        LLAP: ORC production of encoded data, cache usage Sub-task Closed Sergey Shelukhin
        4.
        LLAP: ORC decoding of row-groups Sub-task Resolved Prasanth Jayachandran
        5.
        LLAP: separate decoding thread from read/uncompress thread Sub-task Open Sergey Shelukhin
        6.
        LLAP: implement pause and stop for async data production Sub-task Open Unassigned
        7.
        LLAP: row-level vectorized SARGs Sub-task Patch Available Yohei Abe
        8.
        LLAP: improve high-level cache from prototype Sub-task Resolved Unassigned
        9.
        LLAP: Fix some static vars in the operator pipeline Sub-task Resolved Gunther Hagleitner
        10.
        LLAP: Enable local mode tests on tez to facilitate llap testing Sub-task Resolved Gunther Hagleitner
        11.
        LLAP: Add an execution daemon Sub-task Closed Siddharth Seth
        12.
        LLAP: (IO) consider using reactive framework to string together parts of code Sub-task Open Unassigned
        13.
        LLAP: I'm the decider Sub-task Resolved Gunther Hagleitner
        14.
        LLAP: Widen the scope of ORC readers and related classes Sub-task Resolved Prasanth Jayachandran
        15.
        LLAP: create a reasonable q file test for ORC IO Sub-task Closed Sergey Shelukhin
        16.
        LLAP: initialize IO during service startup, with service configuration Sub-task Closed Gunther Hagleitner
        17.
        LLAP: add check for udfs/udafs to llapdecider Sub-task Resolved Gunther Hagleitner
        18.
        LLAP: add heap mode to allocator (for q files, YARN w/o direct buffer accounting support) Sub-task Resolved Sergey Shelukhin
        19.
        LLAP: design and implement proper metadata cache Sub-task Closed Sergey Shelukhin
        20.
        LLAP: ORC decoding of row groups for complex types Sub-task Open Prasanth Jayachandran
        21.
        LLAP: Merge from trunk to llap branch 2/20/2015 Sub-task Closed Prasanth Jayachandran
        22.
        avoid log locks in operators Sub-task Closed Gunther Hagleitner
        23.
        LLAP: Fix issue with reading last row group of string column Sub-task Closed Prasanth Jayachandran
        24.
        LLAP: use log4j 2 for llap (log to separate files, etc.) Sub-task Closed Prasanth Jayachandran
        25.
        LLAP: Reuse string dictionaries per column per stripe when processing row groups Sub-task Closed Prasanth Jayachandran
        26.
        LLAP: Update launcher, scheduler to work with Tez changes Sub-task Closed Siddharth Seth
        27.
        LLAP: Misc fixes to launch scripts, startup error handling Sub-task Closed Siddharth Seth
        28.
        LLAP: hook up decider + dag utils Sub-task Resolved Gunther Hagleitner
        29.
        LLAP: Update tez dependency in branch Sub-task Closed Siddharth Seth
        30.
        LLAP: uber mode where applicable Sub-task Resolved Gunther Hagleitner
        31.
        LLAP: fix wrapinputformat for client/server; fix server init to use config Sub-task Closed Gunther Hagleitner
        32.
        LLAP: Make sure jar containing the Tez plugins is added as a local resource for the AM Sub-task Closed Unassigned
        33.
        LLAP: Add a MiniLLAPCluster for tests Sub-task Closed Siddharth Seth
        34.
        LLAP: add simple way to determine wether you're running in daemon Sub-task Resolved Gunther Hagleitner
        35.
        LLAP: Add an option to disable uberization Sub-task Closed Gunther Hagleitner
        36.
        LLAP: add config var for uber mode Sub-task Resolved Gunther Hagleitner
        37.
        LLAP: InStream creation for every row group is expensive Sub-task Closed Prasanth Jayachandran
        38.
        LLAP: config not passed to daemon init Sub-task Closed Sergey Shelukhin
        39.
        LLAP: need metrics for llap daemon Sub-task Closed Prasanth Jayachandran
        40.
        LLAP: consider specialized "transient" metadata cache Sub-task Open Unassigned
        41.
        LLAP: Add event logging for execution elements Sub-task Closed Siddharth Seth
        42.
        LLAP: Push work into daemons instead of the current pull Sub-task Closed Siddharth Seth
        43.
        prep object registry for multi threading Sub-task Closed Gunther Hagleitner
        44.
        LLAP: JMX web-service end points for monitoring & metrics Sub-task Closed Gopal V
        45.
        LLAP: Distinguish between memory used by executors and the cache Sub-task Closed Siddharth Seth
        46.
        LLAP: Use a share-nothing scoreboard /status implementation Sub-task Resolved Yohei Abe
        47.
        LLAP: Native Vectorization of Map Join Sub-task Closed Matt McCline
        48.
        LLAP: Make stripe level column readers thread safe Sub-task Closed Prasanth Jayachandran
        49.
        LLAP: fix unit tests Sub-task Closed Sergey Shelukhin
        50.
        LLAP: Decision to use llap or uber is being lost in some reducers Sub-task Closed Gunther Hagleitner
        51.
        LLAP: instrument logging for fragments Sub-task Closed Sergey Shelukhin
        52.
        LLAP: change caches to use unique file ID, not name Sub-task Closed Sergey Shelukhin
        53.
        LLAP: add multi threaded object registry Sub-task Resolved Gunther Hagleitner
        54.
        documentation for llap Sub-task Open Gunther Hagleitner
        55.
        LLAP: Add executor threadpool metrics Sub-task Resolved Prasanth Jayachandran
        56.
        LLAP: Fix exception with HDFS getFileInfo Sub-task Closed Prasanth Jayachandran
        57.
        LLAP: Print ASCII art of LLAP during daemon startup Sub-task Closed Prasanth Jayachandran
        58.
        LLAP: Slider deploy scripts Sub-task Closed Gopal V
        59.
        LLAP: Add llap-daemon to the Tez session if the execution mode is LLAP Sub-task Closed Gopal V
        60.
        LLAP: Add LLAP_DAEMON_LD_PATH to add native libs in Sub-task Closed Gopal V
        61.
        LLAP: Provide per query counters Sub-task Resolved Prasanth Jayachandran
        62.
        LLAP: Provide metrics for producer/consumer queue Sub-task Resolved Prasanth Jayachandran
        63.
        LLAP: disable plan caching Sub-task Resolved Gunther Hagleitner
        64.
        LLAP: Sort out issues with UGI and cached FileSystems Sub-task Closed Prasanth Jayachandran
        65.
        LLAP: Integrate MiniLLAPCluster into tests Sub-task Closed Prasanth Jayachandran
        66.
        LLAP: Use service discovery in the scheduler to determine running daemons Sub-task Closed Unassigned
        67.
        LLAP: Update usage of APIs changed by TEZ-2175 and TEZ-2187 Sub-task Closed Siddharth Seth
        68.
        LLAP: Clean up structures and intermediate data when a query completes Sub-task Closed Siddharth Seth
        69.
        LLAP: Improvements to the Shuffle handler to avoid unnecessary disk scans Sub-task Closed Siddharth Seth
        70.
        LLAP: Avoid fetching data multiple times in case of broadcast Sub-task Closed Siddharth Seth
        71.
        LLAP: Vectorized Field-By-Field Serialize / Deserialize to support new Vectorized Map Join Sub-task Closed Matt McCline
        72.
        LLAP: YARN registry for Auto-organizing Slider instances Sub-task Closed Gopal V
        73.
        LLAP: OrcColumnVectorProducer should handle reading isPresent columns only Sub-task Closed Sergey Shelukhin
        74.
        LLAP: LLAP Cached readers for StringDirectTreeReaders over-read data Sub-task Closed Sergey Shelukhin
        75.
        LLAP: EOFException in reader Sub-task Closed Sergey Shelukhin
        76.
        LLAP: Add rack based scheduling of work Sub-task Open Unassigned
        77.
        LLAP: Handle task rejection from daemons in the AM Sub-task Closed Siddharth Seth
        78.
        fix yarn service registry not found in ut problem Sub-task Open Gunther Hagleitner
        79.
        MiniTez ut fail with missing configs Sub-task Closed Gunther Hagleitner
        80.
        yarn service registry should be an optional dependency Sub-task Resolved Gopal V
        81.
        LLAP: Pass additional JVM args via Slider appConfig Sub-task Closed Gopal V
        82.
        LLAP: NegativeArraySize exception on vector string reader Sub-task Closed Sergey Shelukhin
        83.
        LLAP: Hive sessions run before Slider registers to YARN registry fail to launch Sub-task Closed Gopal V
        84.
        NPE in LLAP logs in heartbeat Sub-task Resolved Unassigned
        85.
        LLAP : investigate showing LLAP IO usage in explain Sub-task Closed Sergey Shelukhin
        86.
        LLAP : add LLAP IO read debug tool Sub-task Open Sergey Shelukhin
        87.
        LLAP: q file test is broken again Sub-task Closed Sergey Shelukhin
        88.
        LLAP: Queued work times out Sub-task Closed Siddharth Seth
        89.
        LLAP: AM should get notifications on daemons going down or restarting Sub-task Closed Unassigned
        90.
        LLAP: Create a fixed size execution queue for daemons Sub-task Closed Prasanth Jayachandran
        91.
        LLAP: Scheduling of work from different queries within the daemon Sub-task Closed Unassigned
        92.
        LLAP : llap_partitioned test is broken Sub-task Closed Sergey Shelukhin
        93.
        LLAP: cache counters are incorrect Sub-task Resolved Prasanth Jayachandran
        94.
        LLAP : NPE in DiskRangeList helper init Sub-task Closed Sergey Shelukhin
        95.
        LLAP : disable ORC trace logging for now Sub-task Resolved Sergey Shelukhin
        96.
        LLAP: VectorMapJoinOperator gets an over-flow on batchSize of 1024 Sub-task Closed Matt McCline
        97.
        LLAP: LowLevelCacheImpl does not match cached ranges properly Sub-task Closed Sergey Shelukhin
        98.
        LLAP: Cache hit rate could be wrong in some cases Sub-task Resolved Prasanth Jayachandran
        99.
        LLAP: read file ID when generating splits to avoid extra NN call in the tasks Sub-task Closed Sergey Shelukhin
        100.
        LLAP: adjust allocation after decompression Sub-task Open Unassigned
        101.
        LLAP: Object Estimator digs in via a deep hashCode() Sub-task Closed Sergey Shelukhin
        102.
        LLAP: Make the low-level IO threadpool configurable Sub-task Resolved Prasanth Jayachandran
        103.
        LLAP: improve how buffers are locked for split Sub-task Closed Sergey Shelukhin
        104.
        LLAP: enable yourkit profiling of tasks Sub-task Patch Available Sergey Shelukhin
        105.
        LLAP: Cancelling tasks fails to stop cache filling threads Sub-task Closed Sergey Shelukhin
        106.
        LLAP: Generate consistent splits and locations for the same split across jobs Sub-task Closed Siddharth Seth
        107.
        LLAP: implement stopping the reader thread Sub-task Resolved Unassigned
        108.
        LLAP: port updates from HIVE-9555 to llap branch in preparation for trunk merge Sub-task Closed Sergey Shelukhin
        109.
        LLAP: query 7 produces corrupted result with IO enabled Sub-task Resolved Sergey Shelukhin
        110.
        LLAP: query 17 tasks fail due to mapjoin issue Sub-task Resolved Gunther Hagleitner
        111.
        LLAP: reducers running in LLAP starve out map retries Sub-task Closed Siddharth Seth
        112.
        LLAP: Use task number, attempt number to cache plans Sub-task Open Unassigned
        113.
        LLAP: Print execution modes in tez in-place UI Sub-task Resolved Prasanth Jayachandran
        114.
        LLAP: Port changes to timestamp stream reader after timezone fix in trunk Sub-task Resolved Prasanth Jayachandran
        115.
        LLAP: Fix ordering of execution modes Sub-task Resolved Prasanth Jayachandran
        116.
        LLAP: BytesBytesMultiHashMap and mapjoin container should reuse refs Sub-task Closed Matt McCline
        117.
        LLAP: Tez heartbeats are delayed by ~500+ ms due to Hadoop IPC client Sub-task Open Siddharth Seth
        118.
        LLAP: GC issues 1 Sub-task Closed Sergey Shelukhin
        119.
        LLAP: switch to sensible cache policy Sub-task Resolved Sergey Shelukhin
        120.
        LLAP: Varchar columns are not handled in encoded readers Sub-task Closed Prasanth Jayachandran
        121.
        LLAP: ORC file contains compression buffers larger than bufferSize (OR reader has a bug) Sub-task Closed Sergey Shelukhin
        122.
        LLAP: Avoid deserializing the plan > 1 times in a single thread Sub-task Closed Gunther Hagleitner
        123.
        LLAP: ORC BIGINT SARGs regressed after Parquet PPD fixes (HIVE-8122) Sub-task Resolved Prasanth Jayachandran
        124.
        LLAP: general cache deadlock avoidance Sub-task Open Sergey Shelukhin
        125.
        LLAP: ORC MemoryManager is singleton synchronized Sub-task Resolved Gopal V
        126.
        LLAP: LLAP IO doesn't get invoked inside MiniTezCluster q tests Sub-task Resolved Siddharth Seth
        127.
        LLAP: Failures from TaskRunnerCallable should be reported Sub-task Closed Siddharth Seth
        128.
        LLAP: start fetching earlier Sub-task Closed Sergey Shelukhin
        129.
        Move fragment execution onto a thread pool Sub-task Open Unassigned
        130.
        LLAP: Support caching of uncompressed ORC data Sub-task Closed Sergey Shelukhin
        131.
        LLAP: Loglevel for daemons as a startup option Sub-task Closed Gopal V
        132.
        LLAP: Clean up encoded ORC tree readers after trunk merge Sub-task Resolved Prasanth Jayachandran
        133.
        LLAP: Certain errors are not reported to the AM when a fragment fails Sub-task Closed Unassigned
        134.
        LLAP: Make use of the timed version of getDagStatus in TezJobMonitor Sub-task Closed Siddharth Seth
        135.
        LLAP: Fix merge conflicts related to HIVE-10067 Sub-task Resolved Prasanth Jayachandran
        136.
        LLAP: orc_llap test fails again Sub-task Resolved Prasanth Jayachandran
        137.
        LLAP: Explain plan diff in llap_partitioned.q test case Sub-task Resolved Prasanth Jayachandran
        138.
        LLAP: Allow the runtime to check whether a task can run to completion Sub-task Closed Siddharth Seth
        139.
        LLAP: Handle errors while sending source state updates to the daemons Sub-task Closed Siddharth Seth
        140.
        LLAP: Fix timeouts for tasks which stay in the pending queue Sub-task Closed Siddharth Seth
        141.
        LLAP: Cache cleaner throws Negative sleep exception Sub-task Closed Sergey Shelukhin
        142.
        Hadoop reflectionutils has issues Sub-task Closed Sergey Shelukhin
        143.
        LLAP: NPE due to failure to find position Sub-task Closed Sergey Shelukhin
        144.
        LLAP: Self-joins with the cache enabled Sub-task Closed Sergey Shelukhin
        145.
        LLAP: IndexOutOfBound in MapJoinOperator Sub-task Closed Sergey Shelukhin
        146.
        LLAP: small improvement to unlocking cache data Sub-task Closed Sergey Shelukhin
        147.
        LLAP: merge trunk to branch 2015-04-15 Sub-task Closed Sergey Shelukhin
        148.
        LLAP: randomness in random machine scheduling is not random enough Sub-task Resolved Gopal V
        149.
        LLAP: query80 fails with vectorization cast issue Sub-task Resolved Matt McCline
        150.
        LLAP: Name IO elevator threads Sub-task Resolved Prasanth Jayachandran
        151.
        LLAP: Tez in-place UI gets misaligned when state is TERMINATING Sub-task Resolved Prasanth Jayachandran
        152.
        LLAP: query 54 doesn't finish in LLAP Sub-task Resolved Siddharth Seth
        153.
        LLAP: Notify AM of pre-emption Sub-task Resolved Prasanth Jayachandran
        154.
        LLAP: Implement Tez SplitSizeEstimator for Orc Sub-task Resolved Prasanth Jayachandran
        155.
        LLAP: scheduling occasionally schedules all the work on one machine Sub-task Resolved Gopal V
        156.
        LLAP: Provide runtime information to daemons to decide on preemption order Sub-task Closed Siddharth Seth
        157.
        LLAP: Make use of additional information to determine run/preemption order Sub-task Open Unassigned
        158.
        LLAP: NPE in scheduler in case of rejected tasks Sub-task Closed Siddharth Seth
        159.
        LLAP: NPE caused by HIVE-10397 Sub-task Resolved Prasanth Jayachandran
        160.
        LLAP: Factor known capacity into scheduling decisions Sub-task Closed Siddharth Seth
        161.
        LLAP: Limit number of threads used to communicate with a single LLAP instance to 1 Sub-task Closed Siddharth Seth
        162.
        LLAP: Abort hive tez processor on interrupts Sub-task Closed Prasanth Jayachandran
        163.
        LLAP: Consider replacing BytesBytesMultiHashMap with new fast hash table code of Native Vector Map Join Sub-task Resolved Matt McCline
        164.
        LLAP: fix container sizing configuration for memory Sub-task Open Vikram Dixit K
        165.
        LLAP: NPE in IO when returning 0 rows with no projection Sub-task Closed Prasanth Jayachandran
        166.
        LLAP: investigate why TPCH Q1 1k is slow Sub-task Resolved Unassigned
        167.
        LLAP: Minor fixes after tez api enhancements for dag completion Sub-task Closed Siddharth Seth
        168.
        LLAP: Tez task is interrupted for unknown reason after an IPC exception and then fails to report completion Sub-task Closed Siddharth Seth
        169.
        LLAP: AsertionError cannot allocate when reading from orc Sub-task Closed Sergey Shelukhin
        170.
        LLAP: Resolve everything in llap-daemon-site.xml Sub-task Closed Gopal V
        171.
        LLAP: Empty PPD splits from Cache throw error Sub-task Resolved Unassigned
        172.
        LLAP: Must reset small table result columns for Native Vectorization of Map Join Sub-task Closed Matt McCline
        173.
        LLAP: Cleanup map join cache when a query completes Sub-task Closed Sergey Shelukhin
        174.
        LLAP: Add test case for all row groups selection with no column projection Sub-task Resolved Prasanth Jayachandran
        175.
        LLAP: a different NPE in shuffle Sub-task Closed Siddharth Seth
        176.
        LLAP: Native Vector Map Join doesn't handle filtering and matching on LEFT OUTER JOIN repeated key correctly Sub-task Closed Matt McCline
        177.
        LLAP: Vector row extraction allocates new extractors per process method call instead of just once Sub-task Resolved Matt McCline
        178.
        LLAP: Invalid containerId prefix Sub-task Resolved Unassigned
        179.
        LLAP: allocator occasionally has a spurious failure to allocate due to "partitioned" locking and has to retry Sub-task Open Sergey Shelukhin
        180.
        LLAP: registry; Tez attempted to schedule to daemon that didn't exist Sub-task Resolved Gopal V
        181.
        LLAP: AM gets stuck completely if one node is dead Sub-task Resolved Unassigned
        182.
        LLAP: AM task communication retry is too long Sub-task Closed Siddharth Seth
        183.
        LLAP: registry logs strange lines on daemons Sub-task Closed Sergey Shelukhin
        184.
        LLAP: investigate why GC with IO elevator disabled is so bad Sub-task Resolved Prasanth Jayachandran
        185.
        LLAP: Wait queue pre-emption Sub-task Closed Prasanth Jayachandran
        186.
        LLAP: Update tez dependency Sub-task Closed Siddharth Seth
        187.
        LLAP: Make use of the task runner which allows killing tasks Sub-task Closed Siddharth Seth
        188.
        LLAP: Add a mechanism for daemons to inform the AM about killed tasks Sub-task Closed Siddharth Seth
        189.
        LLAP: DAGs get stuck at start with no tasks executing Sub-task Resolved Siddharth Seth
        190.
        LLAP: DAG got stuck after reducer fetch failed Sub-task Resolved Siddharth Seth
        191.
        LLAP: Add counters for time lost per query due to preemption Sub-task Open Unassigned
        192.
        LLAP: Log additional debug information in the scheduler Sub-task Closed Siddharth Seth
        193.
        LLAP: fix guava stopwatch conflict Sub-task Closed Siddharth Seth
        194.
        LLAP: task scheduler thread-count keeps growing Sub-task Open Siddharth Seth
        195.
        LLAP: rare NPE in IO Sub-task Resolved Unassigned
        196.
        LLAP: dags get stuck in yet another way Sub-task Closed Prasanth Jayachandran
        197.
        LLAP: Misc changes to daemon scheduling Sub-task Closed Siddharth Seth
        198.
        LLAP: Modify running / wait queues on on fragment finishable state changes Sub-task Closed Siddharth Seth
        199.
        LLAP: Add aging to wait queue tasks Sub-task Closed Prasanth Jayachandran
        200.
        LLAP: Kill any fragments running in a daemon when a query completes Sub-task Closed Siddharth Seth
        201.
        LLAP: Provide current attempt start time for wait queue ordering Sub-task Resolved Siddharth Seth
        202.
        LLAP: Wait queue scheduler goes into tight loop Sub-task Closed Prasanth Jayachandran
        203.
        LLAP: NPE when calling abort on the TezProcessor Sub-task Closed Siddharth Seth
        204.
        LLAP: Exception reported while trying to kill a task Sub-task Open Prasanth Jayachandran
        205.
        LLAP: Improve the way task finishable information is processed Sub-task Closed Siddharth Seth
        206.
        LLAP: perf dips with IO elevator disabled Sub-task Resolved Unassigned
        207.
        LLAP: add pre-fragment and per-table cache details Sub-task Closed Sergey Shelukhin
        208.
        LLAP: Utilities::gWorkMap needs to be cleaned in HiveServer2 Sub-task Closed Sergey Shelukhin
        209.
        LLAP: Daemons should shutdown in case of fatal errors Sub-task Closed Siddharth Seth
        210.
        LLAP: DAGs get stuck in yet another way Sub-task Closed Siddharth Seth
        211.
        LLAP: preemption in AM due to failures / out of order scheduling Sub-task Closed Siddharth Seth
        212.
        LLAP: Better handling of hostnames when sending heartbeats to the AM Sub-task Open Unassigned
        213.
        LLAP: Serialize handling of requests / events for a query within daemons Sub-task Open Siddharth Seth
        214.
        LLAP: Merge from trunk May 28th 2015 Sub-task Closed Gopal V
        215.
        LLAP: bug in split processing in IO elevator causes duplicate rows Sub-task Closed Sergey Shelukhin
        216.
        LLAP: make sure tests pass #1 Sub-task Closed Sergey Shelukhin
        217.
        LLAP: Fixes to TaskReporter after recent Tez changes Sub-task Closed Siddharth Seth
        218.
        LLAP: HIVE-10778 has NPE Sub-task Closed Sergey Shelukhin
        219.
        LLAP: make sure the branch builds on hadoop-1: part 1 (non-llap) Sub-task Closed Sergey Shelukhin
        220.
        LLAP: the return of the stuck DAG Sub-task Closed Sergey Shelukhin
        221.
        LLAP: ArrayIndexOOB exception in new mapjoin Sub-task Resolved Matt McCline
        222.
        LLAP: merge master into branch Sub-task Closed Sergey Shelukhin
        223.
        LLAP: Exception in InputInitializer when creating HiveSplitGenerator Sub-task Resolved Unassigned
        224.
        LLAP: cache QF counters have a wrong value for consumer time Sub-task Closed Sergey Shelukhin
        225.
        LLAP: fix hadoop-1 build for good by removing llap-server from hadoop-1 build Sub-task Closed Sergey Shelukhin
        226.
        LLAP: elevator reads some useless data even if all RGs are eliminated by SARG Sub-task Closed Sergey Shelukhin
        227.
        LLAP: Set java.io.tmpdir correctly for LLAP Slider instance Sub-task Closed Gopal V
        228.
        LLAP: merge master to branch Sub-task Closed Sergey Shelukhin
        229.
        LLAP: make ObjectCache for plans work properly in the daemon Sub-task Patch Available Sergey Shelukhin
        230.
        LLAP: expose what's running on the daemon thru JMX Sub-task Closed Sergey Shelukhin
        231.
        LLAP: support parallel query compilation in HS2 Sub-task Closed Sergey Shelukhin
        232.
        LLAP: recent optimization introduced wrong assert to elevator causing test to fail Sub-task Closed Sergey Shelukhin
        233.
        LLAP: preemption appears to count against failure count for the task Sub-task Resolved Siddharth Seth
        234.
        LLAP: Allow finer control of startup options for LLAP Sub-task Closed Gopal V
        235.
        LLAP: ShuffleHandler + Submit work init race condition Sub-task Closed Siddharth Seth
        236.
        LLAP: Accept --hiveconf parameters for the LlapServiceDriver Sub-task Closed Gopal V
        237.
        LLAP: make sure tests pass #2 Sub-task Closed Sergey Shelukhin
        238.
        LLAP: merge master into branch Sub-task Closed Sergey Shelukhin
        239.
        LLAP: Fix LLAP startup issues due to heap rounding errors Sub-task Closed Gopal V
        240.
        LLAP: fix TestMiniTezCliDriverLocal on the branch Sub-task Resolved Vikram Dixit K
        241.
        LLAP: test auto_sortmerge_join_5 on MiniTez fails with NPE Sub-task Closed Sergey Shelukhin
        242.
        LLAP: fix some tests in the branch and revert incorrectly committed changed out files (from HIVE-11014) Sub-task Closed Sergey Shelukhin
        243.
        LLAP: some MiniTez tests have result changes compared to master Sub-task Closed Sergey Shelukhin
        244.
        LLAP: MiniTez tez_smb_main, tez_bmj_schema_evolution fail with NPE Sub-task Closed Sergey Shelukhin
        245.
        MiniTez mergejoin test fails with Tez input error (issue in merge join under certain conditions) Sub-task Closed Sergey Shelukhin
        246.
        LLAP: disable the flaky TestLlapTaskSchedulerService test Sub-task Closed Sergey Shelukhin
        247.
        LLAP: merge master into branch Sub-task Closed Sergey Shelukhin
        248.
        LLAP: fix TestLlapTaskSchedulerService flakiness Sub-task Closed Siddharth Seth
        249.
        LLAP: SIGSEGV in Off-heap decompression routines Sub-task Closed Sergey Shelukhin
        250.
        LLAP: Cache arena size rounding Sub-task Closed Sergey Shelukhin
        251.
        LLAP: merge master into branch Sub-task Closed Sergey Shelukhin
        252.
        LLAP: more out file changes compared to master Sub-task Closed Sergey Shelukhin
        253.
        Fair ordering of fragments in wait queue Sub-task Closed Prasanth Jayachandran
        254.
        LLAP: spark out file changes compared to master Sub-task Closed Sergey Shelukhin
        255.
        Remove unused LlapUtils class from ql.io.orc Sub-task Closed Prasanth Jayachandran
        256.
        LLAP: clean up shuffle directories when DAG completes Sub-task Closed Sergey Shelukhin
        257.
        LLAP: merge master into branch Sub-task Closed Sergey Shelukhin
        258.
        LLAP: Cache BuddyAllocator throws NPE Sub-task Closed Sergey Shelukhin
        259.
        LLAP: more out file changes compared to master Sub-task Closed Sergey Shelukhin
        260.
        LLAP: occasional NPE in parallel queries in ORC reader Sub-task Closed Sergey Shelukhin
        261.
        LLAP: Fix the LLAP to ORC APIs Sub-task Closed Sergey Shelukhin
        262.
        LLAP: clean up ORC dependencies part 1 Sub-task Closed Sergey Shelukhin
        263.
        LLAP: NPE in AMReporter Sub-task Closed Siddharth Seth
        264.
        LLAP: TaskExecutorService state is not cleaned up Sub-task Closed Siddharth Seth
        265.
        LLAP: PipelineSorter got stuck Sub-task Resolved Gopal V
        266.
        LLAP: investigate locality issues Sub-task Resolved Siddharth Seth
        267.
        LLAP: Execution order within LLAP daemons should consider query-specific priority assigned to fragments Sub-task Closed Siddharth Seth
        268.
        LLAP: Register for finishable state change notifications when adding a task instead of when scheduling it Sub-task Closed Siddharth Seth
        269.
        LLAP: clean up ORC dependencies on object pools Sub-task Closed Sergey Shelukhin
        270.
        LLAP: Hybrid Map-join cache returns invalid data Sub-task Closed Sergey Shelukhin
        271.
        LLAP: Fix API usage to work with evolving Tez APIs Sub-task Closed Siddharth Seth
        272.
        LLAP: Fix API usage to work with evolving Tez APIs - TEZ-2005 Sub-task Closed Siddharth Seth
        273.
        LLAP: move LlapConfiguration into HiveConf and document the settings Sub-task Closed Sergey Shelukhin
        274.
        LLAP: clean up ORC dependencies - move encoded reader path into a separate package and reader Sub-task Closed Sergey Shelukhin
        275.
        LLAP: Fix API usage to work with evolving Tez APIs - TEZ-{2651,2652,2653} Sub-task Closed Siddharth Seth
        276.
        LLAP: make sure JMX view is correct Sub-task Resolved Sergey Shelukhin
        277.
        LLAP: merge master into branch Sub-task Closed Sergey Shelukhin
        278.
        LLAP: ORC-related refactoring changes broke something Sub-task Closed Sergey Shelukhin
        279.
        Disabling llap cache allocate direct is not honored anymore Sub-task Resolved Prasanth Jayachandran
        280.
        LLAP: Fix API usage to work with evolving Tez APIs - TEZ-2678 Sub-task Closed Siddharth Seth
        281.
        LLAP: tez.runtime.compress doesn't appear to be honored for LLAP Sub-task Closed Siddharth Seth
        282.
        LLAP: implement LLAP UI as a separate service - part 1 Sub-task Closed Yuya OZAWA
        283.
        LLAP: move EncodedTreeReaderFactory, TreeReaderFactory bits that rely on orc.encoded, and StreamUtils if needed, to orc.encoded package Sub-task Closed Sergey Shelukhin
        284.
        LLAP: Some counters are incorrect Sub-task Closed Siddharth Seth
        285.
        Allow wait queue comparator to be specified as a classname Sub-task Closed Siddharth Seth
        286.
        LLAP: switch the branch to Tez master from branch Sub-task Closed Sergey Shelukhin
        287.
        LLAP: make sure tests pass #3 Sub-task Closed Sergey Shelukhin
        288.
        LLAP: merge master into branch Sub-task Closed Sergey Shelukhin
        289.
        Create LLAP Monitor Daemon class and launch scripts Sub-task Open Yuya OZAWA
        290.
        LLAP: merge master into branch Sub-task Closed Sergey Shelukhin
        291.
        LLAP: Update tez version to fix build Sub-task Resolved Prasanth Jayachandran
        292.
        LLAP: Merge master into branch Sub-task Closed Prasanth Jayachandran
        293.
        TaskExecutorService can reject work even if capacity is available Sub-task Closed Siddharth Seth
        294.
        LLAP: merge master into branch Sub-task Closed Sergey Shelukhin
        295.
        LLAP: MiniLlapCluster integration broke hadoop-1 build Sub-task Closed Prasanth Jayachandran
        296.
        LLAP: Remove MiniLlapCluster from shim layer after hadoop-1 removal Sub-task Closed Siddharth Seth
        297.
        LLAP: merge master into branch Sub-task Closed Sergey Shelukhin
        298.
        LLAP: Create webapp/llap directory for MiniLlapCluster Sub-task Resolved Prasanth Jayachandran
        299.
        LLAP: Generate golden files for all MiniLlapCluster tests Sub-task Closed Prasanth Jayachandran
        300.
        LLAP: MiniLlapCluster tests are slow Sub-task Closed Prasanth Jayachandran
        301.
        LLAP: TaskCommunicator should not register MultiMRInput Inputs for state updates Sub-task Closed Siddharth Seth
        302.
        LLAP: Exception is ignored if MiniLlap cluster fails to start Sub-task Closed Prasanth Jayachandran
        303.
        LLAP: Merge master into branch Sub-task Resolved Prasanth Jayachandran
        304.
        LLAP: Rerunning MiniLlap does not start as web app port is used Sub-task Resolved Prasanth Jayachandran
        305.
        LLAP: Fix failing tests in MiniLlapCluster Sub-task Closed Prasanth Jayachandran
        306.
        LLAP: Test failures - file.splitmetainfo file not found Sub-task Closed Sergey Shelukhin
        307.
        LLAP: Fix tests related to stats difference Sub-task Closed Prasanth Jayachandran
        308.
        LLAP: Analyze and fix vectorization tests Sub-task Closed Prasanth Jayachandran
        309.
        LLAP: Generate join0.q golden file for java 8 Sub-task Resolved Prasanth Jayachandran
        310.
        LLAP: Fix unionDistinct_1.q and union_view.q Sub-task Closed Sergey Shelukhin
        311.
        LLAP: constprog_dpp.q,explainuser_1.q showing explain diff Sub-task Closed Prasanth Jayachandran
        312.
        LLAP: Remove fileformat_mix.q.out as it is not part of tez test suite Sub-task Closed Prasanth Jayachandran
        313.
        LLAP: Partition column value different for identity_project_remove_skip.q Sub-task Closed Prasanth Jayachandran
        314.
        LLAP: Plan is different for insert1.q Sub-task Closed Sergey Shelukhin
        315.
        LLAP: Fix auto_join_nulls.q,auto_sortmerge_join_6.q test diffs Sub-task Closed Prasanth Jayachandran
        316.
        LLAP: Update errata.txt for HIVE-11871 Sub-task Closed Prasanth Jayachandran
        317.
        LLAP: Fix ordering difference in acid_vectorization_partition.q Sub-task Closed Prasanth Jayachandran
        318.
        LLAP: Fix discrepancies with metadata_only_queries_with_filters.q Sub-task Closed Prasanth Jayachandran
        319.
        LLAP: Remove unused/old golden files Sub-task Closed Prasanth Jayachandran
        320.
        LLAP: merge master into branch Sub-task Closed Sergey Shelukhin
        321.
        LLAP: Merge master into branch (for HIVE-11860) Sub-task Resolved Prasanth Jayachandran
        322.
        LLAP: Update llap golden files after master merge Sub-task Closed Prasanth Jayachandran
        323.
        LLAP: Typo in webapps causing initialization error for MiniLlap tests Sub-task Closed Prasanth Jayachandran
        324.
        LLAP: Update golden files for orc_llap and llapdecider tests Sub-task Closed Prasanth Jayachandran
        325.
        LLAP: Merge branch to hive-2.0 Sub-task Closed Sergey Shelukhin
        326.
        LLAP: merge master into branch Sub-task Closed Sergey Shelukhin
        327.
        LLAP: merge master into branch Sub-task Closed Sergey Shelukhin
        328.
        Llap: Disable web app for mini llap tests Sub-task Closed Prasanth Jayachandran
        329.
        LLAP: Merge master to branch Sub-task Closed Prasanth Jayachandran
        330.
        LLAP: merge master into branch Sub-task Closed Sergey Shelukhin
        331.
        LLAP: Merge master into branch Sub-task Closed Sergey Shelukhin
        332.
        LLAP: merge master into branch Sub-task Closed Sergey Shelukhin
        333.
        LLAP: merge master into branch Sub-task Closed Sergey Shelukhin
        334.
        UGI instances being used in IO elevator threads are incorrect Sub-task Closed Siddharth Seth
        335.
        LLAP: merge master into branch Sub-task Closed Sergey Shelukhin
        336.
        Add javadoc for methods added to public APIs Sub-task Closed Sergey Shelukhin
        337.
        LLAP: merge master into branch Sub-task Closed Sergey Shelukhin
        338.
        LLAP: Make explainuser_1.q test consistent Sub-task Closed Prasanth Jayachandran
        339.
        LLAP: merge master into branch Sub-task Closed Sergey Shelukhin
        340.
        LLAP: update errata for bad branch commits Sub-task Closed Sergey Shelukhin
        341.
        LLAP: Tez counters for LLAP 2 Sub-task Resolved Unassigned

          Activity

            People

            • Assignee:
              sershe Sergey Shelukhin
              Reporter:
              sershe Sergey Shelukhin
            • Votes:
              3 Vote for this issue
              Watchers:
              49 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: