Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-9851

Query status can be unbounded in size

    XMLWordPrintableJSON

    Details

    • Epic Color:
      ghx-label-7

      Description

      We got a report of a query status that was 10s of MB in size. We should avoid this. Probably we should do 2 things: 1) not include the giant debug string in this error message and 2) cap the error message size somewhere to a sane size, e.g. some number of kiloybytes (not sure where the best place is to implement this)

      annot perform hash join at node with id 3. Repartitioning did not reduce the size of a spilled partition. Repartitioning level 2. Number of rows 106305207:nPartitionedHashJoinNode (id=3 op=5 state=RepartitioningBuild #spilled_partitions=0)nPhjBuilder: Hash partitions: 16:n Hash partition 0 <Partition>: ptr=0x1f395c020 Closedn Hash partition 1 <Partition>: ptr=0x4db0a6700 Closedn Hash partition 2 <Partition>: ptr=0xd4145b20 Closedn Hash partition 3 <Partition>: ptr=0x494467860 Closedn Hash partition 4 <Partition>: ptr=0xd4145c40 Closedn Hash partition 5 <Partition>: ptr=0x38efb3be0 Closedn Hash partition 6 <Partition>: ptr=0x4db0a7bc0 Closedn Hash partition 7 <Partition>: ptr=0x14dcd2040 Closedn Hash partition 8 <Partition>: ptr=0x14dcd3500 Closedn Hash partition 9 <Partition>: ptr=0x4febfe40 Closedn Hash partition 10 <Partition>: ptr=0xd4145ec0 Spilledn    Build Rows: 106305207 (Bytes pinned: 0)nn Hash partition 11 <Partition>: ptr=0x7f34acfb8040 Closedn Hash partition 12 <Partition>: ptr=0x14dcd3e00 Closedn Hash partition 13 <Partition>: ptr=0x4944661e0 Closedn Hash partition 14 <Partition>: ptr=0xd4145740 Closedn Hash partition 15 <Partition>: ptr=0x7f1cc7566c00 ClosednProbe hash partitions: 0:nInputPartition: 0x1dfe9a5f0n   Build Partition Closedn   Spilled Probe Rows: 2608nn<BufferPool::Client> 0x12f09320 internal state: {<BufferPool::Client> 0x7f30643f8780 name: HASH_JOIN_NODE id=3 ptr=0x12f09180 write_status:  buffers allocated 0 num_pages: 189834 pinned_bytes: 65536 dirty_unpinned_bytes: 12127043584 in_flight_write_bytes: 3145728 reservation: {<ReservationTracker>: reservation_limit 9223372036854775807 reservation 12441812992 used_reservation 65536 child_reservations 65536 parent:n<ReservationTracker>: reservation_limit 9223372036854775807 reservation 12479561728 used_reservation 0 child_reservations 12479561728 parent:n<ReservationTracker>: reservation_limit 12800000000 reservation 12481593344 used_reservation 0 child_reservations 12481593344 parent:n<ReservationTracker>: reservation_limit 273804165120 reservation 12665946112 used_reservation 0 child_reservations 12665946112 parent:nNULL}n  1 pinned pages: <BufferPool::Page> 0x7f348aab4140 len: 65536 pin_count: 1 buf: <BufferPool::BufferHandle> 0x7f348aab41b8 client: 0x12f09320/0x7f30643f8780 data: 0x15e32000 len: 65536nn  185044 dirty unpinned pages: <BufferPool::Page> 0xc676b020 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0xc676b098 client: 0x12f09320/0x7f30643f8780 data: 0x2ed4a0000 len: 65536n<BufferPool::Page> 0xc6769180 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0xc67691f8 client: 0x12f09320/0x7f30643f8780 data: 0x2ed4b0000 len: 65536n<BufferPool::Page> 0xc676a260 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0xc676a2d8 client: 0x12f09320/0x7f30643f8780 data: 0x2ed4c0000 len: 65536n<BufferPool::Page> 0xc676b660 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0xc676b6d8 client: 0x12f09320/0x7f30643f8780 data: 0x2ed4d0000 len: 65536n<BufferPool::Page> 0xc67685a0 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0xc6768618 client: 0x12f09320/0x7f30643f8780 data: 0x2ed4e0000 len: 65536n<BufferPool::Page> 0xc676bd40 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0xc676bdb8 client: 0x12f09320/0x7f30643f8780 data: 0x2ed4f0000 len: 65536n<BufferPool::Page> 0xc676a580 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0xc676a5f8 client: 0x12f09320/0x7f30643f8780 data: 0x2ed500000 len: 65536n<BufferPool::Page> 0xc676a940 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0xc676a9b8 client: 0x12f09320/0x7f30643f8780 data: 0x2ed526000 len: 65536n<BufferPool::Page> 0xc676a6c0 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0xc676a738 client: 0x12f09320/0x7f30643f8780 data: 0x2ed536000 len: 65536n<BufferPool::Page> 0xc67699a0 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0xc6769a18 client: 0x12f09320/0x7f30643f8780 data: 0x2ed546000 len: 65536n<BufferPool::Page> 0x1652ee8c0 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x1652ee938 client: 0x12f09320/0x7f30643f8780 data: 0x2ed556000 len: 65536n<BufferPool::Page> 0x1652f0ee0 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x1652f0f58 client: 0x12f09320/0x7f30643f8780 data: 0x2ed566000 len: 65536n<BufferPool::Page> 0x1652ef0e0 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x1652ef158 client: 0x12f09320/0x7f30643f8780 data: 0x2ed58a000 len: 65536n<BufferPool::Page> 0x1652f04e0 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x1652f0558 client: 0x12f09320/0x7f30643f8780 data: 0x2ed59a000 len: 65536n<BufferPool::Page> 0x1652f1660 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x1652f16d8 client: 0x12f09320/0x7f30643f8780 data: 0x2ed5aa000 len: 65536n<BufferPool::Page> 0x1652ee6e0 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x1652ee758 client: 0x12f09320/0x7f30643f8780 data: 0x2ed5ba000 len: 65536n<BufferPool::Page> 0x1652eed20 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x1652eed98 client: 0x12f09320/0x7f30643f8780 data: 0x2ed5ca000 len: 65536n<BufferPool::Page> 0x1652ee960 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x1652ee9d8 client: 0x12f09320/0x7f30643f8780 data: 0x2ed5da000 len: 65536n<BufferPool::Page> 0x1652ee0a0 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x1652ee118 client: 0x12f09320/0x7f30643f8780 data: 0x2ed5ea000 len: 65536n<BufferPool::Page> 0x1652f0f80 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x1652f0ff8 client: 0x12f09320/0x7f30643f8780 data: 0x2ed5fa000 len: 65536n<BufferPool::Page> 0x7f2db02315e0 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f2db0231658 client: 0x12f09320/0x7f30643f8780 data: 0x2ed70a000 len: 65536n<BufferPool::Page> 0x7f2db02333e0 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f2db0233458 client: 0x12f09320/0x7f30643f8780 data: 0x2ed71a000 len: 65536n<BufferPool::Page> 0x51d9cbfe0 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x51d9cc058 client: 0x12f09320/0x7f30643f8780 data: 0x2ed72a000 len: 65536n<BufferPool::Page> 0x51d9ca5a0 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x51d9ca618 client: 0x12f09320/0x7f30643f8780 data: 0x2ed73a000 len: 65536n<BufferPool::Page> 0x51d9cc800 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x51d9cc878 client: 0x12f09320/0x7f30643f8780 data: 0x2ed74a000 len: 65536n<BufferPool::Page> 0x51d9cc440 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x51d9cc4b8 client: 0x12f09320/0x7f30643f8780 data: 0x2ed75a000 len: 65536n<BufferPool::Page> 0x51d9caa00 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x51d9caa78 client: 0x12f09320/0x7f30643f8780 data: 0x2ed76a000 len: 65536n<BufferPool::Page> 0x51d9cd660 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x51d9cd6d8 client: 0x12f09320/0x7f30643f8780 data: 0x2ed77a000 len: 65536n<BufferPool::Page> 0x51d9cd200 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x51d9cd278 client: 0x12f09320/0x7f30643f8780 data: 0x2ed78a000 len: 65536n<BufferPool::Page> 0x2c5803ac0 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x2c5803b38 client: 0x12f09320/0x7f30643f8780 data: 0x2ed79a000 len: 65536n<BufferPool::Page> 0x7f2ebf86bf40 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f2ebf86bfb8 client: 0x12f09320/0x7f30643f8780 data: 0x2ed7aa000 len: 65536n<BufferPool::Page> 0x7f2ebf86b360 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f2ebf86b3d8 client: 0x12f09320/0x7f30643f8780 data: 0x2ed7ba000 len: 65536n<BufferPool::Page> 0x7f2ebf86dca0 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f2ebf86dd18 client: 0x12f09320/0x7f30643f8780 data: 0x2ed7ca000 len: 65536n<BufferPool::Page> 0x7f2ebf86abe0 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f2ebf86ac58 client: 0x12f09320/0x7f30643f8780 data: 0x2ed7da000 len: 65536n<BufferPool::Page> 0x7f2ebf86afa0 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f2ebf86b018 client: 0x12f09320/0x7f30643f8780 data: 0x2ed7ea000 len: 65536n<BufferPool::Page> 0x7f2ebf86c6c0 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f2ebf86c738 client: 0x12f09320/0x7f30643f8780 data: 0x2f7e08000 len: 65536n<BufferPool::Page> 0x7f2ebf86dac0 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f2ebf86db38 client: 0x12f09320/0x7f30643f8780 data: 0x2fee2e000 len: 65536n<BufferPool::Page> 0x7f2ebf86ad20 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f2ebf86ad98 client: 0x12f09320/0x7f30643f8780 data: 0x2fee3e000 len: 65536n<BufferPool::Page> 0x7f2ebf86d0c0 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f2ebf86d138 client: 0x12f09320/0x7f30643f8780 data: 0x2fee4e000 len: 65536n<BufferPool::Page> 0x7f2ebf86bae0 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f2ebf86bb58 client: 0x12f09320/0x7f30643f8780 data: 0x2fee5e000 len: 65536n<BufferPool::Page> 0x7f2ebf86a140 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f2ebf86a1b8 client: 0x12f09320/0x7f30643f8780 data: 0x2fee6e000 len: 65536n<BufferPool::Page> 0x7f2ebf86d020 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f2ebf86d098 client: 0x12f09320/0x7f30643f8780 data: 0x2fee7e000 len: 65536n<BufferPool::Page> 0x7f2ebf86bcc0 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f2ebf86bd38 client: 0x12f09320/0x7f30643f8780 data: 0x2fee8e000 len: 65536n<BufferPool::Page> 0x7f2ebf86b0e0 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f2ebf86b158 client: 0x12f09320/0x7f30643f8780 data: 0x2fee9e000 len: 65536n<BufferPool::Page> 0x7f2ebf86df20 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f2ebf86df98 client: 0x12f09320/0x7f30643f8780 data: 0x2feeae000 len: 65536n<BufferPool::Page> 0x7f2ebf86c3a0 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f2ebf86c418 client: 0x12f09320/0x7f30643f8780 data: 0x2feebe000 len: 65536n<BufferPool::Page> 0x7f2ebf86d8e0 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f2ebf86d958 client: 0x12f09320/0x7f30643f8780 data: 0x2feece000 len: 65536n<BufferPool::Page> 0x7f2ebf86a460 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f2ebf86a4d8 client: 0x12f09320/0x7f30643f8780 data: 0x2feede000 len: 65536n<BufferPool::Page> 0x7f2ebf86b720 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f2ebf86b798 client: 0x12f09320/0x7f30643f8780 data: 0x2feeee000 len: 65536n<BufferPool::Page> 0x7f2ebf86b040 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f2ebf86b0b8 client: 0x12f09320/0x7f30643f8780 data: 0x2feefe000 len: 65536n<BufferPool::Page> 0x7f2ebf86a820 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f2ebf86a898 client: 0x12f09320/0x7f30643f8780 data: 0x2fef0e000 len: 65536n<BufferPool::Page> 0x7f2ebf86bc20 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f2ebf86bc98 client: 0x12f09320/0x7f30643f8780 data: 0x2fef1e000 len: 65536n<BufferPool::Page> 0x7f2ebf86da20 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f2ebf86da98 client: 0x12f09320/0x7f30643f8780 data: 0x2fef2e000 len: 65536n<BufferPool::Page> 0x7f2ebf86b180 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f2ebf86b1f8 client: 0x12f09320/0x7f30643f8780 data: 0x2fef3e000 len: 65536n<BufferPool::Page> 0x7f2ebf86b2c0 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f2ebf86b338 client: 0x12f09320/0x7f30643f8780 data: 0x2fef4e000 len: 65536n<BufferPool::Page> 0x7f2ebf86d840 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f2ebf86d8b8 client: 0x12f09320/0x7f30643f8780 data: 0x2fef5e000 len: 65536n<BufferPool::Page> 0x7f3385cf6f80 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f3385cf6ff8 client: 0x12f09320/0x7f30643f8780 data: 0x2fef6e000 len: 65536n<BufferPool::Page> 0x7f3385cf7d40 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f3385cf7db8 client: 0x12f09320/0x7f30643f8780 data: 0xf79e000 len: 65536n<BufferPool::Page> 0x7f3385cf7160 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f3385cf71d8 client: 0x12f09320/0x7f30643f8780 data: 0xf7ae000 len: 65536n<BufferPool::Page> 0x7f3385cf69e0 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f3385cf6a58 client: 0x12f09320/0x7f30643f8780 data: 0xf7be000 len: 65536n<BufferPool::Page> 0x7f3385cf6a80 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f3385cf6af8 client: 0x12f09320/0x7f30643f8780 data: 0x115b0000 len: 65536n<BufferPool::Page> 0x7f3385cf4a00 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f3385cf4a78 client: 0x12f09320/0x7f30643f8780 data: 0x169f4000 len: 65536n<BufferPool::Page> 0x7f3385cf7520 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f3385cf7598 client: 0x12f09320/0x7f30643f8780 data: 0x16a0a000 len: 65536n<BufferPool::Page> 0x7f3385cf5180 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f3385cf51f8 client: 0x12f09320/0x7f30643f8780 data: 0x16a1a000 len: 65536n<BufferPool::Page> 0x7f3385cf52c0 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f3385cf5338 client: 0x12f09320/0x7f30643f8780 data: 0x1772e000 len: 65536n<BufferPool::Page> 0x7f3385cf4c80 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f3385cf4cf8 client: 0x12f09320/0x7f30643f8780 data: 0x1773e000 len: 65536n<BufferPool::Page> 0x7f3385cf50e0 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f3385cf5158 client: 0x12f09320/0x7f30643f8780 data: 0x1774e000 len: 65536n<BufferPool::Page> 0x7f3385cf4aa0 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f3385cf4b18 client: 0x12f09320/0x7f30643f8780 data: 0x1775e000 len: 65536n<BufferPool::Page> 0x7f2ebf86c300 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f2ebf86c378 client: 0x12f09320/0x7f30643f8780 data: 0x1954c000 len: 65536n<BufferPool::Page> 0x7f2ebf86d5c0 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f2ebf86d638 client: 0x12f09320/0x7f30643f8780 data: 0x1955c000 len: 65536n<BufferPool::Page> 0x7f2ebf86a3c0 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f2ebf86a438 client: 0x12f09320/0x7f30643f8780 data: 0x1a406000 len: 65536n<BufferPool::Page> 0x7f2ebf86a640 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f2ebf86a6b8 client: 0x12f09320/0x7f30643f8780 data: 0x1b55a000 len: 65536n<BufferPool::Page> 0x7f2ebf86c1c0 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f2ebf86c238 client: 0x12f09320/0x7f30643f8780 data: 0x1b56a000 len: 65536n<BufferPool::Page> 0x7f2ebf86b220 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f2ebf86b298 client: 0x12f09320/0x7f30643f8780 data: 0xf90e000 len: 65536n<BufferPool::Page> 0x7f2ebf86b7c0 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f2ebf86b838 client: 0x12f09320/0x7f30643f8780 data: 0x1105c000 len: 65536n<BufferPool::Page> 0x7f2ebf86ae60 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f2ebf86aed8 client: 0x12f09320/0x7f30643f8780 data: 0x1678c000 len: 65536n<BufferPool::Page> 0x7f2ebf86ba40 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f2ebf86bab8 client: 0x12f09320/0x7f30643f8780 data: 0x18528000 len: 65536n<BufferPool::Page> 0x7f2ebf86c940 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f2ebf86c9b8 client: 0x12f09320/0x7f30643f8780 data: 0x18840000 len: 65536n<BufferPool::Page> 0x7f2ebf86c120 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f2ebf86c198 client: 0x12f09320/0x7f30643f8780 data: 0x18850000 len: 65536n<BufferPool::Page> 0x7f2ebf86d2a0 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f2ebf86d318 client: 0x12f09320/0x7f30643f8780 data: 0x18cc8000 len: 65536n<BufferPool::Page> 0x7f2ebf86d340 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f2ebf86d3b8 client: 0x12f09320/0x7f30643f8780 data: 0x18cd8000 len: 65536n<BufferPool::Page> 0x7f2ebf86cbc0 len: 65536 pin_count: 0 buf: <BufferPool::BufferHandle> 0x7f2ebf86cc38 client: .........
      

        Attachments

          Activity

            People

            • Assignee:
              rizaon Riza Suminto
              Reporter:
              tarmstrong Tim Armstrong
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: