Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Fixed
-
2.0-incubating
-
None
Description
An mxosrvr process which has been connected to and seen at least one SQL statement given by a user executed will leak memory from the repository thread, The rate of observed leak was about 3 GB during one month. The rate does depend on the amount of activity in the repository thread which in turn depends on the activity in the main thread. In this case the repository thread was periodically executing 2 statements. There was another observed instance of about a 2GB leak after one month.
This is top output
55682 trafodio 20 0 4033m 2.9g 33m S 0.3 2.4 157:16.88 mxosrvr
With gdb we see that the the statement heap of the repository context is leaking
*cli_globals->executorMemory_>memoryList_>nextEntry_>nextEntry_>memoryList_
$104 = {
<NABasicObject> =
,
members of NAMemory:
name_ = "Statement Heap\000\000\001\000\000\000\002",
type_ = NAMemory::DERIVED_MEMORY,
initialSize_ = 8096,
maximumSize_ = 18446744073709551615,
incrementSize_ = 4194304,
parent_ = 0x7f533a92c1f0,
firstBlk_ = 0x7f525ede1060,
allocSize_ = 2393256448,
upperLimit_ = 0,
highWaterMark_ = 2393526936,
intervalWaterMark_ = 2393526936,
allocCnt_ = 458873,
totalSize_ = 2393389344,
blockCnt_ = 648,
thBlockCnt_ = 110,
Memory is allocated with every execution (once a minute), but rarely deallocated.
NAHeap::allocateBlock (this=0x7f3c7f3dbe70, size=32848, failureIsFatal=1) at ../common/NAMemory.cpp:2535
2535 p->segmentId_ = segmentId;
(gdb) bt
#0 NAHeap::allocateBlock (this=0x7f3c7f3dbe70, size=32848, failureIsFatal=1) at ../common/NAMemory.cpp:2535
#1 0x00007f3ca6b2105f in NAHeap::allocateHeapMemory (this=0x7f3c7f3dbe70, userSize=32767, failureIsFatal=1)
at ../common/NAMemory.cpp:3360
#2 0x00007f3ca6b1d025 in NAMemory::allocateMemory (this=0x7f3c7f3dbe70, size=32767, failureIsFatal=1)
at ../common/NAMemory.cpp:1380
#3 0x00007f3ca39cc9e1 in operator new[] (t=32767, h=0x7f3c7f3dbe70) at ../export/NABasicObject.cpp:375
#4 0x00007f3ca575e327 in ExHbaseUMDtrafUniqueTaskTcb::ExHbaseUMDtrafUniqueTaskTcb (this=0x7f3c7f4009b8, tcb=0x7f3c7f3dea68)
at ../executor/ExHbaseIUD.cpp:1758
#5 0x00007f3ca5763c68 in ExHbaseAccessUMDTcb::ExHbaseAccessUMDTcb (this=0x7f3c7f3dea68, hbaseAccessTdb=..., glob=
0x7f3c7f3dc9a0) at ../executor/ExHbaseIUD.cpp:3520
#6 0x00007f3ca573beef in ExHbaseAccessTdb::build (this=0x7f3c7f3ec928, glob=0x7f3c7f3dc9a0)
at ../executor/ExHbaseAccess.cpp:82
#7 0x00007f3ca563c193 in ex_root_tdb::build (this=0x7f3c7f3e81e8, cliGlobals=0x1378240, glob=0x7f3c7f3dc9a0)
at ../executor/ex_root.cpp:209
#8 0x00007f3ca6fb27ac in CliStatement::fixup (this=0x7f3c7f3dbdb0, cliGlobals=0x1378240, input_desc=0x0, diagsArea=...,
doSimCheck=@0x7f3c7fe3973c, partitionUnavailable=@0x7f3c7fe3975c, donePrepare=0) at ../cli/Statement.cpp:2929
#9 0x00007f3ca6fb518a in CliStatement::execute (this=0x7f3c7f3dbdb0, cliGlobals=0x1378240, input_desc=0x0, diagsArea=...,
execute_state=CliStatement::INITIAL_STATE_, fixupOnly=0, cliflags=0) at ../cli/Statement.cpp:3966
#10 0x00007f3ca6f30324 in SQLCLI_PerformTasks(CliGlobals *, ULng32, SQLSTMT_ID *, SQLDESC_ID *, SQLDESC_ID *, Lng32, Lng32, typedef __va_list_tag __va_list_tag *, SQLCLI_PTR_PAIRS *, SQLCLI_PTR_PAIRS *) (cliGlobals=0x1378240, tasks=606,
statement_id=0x2589398, input_descriptor=0x0, output_descriptor=0x0, num_input_ptr_pairs=0, num_output_ptr_pairs=0,
ap=0x7f3c7fe39da0, input_ptr_pairs=0x0, output_ptr_pairs=0x0) at ../cli/Cli.cpp:3301
#11 0x00007f3ca6f31426 in SQLCLI_ExecDirect2(CliGlobals *, SQLSTMT_ID *, SQLDESC_ID *, Int32, SQLDESC_ID *, Lng32, typedef __va_list_tag __va_list_tag *, SQLCLI_PTR_PAIRS *) (cliGlobals=0x1378240, statement_id=0x2589398, sql_source=0x7f3c7fe39ff0,
prepFlags=0, input_descriptor=0x0, num_ptr_pairs=0, ap=0x7f3c7fe39da0, ptr_pairs=0x0) at ../cli/Cli.cpp:3735
#12 0x00007f3ca6fcd8eb in SQL_EXEC_ExecDirect2 (statement_id=0x2589398, sql_source=0x7f3c7fe39ff0, prep_flags=0,
input_descriptor=0x0, num_ptr_pairs=0) at ../cli/CliExtern.cpp:2327
#13 0x00007f3ca9ad5445 in SRVR::WSQL_EXEC_ExecDirect (statement_id=0x2589398, sql_source=0x7f3c7fe39ff0,
input_descriptor=0x0, num_ptr_pairs=0) at SQLWrapper.cpp:363
#14 0x00007f3ca9abc1d0 in SRVR::EXECDIRECT (pSrvrStmt=0x2588d80) at sqlinterface.cpp:4671
#15 0x00007f3ca9a47ef7 in SRVR::ControlProc (pParam=0x2588d80) at csrvrstmt.cpp:768
#16 0x00007f3ca9a47535 in SRVR_STMT_HDL::ExecDirect (this=0x2588d80, inCursorName=0x0,
inSqlString=0x268d078 "update Trafodion.\"REPOS\".metric_query_aggr_table set AGGREGATION_LAST_UPDATE_UTC_TS = CONVERTTIMESTAMP(212340712811423977),AGGREGATION_LAST_ELAPSED_TIME = 60000,TOTAL_EST_ROWS_ACCESSED = 0,TOTAL_EST"..., inStmtType=1,
inSqlStmtType=0, inSqlAsyncEnable=0, inQueryTimeout=0) at csrvrstmt.cpp:450
#17 0x0000000000578eaa in SessionWatchDog (arg=0x0) at SrvrConnect.cpp:974
Attachments
Issue Links
- is duplicated by
-
TRAFODION-2309 Memory leak observed in Repository context
- Closed