[OLIO-53] Database load fails at high user counts

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Component/s: generic
• Assignee: Akara Sucharitakul
• Reporter: Shanti Subramanyam
• Created: 23/Feb/09 21:16
• Updated: 02/Mar/09 21:23
• Resolved: 24/Feb/09 19:40

Description

The current database loader does not load for a user population of over 3000 (this is with the scalefactor set to 100x i.e. the db has 100 times the #concurrent users).

If we edit the dbloader.sh to add a '-Xmx3g' option, the loader works upto a scale of 7000.

Beyond that, the loader fails with:
Feb 17, 2009 9:29:14 PM com.sun.web20.loader.framework.ThreadConnection processBatch
WARNING: com.sun.web20.loader.Person: Retry loading.
java.sql.BatchUpdateException: Duplicate entry 'jjq' for key 'PRIMARY'
at com.mysql.jdbc.PreparedStatement.executeBatchSerially(PreparedStatement.java:1237)
at com.mysql.jdbc.PreparedStatement.executeBatch(PreparedStatement.java:936)
at com.sun.web20.loader.framework.ThreadConnection.flush(ThreadConnection.java:220)
at com.sun.web20.loader.framework.ThreadConnection.processBatch(ThreadConnection.java:166)
at com.sun.web20.loader.framework.Loader$4.run(Loader.java:184)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Feb 17, 2009 9:29:14 PM com.sun.web20.loader.framework.ThreadConnection processBatch
WARNING: com.sun.web20.loader.Person: Failed to update.
java.sql.BatchUpdateException: Duplicate entry 'ycb9vui' for key 'PRIMARY'
at com.mysql.jdbc.PreparedStatement.executeBatchSerially(PreparedStatement.java:1237)
at com.mysql.jdbc.PreparedStatement.executeBatch(PreparedStatement.java:936)
at com.sun.web20.loader.framework.ThreadConnection.flush(ThreadConnection.java:220)
at com.sun.web20.loader.framework.ThreadConnection.processBatch(ThreadConnection.java:166)
at com.sun.web20.loader.framework.Loader$4.run(Loader.java:184)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Error count exceeded threshold of 50! Exiting.
ERROR: Database loader exited with code 2.