Uploaded image for project: 'AdMax'
  1. AdMax
  2. ADMAX-2863

Disk fills up with log file output when DB crashes.

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: Sustaining
    • Component/s: Data Summarization
    • Labels:
      None
    • Environment:

      dc1bos trunk since June 26 (and probably prior)

      Description

      Here's the log file from when it happened... it was 69G so the log file probably does not exist, but Jonah was able to grab the stack trace.

      ./sesummarize.sh_d_3_S_3_T_2011_06_25_a_421,429,653,679,781,796_04:55:15.log

      ----------------------------------------

      2011-06-26 07:50:57.559 (2) [P13T1]: getLock: WarehouseSummarizer.summarize.429: Timeout after 600 seconds. Retrying...

      2011-06-26 07:50:57.559 (2) [P13T1]: Exception [getLock(WarehouseSummarizer.summarize.429, 600)]:com.mysql.jdbc.exceptions.MySQLNonTransientConnectionException: No operations allowed after connection closed.Connection was implicitly closed due to underlying exception/error:

      Last packet sent to the server was 507169 ms ago.

      STACKTRACE:

      com.mysql.jdbc.CommunicationsException: Communications link failure due to underlying exception:

        • BEGIN NESTED EXCEPTION **

      java.io.EOFException

      STACKTRACE:

      java.io.EOFException

      at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:1913)

      at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2304)

      at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2803)

      at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1573)

      at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:1665)

      at com.mysql.jdbc.Connection.execSQL(Connection.java:3170)

      at com.mysql.jdbc.Connection.execSQL(Connection.java:3099)

      at com.mysql.jdbc.Statement.executeQuery(Statement.java:1172)

      at com.carldunham.jst.db.Database.executeQuery(Database.java:356)

      at com.carldunham.jst.db.Database.executeQuery(Database.java:308)

      at com.carldunham.jst.db.Database.getLock(Database.java:499)

      at com.thesearchagency.perf.WarehouseSummarizer$AccountWorker.run(WarehouseSummarizer.java:506)

      at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)

      at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)

      at java.util.concurrent.FutureTask.run(Unknown Source)

      at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)

      at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

      at java.lang.Thread.run(Unknown Source)

        • END NESTED EXCEPTION **

      ----------------------------------------

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              therouxj Jeff Theroux
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated: