ActiveMQ
  1. ActiveMQ
  2. AMQ-1529

The blob of a blob-message will never be deleted

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 5.0.0, 5.1.0, 5.2.0
    • Fix Version/s: 5.6.0
    • Component/s: Message Store
    • Labels:
      None
    • Environment:

      Windows vista, java 6

    • Patch Info:
      Patch Available

      Description

      In a blob-message you can specify that the broker can delete the blob if the message is delivered or outdated.
      But actual onle the message with the reference to the blob is deletet, the use-data in the repository not. The problem is the separation of the blob from the message itself.

      My idea is to extend the persistence store to check at delete if it is a blob message and then delete the blob from the repository. I have done this in the jdbc- and kaha message store (see patch).

      JDBC message store:

      • created a new table with all blob url`s which must be delete if the message is deleted
        ID (primary Key) MSGID (reference to ID in MESSAGE table) URL
      • on message add check if it is a blob-message and if the broker must delete the message add the information to the new table
      • in cleanup a query on that table finds all urls with no existing reference in the MESSAGE table (becaus the message is deleted) and then deletes this blobs

      KAHA message store:

      • TopicSubAck has a new attribute remotBlobUrl
      • on message add check if it is a blob-message and if the broker must delete the message add the url to the TopicSubAck
      • on acknowledge at delete check if the attribute is set an if it is set delete the blob

      THINGS TO DO

      • add a similar mechanism to the other message stores
      • only the first delete is successfull, every other delete request ends in an 500 errorcode from the server

        Activity

        Hide
        Marco Buss added a comment -

        The error 500 occours due to the fact that is an lock on the file.
        I think this happens on jetty becaus the server holds the connection for a time. And so when the message is consumed and a GET is invoked for this file the lock ois set until the connection will be closed.

        Show
        Marco Buss added a comment - The error 500 occours due to the fact that is an lock on the file. I think this happens on jetty becaus the server holds the connection for a time. And so when the message is consumed and a GET is invoked for this file the lock ois set until the connection will be closed.
        Hide
        Vesna Djurdjevic added a comment -

        I have checked the error 500 in the activemq-fileserver.
        The problem is that streams are not closed when the file is uploaded on the server.

        this would fix the problem

        org.apache.activemq.util.RestFilter.java
            protected void doPut(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
        
                // some code missing here...
        
                FileOutputStream out = new FileOutputStream(file);
                try {
                    IO.copy(request.getInputStream(), out);
                } catch (IOException e) {
                    Log.warn(Log.EXCEPTION, e); // is this obsolete?
                    throw e;
                }finally{
                    out.close();
                }
        
                response.setStatus(HttpURLConnection.HTTP_NO_CONTENT); // we return no
                                                                        // content
            }
        

        Also in KahaMessageStore.java should be added removing the blob files when the message is removed.

        Show
        Vesna Djurdjevic added a comment - I have checked the error 500 in the activemq-fileserver. The problem is that streams are not closed when the file is uploaded on the server. this would fix the problem org.apache.activemq.util.RestFilter.java protected void doPut(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { // some code missing here... FileOutputStream out = new FileOutputStream(file); try { IO.copy(request.getInputStream(), out); } catch (IOException e) { Log.warn(Log.EXCEPTION, e); // is this obsolete? throw e; } finally { out.close(); } response.setStatus(HttpURLConnection.HTTP_NO_CONTENT); // we return no // content } Also in KahaMessageStore.java should be added removing the blob files when the message is removed.
        Hide
        Claus Ibsen added a comment -

        AMQ-2430 fixed this.

        Show
        Claus Ibsen added a comment - AMQ-2430 fixed this.
        Hide
        Petr Nídl added a comment -

        This issue is not fixed in 5.7.0.
        org.apache.activemq.command.ActiveMQBlobMessage.deleteFile() method is the only (and I suppose that advisable) way to delete the actual BLOB but it is never called from ActiveMQ classes (not even for messages marked as "deletedByBroker").
        For acknowledged messages you can use simple workaround and define custom acknowledge callback that calls the delete method. But for expired messages I see no elegant way to force ActiveMQ to delete the BLOB. Marco's patch could work but it is not used in 5.7.0 release.
        Is there some other way that broker deletes BLOB messages or am I right that this feature doesn't work?

        Show
        Petr Nídl added a comment - This issue is not fixed in 5.7.0. org.apache.activemq.command.ActiveMQBlobMessage.deleteFile() method is the only (and I suppose that advisable) way to delete the actual BLOB but it is never called from ActiveMQ classes (not even for messages marked as "deletedByBroker"). For acknowledged messages you can use simple workaround and define custom acknowledge callback that calls the delete method. But for expired messages I see no elegant way to force ActiveMQ to delete the BLOB. Marco's patch could work but it is not used in 5.7.0 release. Is there some other way that broker deletes BLOB messages or am I right that this feature doesn't work?

          People

          • Assignee:
            Claus Ibsen
            Reporter:
            Marco Buss
          • Votes:
            1 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development