HornetQ Persistence Concepts
Internal
Overview
This article provides a high level overview of the HornetQ persistence concepts. It will describe what kind of data is persisted, as well as where and when. It will also discuss paging, which is a protection mechanism against running out of memory; persistence is relevant in this context because messages that do not fit in memory go to the filesystem, even if the messages themselves are marked as non-persistent.
There is No Database
Unlike other messaging systems, which do offer the option of storing message data in a relational database, HornetQ does not. For reasons that led to this decision see https://developer.jboss.org/thread/153581. More details in "Messaging persistence in EAP 6.x" https://access.redhat.com/solutions/226743.
What Does HornetQ Persist?
Naturally, HornetQ persists persistent messages, as required by the JMS specification. It also persists some topology information (bindings and JMS information). HornetQ allows sending large messages - a message can be larger than the total amount of memory available to a broker - by fragmenting the messages and storing the fragments on the filesystem, so large message storage is another type of persistence managed by HornetQ. Finally, HornetQ is capable of storing any message, including the non-persistent messages, on the filesystem, when the amount of memory available to the broker is not sufficient to allow handling all messages for a specific address in memory. This mechanism is known as paging and it is describe here.
Persistent Message Journal
All persistent messages must be stored on persistent storage, as mandated by the JMS specification. This is necessary to protect against messaging system failure: a persistent message can be presumably recovered from the persistent storage and re-sent.
HornetQ stores messages in a high-performance append-only file system journal, optimized for message-specific use cases. The journal consists of a set of fixed-size files. Initially, the files are filled with padding, which is progressively replaced with message dat or deletes. When a journal file is full, the next one is used and so on.
Spinning Disks
A great deal of effort has been spent to optimize the journal for spinning hard drives. It's too bad that more and more of these are in process to be replaced with solid state drives, for which the head mechanics is irrelevant.
Bindings
JMS
Large Messages
Non Persistent Messages
Journal
When a node is started for the first time it persists a unique identifier into its journal directory. This ID is needed for proper formation of clusters.
HornetQ Data Directories
The configuration allows the possibility of creating the HornetQ bindings and journal data directories at startup, if they do not already exist. This configuration could be useful in "experimental" mode when one deletes and recreates HornetQ data files for whatever reason, and probably not that useful in production. If the directories exist, they are not re-created, so the "create" options can be left in place, even in a production configuration. However, there is another set of directories (large messages and paging) that will be created automatically if they don’t exist, in absence of any explicit configuration option. For production, it’s probably best if the directories are created manually as part of the installation procedure, and "create-*" options are removed from configuration.