How to set up in memory database?

Overview

Many client systems have for a number of years used "in memory" or "private" databases for the engine/app layers. Private ( in memory) databases can be used for the engine and application layer databases which are effective static between releases. Private ( in memory ) databases allow for many performance enhancements and reduces downtime when upgrading servers.  Private databases should not be used for data layers ( ones that we want to keep data between releases).

Optimizations

• Searches and data loading of an in memory database does not require over the wire SQL calls thus much faster than normal database calls.
• No need to listen to or publish JMS messages.

Limitations

• Each private ( in memory ) database is effectively READONLY or at least no permanent data will be stored. 
• A copy of a private ( in memory) database can only be accessed by one process at a time.
• The engine/app "private" databases are deployed just like they are jar files. Just like if you want to change a installed server jar file you need to deploy the new jar and restart the program as you can just change a jar underneath a running process.

Implementation

• When a new process starts, a copy of the private databases are copied from the "data" directory to the "private/db/" and the application name.

• The environment variable private.db.list defines the list of databases layers that will be treated as "private".

• The private ( in-menory) database is loaded from the database files in "private/db/${application name}/${database name}" on demain. 

Build Process

1. Dump the engine/app databases.
2. Load the dumped databases into HSQLDB database and delete all data that should not be deployed to production. 
3. Close the HSQLDB databases for each layer and store in private database files into the "data" directory.
4. Export the "cleaned" databases into a portable XML format which can be later loaded into any supported database ( Postgres, MySQL, MSSQL, Sybase or Oracle)

Directory Listing

List of full install version of the system. To rollback a previous version of the system, change the symbolic link and restart.

List of a single installation 

List of private databases.