This article looks at After Image Journaling and audit trails in Recital using SET JOURNAL and associated commands.

Overview

After Image Journaling, used in conjunction with a structured backup policy is an effective disaster recovery solution. Any transaction that takes place on a table that is being journaled is logged in the journal file. In the event of a disk crash or similar event in which the table is lost, the journaled transactions can be reapplied to the latest backup copy of the table. Alternatively or additionally, the journal can be used to provide an audit trail to all modifications made to the table data.

NOTE: Recital also provides Before Image Journaling via BEGIN TRANSACTION / END TRANSACTION blocks, allowing unsuccessful transactions to be rolled back to a set saved state.

SET JOURNAL and RECOVER

Regular backups are an essential routine for any system, but in high-transaction environments restoration of the latest backup can still mean a major loss of data. After image journaling can successfully be used as part of your disaster recovery strategy to minimize data loss and down time. Recital after image journaling functionality is based on the use of the SET JOURNAL and RECOVER commands.

SET JOURNAL

SET JOURNAL TO [<.dbj filename> | ()]
SET JOURNAL ON | OFF | ()

The SET JOURNAL command is used to enable the After Image Journaling and audit trail for the active table. The TO <.dbj filename> clause associates the specified transaction journal file with the active table.  If the journal file does not exist, it will be created.  The filename can be substituted with a <expC>, enclosed in round brackets, which returns a valid filename.  If no file extension is specified, ‘.dbj’ is used. When specifying a journal file, it is recommended that the journal file is stored on a different disk than that which the table is stored on, so that if a fatal disk error occurs, then the journal file will not be lost along with the table. 

//Enable journaling for the southwind!orders table
open database southwind
use orders
set journal to /journals/ord_journ

The <.dbj filename> is a standard table.  It contains seven fields that are specific to a journal file, followed by the first 249 fields of the associated table.

The first seven fields in the journal are:

* Dependent on SET CENTURY setting.

The AUD_CMD Command Reference Numbers are as follows:

Since journal files are standard Recital tables, you can use standard Recital commands such as the REPORT command to print audit trails, transaction logs, etc.

//Enable journaling for the southwind!orders table
open database southwind
use orders
set journal to /journals/ord_journ
//.. transactions
close data
//View journaled records
use /journals/ord_journ.dbj

Click image to display full size

Fig 1: Journal Record Example.

The SET JOURNAL TO command without a <.dbj filename> specified closes the active journal file and no further journaling will take place on the active table until the SET JOURNAL TO <.dbj filename> is reissued.

The journaling features are mainly used with shared tables.  It should be noted that there is an overhead in enabling transaction journaling, as records updated in a table are also written to the journal file. When records are appended into a journal file, locking is automatically performed so that multiple users can update the journal concurrently.  The associated table must be opened shareable for this to occur.  Each table can have a journal file associated with it. 

The SET JOURNAL ON | OFF command enables or disables transaction journaling.  This command is primarily used in applications where journaling can be disabled for a certain class of operations. By default, SET JOURNAL is ON, but no journal files are set.

NOTE: Only the first 249 fields of a table can be journaled: subsequent fields are ignored. The maximum number of fields in a Recital table is 256.

RECOVER

RECOVER FROM <.dbj filename> | ()

The RECOVER command uses the journal file to reapply lost transactions to a previous backup of the data after a fatal error such as a disk head crash. The FROM clause specifies the journal file to use. The file name can be substituted with an <expC>, enclosed in round brackets, which returns a valid filename.  If no file extension is specified, then ‘.dbj’ is assumed. 

Regular backups are essential to the successful use of After Image Journaling.   It is also very important to reinitialize the journal file after each backup: either open the journal file as you would a normal table and use the ZAP command, or delete the file completely. If a fatal error occurs, such as a disk head crash, the table and index files must be restored from a backup, then the RECOVER command executed. RECOVER will reapply' all of the transactions in the journal file to the table, and update the indexes.  After the RECOVER command has completed, you can continue with normal processing. 

//Create a backup of the southwind!orders table
//...backup table and associated files
//Reinitialize the journal file
erase /journals/ord_journ.dbj
//Enable journaling for the southwind!orders table
open database southwind
use orders
set journal to /journals/ord_journ
//.. transactions
//Restore the backup of the southwind!orders table
//...restore
//Open the restored backup
open database southwind
use orders
//Reapply the transactions using the journal
recover from /journals/ord_journ.dbj
//Now, enable the journal file again or
//restart with a new backup

Journaling Memo Fields

By default, memo fields - variable length text fields - are not journaled due to the possible storage overhead of multiple copies of potentially large blocks of text. But, if memo journaling is required, the SET MEMOJOURNAL ON command can be used to enable this.

SET MEMOJOURNAL

SET MEMOJOURNAL ON | OFF | ()

The SET MEMOJOURNAL command causes memo fields to be journaled when journaling is set on a table. This command allows the optional logical expression <expL> to be evaluated.  If a value of  .T. is returned, MEMOJOURNAL is set ON.  If a value of .F. is returned, MEMOJOURNAL is set OFF.  By default SET MEMOJOURNAL is OFF.

Like a normal Recital table, the journal holds only a pointer to a data block in an associated memo file, not the actual memo data itself. The journal's memo file has a file extension of .dbm rather than the standard Recital .dbt. Therefore, if the journal is being opened as a table, in order to view the journal's memo data, the SET MEMOEXT command should be used.

//Enable journaling for the southwind!suppliers table
open database southwind
use suppliers
set journal to /journals/sup_journ
//.. transactions
close data
//Set filename extension for memo file
set memoext to '.dbm'
//View journaled records
use /journals/sup_journ.dbj

Summary

The After Image Journaling enabled by the SET JOURNAL and RECOVER commands can be used in conjunction with a strict backup regime to minimize data loss in cases where tables become damaged or irretrievable. Journal files can be accessed like standard Recital tables and provide detailed information about the transactions applied to a table, so can be used for auditing purposes.

 
Key features of the Recital database include:

• SQL-92 and a broad subset of ANSI SQL 99, as well as extensions
• Cross-platform support
• Stored procedures
• Triggers
• Cursors
• Updatable Views
• System Tables
• Query caching
• High-performance
• Single-User and Multi-User
• Multi-Process
• ACID Transactions
• Referential Integrity
• Cascading Updates and Deletes
• Multi-table Joins
• Row-level Locking
• BLOBs (Binary Large Objects)
• UDFs (User Defined Functions)
• OLTP (On-Line Transaction Processing)
• Drivers for ODBC, JDBC, and .NET
• Sub-SELECTs (i.e. nested SELECTs)
• Embedded database library
• Database timelines providing data undo functionality
• Fault tolerant clustering support
• Hot backup

In this article Chris Mavin, explains and details how to use the Recital Database Server with the Open Source Servlet Container Apache Tomcat.

Overview

PHP has exploded on the Internet, but its not the only way to create web applications and dynamic websites. Using Java Servlets, JavaServer Pages and Apache Tomcat you can develop web applications in a more powerful full featured Object Oriented Language, that is easier to debug, maintain, and improve.

Tomcat Installation

There are a number of popular Java application servers such as IBM Web Sphere and BEA WebLogic but today we will be talking about the use of Apache Tomcat 5, the Open Source implementation of the Java Servlet and JavaServer Pages technologies developed at the Apache Software Foundation. The Tomcat Servlet engine is the official reference implementation for both the Servlet and JSP specifications, which are developed by Sun under the Java Community Process. What this means is that the Tomcat Server implements the Servlet and JSP specifications as well or better than most commercial application servers.

Apache Tomcat is available for free but offers many of the same features that commercially available Web application containers boast.

Tomcat 5 supports the latest Servlet and JSP specifications, Servlet 2.4, and JSP 2.0, along with features such as:

• Tomcat can run as a standalone webserver or a Servlet/JSP engine for other Web Servers.

• Multiple connectors - for enabling multiple protocol handlers to access the same Servlet engine.

• JNDI - The Java Naming and Domain Interface is supported.

• Realms - Databases of usernames and passwords that identify valid users of a web application.

• Virtual hosts - a single server can host applications for multiple domain names. You need to edit server.xml to configure virtual hosts.

• Valve chains.

• JDBC - Tomcat can be configured to use any JDBC driver.

• DBCP - Tomcat can use the Apache commons DBCP for connection pooling.

• Servlet reloading (Tomcat monitors any changes to the classes deployed within that web server.)

• HTTP functionality - Tomcat functions as a fully featured Web Server.

• JMX, JSP and Struts-based administration.

Tomcat Installation

In this next two sections we will walk through the install and setup of Tomcat for use with the Recital database server.

To download Tomcat visit the Apache Tomcat web site is at http://jakarta.apache.org/tomcat.
Follow the download links to the binary for the hardware and operating system you require.

For Tomcat to function fully you need a full Java Development Kit (JDK). If you intend to simply run pre compiled JavaServer pages you can do so using just the Java Runtime Environment(JRE).

The JDK 1.5 is the preferred Java install to work with Tomcat 5, although it is possible to run Tomcat 5 with JDK 1.4 but you will have to download and install the compat archive available from the Tomcat website.

For the purpose of this article we will be downloading and using Tomcat 5 for Linux and JDK 5.0, 
you can download the JDK at http://java.sun.com/javase/downloads/index.jsp.

Now we have the JDK, if the JAVA_HOME environment variable isn't set we need to set it to refer to the base JDK install directory.

Linux/Unix:

$ JAVA_HOME= /usr/lib/j2se/1.4/
$ EXPORT $JAVA_HOME

Windows NT/2000/XP:

Follow the following steps:

1. Open Control Panel.
2. Click the System icon.
3. Go to the Advanced tab.
4. Click the Environment Variables button.
5. Add the JAVA_HOME variable into the system environment variables.

The directory structure of a Tomcat installation comprises of the following:

/bin 			- Contains startup, shutdown and other scripts. 
	/common  	- Common classes that the container and web applications can use.
	/conf 		- Contains Tomcat XML configuration files XML files.
	/logs 		- Serlvet container and application logs.
	/server 		- Classes used only by the Container.
	/shared 		- Classes shared by all web application.
	/webapps 	- Directory containing the web applications.
	/work 		- Temporary directory for files and directories.

The important files that you should know about are the following:

• server.xml

The Tomcat Server main configuration file is the [tomcat install path]\conf\server.xml file. This file is mostly setup correctly for general use. It is within this file where you specify the port you wish to be running the server on. Later in this article I show you how to change the default port used from 8080 to port 80.

• web.xml

The web.xml file provides the configuration for your web applications. There are two locations where the web.xml file is used, 
web-inf\web.xml provides individual web application configurations and [tomcat install path]conf\web.xml contains the server wide configuration.

Setting up Tomcat for use

We'll start by changing the port that Tomcat will be listening on to 80.

To do this we need to edit [tomcat install path]/conf/server.xml and change the port attribute of the connector element from 8080 to 80.

After you have made the alteration the entry should read as:

<!-- Define a non-SSL HTTP/1.1 Connector on port 8080 -->
<Connector port="80" maxHttpHeaderSize="8192"

Next we want to turn on Servlet reloading, this will cause the web application to be recompiled each time it is accessed, allowing us to make changes to the files without having to worry about if the page is being recompiled or not.

To enable this you need to edit [tomcat install path]/conf/context.xml and change <Context> element to <Context reloadable="true">.

After you have made the alteration the entry should read as:

<Context reloadable="true">
<WatchedResource>WEB-INF/web.xml</WatchedResource>
</Context>

Next we want to enable the invoker Servlet.

The "invoker" Servlet executes anonymous Servlet classes that have not been defined in a web.xml file.  Traditionally, this Servlet is mapped to the URL pattern "/servlet/*", but you can map it to other patterns as well.  The extra path info portion of such a request must be the fully qualified class name of a Java class that implements Servlet, or the Servlet name of an existing Servlet definition.

To enable the invoker Servlet you need to edit the to [tomcat install path]/conf/web.xml and uncomment the Servlet and Servlet-mapping elements that map the invoker /servlet/*.

After you have made the alteration the entry should read as:

<servlet>
<servlet-name>invoker</servlet-name>
<servlet-class>org.apache.catalina.servlets.InvokerServlet</servlet-class>
<init-param>
<param-name>debug</param-name>
<param-value>0</param-value>
</init-param>
<load-on-startup>2</load-on-startup>
</servlet>

<servlet-mapping>
<servlet-name>invoker</servlet-name>
<url-pattern>/servlet/*</url-pattern>
</servlet-mapping>

If you are you not interested in setting up your own install of Tomcat there are prebuilt versions Tomcat that has all of the above changes already made, and has the test HTML, JSP, and Servlet files already bundled. Just unzip the file, set your JAVA_HOME

Next we will give Tomcat and your web applications access to the Recital JDBC driver.

For the purposes of this article we are going to install the Recital JDBC driver in the /[tomcat install path]/common/lib/ this gives Tomcat and your web applications access to the Recital JDBC driver. The driver can be installed in a number of places in the Tomcat tree, giving access to the driver to specific application or just to the web application and not the container. For more information refer to the Tomcat documentation.

Copy the recitalJDBC.jar which is located at /[recital install path]/drivers/recitalJDBC.jar to the /[tomcat install path]/common/lib/ directory.

Linux:

$cp /[recital install path]/drivers/recitalJDBC.jar /[tomcat install path]/common/lib/

Once you have completed all the steps detailed above, fire up the server using the script used by your platform's Tomcat installation.

Linux/Unix:

[tomcat install path]/bin/startup.sh

Windows:

[tomcat install path]/bin/startup

If you are having problems configuring your Tomcat Installation or would like more detail visit the online documentation a the Apache Tomcat site.

Example and Links

Now we have setup our Tomcat installation, lets get down to it with a JSP example which uses the Recital JDBC driver to access the demonstration database (southwind) shipped with the Recital Database Server.

The example provided below is a basic JDBC web application, where the user simply selects a supplier from the listbox and requests the products supplied by that supplier.

To run the example download and extract the tar archive or simple save each of the two jsp pages individually into /[tomcat install path]/webapps/ROOT/ on your server.

By enabling the invoker Servlet earlier we have removed the need to set the example up as a web application in the Tomcat configuration files.

You can now access the example web application at http://[Server Name]/supplier.jsp if the page doesn't display, check you have followed all the Tomcat installation steps detailed earlier in this article and then make sure both Tomcat and a licensed Recital UAS are running.

Downloads:
Archive: jspExample.tar

Right click and save as individual files and rename as .jsp files:
supplier.txt details.txt

Further Reading on JSP and JDBC can be found at http://www-128.ibm.com/developerworks/java/library/j-webdata/

Final Thoughts

Recital and Apache tomcat are a powerful combination, using Java Servlet technology you can separate application logic and the presentation extremely well. Tomcat, JSP, Java Servlets and the Recital database server form a robust platform independent, easily maintained and administered solution with which to unlock the power of your Recital, Foxpro, Foxbase, Clipper, RMS and C-SAM data.