Recital 10 release notes

Recital is a dynamic programming language with an integrated high performance database particularly well suited for the development and deployment of high transaction throughput applications. Recital 10 further enhances Recital with extensive features and functionality to facilitate its use in fault tolerant high availability systems. Much of the development of Recital 10 was concentrated on performance optimizations and cluster aware functionality to provide an application platform that can be scaled as needed without any application changes.

Key features of Recital 10 include:

Cluster aware database engine that works transparently with drbd, heartbeat, glusterfs and samba
High degree of fault tolerance with self healing indexes
Massive performance improvements
Extensive internals overall and modernization with superior object-oriented capabilities
Chronological data versioning with database timelines
SmartQuery caching
Database Administration Tools
Code and Data Profiling
Better integration with unix/linux command shell
Incorporates a range of new built-in functions compatible with those in the PHP core libraries
Built-in support for outputting data in HTML, XML, and JSON format
Seamless SQL command integration into the Recital scripting language
Much improved Microsoft FoxPRO language compatibility
Numerous extensions and improvements (see below for details)
Very large file support (2^63)

Running Recital

You run recital by typing "recital" or "db" at a shell prompt. Linux also has a man page that describes how to use the recital command.

$ recital
$ recital --help
$ man recital

Starting the Recital Server

You start up the Recital Server and Recital Web on linux using the linux service command. Linux also has a man page that describes how to use the recital server.

# service recital start
# service recital status
# man recitalserver
# man recitalweb

Recital Express Editions

Recital 10 introduces a new single-user developer edition called Recital Express that can be used to develop and test multi-user Recital, Recital Server and Recital Web applications. Once the applications are ready for deployment a commercial license must be purchased. Recital Express, Recital Server Express and Recital Web Express can be used unlicensed for non-commercial purposes only.

New Cluster aware fault tolerant database engine

The Recital 10 database engine is now cluster aware using the SET NETWORK ON command. If data is being accessed on a clustered/network file system (using samba and/or glusterfs in conjunction with drbd and heartbeat) and the master server fails and falls over to a backup server, the Recital database engine maintains file state (open files, active locks etc) and will automatically restore application state against the backup server. A clustered Recital application provides for a high degree of fault tolerance and high availability suited to 24x7 applications.

Recital SmartQuery

Recital 10 introduces new SmartQuery technology which improves SQL SELECT performance dramatically. Most web applications are WORM applications (Write Once Read Many). When SET SMARTQUERY ON is in effect, then data selected with SQL SELECT commands is cached. When the same SELECT query is run again and no modifications have been made to the tables involved in the query, then the cached result set is returned without having to re-execute the query. This improves web application query performance. cached results are deleted automatically when their lifetime expires which defaults to any changes being made within 1 second. You can change the lifetime with the SET SMARTQUERY TO <lifetime_in_seconds> command.

Database Timelines

Recital 10 introduces chronological data versioning known as database timelines. When SET TIMELINE ON is in effect, then all changes made to data are recorded together with the previous data values, who changed the data, when they changed it, and from where they changed it. The timeline can be queried via the SYSTIMESLINES table, and rolled back to a previous data and time if so required. Timelines are particularly useful in Recital Web applications.

Increased variable name length

In versions prior to Recital 10 the maximum length of a variable name was 10 characters. This has been increased to 32 and is consistent with field names and function names .

Increased string length

In versions of Recital previous to 10.0 the maximum length of a string was 8K. This has been extended to 64K and works throughout all of the functions and commands in the product.

Extended Text/Memo Editor

The Text/Memo Editor can now edit and display files of up to 32k lines. Cut, Copy, Paste, Delete and Yank lines has been implemented.

String functions with memos

The character string functions can now be used with memo fields e.g. left( ), right( ), upper( ), lower( ), at( ), rat( ) substr( ) etc. Additionally, the relational operators <, >, <=, >=, = and != can be used to compare memos (LONG VARCHAR) in the same way you would compare character strings.

Self healing indexes

The Recital 10 database engine will now automatically repair any index files if they become corrupted if SET AUTOHEAL ON is in effect (default ON). This operation is transparent to the users and provides for high availability applications. There is now no need to have users logoff to reindex the data.

Reduced memory footprint

Prior to the release of Recital 10 we made a extensive code review and removed any large static memory allocations and changed the engine to dynamically allocate memory as needed.

Dynamically allocated workareas

Prior to the release of Recital 10 workareas (cursors) were pre-allocated by specifying the DB_MAXWKA environment variable in recital.conf file. This resulted in large amounts of memory being allocated for even the smallest and simplest of applications. Recital 10 dynamically allocates a workarea (cursor) the first time it is used. The maximum number of workareas (cursors) that can be used is 256.

Extended shorthand operators

Recital now supports shorthand operators on assignment statements.

+=	add right hand side expression to left hand side variable	var += 10
-=	subtract right hand side expression to left hand side variable	var += 10
*=	multiply right hand side expression by left hand side variable	var *= 10
/=	divide left hand side variable by right hand hand side expression	var /= 10

Static array initializers

Static arrays can now be created and populated using array constants.

private tab = { "hello", "world" }
private tab2 = { { "hello", 10, date()}, {"world", 20, date()} }

myfunc( { 10,20,30,40 } )

Improved data type checking

By default, Recital is a loosely typed language. When SET STRICT is ON, this disables assignments to variables that have not been declared before use. Additionally type checking is in effect when assignments are made.

private nvar as numeric
private cvar as character
private dvar as date
private tvar as datetime
private yvar as currency
private lvar as logical
private avar as array
private ovar as object
private ovar as classname

nvar = 10    // ok
nvar = ""    // error invalid assignment

Initialize variables at declaration time to improve readability

private title = "The application title"
private count = 0

// type check them at runtime too
private title as character = "The application title"
private count as numeric = 0

Database Administration Tools

Recital 10 includes a collection of Database Administration Tools (DAT) that can be used in shell scripts. The DAT are executed using the recitaladmin command from a shell.

recitaladmin shutdown
recitaladmin startup
recitaladmin disable
recitaladmin enable
recitaladmin backup
recitaladmin restore
recitaladmin integrity
recitaladmin suspend
recitaladmin resume
recitaladmin start
recitaladmin stop
recitaladmin restart
recitaladmin logoff
recitaladmin broadcast
recitaladmin --help

Suspend and Resume of Recital

You can now suspend and resume all recital applications; interactive and server. This is particularly useful if you need to take "snapshot backups" at regular intervals without asking users to logoff. www.rsnapshot.org is a good snaphot backup utility that uses rsync and ssh.

# service recital suspend
#     echo perform the backup...
# service recital resume

If you need to handle suspend and resume specifically in your application you can do so using the following new features in Recital 10.

on signal 50 do your_suspend_proc
on signal 51 do your_resume_proc

Then use the "service recital suspend" and "service recital resume" commands. Alternatively send signals using the "kill" command.

kill -50 pid# suspend
kill -51 pid# resume

Improved compiler

In earlier versions of Recital prior to compiling large programs you had to use the SET MAXDBO TO command to specify the maximum size of a compiled program. This was error prone and consumed large amounts of memory even for compilation of a small program file. The compiler now dynamically extends it's allocated memory as needed so the SET MAXDBO command is now no longer required.

Additionally, you can compile your programs in debug mode. Programs that have been compiled in debug mode will provide statement profiling information in the debug.txt file if SET DEBUG ON is in effect when the program is executed. You compile in debug mode using the compile command.

compile filename debug

New database integrity verification tool

The recitalintegrity tool can be used to verify the integrity of tables, indexes and memo files. This new tool when used with the -y option can also repair tables, indexes and memo files.

# recitalintegrity --help     
recitalintegrity - Recital database integrity verification tool     
usage: recitalintegrity [options]    
Options:     
-f table           table to verify
-d directory       directory to verify 
-D database        database to verify 
-r                 recursively process subdirectories 
-y                 repair data if errors 
-q                 Only display errors 
-a                 verbose mode 
--help, -h         Display this information

Major performance improvements

The internals of Recital 10 have been extensively modified to provide greatly improved program execution performance. Internals that have been changed include:

Symbol table hash table sizes
Hashing algorithm used for symbol lookup
All objects, dynarrays and arrays are now passed by reference. They can be returned from procedures/functions even if declared private inside the called procedure
Objects, dynarrays and arrays are now reference counted (similar to java) and automatically released when they have no more references.
The compiler code generation has been enhanced resulting is faster .dbo files with less memory consumption
APPEND FROM now locks the complete table and appends in the new records which reduces record locks and is particularly noticeable on clustered/network file systems.
General network I/O optimizations and improving performance of the REPLACE and SQL UPDATE statements.
Major performance improvement on indexes with large numbers of duplicate keys
Major performance improvements for network file systems and clusters.
Improved compiler optimizations resulting in faster code

Code and data profiler

Code and Data profiling has been introduced in Recital 10. See this article for details.

Using Recital with MySQL, Oracle, DB2, and Postgres

Recital 10 has a large number of functions that are familiar to PHP developers. The most commonly used PHP functions for working with MySQL databases are the mysql_xxx() functions. These functions are available in Recital 10 and can be used to access MySQL databases in both Recital and Recital Web. Note that the rdo_xxx() functions are functionally identical to their mysql_xxx() counterparts, but they can be used with Recital, Oracle, DB2, MySQL or PostgreSQL. See this article for details.

Enhanced object-oriented programming functionality

Recital is a dynamic programming language particularly suited to the development of database applications. While Recital still supports standard procedural programming, new extensions to the language give you the power and flexibility of object-oriented programming. Object-oriented design and object-oriented programming represent a change in focus from standard procedural programming. Prior to Recital 10 being released, object-oriented programming (OOP) support in Recital was somewhat limited. In Recital 10 the whole OOP engine has been overhauled and is now much more advanced and flexible. See this articlefor details.

Enhanced Recital command window

When using Recital in interactive mode, the command window has been enhanced. Pressing ^w pops up a textedit window which can be used for editing long commands (particularly useful for SQL commands).
Pressing the Return key echoes it to the output window (useful for clearing text) when using interactive commands.
Improved command history. When using the up-array and down-array keys from the keyboard to view and/or execute previous commands, duplicate commands are ignored.
When developing/testing Recital program files from the interactive command window, if an error occurs you can edit the offending program by selecting 'edit' from the error dialog. The file will be opened at the offending line number.
Command autocompletion using the TAB key

Program editor enhancements

Editing Recital .prg files inside the Recital Workbench on linux will now color code them using vim.

Use Recital in shell scripts

Recital 10 enhances Recital by enabling it to be used in bash shell scripts and in shell commands which use pipes and/or redirect stdin and stdout. If stdin is not redirected then recital will startup and operate as normal in a terminal window. Additionally you can use heredoc to denote a block of recital commands that should be executed. Note that when used in this manner, no UI commands can be executed and no user interaction is allowed.

# recital < mrprog.prg

# recital < myprog.prg > myoutput.txt

# recital > myoutput.txt <

use customers

list structure

END

# echo "select * from sales!customers where overdue" | recital | wc -l

Individual commands can be executed in shell scripts.

# recital -c "create database sales"

# recital -c "create table sales!invoices (id int, name char(25), due date)"

Expressions can be evaluated and used in shell scripts.

# VER=`recital -e "version(1)"`

You can view what command line options are available by typing:

# recital --help

Better Unix/Linux command line integration

Recital 10 supports use of `backticks` in strings.

echo "The default directory is `pwd`"
echo "There are `ls -l *.dbf | wc -l` tables in this directory"

Automatic data type conversions in character expressions

When non-character data is added to strings, the data is automatically converted to a string similar to the etos() functions operates. SET STRICT OFF must be in effect to enable this.

echo "This string can add numerics and dates etc " + 100.89 + " " + date()

Conditional loading of shared libraries

The require_once( Filename ) function will load a Recital library (.prg extension) or C shared library only once. If the library is already loaded the function is ignored. Shared libraries stay in memory until there are no further references to them from any Recital, Recital Server or Recital Web application.

Automatic loading of extension libraries

Recital and C API libraries that reside in the /opt/recital/extensions directory will automatically be loaded into shared global memory by Recital, Recital Web and Recital Server. This results in a big performance improvement when using class libraries and is particularly noticeable when running web applications with Recital Web. Extension libraries can be written in Recital (.prg extension) or in C using the Recital C API in which case they will have a ".so" file extension on linux an a ".dll" extension on windows. Extensions stay in memory until there are no further references to them from any Recital, Recital Server or Recital Web application.

SQL performance improvements

SQL SELECT improved query optimizer

Handles index restriction using filtered indexes (INDEX ... FOR ... )
Handles index restriction with > and >=
Dramatic improvements to nested SELECT statements
SmartQuery caching

SQL Select extensions

SELECT * FROM table INTO JSON

SELECT * FROM table INTO OBJECT

SELECT * FROM table INTO HTML

SELECT * FROM table INTO XML

New commands

COPY DATABASE TO [ IF [ NOT ] EXISTS ]
This command is used to copy an existing database to a new database. By default an error will be returned if the target database already exists. Specifing the optional IF NOT EXISTS keywords no error will be returned if the traget database already exists. If the optional IF EXISTS keywords are specfied and the target databases already exists, then it will be removed before the copy. Both the databases must be closed before they can be copied.

SET AUTOHEAL ON/OFF
Enables or disables auto-healing of indexes.

SET FSTATCACHE ON/OFF
Enables or disables caching of file information on a network file system.

SET TXLEVEL TO COMMITTED | UNCOMMITTED
Sets the transaction isolation level for SQL commands.

SET NETWORK ON/OFF

SET NETWORK TO

Enable or disables Recital Clustered File system support which provides superior performance on networked systems using glusterfs or samba.

SET TIMELINE ON/OFF

Enables or disables timelines for the active table.

CLEAR TIMELINE

Clears the timeline for all tables.

LIST TIMELINE [RANGE date [,date] [FOR condition]

Lists the timeline for all or selected tables over a datetime range.

ROLLBACK TIMELINE RANGE date [,date] [FOR condition]
Rolls back data to a previous timeline for all or selected tables over a datetime range.

SET TIMESTAMP TO

Enables a timestamp to be inserted into a field of a record as it is being appended into a table.

SET SMARTQUERY ON/OFF
SET SMARTQUERY TO
Enables or disable SmartQuery caching.

CLEAR SMARTQUERY
Clears the SmartQuery cache for all tables.

EXPLAIN

Explains the execution of a SQL SELECT statement and provides details of how the query is being processed.

FOREACH

Provides a convenient way to iterate over elements in associative or static arrays.

ECHO

Similar to the existing ? command but the string output can contain C-style escaped characters e.g. "\n" and php-style $variable and {expression} substitutions in character strings.

OPTIMIZE

This new command is used to optimize a table by it's primary key. The table is physically ordered by the specified primary key to give better performance when reading rows from the table. If the optional PACK keyword is specified then deleted rows are removed from the table during the optimization process. The optional ALL keyword can be specified to optimize all open tables or the optional DATABASE key word can used to optimize all tables in a specified database. The table being optimized must have a primary key.

SET DATADIR TO [directory]

This new command is used to specify a directory where database tables, memos, indexes, and dictionary files are located. When a table is being opened this directory is searched first before the current directory and the file search path to locate the table and its associated files. This allows the database tables to be relocated to a different file system without the need to change an existing application.

SET DATE TO ISO

Recital now supports SET DATE TO ISO which will set date to a YYYY-MM-DD format. International Standard ISO 8601 specifies numeric representations of date and time. This standard notation helps to avoid confusion in international communication caused by the many different national notations and increases the portability of computer user interfaces. In addition, these formats have several important advantages for computer usage compared to other traditional date and time notations.

Enhancements to existing commands

APPEND TYPE CSV

Appends from a csv file

COPY TYPE CSV

Copies to a csv file

New functions

Outputs an object or dynarray variable as html

print_html( object as object )

Outputs an object or dynarray variable as xml

print_xml( object as object )

Outputs an object or dynarray variable as json

print_json( object as object )

Create a dynarray

object = array( [ key => value [, key => value] ] )

Create an anonomous object

object = object( [ property => value [, property => value] ] )

Run linux commands (optionally piped together) and return the output

character = pipetostr( "linux | commands | like | this" )

Save an object or dynarray to an external .obf file (object serialization)

saveobject(oName as object, cFilename as character)

Load an object or dynarray from an external .obf file (object deserialization)

object = loadobject(cFilename as character)

Time function with msecs granularity

numeric = timeofday( [mode as numeric] )

XML functions

character = xml_encode(da as object [, roottagname as character, rowtagname as character] )

object = xml_decode(xmlstring as character [, roottagname as character, rowtagname as character] )

character = xml_scatter( [wka as numeric] )
numeric = xml_gather( xmlstring as character )

Linux messaging functions (hint: use with xml_encode() and xml_decode() to pass objects between processes)

numeric = mqcreate(name, as character, mode as numeric)

numeric = mqopen(name as character, mode as numeric)

mqclose(mqdes as numeric)

numeric = mqsend(mqds as numeric, data as character)

character = mqreceive(mqdes as numeric)
numeric = mqcurmsgs(mqdes as numeric)

TCP/IP socket function

fp = fsockopen(address as character, port as numeric) as numeric

( use fputs(), fgets(), fflush(), fread(), fclose() with this function )

fopen() function enhanced

The fopen() function can now be used to open http://www.xxx.com/page URL's

The fopen() function can also be used to open linux fifo pipes with fifo://pipename
( use fputs(), fgets(), fflush(), fread(), fclose() with this function )

URL function

character = geturl( url as character [, timeout as numeric] )

C-style printf(), sprintf() and fprintf() functions

printf( format as character [, args...])

character = sprintf( format as character [, args...])

fprintf( fp as numeric, format as character [, args...])

The format can contain any of the following formatting characters:

%s %d %f %y %t %T %l %L

Each formatting character sequence can optionally contain a width and decimal places (only relevant for numeric)

Left justify by placing a '-' directly following the '%' character e.g. %-20s or %-14.2f.

When %s is specified, the corresponding argument is converted to character format (similar to specifying etos().

%t and %T are for formating datetime data types.

%y is for formatting currency data types

%l and %L is for formatting logical datatypes.

tostring() function (synonymous with etos()

character = tostring( expression )

Returns the current transaction isolation level
character = txlevel()

Fills an array with elements of a string separated by newline (default)
numeric = alines(arrayname, string as character [,flags as numeric [, separator as character] ] )

Returns a string describing the last operating system error message. If the optional error number is
specified then the related operating system error message will be returned.
character = strerror( [ errno as numeric ] )

Returns all the fields in the current row as a string. The string will begin with the uniquied row identifier and then the deleted flag, followed by the data in the record. An optional workarea can be specified, otherwise the current workarea will be used.
character = rtos( [workarea as numeric] )

Returns true if the session is running from a terminal (not a batch job)
logical = isatty()

Returns information showing who has a file open and whether it is exclusive or shared.
character = lockinfo(filename as character)

New functions to handle ini / configuration files

Sets a key/value pair in a section of an ini/configuration file

ini_set(Section as character, Key as character, Value as character[, Filename as character]) as logical

Gets a value for a key in a section of an ini/configuration file

ini_get( Section as character, Key as character[[[, Default as character], Filename as character], Macros as logical]) as character

Sets all key/value pairs for a section of an ini/configuration file

ini_setsection(object as object, Section as character[, Filename as character]) as logical

Gets all key/value pairs for a section of an ini/configuration file

ini_getsection(Section as character[[, Filename as character], Macros as logical]) as object

A typical ini/configuration file looks like this:

# comment lines are preceeded by a '#' 
; or a ';'
#
# "Sections" are enclosed in [ and ]
[global]
; key/value pairs are contained in the sections and are written like this
 key1 = value
 key2 = value2
; You can include macros in the key/value pairs.
 key3 = This is $(key2) and $(key1)

Notes:

If "Filename" is not specified then it will default to the recital system configuration file /etc/recital.conf
It "Macros" is not specified both ini_get( ) and ini_getsection( ) will not expand macros. Set it to true if you want macros expanded
ini_getsection( ) returns a dynamic array which allows you to reference key/value pairs, update them, and write them back to the configuration file.
You should set the file permissions on /etc/recital.conf to 755 to prevent changes to the recital system configuration file

New PHP-compatible functions to reduce learning curve for PHP developers

(Click here for function details and examples)

number = strcmp( expC1, expC2 )

number = strcasecmp( expC1, expC2 )

number = strpos( expC1, expC2 )

logical = in_array( expC, expA )

character = strstr( expC1, expC2 )

number = strlen( expC1 )

unset( variable )

isset( variable )

die( exp )

exit( exp )

object = array( )

character = base64_encode( expC )

character = base64_decode( expC )

number = filesize( expC )

logical = function_exists( expC )

print_r( expO )

character = str_replace( needle, replace, haystack )

character = strtolower( expC )

character = strtoupper( expC )

character = ucfirst( expC )

character = str_pad( expC, expN [, expC] )

character = html_entities( expC )

character = html_entity_decode( expC )

require( Filename )

require_once( Filename )

include( Filename )

include_once( Filename )

character = strftime( format [, datetime ])

Determine if a variable is of a specific type
logical = is_array( exp )
logical = is_object( exp )
logical = is_string( exp )
logical = is_int( exp )
logical = is_float( exp )

Wraps a string to given line width (defaults to 80)
character = wordwrap( character string [, numeric line_length [, character break_char [, logical cut ] ] ] )

The implode() function returns a string from the elements of an array
character = implode(separator as character, arrayname)

The explode() function breaks a string into an array
array = explode(separator as character, string as character)

Bug fixes

Numerous bug fixes.

Deprecated features

The following features are no longer supported and have been removed from Recital 10

The ASSIST command and the shell dba script
The INFO command and the dbinfo script
The DESIGN command and the dbdesign script
SET MAXDBO command is ignored
DB_MAXWKA environment variable in recital.conf is ignored
shell commands dbc, dbvideo
The Recital database library has been discontinued. The Recital/SDK C extensions API should be used instead.