Problem:        How To Commit Table Data to Storage Immediately

Solution:        Newer PalmOS devices, and Windows Mobile/PocketPC devices using the PDB database format, employ a system of in-memory caching of table data for improved performance.  Changes to cached tables are automatically committed, or saved, back to storage when the application is closed.  In order to prevent loss of data due to a system reset before the application is closed, you can force the cached data to be committed immediately on a table by table basis, either by using the Tables("tablename").CommitData method that is new in Satellite Forms 7.1, or by using the PalmOS DmSyncDatabase extension or the PocketPC CommitDatabase extension.

A. Overview

The original PalmOS memory system stored non-built-in applications and data in battery-powered RAM memory.  Everything worked well as long as the PalmOS device had battery power, but if the batteries died then the contents of RAM (including applications and data) were lost.  A standard reset would not lose any data, because the contents of RAM were not cleared, leaving apps and data intact.

A few years ago a new PalmOS memory system was introduced in which applications and data were stored in non-volatile memory, keeping applications and data intact even if the batteries go dead.  The main impetus for this new system, called the Non-Volatile File System (NVFS), was to enable "on the go" swapping of removable batteries in Palm Treo smartphones, without losing data such as phone numbers, appointments, and so on.  The overall advantages of the NVFS memory system were readily apparent, and so NVFS was quickly adopted on most other non-Treo PalmOS 5.x devices as well, becoming the new normal memory system.

While the goals of NVFS were certainly admirable, and in general terms it does work well, the actual impementation of NVFS has a few holes that make it necessary to consider data protection strategies in your application.  In basic terms, the NVFS system caches database tables that are open (in use) in regular RAM memory, and then commits (saves) the modified databases back to non-volatile storage when the database is closed (when the application ends) and when the power is turned off.  This is all performed automatically behind the scenes by the PalmOS.  The problem comes when the device is reset (due to a pin reset or to a system crash): if the cached databases have not been committed to storage before the reset, then all of the modified data is lost when the device restarts.

For Windows Mobile/PocketPC devices, the Satellite Forms PDB database format uses a similar system of caching modified data in memory for enhanced performance.  While the applications databases are open, changes are cached in RAM, and when the application closes the modified data is committed to storage.  This process of committing data to storage is what causes the delay when a PocketPC PDB application is closed.  Under this PDB data caching system, a device reset/crash can also cause data loss if the data has not been committed to storage before the reset.

The implication of this data loss on reset behaviour is that in order to ensure your modified application data is saved to non-volatile storage, your application may need to periodically force its data to be committed to storage before the application closes.

Starting with Satellite Forms 7.1, a new Tables("tablename").CommitData method is available to perform this task.  At any point in your application you can immediately commit any specified table to storage by calling the Tables("tablename").CommitData method.  For versions of Satellite Forms prior to 7.1, two extensions are available to provide this capability.  The DmSyncDatabase extension for PalmOS provides this capability with the DmSyncDatabase(Tables("tablename").Index) script method.  It is available for download from the Products & Support | Support Files | Software Updates & Patches section of the Satellite Forms website, and is also included as a standard extension in SatForms 7.0.  The CommitDatabase extension for PocketPC provides this same capability for the Windows Mobile/PocketPC platform, using the CommitDatabase(Tables("tablename").Index) script method.  The CommitDatabase extension is also available for download from the Products & Support | Support Files | Software Updates & Patches section of the Satellite Forms website.

The natural question that comes to many developers' minds is "when or how often should I commit the table data in my application to prevent data loss?"  The simplest answer is that it depends on the application.  Because the actual process of committing a table to storage does take a little time and is not instantaneous, there is a tradeoff between ensuring complete data protection versus acceptable application performance.  It is up to you as the developer to determine the optimal blend of data protection and performance for your application, and to use the CommitData table method accordingly.  For example, an inventory scanning application may want to commit the data table after each new record is scanned, while a sales entry app may wish to wait until a multi-item order is completed before committing the data.  A medical reference application may not need to commit data at all, and just rely on the automatic committing of data when the application is closed.  There is no single recommended approach to committing application data for all applications: it is dependent on what your application does and how important it is to ensure that modified data is saved.

B. Usage

(a) Satellite Forms 7.1 and higher

The recommended approach when using Satellite Forms 7.1 and higher is to call the new Tables("tablename").CommitData method on your desired table(s).  This method works the same on both the PalmOS and Windows Mobile/PocketPC platforms, enabling you to use the same script code on both platforms.  Here is the method prototype:

CommitData

(b) Satellite Forms versions prior to 7.1

For Satellite Forms versions prior to 7.1, the CommitData table method is not available.  The table committing functionality is available through the DmSyncDatabase extension for PalmOS, and the CommitDatabase extension for PocketPC.  In order to seamlessly integrate the table commit procedure into your pre-7.1 SatForms application for both the PalmOS and PocketPC platforms, follow this approach which is a combination of using the extension functions and global scripts.  This allows your form level scripts to use the same script code for either/both platforms, and follows the approach presented in the KnowledgeBase article How To use Global Functions & Subs to replace extension functions not available on the current target platforms.

• First, add the DmSyncDatabase extension to your PalmOS application target, using the Manage Extensions toolbar.
• Next, add the CommitDatabase extension to your PocketPC PDB application target, using the Manage Extensions toolbar.
• In the PalmOS build target, create a new Private (not Shared) Global Sub named CommitData:

       'CommitData global sub for PalmOS uses DmSyncDatabase

       'the table index is passed to this function as a string variable

       Sub CommitData (tIndex)

         DmSyncDatabase(tIndex)

       End Sub

• In the PocketPC build target, also create a new Private (not Shared) Global Sub named CommitData:

       'CommitData global sub for PocketPC uses CommitDatabase

       'the table index is passed to this function as a string variable

       Sub CommitData (tIndex)

         CommitDatabase(tIndex)

       End Sub

• In the form-level scripts for either platform, you can commit a specified table to storage by calling the CommitData global sub like this:

       'save our updated table to storage right now

       'using the CommitData global sub

       CommitData( Tables("tablename").Index )

C. Additional Data Protection Strategies

If you want to add some additional data security to your application, you could copy data tables to an installed memory card (or in the case of the Palm Tungsten T5, the built in flash memory area, and in the case of the Palm LifeDrive, the built in hard drive).  In that way, if the data files are lost due to whatever reason, they could be recovered from the memory card backup.  There are many low cost or even free backup utilities available (Google for 'Palm memory card backup' for some examples), which are user-run utilities.  To programmatically handle backups of data in your app to a memory card without user intervention, you can use the PalmDataPro SFTextFile extension for PalmOS (http://www.palmdatapro.com/itm00116.htm) or SFTextFile-PPC for PocketPC (http://www.palmdatapro.com/itm00152.htm).

Keywords:        commit, save, write, cache, reset, dmsyncdatabase, commitdatabase, NVFS, storage, protection

KB ID: 10074 

Updated: 2007-09-13

Satellite Forms KnowledgeBase Online

Satellite Forms Website Home