5.1 JDO Concepts

Being proficient at using JDO to build applications requires a good understanding of the concepts underlying JDO. Some of these were introduced in previous chapters to varying degrees, and others are dealt with in greater detail in the chapters that follow.

5.1.1 Persistence-capable

The concept of a persistence-capable class was explored in detail in Chapter 3. It's the basic mechanism that enables JDO to transparently persist instances of a Java class in a datastore. Any Java class that implements the JDO PersistenceCapable interface and adheres to the JDO programming style is said to be persistence-capable and can be used with any JDO implementation without recompilation.

Typically, a developer would use vendor-supplied tooling to either generate a persistence-capable class from scratch or take an existing Java class and make it persistence-capable (through source code or byte code manipulation).

5.1.2 JDO Metadata

As introduced in Chapter 3, JDO requires additional metadata to be defined for each persistence-capable class. This metadata denotes that a given Java class is actually persistence-capable, whether it has a persistence-capable superclass, and how the class and fields of the class should be treated by the JDO implementation.

The XML file containing the metadata file should contain the following:

• A header that specifies the location of the JDO Metadata XML DTD. The location can be the URL as published by SUN Microsystems, or it can be some DTD file stored elsewhere:

   <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE jdo SYSTEM "jdo.dtd"> 

• A <jdo> element, within which the metadata for the persistence-capable classes is contained:

   <jdo> </jdo> 

• The <jdo> element should contain one or more <package> elements, each having a name attribute that specifies the name of the package. A <package> element contains the metadata for the persistence-capable classes contained in that package:

   <package name="com.corejdo.examples.model"> <\package> 

• A <package> element should contain one or more <class> elements, each having a name attribute that specifies the name of the Java class. A <class> element can have a number of additional attributes: identity-type ; objectid-class ; requires-extent , and persistence-capable-superclass that define how the persistence-capable class should be treated by the JDO implementation:

   <class name="Author"> </class> 

• A <class> element can contain one or more <field> elements, each having a name attribute that specifies the name of the field. A <field> element can have a number of additional attributes: persistence-modifier ; primary-key ; default-fetch- group , embedded , and null-value that define how the field should be treated by the JDO runtime:

   <field name="books"> </field> 

• A <field> element can contain <collection> , <map> , or <array> elements, depending on the type of the field. These elements specify additional metadata about the field (such as the class of the instances in a collection):

   <collection element-type="com.corejdo.examples.model.Book"> </collection> 

JDO defines various defaults for the optional attributes that avoid the need to specify them explicitly. This keeps the average metadata file short and simple. In addition to the elements outlined above, the DTD also allows vendors to specify extensions using the <extension> element, which has vendor-name , key , and value attributes.

The metadata for a persistence-capable class should be located in a metadata file specific to the class or in a file that contains the metadata for all persistence-capable classes in a given package hierarchy. A metadata file specific to a single class should be named after the class itself, but with a .jdo suffix. For example, the metadata for the Author class could be located in a file called Author.jdo . Alternatively, the metadata could be located in a file called package.jdo . This file would then contain the metadata for the Author class as well as any other persistence-capable class in the same package hierarchy. In both situations, the metadata file should be available at runtime as a resource that can be loaded by the same class loader that loaded the class itself (i.e., it should be in the classpath ). For example, the metadata for the class com.corejdo.examples.model.Author should be contained in one of the following files:

• package.jdo

• com/package.jdo

• com/corejdo/package.jdo

• com/corejdo/examples/package.jdo

• /corejdo/examples/model/package.jdo

• com/corejdo/examples/model/Author.jdo

The metadata is searched for in the above sequence, and the first definition that is found is used.

As explained in Chapter 3, the initial JDO 1.0 specification did not mandate the location and name of the XML metadata files. To aid in portability, the 1.0.1 maintenance release of the specification changed this to what has just been described.

Prior to this release, the name of a metadata file that contained multiple classes was the package name itself with a .jdo suffix, and it was located in the directory that contained the package directory. In the Author example, it would have been called model.jdo and would have been located in the com/corejdo/examples/ directory. Some JDO implementations may still use this naming convention.

Appendix B provides greater details on the exact syntax of the JDO metadata XML file and the purpose of each element and attribute.

5.1.3 Default fetch group

When a persistent object is retrieved from the datastore, the JDO implementation initially retrieves the values of those fields that are part of what is referred to as the default fetch group for the class. If a persistent object has been retrieved from the datastore, then as a minimum the values of all the fields that are part of the default fetch group are in memory. This concept allows the JDO implementation to optimize access to the fields in the default fetch group because it doesn't have to mediate access to each field individually once a persistent object has been retrieved. For fields not in the default fetch group, the JDO implementation mediates each access, ensuring that the field has first been retrieved from the datastore.

By default, fields of primitive types, java.util.Date and supported system classes from java.lang and java.math packages, automatically are part of the default fetch group. A field can be explicitly added to the default fetch group by setting the default-fetch-group attribute to true in the metadata for the field. If a field that references another persistent object is made part of the default fetch group, then effectively the JDO implementation retrieves the referenced persistent object also.

5.1.4 Persistence by reachability

Chapter 3 introduced the concept of persistence by reachability. This simply means that instances of persistence-capable classes or supported-system classes that are referenced from an instance that is itself made persistent is implicitly made persistent also, and so on for the entire graph of reachable instances. Likewise, any transient instance that is later added to an already persistent instance is made persistent, too.

JDO defines that the persistence by reachability calculation is initially done during PersistenceManager.makePersistent() and then redone during Transaction.commit() . Therefore, a transient instance that is added to a graph after makePersistent() does not itself become persistent until commit() “ or as a result of a later call to makePersistent() .

Figure 5-1 shows a graph of instances and identifies those that would be implicitly made persistent.

Figure 5-1. Persistence by reachability.

Even if the graph of reachable instances is cyclic in nature, or a particular instance is referenced from multiple others or is already persistent, the JDO implementation ensures that each is made persistent only once.

Persistence by reachability negates the need for the application to explicitly track all instances that it needs to make persistent as it creates them; instead, it just needs to make the root instance of a graph persistent, and the JDO implementation implicitly takes care of the rest.

5.1.5 First-class and second-class objects

JDO uses the concept of first-class and second-class objects to differentiate how a JDO implementation manages instances of user -defined, persistence-capable classes (persistent objects) versus instances of supported-system defined classes (like Java collection classes) and arrays.

A first-class object is an instance of a persistence-capable class; it has a JDO object identity and can be stored in the datastore by itself. A second-class object does not have a JDO object identity; it can be stored only in the datastore as belonging to a first-class object.

As an example, consider a persistence-capable class that contains a java.util.HashSet . Instances of the persistence-capable class are first-class objects; instances of java.util.HashSet are second-class. Essentially, instances of supported-system classes or arrays are not themselves persisted in the datastore; rather a representation is persisted as part of the owning first-class object.

Because second-class objects can be persisted only as belonging to a first-class object, it is not possible for multiple first-class objects to reference the same second-class object in the datastore. As an example, at some point within a transaction, multiple in-memory persistent objects may reference the same java.util.HashSet instance. However, when the transaction commits and the persistent objects are persisted in the datastore, each creates its own representation of the contents of the java.util.HashSet . When retrieved again into memory, each persistent object gets its own java.util.HashSet instance.

In addition, it is not necessary to explicitly delete second-class objects. When a first-class object is deleted using the deletePersistent() method on PersistenceManager, the JDO implementation automatically ensures that the representation of any second-class objects is also deleted from the datastore. Of course, if a java.util.HashSet contains references to other persistent objects, only the representation of the java.util.HashSet is deleted; the referenced persistent objects are not deleted because they are first-class objects in their own right.

5.1.6 Object identity

Chapter 3 introduced the concept of JDO object identity and the different types of identity that can be used with JDO. Essentially, every persistent object has a unique JDO object identity. An in-memory instance is associated with its representation in the underlying datastore via this identity. Object identity is central to how a JDO implementation manages and persists instances of persistence-capable classes.

JDO defines three types of object identity:

• Datastore identity: A persistent object's identity is automatically generated.

• Application identity: A persistent object's identity is determined by the values of certain fields (those whose primary-key attribute is true in the metadata for the class).

• Non-durable identity: A persistent object's identity is not persistent at all and is only valid while in-memory.

By default, a persistence-capable class is assumed to have a datastore identity. The identity type of a persistence-capable class can be explicitly specified using the identity-type attribute in the class's metadata. The attribute can be data-store (the default), application , or nondurable .

If using the application identity, the objectid-class attribute is also required to identify the user-defined class used to represent the persistence-capable class's identity.

5.1.7 Object lifecycle

JDO defines a number of states that an in-memory instance of a persistence-capable class can be in. These states are used by the JDO implementation to keep track of whether the fields of a given instance have been retrieved from the underlying datastore within the current transaction, whether any fields have been modified by the application or if the application has deleted it. The transition from state to state is referred to as an object's lifecycle and the JDO specification denotes which transitions are valid and which aren't.

As explained in Chapter 3, these are the basic states:

• Transient: A non-persistent, normal Java object.

• Persistent: A persistent object whose fields have been retrieved from the datastore in the current transaction.

• Hollow: A persistent object whose fields have not yet been retrieved from the datastore.

The object lifecycle is of real concern only to a JDO implementation; a JDO application itself needs to do nothing. However, using the InstanceCallbacks interface or via a JDOHelper class, an application can interact with the JDO implementation as certain state transitions occur or interrogate the state of a particular instance of a persistence-capable class.

See Chapter 4 for a comprehensive description of all the states and state transitions that make up a persistent object's lifecycle.

5.1.8 Transactions

Typically, persistent objects must be created, retrieved, modified, or deleted within the context of a transaction. The underlying datastore uses transactions to enforce concurrency control between multiple applications and ensure that, upon commit, all changes within a transaction get written to the datastore or none of them do (in the case of failure).

JDO supports both non-managed and managed transaction control. For non-managed transactions, the application itself must explicitly start and end a transaction. For managed transactions, transactions are implicitly started and ended by an external coordinator (a l   the Java Transaction Service). Managed transactions are typical if using JDO within the J2EE environment and are covered in more detail in Chapter 9.

Because different datastores use different approaches to managing concurrency control, JDO does not explicitly define the semantics of how concurrency control should be enforced or when concurrency conflicts should be detected . It instead defines the concept of a datastore transaction and simply says that a datastore transaction should exhibit ACID properties. See Chapter 3 for more details on ACID transactions.

5.1.8.1 Non-transactional read/write

Basic database principles dictate that a transaction should be short-lived to avoid concurrency conflicts between different applications running concurrently. If an application begins a transaction and retrieves persistent objects, it can potentially block other applications from accessing or updating those same persistent objects.

For applications that need to access persistent objects over an extended period of time, JDO specifies a number of optional features that allow access to persistent objects outside of a normal transaction and therefore avoid potential concurrency issues.

The non-transactional read and write features allow an application to retrieve persistent objects and to update them without first beginning a transaction. The following PersistenceManagerFactory options denote whether these features are supported by a JDO implementation:

• javax.jdo.option.NontransactionalRead

• javax.jdo.option.NontransactionalWrite

Persistent objects accessed in this way are referred to as being non-transactional.

Because the persistent objects are accessed without using a transaction, the underlying datastore is not going to enforce any concurrency control. After it is retrieved into memory, a persistent object is potentially no longer in sync with the underlying datastore; another application might have modified or even deleted the persistent object after it was retrieved.

A persistent object can also be modified outside of a transaction, but if the instance is later accessed within a database transaction, any changes are lost (the persistent object is effectively re-retrieved from the datastore within the current transaction). If using optimistic transactions, then it is possible to preserve changes.

Non-transactional support is useful for applications that want to keep persistent objects that are rarely modified in-memory for reference purposes to avoid the overhead of having to keep retrieving them from the datastore within each transaction. However, it is the application's responsibility to ensure that the in-memory instances are kept up-to-date with respect to changes in the underlying datastore.

The methods setNontransactionalRead() and setNontransactionalWrite() on Transaction and PersistenceManagerFactory can be used to explicitly enable or disable these features independently.

5.1.8.2 Optimistic transactions

Similar to non-transactional read and write, this feature allows an application to retrieve and modify persistent objects over a prolonged period of time without requiring an actual datastore transaction for the entire duration. But unlike non-transactional read and write, it does ensure transactional consistency. The following PersistenceManagerFactory option denotes whether this feature is supported by a JDO implementation:

• javax.jdo.option.Optimistic

Using an optimistic transaction is similar to using non-transactional read in that persistent objects are retrieved from the datastore and managed in memory as non-transactional instances outside of an actual datastore transaction. This means that any other application can modify the persistent objects after they have been retrieved, in which case the in-memory instances are no longer in sync with the actual persistent objects in the datastore. With optimistic transactions, however, the JDO implementation ensures that any conflicting updates are detected at commit time.

When a persistent object is first retrieved within an optimistic transaction, it becomes a non-transactional instance. If modified, the in-memory instance is marked as being modified and transitions to being transactional. The same is true for new or deleted persistent objects. An application can also choose to explicitly make non-transactional instances transactional using the makeTransactional() method on PersistenceManager .

At commit time, all transactional instances are checked against the datastore to ensure that no other application has modified them because they were first retrieved. If there has been a conflicting update, a JDOOptimisticVerificationException is thrown and the transaction is automatically rolled back, in which case an application must retry in a new transaction.

JDOOptimisticVerificationException

This exception was introduced in the JDO 1.0.1 maintenance release. Prior to this, if a conflict was detected upon commit, a JDOUserException was thrown. However, this leaves the transaction in an indeterminate state as far as the application is concerned , because it is not possible to determine whether the commit indeed failed because of a conflict or because of user error. If user error caused the failure, some corrective action might be taken and the transaction retried. Also, assuming that a conflict is involved, there is no way to determine which persistent objects caused it.

The JDO 1.0.1 maintenance release addresses these issues by introducing the JDOOptimisticVerificationException . This differentiates the failure of an optimistic transaction as a result of a conflict from a failure as a result of user error. The exception contains nested JDOOptimisticVerificationException instances, one for each persistent object that caused the conflict. Also the exception is a fatal rather than non-fatal exception, therefore, the transaction is automatically rolled back.

The method setOptimistic() on Transaction and PersistenceManagerFactory can be used to explicitly enable or disable this feature.

5.1.8.3 Retaining values

By default, at the end of a transaction, all in-memory persistent objects transition to being "hollow" and if needed are re-retrieved from the datastore within the next transaction. This default behavior can be changed.

The retain values feature allows an application to retain the fields of persistent objects in memory after a transaction ends. The following PersistenceManagerFactory option denotes whether this feature is supported:

• javax.jdo.option.RetainValues

If using retain values, at the end of a transaction, all in-memory persistent objects transition to being non-transactional rather than hollow. Assuming that non-transactional read has been enabled, an application can continue to access the fields of the persistent objects, although they are no longer in sync with the underlying datastore.

The method setRetainValues() on Transaction and PersistenceManagerFactory can be used to explicitly enable or disable this feature.

5.1.8.4 Restoring values

The restore values feature is a mandatory JDO feature and can used to revert the fields of in-memory persistent objects on rollback to their values as of the beginning of the transaction or the point in time they were made persistent within the transaction.

If used by itself, then on rollback the fields of any persistent objects that where made persistent within the transaction will be reverted to their original values and the persistent objects will transition to being transient instances again.

If used in conjunction with the RetainValues option, then on rollback the fields of all in-memory persistent objects will be reverted and they will transition to being non-transactional rather than hollow, as per the behavior for RetainValues on commit.

If restore values is not used then the fields of any modified persistent objects retain the modified values, even though these have not been committed to the datastore.

The method setRestoreValues() on Transaction and PersistenceManagerFactory can be used to explicitly enable or disable these options.

5.1.8.5 Transient transactional

In addition to managing persistent objects transactionally , JDO has an optional feature that allows an application to make transient (i.e., non-persistent) instances of persistence-capable classes transactional.

The transient transactional feature allows an application to specify that changes to a transient instance of a persistence-capable class should be managed transactionally by the JDO implementation. An application can make a transient instance transactional so that changes made to the instance during a transaction are undone if the transaction rolls back. The following PersistenceManagerFactory option denotes whether this feature is supported:

• javax.jdo.option.TransientTransactional

A transient instance is made transactional using the make transactional methods defined on PersistenceManager . Any modification of a transient transactional instance during a transaction is subject to rollback. Any modification made outside of a transaction is not. A transient transactional instance remains transactional until explicitly made non-transactional using the make non-transactional methods defined on PersistenceManager .

On rollback, a transient transactional transient instance's fields are reverted to the values they had at the beginning of the current transaction (or the values they had when they were made transactional, if this occurred within the current transaction).

Transient transactional can be useful for applications that need to keep changes made to transient instances transactionally consistent with changes made to persistent objects.