10.2 Implementing PersistenceCapable

10.2 Implementing PersistenceCapable

To make transparent persistence possible, the JDO specification defines the contract between PersistenceCapable , the interface that all persistent classes have to implement, and PersistenceManager , the central interface of a JDO implementation. But don't worry, how a JDO implementation works inside is not explained in this chapter. Because JDO gets direct access privileges for fields and classes, the contract is examined under security aspects. On the one hand, the PersistenceManager must be able to read and write fields and gather information about persistence-capable classes, like field types. This is deliberately not implemented using Java Reflection API, but by a more direct interfacing to get highest performance. On the other hand, it should not be possible for anybody to read or modify persistence-capable instances. Restrictions such as "private" and so on must be taken into consideration.

The code necessary for persistence-capable classes may be added to the Java source code by tools before a class is compiled or after compilation by modifying byte code. It is a misunderstanding that JDO implementations are required to modify byte code. In essence, the JDO expert group has focused high portability and support of various kinds of possible JDO implementations.

Using source code enhancement, as illustrated in Figure 10-1: Source code and byte code enhancement, the Java source files are processed like a C-preprocessor on a text basis. To implement such a source code enhancer , a full Java parser is required because field access and reserved word must be recognized. A simple line-wise processing is not sufficient. This procedure has the advantage that one can look at the resulting source code and debugging is easy, but today's IDEs do not really support source-code preprocessing well enough. It's like early Java Server Page (JSP) development: Without the right hooks in the development environment, edit-compile-debug cycles become a pain.

Figure 10-1. Source code and byte code enhancement.

Byte code enhancement uses the Java compiler's output and processes *.class files. This standardized format, which all Java compilers have to create and which can be interpreted by all Java virtual machines, is modifiable much simpler than source code. It is even possible to enhance the byte code at class load time of the application, like JSPs are compiled at Web page load time by the JSP engine.

A third approach creates persistence-capable classes completely from other sources than Java code. This might be from a database table layout or from XML schema declarations (XSD). The automatically created source code can be persistence-capable inherently .

10.2.1 The reference enhancer

The reference enhancer has been an idea to provide an umbrella for different technologies that existed before 1999. Essentially, the API that was needed to read, modify and create persistence-capable instances could have been vendor-specific, but JDOs were not accepted by a broad developer community in that case. During the first phase of the JDO specification, it was determined that differences in persistence technologies were insignificant and that the reference implementation could also provide a reference enhancer. By specifying the PersistenceCapable interface and the contract between persistence-capable classes and the PersistenceManager, the JDO specification achieves the goals outlined in the following sections.

10.2.1.1 Binary compatibility

This enables classes to be used by different JDO implementations, after they are enhanced. Persistence-capable applications can change the JDO persistence service without re-compilation or re-enhancement. The JDO compatibility test kit (TCK) takes this into account: Some special tests explicitly check those modifications, which make a class persistence-capable, and compare the byte code with reference classes. A JDO implementation can additionally extend the classes, which allows vendors to tune performance for a specific database system or add other vendor-specific features. The reference enhancer is the smallest common denominator.

10.2.1.2 Simplicity

The class must be enhanced by applying minimal code changes. The algorithms defined to make code persistence-capable or persistence-aware are restricted to exchanging only single opcode ^[1] in the byte code or simply adding a method to a class. No complex flow-analysis or structural changes should be needed to make a class persistence-capable.

^[1] Java byte code is made of different sections for strings, constants, methods or debug information. In a method's section of a class, instructions start commonly with a single byte instruction or opcode, therefore the name "byte code."

10.2.1.3 No reflection permission

Reflection is needed for JDO implementations only to instantiate an application class. This is done by calling ClassLoader.getClass(String name) . All other meta-information about the classes' fields, key classes, and so on are provided by a separate interface that is inaccessible other than by the JDO implementation classes.

10.2.1.4 No set/get methods or persistent base class

The application code is not required to implement any methods to become persistence-capable. The idea is to tag a class as persistent, and the rest is done like Java serialization. A persistence-capable class is also not required to derive from a special, persistent class or interface. The inheritance hierarchy doesn't have to be changed.

10.2.1.5 Field navigation by natural Java syntax

JDO defines names for set and get methods and the rules to convert field access in applications to method invocation. To make a class persistence-aware, field access is changed automatically from this:

 String title = book.title; 

to this with help from a tool:

 String title = book.getTitle(); 

10.2.1.6 Automatic tracking

The application is no longer responsible for detecting modified data and updating the data in a datastore or for propagating state changes to the JDO implementation. This is done in JDO set()- methods, which are added automatically at enhancement. These methods contain the required code to detect modifications, set flags, and propagate state changes.

10.2.1.7 Support of Java field attributes

The application class may use regular field modifiers, such as private , public , or even transient for any persistent fields. In XML, the metadata default attribute for fields may be overridden.

10.2.1.8 High performance

Compared to other persistence technologies, which may use Java Reflection to read or modify persistent instances, JDO provides great performance. As long as an instance is transient or after valid data is loaded from the datastore, it performs like any other Java instance. A factory inside of the persistence-capable class is used to create new (hollow) instances without the need for Java Reflection.

10.2.1.9 Same classes for different JDO implementations

The persistence-capable application code can be used together with different PersistenceManager instances at the same time (not the same persistent instances, but the code).

10.2.1.10 Debugger support

A JDO enhancer implementation is required to modify the byte code in such a way that any Java debugger can still be used to debug the application code.

10.2.2 Principles of operation

What has all this to do with security? The main problem is allowing a JDO implementation to access private fields, while protecting other code from doing the same. That is the reason for an essential principle in the contract between the persistence-capable class and the PersistenceManager defined in the JDO specification: "double dispatch." The next paragraphs shed some light into the implementation details of this contract.

10.2.2.1 Double dispatch

Every persistence-capable class has a StateManager reference, which is the mediator between itself and the corresponding PersistenceManager . The StateManager is a JDO service provider interface used for dispatching. Principally, the StateManager's methods could have been implemented in the PersistenceManager or in the PersistenceCapable interface.

10.2.2.2 Making instances persistent

At the moment that a transient instance becomes persistent by a call to pm.makePersistent(pc) , the JDO implementation first must get the right to access persistent instances to read out fields and store the data persistently. The transactional behavior of the persistent instance must be controlled by the JDO implementation as well, because the previous state of the instance must be restored at transaction rollback. At this point, the StateManager of the persistent instance is set, and control is granted to the JDO implementation. Just at this point, a security check is made through Java's regular security API, to check the JDO implementation's access permissions. If the JDO's code base is allowed to set the StateManager reference at the persistent instance, it also gets access to all the rest, as illustrated in Figure 10-2: Setting the StateManager. This ensures that only authorized implementations can use the StateManager interface.

Figure 10-2. Setting the StateManager .

The JDO implementation checks the initial setting of the StateManager through JDOPermission("setStateManager") .

The implementation of the StateManager interface is not defined by JDO. In principle, there can be one StateManager instance per persistence-capable instance, one instance for each persistence-capable class, or one instance per PersistenceManager instance.

10.2.2.3 Reading fields from persistent instances

When the instance is made persistent, the PersistenceManager may need to copy fields from the instance into some internal data cache or into the storage. This may happen to save the field data for later restoration in case of rollback or at commit time. The PersistenceManager is not allowed to read directly, because any other class could do the same. Instead, it may trigger the operation, and field data is provided via the StateManager interface.

As indicated in Figure 10-3: Reading fields, the PersistenceManager first calls the PersistenceCapable jdoProvideFields() method to trigger the read-out by the StateManager , which then receives the data from the persistence-capable instance. The idea is based on the protection of the StateManager : If foreign code cannot set the StateManager reference of the persistence-capable instance, it cannot access fields. The costly Java security check is necessary only once, when a transient instance becomes persistent or when a hollow instance is created.

Figure 10-3. Reading fields.

The StateManager interface provides all methods to get or provide data from or to the persistent instance. Some other methods are called for state changes and modification tracking, which are covered in the next section. The interface might support other applications as well, for instance the Java XML Data Binding (JAXB), which also reads and writes data from/to Java objects (called marshaling and unmarshaling ).

10.2.3 Tracking field access

Another important aspect is tracking field modifications and initial access of hollow instances. Resolving hollow instances on demand and modification tracking makes database application development with JDO extremely simple. It can be a horror to find a missing line of code in a huge application, which causes some data to not be written back into the datastore, or to track down performance problems because data is written too often.

Because many developers are afraid of byte-code post-processing, the actual code modifications to accomplish persistence awareness are explained here. First, any field access must be converted into a method invocation. As mentioned earlier, the basic idea is to change automatically a line like the following:

 String title = book.  title  ; 

into this by a tool:

 String title = book.get  Title  (); 

Add a method getTitle() that reads the corresponding data from the datastore on demand, and the state changes from hollow to persistent-clean in this case. To make persistence-aware classes compatible between different JDO implementations, the name is defined by JDO and the method signature is not the same:

 String title = Book.jdoGet  title  (book); 

The rule is to add the prefix jdoGet , add the field name, and declare it as a static method with the same access modifiers in the class. The set methods are similar:

 book.  title  = "new title"; 

In a persistence-aware class becomes:

 Book.jdoSet  title  (book,"new title"); 

To make things clearer, the code can be put into a small method, containing just the above lines. The code can be "disassembled" by the javap tool after it has been processed by the enhancer. Before enhancement is applied, the field assignment disassembles to the following:

 0 aload_0 1 getfield #2 <Field Book book> 4 ldc #3 <String "new title"> 6  putfield  #4 <Field java.lang.String  title  > 9 return 

This is the same modification method after the enhancer modified the code to look like this:

 0 aload_0 1 getfield #2 <Field Book book> 4 ldc #3 <String "new title"> 6  invokestatic  #5 <Method void                 jdoSet  title  (Book,java.lang.String)> 9 return 

The mutation of the code does not change its length. Both return statements are at position 9, and debugging information will not change because mapping of code addresses and source lines is not altered . During access of persistent fields, the timing is changed a bit. So what happens inside those set and get methods? The JDO specification suggests a default implementation, but it is not mandatory. The goal is to run as fast as possible in case of transient, persistent-clean, and persistent-dirty for fields in the default fetch group.

Default Fetch Group (DFG)

A fetch group declares a number of fields retrieved from a datastore together. Often, data is organized in sections, clusters, or tables, and values can be fetched in a group more efficiently . JDO provides two different types: fields in the default fetch group and fields in other fetch groups. The advantage of the DFG is primarily a flag field that is checked at field access. If that flag is set, the field value is returned directly without calling the JDO implementation. It is not specified how these other groups are declared in the XML metadata, because it is strongly implementation-specific. For example, the Book class might contain a reference to some complex class that has subclasses ( Category , CategoryList , and so on). The retrieval of that reference can become expensive in a relational database, because a join query is needed. By declaring the reference in another fetch group, retrieval is delayed until the reference is directly requested by the application.

If a persistent instance needs to be filled with data from a datastore, the time necessary to fill the instance is much lengthier than the code in the methods. The proposed jdoGettitle() method might be implemented like this:

 final static String jdoGet  title  (Book b) {   if (book.jdoFlags <= READ_WRITE_OK) {     return book.  title  ;   }   StateManager sm = book.jdoStateManager;   if (sm != null) {     // hollow ?     if (sm.isLoaded(book,jdoInheritedFieldCount+1))       return book.  title  ;     return sm.getStringField(book,               jdoInheritedFieldCount+1,book.  title  );   } else {     // transient     return book.  title  ;   } } 

First, the method checks jdoFlags for fields in the default fetch group. If the instance is already filled with valid data from the datastore, the instance returns the field. If the StateManager reference was not set yet, the instance is transient and the field can be returned (else part). If a StateManager is set, the field might be read and returned, or the field is completely managed by the JDO implementation and access is always mediated. Here is a summary:

1. The instance is "clean" or "transient."

2. There is no StateManager .

3. The instance is hollow, or the field has not yet been read. Access causes the field to be read.

4. Field data is kept in some "shadow" memory, and access is always mediated.

The jdoSettitle() method works similarly and can be implemented in the following way:

 final static void jdoSet  title  (Book b, String value) {   if (book.jdoFlags == READ_WRITE_OK) {     book.  title  = value;     return;   }   StateManager sm = book.jdoStateManager;   if (sm != null) {     sm.setStringField(book,               jdoInheritedFieldCount+1,               book.  title  ,               value);   } else {     // transient     book.  title  = value;   } } 

Again, the set method checks jdoFlags and simply returns whether the instance is persistent-dirty already. In this case, the title field can be set directly. If a StateManager is set, the appropriate method is called, or else the instance is transient and the field can just be set.

The sequence of calls between PersistenceCapable and StateManager allows for flexibility for various kinds of implementations. One extreme can have all fields in the default fetch group. Reading a field causes all other fields to be filled with valid data. Any further read access simply returns the corresponding field. The other extreme delegates every access to its JDO implementation. O/R mapping tools probably support a flexible configuration somewhere amid, depending on the mapping of fields to tables. An object-oriented database might always read and write entire objects.

10.2.4 Metadata access

The JDOImplHelper class helps to gather information about persistence-capable classes. It is another service class and is not intended for application use. At the point when a class is initially loaded by the JVM, it registers itself in JDOImplHelper . Later, JDO implementations can query the JDOImplHelper singleton for persistent classes and their field attributes. Again, this indirection is needed because of security issues.

The JDOImplHelper.getInstance() method checks for JDOPermission("getMetadata") rights and executes only the code of JDO implementations that have been granted access. If this check was not made, anyone can get information about field names or object identity classes that can be compared to RuntimePermission("accessDeclaredMembers") .

10.2.5 Policy file

The JDO implementation must be granted privileges to get access to the PersistenceCapable.setStateManager() and JDOImplHelper.getInstance() methods by adding the following lines to a policy file:

 grant codeBase "file:  /home/myJDOImpl  " {   permission javax.jdo.spi.JDOPermission         "getMetadata";   permission javax.jdo.spi.JDOPermission         "setStateManager"; }; 

10.2.6 Security problems

Unfortunately, the PersistenceCapable interface opens up some security gaps. Because any persistence-capable instance is able to return its PersistenceManager by the following:

 PersistenceManager pm =      JDOHelper.getPersistenceManager(book); 

any other application part, even some code that has nothing to do with persistence, can operate on the datastore. Some code may simply delete the instance from the datastore, although it had never been intended. Additionally, other parts of the application can peek at the datastore because Transaction , Extent , Query , and PersistenceManagerFactory are easily reached via the PersistenceManager .

This may sound a bit paranoid . On the other hand, JDO leads developers to use persistent objects throughout the application. Compared to direct JDBC programming, which often leads to home-grown persistence layers , persistence-capable instances pass their database connection around. Maybe such a security gap does not really allow malicious code to be executed, but an API that's too exposed can lead to an application with messed up control paths. To prevent such a mess, making instances transient after transaction completion is recommended. Under some circumstances, it can be an option to pass around JDO object identities instead of persistent instances.

Another security problem can come from JDOQL: JDO allows querying private fields outside of a class or even outside of a package. For example, if the Book class had a private field purchasePrice , a simple binary search could find the price for a book. Taking the reference of a persistent book instance, the search method can get the PersistenceManager , create Extent and Query instances, and run queries. All this has to take place in a running transaction, or a new Persistence-Manager must be instantiated from the PersistenceManagerFactory .

10.3 Application Security

The reasons mentioned in the previous section lead to requirements for a clean application design and, if applicable , to copy instance data or to make instances transient before passing them to other application layers. These are the results of the previous section:

• Access modifiers public , private , and protected are not changed by the JDO reference enhancer. Fields are still not accessible by other classes, although JDO implementations are authorized to do so by security policies.

• The PersistenceManager is reachable from persistent instances. Those instances can be deleted from or inserted into the datastore.

• The other JDO interfaces, Transaction, PersistenceManagerFactory , Extent, and Query can be reached, and other operations can be performed than those available at the persistent instance.

• Queries can be executed with expressions that contain private fields outside of the owning class.

JDO does provide some basic application security, which can be used to prevent some flaws mentioned above.

10.3.1 User /password security

When JDO is used in a non-managed environment, the application can provide the user and password to open a connection to an underlying datastore. The code looks like this:

 Properties props = new Properties(); props.setProperty("javax.jdo.option.  ConnectionUserName  ",                        username); props.setProperty("javax.jdo.option.  ConnectionPassword  ",                        password); ... other properties ... PersistenceManagerFactory pmf =        JDOHelper.getPersistenceManagerFactory(props); 

Additionally, the PersistenceManagerFactory can create instances of PersistenceManager with different user contexts. As mentioned in Chapter 5, it is not possible to change the properties of a PMF after a first PersistenceManager has been created.

 Properties props = new Properties(); ... other properties ... PersistenceManagerFactory pmf =        JDOHelper.getPersistenceManagerFactory(props); PersistenceManager pm = pmf.getPersistenceManager(  username, password  ); 

In the second variant, JDO implementations must keep track of connection pooling and user/manager mapping. If different PersistenceManager instances are used with the same user identity, the same database connection might be used. As illustrated in Figure 10-4: User context and connections, the PersistenceManagerFactory created three PersistenceManager instances, two of them sharing the same user context.

Figure 10-4. User context and connections.

How this mapping is done is not defined by JDO.

10.3.2 Managed environments

In managed environments, the user or session can be authorized by the container, and PersistenceManagerFactory instances are returned by a JCA resource adapter, as explained in Chapter 9:

 PersistenceManagerFactory pmf =    (PersistenceManagerFactory)new InitialContext().         lookup("java:comp/env/jdo/example") 

The resource adapter is responsible for providing the security context or getting the security credentials from the container. This means that a session can share its access rights, user information, and so on with different services in the container, like sharing a transaction manager across multiple, distributed transactions. Likewise, security checks can be more restrictive in a managed environment, so that only specific classes are allowed to access the persistence service. This would also prevent code from taking the PersistenceManager of a persistent instance and creating new connections to a datastore, as mentioned before.

10.3.3 Application-specific authorization

Some application types require a different layer of security on top of the data-store's native security layer. For example, to manage Internet chat room users by database user administration would be overkill. In this case, implementing a simple user administration by means of transparent persistence is recommended. Whenever rights, groups, and users should be managed by the application itself, it is advised to do so. It provides datastore independency and keeps the domain object model free of vendor-specific code.