This last group of custom functions requires some explanation, and perhaps some evangelism as well. These functions are complex; our intent was to demonstrate, with a set of real examples from our own work, how its possible to use custom functions and the power under the hood of this often under-appreciated feature.
This fnXML*** set of functions all exist to help manipulate a block of XML data, stored in a text field or variable. Together they represent a kind of small API (Application Programming Interface), and support some specific programming methodologies. In that way, this suite of functions is less general purpose than the earlier ones in this chapter.
The strength of these functions lie in their capability for manipulating complex data structures. You can add, delete, update, and extract data from a tree of nested data.
An API is a term borrowed (perhaps with a degree of liberty) from other programming environments like C# and Java. It refers to a set of instructions that are largely independent and allow programmers to accomplish some set of functions without having to know how the API itself was constructed. Both Apple and Microsoft provide a large range of APIs with their operating systems, theres an Apache API for extending its capabilities as a web server, and theres even a FileMaker API for writing plug-ins. The idea here is that an API provides a framework and hooks into some set of functionality that you as a developer intend to leverage and reuse, without necessarily needing to understand all the details of how it works internally.
A data tree is a powerful programming concept also borrowed from other environments. Abstractly, it is essentially a data structure that can hold multiple values. These values can be referenced by their positions within the tree (or perhaps array, if you accept a loose interpretation of the term) and related in similar ways to a set of FileMaker tables. In crude terms, this data structure can be thought of as a database within a database.
Data trees and arrays are useful for a variety of things, including storing a simple list of values, efficiently moving multiple values as a single block from place to place in a system, and for dealing with variable length data structures where it would be impractical or impossible to define fields or variables enough to hold them.
Perhaps an example would help; this is a simple one-dimensional array, and represents a simple one-level hierarchy of tiered data:
[ red | green | blue | yellow ]
In this scenario, most FileMaker developers would chooserightlyto work with either a repeating variable or a return-delimited list and the GetValues() function. (Weve included list handlers in this book as well.)
But there are times when a one-dimensional array isn enough for your needs. Consider a scenario where you need to store colors and, say, shirt sizes and quantities. Your pipe-delimited array will need some new delimiter characters:
[ red; large; 20 | green; large; 10 | blue; medium; 15 ... and so on ]
As you can see, even a two-dimensional array can start to feel complex.
This is where XML can come in. How does one describe an N-dimensional data structure in a way that can be interpreted by both humans and across multiple platforms?
Note that the issue is twofold: XML holds information about its data, as well as providing a structure in which to store it.
Entire books have been written on XML and a complete discussion of it extends beyond the scope of this book; however, suffice it to say we chose XML for three reasons. First, its self-documenting: Instead of identifying something by its position within an array, XML uses tags to clearly label data.
SiliconValleyT 122 red Extra Large 100 Large 200 1010 122 1011 142
Imagine trying to represent that data structure in a flat, delimited text list, and the mind boggles. Note too that even if you e unfamiliar with XML or with the data that this block is meant to express, you can infer a great deal by simply reading it.
Having now decided on an API to manage a data tree, and having selected XML as a data format, this now brings us to FileMaker. Weve worked with various sorts of arrays and trees for many years, but working with return-delimited lists or temporarily shoving things into makeshift data tables has never completely fit the need.
FileMaker 7 introduced script parameters and FileMaker 8 introduced script variables. Neither of them support anything other than one value (which can admittedly be a repeating value); if you want to pass more than one piece of data by either of these two features of FileMaker, youll need to use a block of text and delimit it somehow. Then the second part of the process will be writing a parsing routine that extracts your multiple values from this data block.
This problem is a good candidate for one or more custom functions. Rather than writing a series of parsing routines throughout a database, we suggest building a set of array handlers that can abstract and centralize the entire set of functionality you need.
The approach weve developed here is admittedly complex. We wanted to get more than just a simple container of one or two dimensional data: We wanted to be able to name the values in our data and to hierarchically organize them to N layers deep. We have created a path syntax (inspired by XPath, for those of you familiar with ithttp://en.wikipedia.org/wiki/Xpath) that can pull a variety of structured data from an XML source.
The six main functions do the following:
fnXMLselect Extracts a block of data from an XML source.
fnXMLupdate Replaces a block of data within an XML source with a new value.
fnXMLinsert Inserts a new block of XML into an XML source.
fnXMLdelete Deletes a block of XML from within an XML source.
fnXMLclean Strips an XML block of extraneous characters and formatting.
fnXMLformat Adds tab characters and return carriages to a block of XML for easy display/reading.
fnXMLselect() is the most powerful of the four functions and is used by three of the others. It takes two parameters, xmlSource and path, and returns a block of XML extracted from the value passed into xmlSource.
The path syntax is specific and is the key to understanding how to use all four functions. An example (referring to the preceding shirt inventory example) might be:
inventory/shirt/color
This path would return the value for the first color element of the first shirt of the first inventory it finds. Think of this path exactly like a tree or a file directory structure. In the preceding case, this path would return
red
The path inventory/shirt/sizes would return from the XML on pages 240-241:
Extra Large 100 Large 200
The path syntax here is specific and drives the logic of what data you manipulate within the block of XML. Valid syntax includes
node/subnode
(a tree of any depth comprised of simple path nodes)
node/subnode/
(function will strip trailing slashes)
node[2]/subnode
(specify an integer to take the Nth occurrence of a node; 1-based)
node[attribute="foo"]/subnode
You can specify finding an occurrence of a node where a child attribute node contains specified data.
Note: This syntax does not support XML attributes.
Supported:
Not supported:
This syntax also does not support the empty/close shortcut style:
This syntax supports the use of carriage returns, tabs, and spaces within XML values. It will not strip them out, and shouldn produce bugs when encountered; however, using fnXMLinsert() and fnXMLdelete() in combination with XML formatted with such characters may end up looking fairly ugly. It should retain functionality, however.
Imagine a scenario where you want to create an audit trail. (Using auto-enter by calculation functions, you can trigger a second field to update itself when a given field is updated by the user.)
Rather than having to create double the amount of fields in your database, youll want to store the audit information likely in one field. Likewise, you need to be able to store multiple values for a single field: who edited a field, what the old value was, what the new value is, and so on.
These functions would be perfectly suited for just such a scenario: use fnXMLinsert to store information into the audit trail field, and then use fnXMLselect to extract it for a rollback if necessary. The data structure might look something like this:
<audittrail>12 CustomerID 1001 Address 123 Main Street 100 Center Drive Molly Tully 11/12/2005 11:10:14
You can then extract the old value with a query like so:
fnXMLselect ( auditTrail; audittrail/record[id=12]/field[name=Address]/oldValue )
This function will return: 123 Main Street.
The following, finally, are the functions that comprise the suite of tools within this API.
This function calls the fnXMLclean_sub() function and is used to supply default initial values to that function.
Example:
Assume a field exists, xmlSource, that includes XML data formatted with return characters and tabs or spaces:
California San Mateo downtown
fnXMLclean ( xmlSource ) returns
California San Mateo downtown
Note that all text formatting (font, size, style, and color) will be removed as well.
Code:
// strips extraneous characters between end tags and start tags // input: text with propervalue XML embedded // output: cleaned XML // dependencies: this is the calling function to fnXMLclean_sub that needs a second parameter initialized TextFormatRemove ( fnXMLclean_sub ( xmlSource; 1 ))
This function takes a block of XML with some amount of extraneous characters sitting between end tags and start tags in the form of spaces, tabs, and carriage returns, and returns a block of XML stripped of all such detritus.
Note that it uses a subfunction to determine white space characters. If ever ones logic needed to be extended, this would easily allow for such.
Example:
Assume a field exists, xmlSource, that includes XML data formatted with return characters and tabs or spaces:
California San Mateo downtown
fnXMLclean ( xmlSource ) returns
California San Mateo downtown
Code:
// iterates through a block of XML and removes extraneous characters // between close and open tags. // input: text block of XML // afterStartTag = initial value should be 1. 1 if the prior tag was a start tag ( or at the start of the processing of the data ), 0 if the prior tag was an end tag. // output: text - clean XML // dependencies: uses fnIsWhitespace Let ([ nexttag = Position( xmlSource;"<";1;1 ); ending = Position( xmlSource;">";1+nexttag;1 ); isStartTag = If( Middle( xmlSource;nexttag+1;1 )="/";0; 1 ) ]; If(nexttag = 0 or ending=0; // Error case, or when source is empty If(fnIsWhitespace ( xmlSource ); "" ; xmlSource ); Let ([ rest = fnXMLclean_sub ( Right( xmlSource;Length( xmlSource )-ending );isStartTag ); start = Left( xmlSource;nexttag-1 ); tag = Middle( xmlSource;nexttag; ending-nexttag+1 ) ] ; Case ( isStartTag=0 and afterStartTag = 1; start; fnIsWhitespace( start ); ""; start) // end Case & tag & rest ) // end Let ) // end If ) // end Let
This function, along with its sibling functions fnXMLselect, fnXMLupdate(), fnXMLinsert(), exists to help manipulate an XML data structure.
This particular function will remove a block of data as controlled by the path parameter.
Examples:
fnXMLdelete ( xmlSource; "first_name" )
The result would be that within xmlSource, the
Consider the following source XML:
source XML =100 foo
for fnXMLdelete ( xmlSource; "root/branch" ) the result would be
Code:
// Function removes a block of XML as controlled by a path. // dependencies // makes use of two global vars => $$xmlSourceValueStart and $$xmlSourceValueEnd. Needs to initialize them to ZERO. // uses fnXMLselect to find the proper position within xmlSource // input: // xmlSource = a block of xml. Syntax is strict. Only usepairs. // path = text string defining the hierarchical tree that should be used to point to a specific branch or node within the block of XML. // see fnXMLselect comments for syntax options and examples. // output: // xmlSource with data removed Let([ $$xmlSourceValueStart = 0; $$xmlSourceValueEnd = 0; vValueLength = 0; vSourceLength = Length ( xmlSource ); vBlock = fnXMLselect ( xmlSource ; path ); vValuePosition =Position(xmlSource;"<";$$xmlSourceValueStart;-1)-1; vBlockLength = $$xmlSourceValueEnd + ($$xmlSourceValueStart-vValuePosition)*2+1; end_position = vValuePosition + vBlockLength + 1; position_of_following_CR = Position(xmlSource;"¶";end_position;1); size_of_trailing_string = position_of_following_CR- end_position; trailing_string = Middle(xmlSource;end_position;size_of_trailing_string+1); vBlockLength = If(Trim(trailing_string)="¶";vBlockLength+size_of_trailing_string+1;vBlockLength) ]; // end variable declaration Left ( xmlSource; vValuePosition ) & Middle ( xmlSource; vValuePosition + vBlockLength + 1; vSourceLength + vValueLength) )// end let
This function calls the fnXMLformat_sub() function and is used to populate default initial values.
Example:
Assume a field exists, xmlSource, that holds
California San Mateo downtown
fnXMLformat_sub ( xmlSource ) returns
California San Mateo downtown
Code:
Replace ( fnXMLformat_sub ( xmlSource; 1 ; "" );1;1;"")
This function is used to format a block of XML into an easy-to-read form. It inserts tab and carriage return characters, and colors the XML tags.
Notice that it uses subfunctions for the color choice, tab character, and for determining if there is already some whitespace (spaces, tabs, and return characters) in the block of XML.
Further, this function uses two parameters for keeping track of its recursions. This is a sub-function that should never be called by anything other than its enclosing fnXMLformat() function.
Example:
Assume a field exists, xmlSource, that holds
California San Mateo downtown
fnXMLformat_sub ( xmlSource; 1; "" ) returns
California San Mateo downtown
Code:
[View full width]
// formats xmlSource with tab and return characters // input: // xmlSource = text block of raw, unformatted xml // afterStartTag = initial value should be 1. 1 if the prior tag was a start tag (or at the start of the processing of the data), 0 if the prior tag was an end tag. // indent = initial value should be set to ""; used for recursion to store iterative data // output: formatted xmlSource // dependencies: uses fnTab, fnIsWhitespace, fnTextColor Let([ nexttag = Position(xmlSource;"<";1;1); ending = Position(xmlSource;">";1+nexttag;1); isStartTag = If(Middle(xmlSource;nexttag+1;1)="/";0; 1 ); indentStep = fnTab ]; If(nexttag = 0 or ending=0; // Error case, or when source is empty If( fnIsWhitespace(xmlSource); "" ; xmlSource); Let ([ newIndent = If(isStartTag; indent & indentStep; Left(indent;Length(indent)-Length (indentStep))); rest = fnXMLformat_sub (Right(xmlSource;Length(xmlSource)-ending);isStartTag;newIndent); start = Left(xmlSource;nexttag-1); tag = Middle(xmlSource;nexttag+1; ending-nexttag-1) ] ; Case (isStartTag=0 and afterStartTag = 1; start; fnIsWhitespace(start); ¶ & If(isStartTag; indent; newIndent); start) // end Case & "<" & fnTextColor(tag;"blue") & ">" & rest ) // end Let ) // end If ) // end Let
This particular function will create a block of data as controlled by the path and value parameters.
Examples:
fnXMLinsert ( xmlSource; "first_name"; "Alexander" )
The result would be that within xmlSource, a new
Consider the following source XML:
source XML =
100 foo
for fnXMLinsert ( xmlSource; "root/branch; "999
500 foo 99911/20/2005
Code:
// Function creates a block of XML with value as controlled by a path. // dependencies // makes use of two global vars => // $$xmlSourceValueStart and $$xmlSourceValueEnd. Needs to initialize them to ZERO. // uses fnXMLselect to find the proper position within xmlSource // input: // xmlSource = a block of xml. Syntax is somewhat strict. // Only usepairs. // path = text string defining the hierarchical tree // that should be used to point to a specific branch // or node within the block of XML. // see fnXMLselect comments for syntax options and examples. // value = a block of text or xml // output: // xmlSource with new data Let ([ $$xmlSourceValueStart = 0; $$xmlSourceValueEnd = 0; vValueLength = Length ( value ); vSourceLength = Length ( xmlSource ); vBlock = fnXMLselect ( xmlSource ; path ); vValuePosition = $$xmlSourceValueStart; vBlockLength = $$xmlSourceValueEnd ]; // end variable declaration Left ( xmlSource; vValuePosition ) & value & Middle ( xmlSource; vValuePosition + vBlockLength+ 1; vSourceLength + vValueLength) & "¶position:"&vvalueposition ) // end let
fnXMLselect() serves as both a subfunction for three of the other XML-parsing functions and as the means by which developers can extract data from an XML block.
Its primary mission is to take a path parameter (discussed in detail in the preceding pages) and return a block of XML extracted from a larger XML data source.
It also makes use of two global variables to keep track of where within the source XML a given block starts and ends.
Examples:
fnXMLselect ( xmlSource; "last_name" )
which might return Smith.
Another call might look like this:
fnXMLselect ( xmlSource; "new_record_request/invoice/fkey_customer" )
and might return C_10012 as a customer ID.
Another approach can make use of filtering, using a square bracket construction similar to an XPath predicate:
fnXMLselect ( xmlSource; "new_record_request/invoice[date="11/11/2005"]/fkey_customer" )
This might return a different customer ID.
Consider the following source XML:
source XML =
100 foo 200 xyz
for path root/branch:
result =
100 foo
for path root/branch/num:
result =
100
for path root/branch[2]:
result =
200 xyz
for path root/branch[num="200"]:
result =
200 xyz
for path root/branch[num="200"]/text:
result =
xyz
Code:
// Function returns a block of XML as controlled by a path. // It recursively iterates through a block of XML until it reaches the end of the path parameter and returns a "child" block of XML. // dependencies: makes use of two global vars => // $$xmlSourceValueStart and $$xmlSourceValueEnd. // Expects them to start at ZERO // input: // xmlSource = a block of xml. Syntax is somewhat strict. Only usepairs. // path = text string defining the hierarchical tree that should be used to point to a specific branch or node within the block of XML. // see below for syntax options and examples. // output: // text = a block of text as extracted from xmlSource Let ([ // Path Values vPathLength = Length ( path ); vPath = Case ( Right ( path; 1) = """/"; Left ( path; vPathLength - 1); path ); // strips trailing slash if necessary vPathNodeCount = PatternCount ( vPath; "/") + 1; // counts the nodes in the path provided vPathSlashPosition = Position ( vPath; "/"; 1; 1); vPathNew = Middle ( vPath; vPathSlashPosition + 1; vPathLength - vPathSlashPosition ); // crops the first root of the path out... // this will be passed recursively back into the function vPathFirstNode = Case ( vPathNodeCount > 1 ; Left ( vPath; vPathSlashPosition - 1 ); vPath ); // the inverse of vPathNew, takes the first node and drops the rest // Isolate Expression Info vPathLBracketPosition = Position ( vPathFirstNode; "[";1; 1 ); vPathRBracketPosition = Position ( vPathFirstNode; "]";1; 1 ); vPathExpression = Middle ( vPath; vPathLBracketPosition + 1; vPathRBracketPosition - vPathLBracketPosition - 1 ); // Isolate First Tag within Path vPathRTagPosition = Case ( vPathLBracketPosition > 1; vPathLBracketPosition - 1; Length ( vPathFirstNode ) ); vPathTag = Left ( vPath; vPathRTagPosition ); // Expression Checks // test to see if bracketValue is an INTEGER // or if it is a NAME-VALUE pair, by checking for an "="char. // in the case that its a name-value pair, extract the // search tag in question. vExpressionValue = Case ( PatternCount ( vPathExpression; "=" ) > 0; Let ([ vExpressionTagEnd = Position ( vPathExpression; "=";1; 1) - 1; vExpressionTag = Trim ( Left ( vPathExpression; vExpressionTagEnd ) ); vExpressionLPosition = Position ( vPathExpression; "=";1; 1) + 2; vExpressionValueLength = Length ( vPathExpression ) - vExpressionLPosition; vExpressionValue = Trim ( Middle ( vPathExpression; vExpressionLPosition; vExpressionValueLength ) ); vExpressionResult = "<" & vExpressionTag & ">" & vExpressionValue & "" & vExpressionTag & ">" ]; // end variable declaration vExpressionResult ); // end let // else if there is no "=" within vPathExpression // return "IsInteger" as a control check for blockOccurence "IsInteger" ); // end case { vExpressionValue } // vParentCount will determine which parent node contains // the search string in question. should end up with an integer // to be used as an occurrence variable. vExpressionTagPosition = Position ( xmlSource; vExpressionValue;1; 1); vCropSource = Left ( xmlSource; vExpressionTagPosition ); vParentCount = PatternCount ( vCropSource; "<" & vPathTag & ">" ); // Set Occurrence // The following is used in the position functions below // for extracting blocks. This controls which of a given block // is taken, the first, second, third, etc. // If vPathExpression is an integer, use that for blockOccurrence. // If theres something in searchTagString, use the vParentCount as // blockOccurrence. Otherwise use 1. vBlockOccurrence = Case ( vExpressionValue ? "IsInteger"; vParentCount; vPathExpression > 1; vPathExpression; 1 // default to 1 ); // end case { vBlockOccurrence } // Extract XML between vPath tags vBlockStartChar = Position ( xmlSource; "<" & vPathTag & ">"; 1; vBlockOccurrence) + Length ( "<" & vPathTag & ">"); vBlockEndChar = Position ( xmlSource; "" & vPathTag & ">"; vBlockStartChar ; 1 ); vBlockLength = vBlockEndChar - vBlockStartChar ; vBlockRaw = Middle ( xmlSource ; vBlockStartChar ; vBlockLength ); // Trim excess ¶ and space chars by excluding all but the block itself vBlockPositionStart = Position ( vBlockRaw; "<"; 1; 1); vBlockCountClose = PatternCount ( vBlockRaw; ">" ); vBlockPositionEnd = Case ( vPathNodeCount = 1; Length ( vBlockRaw ); Position ( vBlockRaw; ">"; 1; vBlockCountClose) ); vBlockTrimmed = Middle ( vBlockRaw; vBlockPositionStart; vBlockPositionEnd - vBlockPositionStart + 1 ); // update global pointer to track beginning position // of block and value within xmlSource // ( allows for update, delete, insert ) $$xmlSourceValueStart = $$xmlSourceValueStart + vBlockStartChar - 1; $$xmlSourceValueEnd = vBlockLength; // Error Checking vErrorHasCloseTag = vBlockEndChar; vErrorHasTag = PatternCount ( xmlSource; "<" & vPathTag & ">"); vErrorHasNameValueMatch = Case ( vExpressionValue = "IsInteger""; "Ignore"; Case ( PatternCount ( xmlSource; vExpressionValue ) > 0; "Ignore"; "No Match") ); vErrorOutOfBounds = Case ( vExpressionValue ? "IsInteger""; "Ignore"; Case ( PatternCount ( xmlSource; "<"& vPathTag & """>" ) < vBlockOccurrence; "Out Of Bounds"; "Ignore")); //Result //--> if you need to debug, just comment out the case below // and place one of your variables next to result. vResult = Case ( IsEmpty ( xmlSource ) and IsEmpty ( path ); "Error: Missing Parameters ( xmlSource; path )"; IsEmpty ( xmlSource ); "Error: Missing Parameter ( xmlSource )"; IsEmpty ( path ); "Error: Missing Parameter ( path )"; vErrorHasTag = 0; "Error: Invalid Tag (" &vPathTag & ")"; vErrorHasCloseTag = 0; "Error: No Close Tag ("& vPathTag & ")"; vErrorHasNameValueMatch ? "Ignore"; "Error: Invalid Name/Value "( " & vExpressionValue & ")"; vErrorOutOfBounds = "Out Of Bounds""; "Error: Invalid Index (" & vPathTag & "[" & vBlockOccurrence & "])"; // now return valid xml block in the case that // no error is returned; recursive if additional nodes exist vPathNodeCount > 1; fnXMLselect ( vBlockRaw; vPathNew ); vBlockTrimmed // default to value ) // end case {result} ]; // end variable declaration vResult ) // end let
This particular function will replace a block of data within the source XML with the contents of the "value" parameter.
Examples:
fnXMLupdate ( xmlSource; "last_name"; "Smith" )
The result would be that within xmlSource, the first
Consider the following source XML:
source XML =
100 foo 200 xyz
for fnXMLupdate ( xmlSource; "root/branch/num"; 500 ) the result would be
500 foo 200 xyz
Code:
// Function replaces a block of XML with value as controlled by a path. // dependencies // makes use of two global vars => // $$xmlSourceValueStart and $$xmlSourceValueEnd. // Needs to initialize them to ZERO. // uses fnXMLselect to find the proper position within xmlSource // input: // xmlSource = a block of xml. Syntax is somewhat strict. // Only usepairs. // path = text string defining the hierarchical tree // that should be used to point to a specific branch or // node within the block of XML. // see fnXMLselect comments for syntax options and examples. // value = a block of text or xml // output: // xmlSource with new data Let ([ $$xmlSourceValueStart = 0; $$xmlSourceValueEnd = 0; vValueLength = Length ( value ); vSourceLength = Length ( xmlSource ); vBlock = fnXMLselect ( xmlSource ; path ); vValuePosition = $$xmlSourceValueStart; vBlockLength = $$xmlSourceValueEnd ]; // end variable declaration Left ( xmlSource; vValuePosition ) & value & Middle ( xmlSource; vValuePosition + vBlockLength + 1; vSourceLength + vValueLength) & "¶position:"&vvalueposition ) // end let
: FileMaker Specifications
FileMaker 8 Product Line
Specifications and Storage Limits
Field Types and Import/Export Formats
: Calculation Functions
Working with Calculations Primer
Calculation Signatures
Calculation Functions
: Custom Functions
Custom Functions Primer
Useful Custom Functions
: Script Steps
Scripting Primer
Script Step Reference
: Quick Reference
FileMaker Error Codes
FileMaker Keyboard Shortcuts
FileMaker Network Ports
FileMaker Server Command Line Reference
FileMaker XML Reference
: Other Resources
Where to Go for More Information