Many of the patterns and best practices presented in this book assume a certain level of knowledge in the area of Web services. It is assumed you have experience designing and developing in at least one object- or component-oriented language, as well as a basic knowledge of XML. However, I don't insist that you have any experience with designing and developing XML Web services. This section should provide you with enough information to get you started and to help you better understand those patterns that use Web services for their implementation choice. That said, I highly recommend that you acquire deeper architectural knowledge of Web services and their protocols. Understanding the protocols that surround Web services will not only aid you in better understanding the elements in this book but will also help you in your design and later debugging efforts.
This book mainly focuses on implementing Web services using the SOAP protocol over HTTP. Throughout the book, I will provide explanations in the form of technology backgrounders specifically in those areas of SOAP considered essential to grasping the implementation of the pattern presented. I have found that having material devoted strictly to SOAP at my side is not a bad idea. One good resource is SOAP: Cross Platform Web Service Development Using XML , by Scott Seely (Prentice Hall, 2001). The specification and tutorials on the World Wide Web Consortium (W3C) Web site at http://www.w3.org/TR/2001/WD-soap12-part0-20011217/ are another great source of information regarding SOAP.
What Is an XML Web Service?
XML Web services are somewhat loosely defined in the most general sense. I am not talking about .NET Web services because .NET provides the tools to generate both a Web services client and a server with little effort. What I'm referring to is the overall concept and technology behind it. You can construct an XML Web service without the use of any Web service-generation tool. You do not even require some of the more specific protocols, such as WSDL, to be successful. The idea of an XML Web service should be understood first before diving into each "recommended" protocol. Using its vanilla definition, XML Web services are simply distributed "invokeable" services that use XML to send data across a standard and accepted protocol. XML is also used to describe the service and the parameters that the service requires to execute remotely.
Instead of using proprietary protocols in a proprietary format, such as those employed by the likes of COM or many other now legacy distributed mechanisms, Web services simply use what is already open to the public. Not only is HTTP and XML open , but you now can actually make sense of the data packets you send to and receive from Web services because they are now self-describing and actually quite readable. Gone are the days of using proprietary binary readers to make sense of distributed data streams. Using Notepad will do just fine. Of course, for complex data structures, I'm oversimplifying it a bit, but not by much.
You can architect your own version of a Web service without the need for any specific protocols by simply using XML over HTTP. However, to distribute and comply better with the rest of the world, you must understand some of the other protocols used when building Web services. Also, to gain a true understanding of how to design Web services, you really have to understand the primary language that both describes the service and is used for all data packets. This is, once again, XML. Aside from that, you should have a general understanding of many of the protocols used to search, discover, format, transfer, and interpret the elements of a Web service. Do you have to understand each protocol in detail? No. However, the better you understand the likes of SOAP and WSDL, the more successful you will be in invoking Web services that require complex data types, such as those used for transporting relational rows of data or complex custom objects.
The exact definition of an XML Web service varies, depending on whom you ask. Some are simply exposed functionality points using standard Web protocols. In most cases, you will find SOAP being that protocol. In addition to SOAP, two other primary protocols have emerged: WSDL and Universal Discovery Description and Integration (UDDI). SOAP is the XML language for invoking a service but WSDL is how you describe to the outside world what you have to offer. WSDL allows clients to be built either by statically or dynamically accessing a service's WSDL document. For those familiar with COM, its acts as a "type library" of sorts to provide the details of the Web service, its methods , and the data types it accepts and returns. With SOAP, XML, and WSDL you provide, describe, and use fully distributable and invokeable services from anywhere on the Internet or even your own intranet. But how does one find all of the Web services on a particular server or anywhere on the Internet, for that matter? This is where UDDI comes in. Web services are registered so that users can find them wherever they may be. Once a service is found, its WSDL file can be accessed (in the case of .NET Web services, by specifying the Web service *.asmx followed by ? WSDL , e.g., http://www.companyxyz.com/ productx .asmx?WSDL), which the client then uses to access that functionality. How the WSDL information is retrieved or how Web service consumers are built is completely vendor-dependent . Microsoft has provided a very simple means of both producing and consuming Web services using Visual Studio .NET. Figure 1.6 shows a sample WSDL file that is built in and to be used from within Visual Studio .NET using C#.
Figure 1.6. A sample Web service WSDL via Visual Studio .NET when adding a Web reference from Visual Studio .NET 1.0 (2001).
If you are using Visual Studio .NET 1.0 by selecting View Contract in Figure 1.6, the appropriate Web service proxy will be generated and added to the project. From this proxy, a Web service can then be instantiated and called, just like any other .NET object (with a little help from a proxy generator or a little elbow grease). This is one way to implement this behavior. If you do not have the luxury of using Visual Studio .NET, the proxy can also be generated with command tools provided by Microsoft (such as sproxy.exe) or "manually consumed" using something like Microsoft's SOAP Toolkit. Using the SOAP toolkit, you obtain direct control over the SOAP requests sent and received against a Web service, albeit with slightly more work. If you are using Visual Studio .NET 2003 edition, you will have a slightly different Web references interface. The latest version of Visual Studio .NET has simplified this interface and included computer and even network browsing features (Figure 1.7). This radically simplifies the work required to even find a Web service, whether it exists somewhere on your machine (Figure 1.8) or somewhere on the Internet.
Figure 1.7. Using Visual Studio .NET, you can now browse for Web services locally or remotely.
Figure 1.8. A sample Web service local machine listing (using 2003) ”each providing WSDL to bind to.
With XML, SOAP, and WSDL, vendors can now provide their own implementations of these open standards in any way they see fit. The real benefit, however, is that the standards do exist without the traditional disputes over distributed wire protocols. Will there be vendor-specific differences in what you can or cannot do? Yes, I don't know whether we will ever get completely away from that. Just look at the differences in the J2EE space with all of the vendor implementations of EJB servers (e.g., BEA WebLogic or WebSphere). Finally a "standard bar" has been raised for what makes up a distributed service, how to access it, and how to use it. For more information on WSDL, please refer to the WSDL specification at http://www.w3.org/TR/wsdl.
Web Services and SOAP
When given any XML parser, such as a Microsoft's XML DOM component, anyone can pass a function name embedded into an XML message. Therefore, you do not need SOAP or any formal protocol around XML Web services to accomplish this. The designers of such a system could provide a specification for the outlay of the function parameters and even return XML after the function is called if they wanted. Standard XML Web services just make it easier. In fact, I helped build such a system before SOAP was a standards recommendation. SOAP simply standardizes this type of RPC mechanism. SOAP is the communication protocol used by XML Web services. Is SOAP required? No. Is it recommended? Yes. Only because SOAP provides a standardized means of describing the data elements that make up the messages sent to SOAP servers, including .NET servers hosting Web services. Along with WSDL and UDDI, SOAP is another element in the distributed arm of .NET and the Web services it hosts . Besides being a mechanism for invoking RPCs, it can also be used as a wrapper around entire XML documents or binary data, as is the case with the DIME protocol from Microsoft. This greatly expands what can be done with this protocol and will continue to expand as XML Web services evolve .
As mentioned, when using Web services, SOAP is not the only protocol that can be used to call Web services. For example, a simple HTTP GET request can be made as long as it is passing and receiving simple data types, such as strings. However, when you begin to work with more complex types, such as ADO DataSets (covered later in this book), it becomes necessary to use a protocol such as SOAP and more specifically, XML schemas to describe such data as referenced in the SOAP message. SOAP was designed for simplicity and elegance , and it does not do some of things you might expect. Some of the elements not covered (at this time) are:
SOAP currently does not outline these specific elements like other remote protocols, such as DCOM and CORBA. But it does what it was intended to do ”provide us with a simple, standard specification to drive the description of the data types and methods for using and producing Web services. It is important to note, however, that a SOAP "message" only specifies the XML format for the package sent across the wire. As long as you have a well- formed XML document encapsulated in the necessary SOAP tags, you have a valid SOAP message. Any compliant SOAP implementation on the server should understand the request and should service it. In fact, although the default application protocol for SOAP is HTTP, it does not require it. That's the simplicity of it. Other protocols, such as SMTP, FTP, and even asynchronous messaging protocols, will soon be supported. Because HTTP is the Internet protocol, it is also the default for SOAP. This is why when you hear about XML Web services, it is usually assumed that there will be a one-to-one correspondence with both SOAP and HTTP.
To truly understand SOAP, you first have to understand a few basic elements of the language that makes up a SOAP message. Once again, that language is XML. Most interaction with SOAP usually takes place through a toolkit or service of some kind. For example, the Microsoft SOAP Toolkit (Java has its own) hides the exact format of a SOAP message by providing high-level programmatic client objects to use. These clients provide the code to serialize, deserialize , send, and receive SOAP messages to and from a corresponding SOAP server, such as a .NET Web service. This simplifies developers' lives but it also shelters them from understanding the format in which their messages are bundled. Knowing such details will help your debugging and designing efforts in the future, and it is recommended that you become at least slightly acquainted with such details.
To do so, you first need to understand a few XML items. These "XML Basics" are included in Table 1.1.
Why will SOAP succeed in widespread fashion where others have failed? It already has. This is not to say that protocols such as COM or CORBA have failed. They will continue to be used for many years to come. There just has not been any one distributed protocol that has been adopted by so many different vendors as SOAP has. By the time this book is published, I am sure there will be even more.
Table 1.1. XML tags essential for understanding SOAP-formatted messages
So what does a SOAP message look like? Unfortunately, I don't have room to explain every element. Fortunately, SOAP's specification is rather simple, compared with other distributed protocols. To format a SOAP message and send it to a Web service requires knowledge of only a few XML tags. After all, SOAP is merely XML. Here is a sample SOAP message calling a Web service method, GetSchemaTemplate (). As its name implies, this Web method returns an XML schema from a database formatted as a .NET DataSet from ADO.NET.
Listing 1.1 A sample SOAP message captured using SOAP Trace Utility.
<?xml version="1.0" encoding="utf-8" ?> <soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/xmlSchema-instance" xmlns:xsd="http://www.w3.org/2001/xmlSchema"> <soap:Body> <GetSchemaTemplate xmlns="http://www.companyxyz.com/productx.net"> <sSchemaName>FinancialPacket</sSchemaName> <sType>CreditCard</sType> <bUseCaching>false</bUseCaching> </GetSchemaTemplate> </soap:Body> </soap:Envelope>
Besides the SOAP namespace, the only two SOAP tags are Body and Envelope. In fact, just by looking at this sample you should have an idea of what most SOAP messages generally look like. This message, in particular, is used to call a Web method named GetSchemaTemplate , passing in the name of the schema, the type, and whether to cache the results. Each parameter is passed as in inset tag of the Web method tag. This is enclosed by the <soap:Body> tag, which contains the contents of the RPC desired. The <soap:Body> is then enclosed in a <soap:Envelope> tag. The envelope is what encloses the entire SOAP message and is where you will encompass any namespaces declarations, such as:
Optionally, the SOAP envelope may also contain a SOAP header that is not passed as part of the RPC but rather mimics behavior similar to that of HTTP headers. Unlike the body and envelope tags, headers are not mandatory. SOAP headers can be used to pass information such as security credentials or other message-oriented meta information. They are used to avoid having to add additional parameters to each RPC or in this case, each Web method called. I use SOAP headers throughout this book and will explain their use in detail in the upcoming chapter on technology backgrounders.
Security in SOAP?
Formal .NET training may not quite cover all of the areas required in designing mission-critical Web services-based systems. This is especially true of security. Security policy dictates whether any system will be secure. Unfortunately, formal training may not provide the level of aid in developing a specific .NET security policy. I don't have nearly enough room to exhaust a topic as large as security, especially when it comes to .NET and Web services where it has been enhanced at multiple levels (authentication, authorization, configuration, and code access security). SOAP is only beginning to outline security elements, such as the ws-security specifications just now becoming widely known. This places some of the burden onto the developers and designers of the system. This is great for the end user but the key here is the "shift of burden" comment. Currently, that security burden rests on the developers and administrators. This means that IT must develop a level of comfort with application security that may not currently exist when deploying a Web services application.
A positive side effect to the robustness of .NET security is the fact that the CLR now relieves other areas of concern for IT. Due to its managed environment; the applications that run under the control of the CLR now have the advantage of granular type-safe controls. This includes protecting the system from attacks such as employing "stack walking." This protects the system not only from the immediate called operation but also from all callers of that immediate caller. In the end, this will ease some of the time spent due to attacks from outside intruders.
The depth at which security is supported in the runtime is impressive. The CLR offers granular permission-level control, even at the object level, and allows developers to react to specific permissions granted in their application code. Some of the areas you can control in .NET are:
Administrators can provide the overall policy and leave some responsibility to the developer to tighten further as needed. Granular security also means that applications that should easily run in production may not. During deployment, laying out security tends to be one of those overlooked items that will always bite you. Now imagine a security design at the level at which .NET has outlined. The better prepared and informed those people managing policy are, the more you are assured of deployment success.
No longer can those leading your security infrastructure be unaware of each security element at the application level. To be better prepared, developers can no longer simply learn .NET's new languages, its pure object-orientation model, or the SOAP protocol upon which complex messages will be sent to Web methods. Developers also must cooperate with IT in the permission schemes employed and the levels at which they will enforce security. This includes but is not limited to whether developers will employ role-based security, what authentication method they will use (e.g., IIS, Forms, Passport, etc.), or the code access level they require in their applications. Then they can roll their applications into the appropriate .msi file and deploy with whichever tool they see fit.
Training isn't the only area of focus here. Currently as far as SOAP is concerned , SOAP assumes that security is a transport or application issue and does not define security semantics specifically. This is changing, because there are specifications already released to add security semantics to SOAP (see ws-security). Cooperation among the infrastructure and application development teams must be better than ever. It also would be prudent to bring in an outside security consultant to help evaluate your policies and better prepare your team for this new model. Only this will help guarantee that applications will run with production when they are supposed to and not fail to load just because someone was misinformed.
WSDL is based on the XML schema standard and uses it to describe its services. Because WSDL uses XML and is language-neutral, it is perfect for Web services that are not implemented in the same language. There is no guarantee that everyone will implement his or her Web services using .NET, although I know Microsoft would like it to be that way. WSDL uses a schema to describe where each service is and what protocols to use to access it. For example, if your Web method is like the previous example, the WSDL will inform the consumer of the service that the SOAP protocol is one protocol supported by the GetSchemaTemplate. However, because this particular method uses simple data types in the method, other protocols such as an HTTP POST or GET can also be used. If the data types of the parameters are complex, it is recommended that SOAP be used, and the WSDL will reflect that. This helps the consumer bind to the correct protocol before sending any messages to the server. This binding typically is done statically but can also be done dynamically if need be. You can build a SOAP request simply by looking at the WSDL contents, which is accessed as specified earlier in this chapter (i.e., ? WSDL) or with a WSDL file located somewhere on the Web service server. Luckily for the developer, this usually isn't necessary because many tools exist for reading and binding to WSDL files through the proxy generated, making your life much easier. In Visual Studio .NET, you can read a WSDL file and build a proxy file based on its output. With the proxy, you can then instantiate a Web service just like any other class and call its Web methods as you would normally call a local object. Knowing the plumbing underneath this proxy-generated code may save your life at some point.
But how do you find all of the WSDL content out there in the first place? Hopefully, that answer will be UDDI. This acts as a "Yellow Pages" for Web services. This is optional, however, because you can produce Web services for consumption without ever publishing them to the outside world. This will be the case for those services intended for specific clients. For example, there are Web servers not necessarily published in some public search engines. If you intend to be used by the general public or just want to be found, UDDI is how you do it.
Like everything else described thus far, UDDI is just an XML file. It is used to describe a business or services it offers (Web services, that is). Web services are described by using what are called tModels or Type Models, in "UDDI-speak." In most cases, the model will simply contain a WSDL file that describes the interface to the Web service. UDDI also offers means by which to search for the services you desire based on business type or physical location of that business. Once you find that business, you can search deeper for the specific Web service you are looking for by using what is called the WS-Inspection Language . For more information on either UDDI or the WS-Inspection Language, please refer the following links: