A Guide to XML and Its Technologies

Brian Randell and Aaron Skonnard
DevelopMentor

September 1999

Summary: Introduces XML and the family of supporting XML technologies. Discusses how each of the core technologies fits into the big picture and what some of the supporting XML technologies are all about. (15 printed pages)

Every day it seems like more and more developers are convinced that the Extensible Markup Language (XML) will radically change the software industry. If you ask one of them to explain the details of how or why this is going to happen, chances are that they will fumble through an explanation chock-full of XML acronyms, leaving you curious but definitely not satisfied.

The main obstacle to gaining a solid understanding of XML is XML's rapid evolution. If you point your browser to the World Wide Web Consortium (W3C) XML Web site , you can witness for yourself the vast number of XML technologies and their corresponding publications. Aside from the fact that W3C rhetoric is very unforgiving, it's very difficult for developers to assimilate so much information on a given technology without clearly understanding how the different pieces fit together.

This article introduces you to XML and the family of supporting XML technologies. It will not only give you a high-level understanding of why XML is important, but also explain how each of the supporting technologies fits into the big picture. After reading this article, you'll at least know what each of the XML acronyms mean and you'll be more prepared to start digging deeper into the XML technology that interests you.

So What Is XML, Anyway?

Answering this question has been a popular pastime for most authors discussing XML of late. As you've probably figured out, XML is the official acronym for the Extensible Markup Language. They probably used the letter X instead of the letter E simply because XML sounds much sexier than EML. When you go beyond the acronym, you find more than just a markup language. You find a family of technologies that can be leveraged to build highly extensible and interoperable software solutions.

Where Did It Come From?

XML derived from the Standard Generalized Markup Language (SGML). You can use both XML and SGML to create self-describing documents. Both languages use textual markup (tags) to describe data so that other applications or tools (like an SGML or XML parser) can correctly read the information and then do interesting things with it. XML is a simplified version of SGML, more suitable for use on the Web.

XML Syntax

XML defines the syntax for describing your data. The following is valid XML:

<hamburger name="CowBurger" lowfat="dream on"/>

Unlike other markup languages, XML is case-sensitive. A <hamburger> element is different than a <Hamburger> element. XML also cares about white space (it doesn't ignore white space, as other languages do), and it must deal carefully with special characters that could potentially confuse the document structure (like < and >).

An XML document is considered well-formed if it contains exactly one root element (the document element), and all the child elements are nested properly within each other. This means that both the begin and end tags of a given element should exist within the body of the same parent element. The following is an example of a well-formed XML document (hamburger.xml):

<?xml version="1.0"?>
<hamburgers>
  <hamburger lowfat="dream on">
    <name>CowBurger</name>
    <description>Greasy and good.</description>
    <price>2.99</price>
  </hamburger>
</hamburgers>

Who Defines the Tags?

After reading the previous section, you now know just about everything you need to know about XML syntax. There's not much to it. It's simple and elegant.

You may have noticed that XML looks very similar to the Hypertext Markup Language (HTML). Both XML and HTML use the same markup language syntax to define begin and end tags and attributes. HTML is essentially a specific case of an XML language with predefined elements and behavior. These elements and their associated behaviors define how a given document will be rendered within a Web browser and used by the end user.

In the same manner that HTML provides a universal method to create user interfaces, XML offers a universal way to describe and work with data. XML allows developers to create their own XML vocabularies that are customized for describing their particular data structures. If a developer is writing software for a fast food chain, a hamburger element might come in handy for describing certain greasy snacks.

Once developers harness the power of XML to describe their data, they can easily interoperate with any other homogenous or heterogeneous system that also understands XML. Likewise, the developer can consume data from any other system as long as it's also described using XML. A developer who leverages XML no longer needs to worry about platform, operating system, language, or data store differences when interoperating with other systems. XML becomes the least common denominator for system interoperability.

XML Namespaces

Because XML is truly about interoperability and everyone is free to create their own XML vocabularies, everything would start to break down rather quickly if different developers chose identical element names to represent conceptually distinct entities. To safeguard against these potential conflicts, the W3C introduced namespaces into the XML language.

You use XML namespaces to provide a context for your XML document elements. XML namespaces allow developers to resolve elements to a particular implementation semantic. In the case of our hamburger example, the price element on one system might represent the customer sales price, but on another system it might represent the store's purchase price. The following example illustrates how namespaces can help resolve any potential ambiguity:

<?xml version="1.0"?>
<hamburgers       
    xmlns:purchase="http://fastfood.org/franchise/prices"
    xmlns:sales="http://fastfood.org/customer/prices"
>
  <hamburger lowfat="dream on">
    <name>CowBurger</name>
    <description>Greasy and good.</description>
    <purchase:price>0.99</price>
    <sales:price>2.99</price>
  </hamburger>
</hamburgers>

How Do I Use It?

The XML syntax is not difficult to understand. Leveraging XML data and doing something useful with it is much more challenging.

To do something useful with XML data, we need to be able to programmatically process the XML file. The W3C defines the term XML processor as a software module capable of reading XML documents and providing access to their content and structure. Microsoft's flagship XML processor is called Microsoft XML (MSXML) 2.0. MSXML 2.0 ships with Internet Explorer 5 and is also freely available as a stand-alone redistributable file.

One of the main advantages of adopting XML as the universal standard for describing data is that any XML processor should give us the functionality we need to accomplish this goal. Developers should rarely (if ever) need to write their own XML processors. In theory, developers should be able to leverage the best processor on the market for their particular environments while avoiding compatibility issues.

With a standard XML processor, you can programmatically read any XML document (such as hamburger.xml) and access any element name, body, or attribute. Even if you produced the XML document on a Windows-based system, you could easily ship it off to a mainframe system and use the mainframe's XML processor to interact with the same data. This illustrates the true beauty of XML. XML as a technology is not a silver bullet that solves all of your software problems. It is, however, an open and effective mechanism for exchanging structured data between your own applications and the applications of others.

XML Core Technologies

It is reasonable to create XML documents and use them completely on their own. However, the true potential of XML becomes apparent when you examine the host of supporting technologies that exist and are currently emerging. You will not necessarily need to use all the technologies covered throughout the rest of this article. They are presented solely to help you understand how they fit together as part of the overall XML strategy.

Validation Technologies

As you've seen, XML provides a very flexible syntax for describing well-formed documents. Because of this flexibility, we need some way to validate that a particular class of XML document adheres to a format we expect. For example, the following XML document is well-formed and valid:

<?xml version="1.0"?>
<hamburgers>
  <hamburger lowfat="dream on">
    <hamburger lowfat="maybe">
      <name>CowBurger</name>
      <description>Greasy and good.</description>
      <price>2.99</price>
      <price>3.99</price>
    </hamburger>
  </hamburger>
</hamburgers>

However, there are some application-level problems with this document. Notice that there is a hamburger element within another hamburger element. While there is nothing inherently wrong with this XML structure, in the case of this example, it doesn't make sense. Also, notice that there are multiple price elements within the inner hamburger element. Which price is correct? This probably indicates a bug somewhere in the system. To catch problems like this, it would be useful to have a standard mechanism for validating an XML document against some set of predefined rules.

Schemas

A schema is basically a set of predefined rules that describe a given class of XML document. A schema defines the elements that can appear within a given XML document, along with the attributes that can be associated with a given element. It also defines structural information about the XML document, such as which elements are child elements of others, the sequence in which the child elements can appear, and the number of child elements. It can define whether an element is empty or can include text as well as default values for attributes.

Document Type Definitions (DTDs) and XML-Data are both examples of specifications that outline how to describe XML document schemas.

Document Type Definitions

The DTD language was invented specifically for defining validation rules for SGML documents. Since XML is a simplified subset of SGML, DTDs can also be used to define XML validation rules. An XML processor can use the DTD at run time to validate a given XML file against the predefined XML schema.

The DTD syntax can be a bit overwhelming and esoteric at times. DTDs use different syntactical elements, like exclamation points, parenthesis, asterisks, angle brackets, and many others, to define what elements are required in an XML document, what elements are optional, how many occurrences of a given element are allowed, and so forth. DTDs also describe the relationship between elements and how attributes relate to different elements.

Below is the DTD (hamburger.dtd) for the previously listed hamburger.xml file:

This document states that the hamburgers element can contain many hamburger elements. Each hamburger element must contain a lowfat attribute and three sub-elements, all of the type #PCData (parsed character data). Documents that conform to this DTD would then add the following line:

<!DOCTYPE hamburgers SYSTEM "hamburger.dtd">

This statement tells the parser to validate the content of the XML document against the schema described in the DTD.

Although MSXML 2.0 supports DTDs, you'll find that working with them can be a taxing endeavor. The syntax is ugly and hard to use. Notice that DTD syntax is not valid XML. For this reason, XML processors must support the DTD syntax for describing schemas along with the XML syntax for reading documents. If we described schemas using XML, however, XML document validation would be much easier to deal with for developers and, especially, XML tool vendors. The W3C is currently considering several alternate specifications that will alleviate the shortcomings of DTDs and provide enhancements to the grammar definition process.

XML-Data

XML-Data is one proposed implementation of an XML schema language. In Microsoft circles, XML-Data schemas are loosely referred to as XML schemas, not to be confused with DTD schemas. An XML-Data schema is a well-formed XML document. The XML-Data language is based on the XML-Data DTD, which specifies the expected format for the schema definition. Because XML-Data schemas are simply XML documents, any tool you use to work with XML documents can also be used to work with XML-Data schema definitions.

The following XML-Data schema produces a similar schema to the one defined above by hamburger.dtd:

<?xml version="1.0"?>
<Schema xmlns="schemas-microsoft-com:xml-data">
  <ElementType name="name" />
  <ElementType name="description" />
  <ElementType name="price" />
  <AttributeType name="lowfat" />
  <ElementType name="hamburger" />
    <element type="name" maxOccurs="1" />
    <element type="description" maxOccurs="1" />
    <element type="price" maxOccurs="1" />
    <attribute type="lowfat" maxOccurs="1" />
  </ElementType>
  <ElementType name="hamburgers" model="closed">
    <element type="hamburger" maxOccurs="*" />
  </ElementType>
</Schema>

Elements and attributes are defined in an XML-Data schema by specifying <ElementType> and <AttributeType> elements, respectively. These provide the definition and type of the element or attribute. An instance of an element or an attribute is declared using <element> or <attribute> tags. You can define how many occurrences of a given element are allowed through the minOccurs/maxOccurs attributes. The schema XML structure defines where the element is allowed to exist within the XML document (a <hamburgers> element contains <hamburger> elements, and so on).

Microsoft provides support for XML-Data through MSXML 2.0. According to the Microsoft XML SDK documentation, the XML Schema implementation that ships with Internet Explorer 5 is based primarily upon the XML-Data Note posted by the W3C in January 1998, and the Document Content Description (DCD) note . XML Schemas in Internet Explorer 5 provide support for the subset of XML-Data that coincides directly with the functionality expressed in the DCD, albeit in a slightly different XML grammar.

Processor (API) Technologies

As mentioned earlier, to do something useful with XML, you must be able to programmatically access the data. A software module capable of reading XML documents and providing access to their content and structure is referred to as an XML processor or an XML API.

While developers are free to implement their own XML APIs, it is in their best interests to leverage industry-accepted standard APIs. By accepting an industry standard API, a developer can write code for a given API implementation that should be capable of running under any other compliant implementation of the same API without modifications.

There are two main API specifications that have gained popularity among developers today and are striving to become industry standards: the Document Object Model (DOM) and the Simple API for XML (SAX).

DOM

The Document Object Model is a defined standard for programmatically accessing the structure and data contained in an XML document. The W3C has approved the DOM Level 1 specification as a recommendation.

The DOM is based on an in-memory tree representation of the XML document. When an XML file is loaded into the processor, it must build an in-memory tree that correctly represents the document (see Figure 1). The DOM also defines the programmatic interface (including the names of the methods and properties) that should be used to programmatically traverse an XML tree and manipulate the elements, values, and attributes.

Figure 1. XML in-memory representation

MSXML 2.0 fully supports the DOM and provides an easy-to-use object model for interacting with the in-memory tree. Here is a simple VB example of how you could traverse all of the child elements of the root document element using MSXML:

Set xmlDoc = CreateObject("MSXML.DOMDocument")
bSuccess = xmlDoc.load("hamburger.xml")
If bSuccess Then
   For Each node in xmlDoc.documentElement.childNodes
       val = node.text
   Next
End If

SAX

One of the major downsides to the DOM standard is the overhead involved in loading the entire XML document into memory. For very large data files this can become problematic. If you're transmitting large amounts of XML data around the network or Internet, you may not want to wait for the entire file to finish transmitting before you begin to process the file. As XML developers became painfully aware of this reality, they joined efforts (from the XML-DEV mailing list) and began a collaborative effort to form an alternate specification called SAX. While SAX is still in its preliminary stages, it's quickly gaining popularity because of the performance benefits.

SAX is a very simple XML API (hence the name, Simple API for XML) that allows developers to take advantage of event-driven XML parsing. Unlike the DOM specification, SAX doesn't require the entire XML file to be loaded into memory. The idea is quite simple. As soon as the XML processor finishes reading an XML element, it calls into one of your custom event handlers to just-in-time process the element and its associated data. While this can greatly improve performance, developers do lose a degree of flexibility. For more information on this evolving technology, go to http://www.megginson.com/SAX/sax.html.

Transformation Technologies

Once you start using the standard DOM API to interact with your XML data, you'll notice that it becomes quite tedious to extract specific pieces of data from large documents or to transform certain parts of an XML document into another format (such as HTML).

For example, assume you wish to find all of the lowfat hamburger price elements. In order to do this with the standard DOM API, you must manually write the code that traverses the entire tree looking for those specific elements that match the criteria (in this case price elements that are within hamburger elements where lowfat=yes). In another example, assume you wish to transform all hamburger elements and their associated data into a simple HTML table for the user to interact with. Using the standard DOM API, you must manually traverse the tree and create the output string containing the HTML table.

In order to simplify and standardize how one accomplishes these tasks, the W3C introduced a specification for XML transformations called the Extensible Stylesheet Language (XSL) and a simple query language referred to as XSL Patterns.

XSL Patterns

A pattern is a string, which selects a set of nodes in an XML tree. The selection is relative to the current node that the pattern is applied to. The simplest pattern is an element name; it selects all the child elements of the current node with that element name. For example, the pattern hamburger selects all the hamburger child elements of the current node.

The pattern syntax is complete. It allows you to identify the context of where a given element lives within a document (for example, price elements that live within hamburger elements). It also provides powerful filtering syntax to help identify specific nodes that match a given criteria (for example, where lowfat=yes). To find all of the lowfat hamburger price elements within a hamburgers element, you would use the following pattern string:

/hamburgers/hamburger[@lowfat="yes"]/price

When a pattern is applied to a given node, it simply returns a list of nodes that match the given pattern. This simplifies your life as a developer because you no longer need to manually traverse the tree to find what you're looking for.

MSXML 2.0 supports the pattern syntax as defined in described in Section 2.6 of the Extensible Stylesheet Language (December 18th Working Draft) . The IXMLDOMNode interface implemented in MSXML 2.0 offers two methods, SelectNodes and SelectSingleNode, both of which accept a pattern string as a parameter. For example, the following line of code would retrieve all of the price nodes that match the given criteria:

Set nodeList = rootNode.selectNodes("hamburger[@lowfat="yes"]/price")

XSL

XSL Patterns help identify certain nodes within a given XML document, but it's still up to the developer to do something interesting with those matching nodes. XSL helps simplify the process of performing one of the most common XML tasks: transforming nodes from an XML format into another format. The need for this originated on the Web as developers wanted to take their XML data and transform it into HTML for the user to view.

But XSL goes far beyond that. XSL is very useful for defining transformations from a given XML format to another distinct XML format. This makes interoperability much more feasible. If someone sends you an XML document that doesn't use the exact XML vocabulary that your system expects, simply perform an XSL transformation to produce the desired vocabulary. With the simplicity of XML, developers no longer have to agree on a universal vocabulary for describing a certain type of data.

An XSL file contains a list of declarative templates that define the transformation rules. Each template defines exactly how you wish to transform a given node from the source document to a node (or some other type of data) in the output document. You use XSL Patterns within a template to define which portions of the document the template applies to.

For example, to transform the following hamburger XML file:

<?xml version="1.0"?>
<hamburgers>
  <hamburger lowfat="dream on">
    <name>CowBurger</name>
    <description>Greasy and good.</description>
    <price>2.99</price>
  </hamburger>
</hamburgers>

into the following HTML file:

<html>
<body>
<h1>hamburgers</h1>
<ol>
  <li>CowBurger, $2.99, Greasy and good.</li>
</ol>
</body>
</html>

you would use the following XSL file:

<?xml version=“1.0”?>
<xsl:stylesheet xmlns:xsl=“ http://www.w3.org/TR/WD-xsl “>
  <xsl:template match=“/”>
    <html>
    <body>
    <h1>hamburgers</h1>
    <xsl:for-each select=“hamburgers[@lowfat=“dream on”]>
      <li><xsl:value-of select=“name”/>, <xsl:value-of select=“price”/>, 
      <xsl:value-of select=“description”/></li>
    </xsl:for-each>
    </body>
    </html>
  </xsl:template>
</xsl:stylesheet>

Notice how XSL Patterns are utilized within the match and select attributes of the different XSL elements to identify a set of elements. The text within the <xsl:template> tags defines the transformation rule for that set of nodes. It's interesting that XSL uses a standard XML vocabulary to define the transformation process.

Linking Technologies

Many would argue that the true power of the Internet is found in the HTML anchor element:

<A HREF="http://www.someserver.com">some link</A>

The anchor element allows developers to establish links from one HTML page to another, defining a relationship between the documents. This provides a way for the user to retrieve additional data on the same subject as the current page. This is typically how users find the data they are looking for. They go to a page that seems to match what they're after but then they see a link to another document that's an even better match.

The Web is based on this fundamental principal of establishing relationships (through links) between distinct data files. As the industry moves towards XML to describe data, it seems only natural that we'll need a similar mechanism to describe relationships between distinct XML documents and maybe even between distinct elements within a single XML document.

XLink

XML Linking 1.0 (XLink) is a new W3C initiative that outlines the syntax for defining XML links. According to the XLink 1.0 requirements document, an XML link is the specification of an explicit relationship between resources or portions of resources, as well as XLink-defined descriptive information. The specific identification methods that locate various types of data (such as URIs, XPointers, and graphical coordinates) are outside the scope of XLink.

Here is an example of a simple XML link:

<hamburger xml:link="simple" href="http://fastfood.org/hamburger.htm">
</hamburger>

XPointer

As you read in the previous paragraph, XLink depends on different mechanisms for identifying resources (such as a Uniform Resource Identifier) that you wish to link to. Another W3C initiative called XPointer specifies constructs for addressing the internal structures of XML documents. In particular, it provides for specific reference to elements, character strings, and other parts of XML documents, whether or not they bear an explicit ID attribute.

An XPointer consists of a series of location terms, each of which specifies a location, usually relative to the location specified by the prior location term. Each location term has a keyword (such as id, child, ancestor, and so on) and can have arguments, such as an instance number, element type, or attribute. For example, the XPointer:

child(2,hamburger) 

refers to the second child element whose type is hamburger.

Other XML-Related Technologies and Vocabularies

The technologies presented up to this point really represent the core XML technologies. While you may feel that you've got enough to start dealing with, this paper would not be complete without at least briefly introducing some related XML technologies and vocabularies that are gaining popularity today. Many of these technologies are currently evolving with various W3C working groups.

Mathematical Markup Language (MathML)

MathML is an XML application for describing mathematical notation and capturing both its structure and content. The goal of MathML is to enable mathematics to be served, received, and processed on the Web, just as HTML has enabled this functionality for text. Take a look at the following MathML example, taken directly from the W3C specification. The following mathematical equation:

x2 + 4x + 4 =0

is represented in MathML using the following XML vocabulary:

<apply>
   <plus/>
   <apply>
      <power/>
      <ci>x</ci>
      <cn>2</cn>
   </apply>
   <apply>
      <times/>
      <cn>4</cn>
      <ci>x</ci>
   </apply>
   <cn>4</cn>
</apply>

SMIL

Synchronized Multimedia Integration Language (SMIL, pronounced "smile") is an XML-based language for describing multimedia presentations. SMIL allows a set of independent multimedia objects to be integrated into a synchronized multimedia presentation. HTML+TIME is another industry initiative that leverages SMIL functionality to add multimedia "timing" behaviors (such as timeline descriptions) to your HTML pages. Internet Explorer 5 currently provides a sample HTML+TIME implementation. The following is an example of a Web page that contains a sequential timeline. Each P element within the timeline DIV will appear after the previous P element disappears.

<HTML>
<HEAD>
<STYLE>
    .time { behavior:url(#default#time); }
</STYLE>
</HEAD>
<BODY>
<DIV CLASS=“time” t:timeline=“seq”> 
   <P class=“time” t:dur=“1”>
   This appears for one second and goes away
   </P>
   <P class=“time” t:dur=“1”>
   This appears after one second, remains visible for one second 
   and goes away
   </P>
   <P class=“time” t:dur=“1”>
   This appears after two seconds, remains visible for one second 
   and goes away
   </P>
</DIV>
</BODY>
</HTML>

Vector Markup Language (VML)

The Vector Markup Language (VML) is an application of XML that defines a format for the encoding of vector information, together with additional markup to describe how that information may be rendered on the screen. VML supports the markup of vector graphic information in the same way that HTML supports the markup of textual information. Some Microsoft products (such as Microsoft Powerpoint® 2000) support exporting files to HTML using VML to describe graphic information. The following is a simple VML example that defines a shape:

<v:shape style=‘top: 0; left: 0; width: 250; height: 250’
       stroke=“true” strokecolor=“red” strokeweight=“2” fill=“true”
       fillcolor=“green” coordorigin=“0 0” coordsize=“175 175”>
<v:path v=“m 8,65     
  l 72,65,92,11,112,65,174,65,122,100,142,155,92,121,42,155,60,100
  x e”/>
</v:shape>

Channel Definition Format (CDF)

XML Fragments

The XML specification supports logical documents composed of possibly several entities. It's very common to view or edit one or more of the entities or even part of an entity while having no interest in viewing or editing the entire document. We need a way, however, to provide the client with enough contextual information about the larger document that the document fragment lives within without the client having to interact with the entire document. XML Fragments define a standardized mechanism for accomplishing this goal.

XHTML

XHTML is a family of new XML-conforming HTML documents. XHTML documents are designed to work in conjunction with XML processors. Most HTML files today that you view in a Web browser are not well-formed XML documents. For example, it's very common to have a begin <LI> tag without a subsequent end </LI> tag. Hence, it becomes very difficult, if not impossible, to interact with HTML using standard XML tools.

XHTML documents are well-formed XML, so they are readily viewed, edited, and validated with standard XML processors. This also makes it much easier for lightweight clients (like Palm-size PCs) to deal with error conditions.

As stated in the XHTML specification, the XHTML documents conforming to XHTML 1.0 will be more likely to interoperate within and among various XHTML environments. The XHTML family is the next step in the evolution of the Internet. By migrating to XHTML today, content developers can enter the XML world, with all of its attendant benefits, while still remaining confident of their content's backward and future compatibility.

Summary

There are many other XML-related technologies besides the ones listed above, but we'll leave those for you to peruse on your own. As you can see, there is currently a huge, industry-wide effort behind the evolution of XML and its family of technologies.

XML is destined to change the way the world writes interoperable software going into the future. For a more thorough discussion of how XML will impact software component technology, see Lessons From the Component Wars: An XML Manifesto.

This vast amount of XML information is often difficult to assimilate because it's hard to see the bigger picture of how everything fits together. This paper introduced you to XML and the family of supporting XML technologies. By now you should understand how each of the core technologies fits into the big picture and what some of the supporting XML technologies are all about. Now you're ready to start digging into the XML technology that interests you the most. Enjoy!

References

W3C Specifications and Recommendations

• Extensible Markup Language (XML) 1.0 (W3C Recommendation)
• Namespaces in XML (W3C Recommendation)
• XML-Data (W3C Note)
• XML-Schema Part 1: Structures (W3C Working Draft)
• XML-Schema Part 2: Datatypes (W3C Working Draft)
• Document Object Model (DOM) Level 1 Specification
• Resource Description Framework (RDF) Model and Syntax Specification
• XML Pointer Language (XPointer)
• XML Fragment Interchange Requirements Version 1.0
• Mathematical Markup Language (MathML^TM) 1.01 Specification
• XML XLink Requirements Version 1.0
• SMIL^TM: Synchronized Multimedia Integration Language
• Vector Markup Language (VML)
• XHTML^TM 1.0: The Extensible HyperText Markup Language
• Channel Definition Format (CDF)

Books

XML In Action, by William J. Pardi, available at Microsoft Press . ISBN:0-7356-0562-9