WORLD METEOROLOGICAL ORGANIZATION
COMMISSION FOR BASIC SYSTEMS
MEETING OF EXPERT TEAM ON DATA REPRESENTATION AND CODES
PRAGUE, CZECH REPUBLIC, 22-26 APRIL 2002
METEOROLOGICAL DATA AND XML
(Submitted by Charles Sanders, Australia)
Summary and Purpose of Document
XML (eXtensible Markup Language) is a standard for communicating structured data. Use of XML is rapidly expanding, but WMO has not as yet taken a leading role in developing XML standards for meteorological information. The document analyze the present use of XML in Meteorology and try to propose options for WMO actions with a view to recommend a WMO standard for the exchange of data and metadata in XML.
The Meeting will examine the document on the use of XML to exchange meteorological information. The Meeting will discuss possible requirements and benefits. It may derive the appropriate statements for standardization and make recommendations on the practical use of XML in the framework of WMO programmes.
1) Introduction to XML;
5) Example of XML from MEDS;
6) A possible translation from BUFR to CREX to Tree-table-XML;
7) Examples of possible XML file structures.
Meteorological Data and XML
(Mal Gorman, James Kelly, Chris Ryan and Charles Sanders)
XML (eXtensible Markup Language) is a standard for communicating structured data. Use of XML is rapidly expanding, but WMO has not as yet taken a leading role in developing XML standards for meteorological information. With the notable exception of the US Navy, the use of XML in meteorological operations has been fairly limited.
If WMO does not take action in the near future to recommend a standard or standards, there is a risk that several competing and incompatible “standards” will be developed. There are a great many options, ranging from doing nothing and leaving it up to individual meteorological services to make their own decisions, through to attempting to devise a WMO standard for the exchange of data and metadata in XML.
If WMO does take a lead in developing a Meteorological standard for XML, there are many options. The possibilities range from generic XML standards that treat most (or all) of the metadata as data, to much more specific standards that specify exactly what metadata is permitted. It is not always clear what is data and what is metadata, and the amount and range of possible metadata is very large. There are also issues of language (English only, multiple languages, a specialist language) and design to be resolved. Some data (eg station dictionaries) change slowly, while others are much more volatile, so different approaches may be needed.
If a meteorological XML standard is correctly defined, it should be possible to ensure compatibility with and produce automatic translators between standard WMO codes (such as CREX or BUFR data or GRIB metadata) and XML.
WMO data can be split into two broad types, namely slowly changing data such as station dictionaries, codes manuals, and bulletin catalogues, and more volatile data such as observations and forecasts. Different approaches may be needed for these different areas.
Traditionally weather data has been exchanged in telegraphic formats, designed for low speed, data communication lines. Information about the data (the “metadata”) has not been transmitted. It has been assumed that the recipient understands the abbreviated message format, and has access to any metadata needed to determine the meaning of the data.
XML is designed for the “internet” world where the bandwidth of the transmission line is less of an issue, and the aim is to make the message as self-contained as possible. XML contains “information about the data” (metadata) as well as the data itself. XML is flexible, extensible, well suited to weather data, and worthy of WMO attention as an exchange format.
XML has the potential to greatly simplify the exchange of data and metadata between the meteorological community and users of meteorological data, and also to a lesser extent within the meteorological community. Standard, freely available software, such as newer web browsers, can be used to display limited amounts of XML data clearly and simply. More complex software can be used to search XML data files for particular items of information, and to process these in various ways.
XML is really a meta-language. An XML schema, or a Document Type Definition, is used to define an XML “vocabulary” for use in a particular subject area, or for a particular purpose. This permits checking the XML for conformance to a pre-defined standard. A more detailed introduction to XML and XML processing is provided in Appendix 1.
Strategic Options for WMO
The various WMO committees involved will have to make a number of strategic decisions in the near future. The decisions involved include:
The broad, overall policy on whether to follow the crowd, or to be a lead player in the development of XML.
Whether to adopt an English-based XML vocabulary, or try to develop a more “international” standard.
Whether to develop a small set of general-purpose XML schemas, or a larger set of more specialized schemas.
Whether to create or adopt a very rigid XML standard, or to opt for a more generalized and flexible standard.
Any WMO defined standard for the use of XML in meteorology is unlikely to meet all needs. Some mechanism to allow additions or alterations to the standard for internal, bilateral, or multilateral basis will probably have to be devised.
The following discusses the options, mainly from the perspective of data representation and codes.