YAML

From Wikipedia, the free encyclopedia - View original article

 
Jump to: navigation, search

YAML (play /ˈjæməl/, rhymes with camel) is a human-readable data serialization format that takes concepts from programming languages such as C, Perl, and Python, and ideas from XML and the data format of electronic mail (RFC 2822). YAML was first proposed by Clark Evans in 2001,[1] who designed it together with Ingy döt Net[2] and Oren Ben-Kiki.[2] It is available for several programming languages.

YAML is a recursive acronym for "YAML Ain't Markup Language". Early in its development, YAML was said to mean "Yet Another Markup Language",[3] but it was then reinterpreted (backronyming the original acronym) to distinguish its purpose as data-oriented, rather than document markup.

Contents

Features

YAML syntax was designed to be easily mapped to data types common to most high-level languages: list, associative array, and scalar.[4] Its familiar indented outline and lean appearance make it especially suited for tasks where humans are likely to view or edit data structures, such as configuration files, dumping during debugging, and document headers (e.g. the headers found on most e-mails are very close to YAML). Although well-suited for hierarchical data representation, it also has a compact syntax for relational data as well.[5] Its line and whitespace delimiters make it friendly to ad hoc grep/Python/Perl/Ruby operations. A major part of its accessibility comes from eschewing the use of enclosures like quotation marks, brackets, braces, and open/close-tags which can be hard for the human eye to balance in nested hierarchies.

Examples

Sample document

Data structure hierarchy is maintained by outline indentation.

 --- receipt:     Oz-Ware Purchase Invoice date:        2007-08-06 customer:     given:   Dorothy     family:  Gale  items:     - part_no:   A4786       descrip:   Water Bucket (Filled)       price:     1.47       quantity:  4      - part_no:   E1628       descrip:   High Heeled "Ruby" Slippers       size:      8       price:     100.27       quantity:  1  bill-to:  &id001     street: |             123 Tornado Alley             Suite 16     city:   East Centerville     state:  KS  ship-to:  *id001  specialDelivery:  >     Follow the Yellow Brick     Road to the Emerald City.     Pay no attention to the     man behind the curtain. ... 

Notice that strings do not require enclosure in quotations. The specific number of spaces in the indentation is unimportant as long as parallel elements have the same left justification and the hierarchically nested elements are indented further. This sample document defines an associative array with 7 top level keys: one of the keys, "items", contains a 2 element array (or "list"), each element of which is itself an associative array with differing keys. Relational data and redundancy removal are displayed: the "ship-to" associative array content is copied from the "bill-to" associative array's content as indicated by the anchor ( & ) and reference ( * ) labels. Optional blank lines can be added for readability. Multiple documents can exist in a single file/stream and are separated by "---". An optional "..." can be used at the end of a file (useful for signaling an end in streamed communications without closing the pipe).

Language elements

Basic components of YAML

YAML offers both an indented and an "in-line" style for denoting associative arrays and lists. Here is a sampler of the components.

Lists

Conventional block format uses a hyphen+space to begin a new item in list.

  --- # Favorite movies  - Casablanca  - North by Northwest  - The Man Who Wasn't There 

Optional inline format is delimited by comma+space and enclosed in brackets (similar to JSON).[6]

  --- # Shopping list  [milk, pumpkin pie, eggs, juice] 

Associative arrays

Keys are separated from values by a colon+space.

  --- # Indented Blocks, common in YAML data files, use indentation and new lines to separate the '''key: value''' pairs    name: John Smith    age: 33  --- # Inline Blocks, common in YAML data streams, use comma+space to separate the '''key: value''' pairs between braces  {name: John Smith, age: 33} 

Block literals

Strings do not require quotation.

Newlines preserved
  --- |    There once was a man from Darjeeling    Who got on a bus bound for Ealing        It said on the door        "Please don't spit on the floor"    So he carefully spat on the ceiling 

By default, the leading indent (of the first line) and trailing white space is stripped, though other behavior can be explicitly specified.

Newlines folded
  --- >    Wrapped text    will be folded    into a single    paragraph      Blank lines denote    paragraph breaks 

Folded text converts newlines to spaces and removes leading whitespace.

Hierarchical combinations of elements

Lists of associative arrays
 - {name: John Smith, age: 33} - name: Mary Smith   age: 27 
Associative arrays of lists
 men: [John Smith, Bill Jones] women:   - Mary Smith   - Susan Williams 

Advanced components of YAML

Two features that distinguish YAML from the capabilities of other data serialization languages are Relational trees and Data Typing.

Relational trees

References

For clarity, compactness, and avoiding data entry errors, YAML provides node anchors ( & ) and references ( * ). References branch the tree to the anchor and work for all data types (see the ship-to reference in the example above).

Below is an example of a queue in an instrument sequencer in which two steps are reused repeatedly without being fully described each time.

 # sequencer protocols for Laser eye surgery --- - step:  &id001                  # defines anchor label &id001     instrument:      Lasik 2000     pulseEnergy:     5.4     pulseDuration:   12     repetition:      1000     spotSize:        1mm  - step: &id002     instrument:      Lasik 2000     pulseEnergy:     5.0     pulseDuration:   10     repetition:      500     spotSize:        2mm  - step: *id001                   # refers to the first step (with anchor &id001) - step: *id002                   # refers to the second step - step: *id001 - step: *id002 

Data types

Explicit data typing is seldom seen in the majority of YAML documents since YAML autodetects simple types. Data types can be divided into three categories: core, defined, and user-defined. Core are ones expected to exist in any parser (e.g. floats, ints, strings, lists, maps, ...). Many more advanced data types, such as binary data, are defined in the YAML specification but not supported in all implementations. Finally YAML defines a way to extend the data type definitions locally to accommodate user defined classes, structures or primitives (e.g. quad precision floats).

Casting data types

YAML autodetects the datatype of the entity. Sometimes one wants to cast the datatype explicitly. The most common situation is a single word string that looks like a number, boolean or tag may need disambiguation by surrounding it with quotes or use of an explicit datatype tag.

 --- a: 123                     # an integer b: "123"                   # a string, disambiguated by quotes c: 123.0                   # a float d: !!float 123             # also a float via explicit data type prefixed by (''' !! ''') e: !!str 123               # a string, disambiguated by explicit type f: !!str Yes               # a string via explicit type g: Yes                     # a boolean True h: Yes we have No bananas  # a string, "Yes" and "No" disambiguated by context. 
Other specified data types

Not every implementation of YAML has every specification-defined data type. These built-in types use a double exclamation sigil prefix ( !! ). Particularly interesting ones not shown here are sets, ordered maps, timestamps, and hexadecimal. Here's an example of base64 encoded binary data.

  ---  picture: !!binary |   R0lGODlhDAAMAIQAAP//9/X   17unp5WZmZgAAAOfn515eXv   Pz7Y6OjuDg4J+fn5OTk6enp   56enmleECcgggoBADs=mZmE 
Extension for user-defined data types

Many implementations of YAML can support user defined data types. This is a good way to serialize an object. Local data types are not universal data types but are defined in the application using the YAML parser library. Local data types use a single exclamation mark ( ! ).

 --- myObject:  !myClass { name: Joe, age: 15 } 

Syntax

A compact cheat sheet (actually written in YAML) as well as a full specification are available at the YAML web site. The following is a synopsis of the basic elements.

YAML requires that colons and commas used as list separators be followed by a space so that scalar values containing embedded punctuation (such as 5,280 or http://www.wikipedia.org) can generally be represented without needing to be enclosed in quotes.

Two additional sigil characters are reserved in YAML for possible future standardisation: the at sign ( @ ) and accent grave ( ` ).

Comparison to other data structure format languages

While YAML shares similarities with JSON, XML and SDL, it also has characteristics that are unique in comparison to many other similar format languages.

JSON

JSON syntax is a subset of YAML version 1.2, which was promulgated with the express purpose of bringing YAML "into compliance with JSON as an official subset."[7] Though prior versions of YAML were not strictly compatible,[8] the discrepancies were rarely noticeable and most JSON documents can be parsed by some YAML parsers such as Syck.[9] This is because JSON's semantic structure is equivalent to the optional "inline-style" of writing YAML. While extended hierarchies can be written in inline-style like JSON, this is not a recommended YAML style except when it aids clarity.

YAML has many additional features lacking in JSON, including extensible data types, relational anchors, strings without quotation marks, and mapping types preserving key order.

XML and SDL

YAML lacks the notion of tag attributes that are found in XML and SDL (Simple Declarative Language). For data structure serialization, tag attributes are, arguably, a feature of questionable utility since the separation of data and meta-data adds complexity when represented by the natural data structures (associative arrays, lists) in common languages.[10] Instead YAML has extensible type declarations (including class types for objects).

YAML itself does not have XML's language-defined document schema descriptors that allow, for example, a document to self validate. However, there are several externally defined schema descriptor languages for YAML (e.g. Doctrine, Kwalify and Rx) that fulfill that role. Moreover, the semantics provided by YAML's language-defined type-declarations in the YAML document itself frequently relaxes the need for a validator in simple, common situations. Additionally, YAXML, which represents YAML data structures in XML, allows XML schema importers and output mechanisms like XSLT to be applied to YAML.

Indented delimiting

Because YAML primarily relies on outline indentation for structure, it is especially resistant to delimiter collision. YAML's insensitivity to quotes and braces in scalar values means one may embed XML, SDL, JSON or even YAML documents inside a YAML document by simply indenting it in a block literal:

 --- example: >         HTML goes into YAML without modification message:  |         <font name='times' size=10>         <p><i>"Three is always greater than                two, even for large values of two"</i>         </p><p>    --Author Unknown    </p></font> date: 2007-06-01 

YAML may be placed in JSON and SDL by quoting and escaping all interior quotes. YAML may be placed in XML by escaping reserved characters ( <, >, &, ', " ), and converting whitespace; or by placing it in a CDATA-section.

Non-hierarchical data models

Unlike SDL, and JSON, which can only represent data in a hierarchical model with each child node having a single parent, YAML also offers a simple relational scheme that allows repeats of identical data to be referenced from two or more points in the tree rather than entered redundantly at those points. This is similar to the facility IDREF built into XML.[11] The YAML parser then expands these references into the fully populated data structures they imply when read in, so whatever program is using the parser does not have to be aware of a relational encoding model, unlike XML processors which do not expand references. This expansion can enhance readability while reducing data entry errors in configuration files or processing protocols where many parameters remain the same in a sequential series of records while only a few vary. An example being that "ship-to" and "bill-to" records in an invoice are nearly always the same data.

Practical considerations

YAML is line-oriented and thus it is often simple to convert the unstructured output of existing programs into YAML format while having them retain much of the look of the original document. Because there are no close-tags, braces, or quotation marks to balance, it is generally easy to generate well-formed YAML directly from distributed print statements within unsophisticated programs. Likewise, the white space delimiters facilitate quick-and-dirty filtering of YAML files using the line oriented commands in grep, awk, perl, ruby, and python.

In particular, unlike mark-up languages, chunks of consecutive YAML lines tend to be well-formed YAML documents themselves. This makes it very easy to write parsers that do not have to process a document in its entirety (e.g. balancing open- and close-tags and navigating quoted and escaped characters) before they begin extracting specific records within. This property is particularly expedient when iterating in a single, stateless pass, over records in a file whose entire data structure is too large to hold in memory, or for which reconstituting the entire structure to extract one item would be prohibitively expensive.

Counterintuitively, although its indented delimiting might seem to complicate deeply nested hierarchies, YAML handles indents as small as a single space, and this may achieve better compression than markup languages. Additionally, extremely deep indentation can be avoided entirely by either: 1) reverting to "inline-style" (i.e. JSON-like format) without the indentation; or 2) using relational anchors to unwind the hierarchy to a flat form that the YAML parser will transparently reconstitute into the full data structure.[citation needed]

Security

YAML is purely a data representation language and thus has no executable commands.[12] This means that parsers will be (or at least should be) safe to apply to tainted data without fear of a latent command-injection security hole. For example, because JSON is native JavaScript it is tempting to use the JavaScript interpreter itself to evaluate the data structure into existence, leading to command injection holes when inadequately verified. While validation and safe parsing is inherently possible in any data language, implementation is such a notorious pitfall that YAML's lack of an associated command language may be a relative security benefit.[citation needed]

However, YAML allows language specific tags so that arbitrary local objects can be created by a parser that supports those tags. Any YAML parser that allows sophisticated object instantiation to be executed opens the potential for an injection attack. Perl parsers that allow loading of objects of arbitrary class create so-called "blessed" values. Using these values may trigger unexpected behavior, e.g. if the class uses overloaded operators. This may lead to execution of arbitrary Perl code.[citation needed]

The situation is similar for Python parsers. According to the PyYAML documentation[citation needed]:

Note that the ability to construct an arbitrary Python object may be dangerous if you receive a YAML document from an untrusted source such as the Internet. The function yaml.safe_load limits this ability to simple Python objects like integers or lists.

Data processing and representation

The XML[13][14] and YAML specifications[15] provide very different logical models for data node representation, processing, and storage.

XML: The primary logical structures in an XML instance document are: 1) Element; and 2) Attribute.[16] For these primary logical structures, the base XML specification does not define constraints regarding such factors as duplication of elements or the order in which they are allowed to appear.[17] In defining conformance for XML processors, the XML specification generalizes them into two types: 1) validating ; and 2) non-validating.[18] The XML specification asserts no detailed definitions for: an API; processing model; or data representation model; although several are defined in separate specifications that a user or specification implementor may choose independently. These include the Document Object Model and XQuery.

A richer model for defining valid XML content is the W3C XML Schema standard.[19] This allows for full specification of valid XML content and is supported by a wide range of open source, free and commercial processors and libraries.

YAML: The primary logical structures in a YAML instance document[20] are: 1) Scalar; 2) Sequence; and 3) Mapping.[21] The YAML specification also indicates some basic constraints that apply to these primary logical structures. For example, according to the specification, mapping keys do not have an order. In every case where node order is significant, a sequence must be used.[22]

Moreover, in defining conformance for YAML processors, the YAML specification defines two primary operations: 1) Dump; and 2) Load. All YAML-compliant processors must provide at least one of these operations, and may optionally provide both.[23] Finally, the YAML specification defines an information model or "representation graph" which must be created during processing for both Dump and Load operations, although this representation need not be made available to the user through an API.[24]

Implementations

Pitfalls and implementation defects

Portability

Simple YAML files (e.g. key value pairs) are readily parsed with regular expressions without resort to a formal YAML parser. YAML emitters and parsers for many popular languages written in the pure native language itself exist, making it portable in a self-contained manner. Bindings to C-libraries also exist when speed is needed.

C libraries

C++ library

Bindings

Native implementations and C libraries bindings for YAML exist for the following languages:

Actionscript
as3yaml A direct port of jvyaml for Actionscript 3.
C++
C++ wrapper for LibYaml.
see also the native C++ library yaml-cpp.
Erlang
yamler based on LibYAML, load only.
Go
goyaml based on LibYAML but written entirely in Go.
Haskell
YamlReference implementation of YAML syntax and utilities.
yaml binding to LibYAML.
HsSyck interface to SYCK.
json2yaml conversion.
Java
jvyaml based on Syck API, and patterned off RbYAML.
JYaml pure small Java implementation.
SnakeYAML YAML 1.1 parser and emitter for Java 5.
JavaScript
JS-YAML native PyYAML port. The most advanced YAML 1.1+ parser for JS.
CommonJS js-yaml Covers a small subset of YAML.
javascript-yaml-parser.
jsyaml code on stackoverflow.
Lua
Lua-Syck.
yaml LibYAML binding.
.NET Framework
Yaml Library for .NET (C#).
YAML Parser in C# An almost feature complete YAML Parser written in C#.
YAML for .NET, Visual Studio and Powershell.
OCaml
OCaml-Syck.
Objective-C
YAML.framework based on recommended LibYAML.
syck/ext/cocoa.
Perl
YAML is a common interface to several YAML parsers.
YAML::Tiny implements a useful subset of YAML; small, pure Perl, and faster than the full implementation.
YAML::Syck Binding to SYCK C-library. Offers fast, highly featured YAML.
YAML::XS Binding to LibYaml. Better yaml 1.1 compatibility.
PHP
Spyc is a pure PHP implementation.
PHP-Syck (binding to SYCK library).
Symfony YAML Component was initially released as part of the Symfony framework.
PECL Yaml (binding to LibYAML library).
Python
PyYaml Highly featured. Pure Python or optionally uses LibYAML.
PySyck Binding to SYCK C-Library.
Ruby
(YAML included in standard library since 1.8. based on SYCK)
Ya2YAML with full UTF-8 support.
ZAML far faster than default library.
RbYAML A YAML parser in pure Ruby.
R
CRAN YAML (binding to LibYAML library).
Tcl
Available as of Tcl 8.4.
XML
YAXML (currently draft only).

See also

Other human-readable serialization formats include:

Notes and references

  1. ^ Evans, Clark (May 11, 2001). "YAML Draft 0.1". Yahoo! Tech groups: sml-dev. http://tech.groups.yahoo.com/group/sml-dev/message/4710. Retrieved 2008-08-02. 
  2. ^ a b "YAML Ain't Markup Language: About". http://www.yaml.org/about.html. Retrieved 2010-06-16. 
  3. ^ "Yet Another Markup Language (YAML) 1.0". http://yaml.org/spec/history/2001-08-01.html. Retrieved 2008-11-24. 
  4. ^ For purposes of this article, the terms (list and array), (associative array, hash, dictionary and mapping) and (string and scalar) are used interchangeably. Such usage is a simplification and may not be correct when specifically applied to some programming languages.
  5. ^ A hierarchical model only gives a fixed, monolithic view of the tree structure. For example, either actors under movies, or movies under actors. YAML allows both using a relational model.
  6. ^ http://redhanded.hobix.com/inspect/yamlIsJson.html
  7. ^ YAML Version 1.2
  8. ^ The incompatibilities were as follows: JSON allows extended character sets like UTF-32 and had incompatible unicode character escape syntax relative to YAML; YAML required a space after separators like comma, equals, and colon while JSON does not. Some non-standard implementations of JSON extend the grammar to include Javascript's /*...*/ comments. Handling such edge cases may require light pre-processing of the JSON before parsing as in-line YAML. See also [1].
  9. ^ Parsing JSON with SYCK. Note that e.g. Symfony's YAML parser does not support line breaks inside [] or {} structures, which is a major incompatibility with JSON.
  10. ^ In Markup Languages, attribute values in an open-tag must be handled separately from the data value enclosed by the tags. Typically, to hold this in a data structure means each node is an object with storage for the tag-name plus an associative array for any possible named attributes and their values, and then a separate scalar for holding any enclosed data. YAML treats these even-handedly: each node is simple type, usually a scalar, list, or associative array.
  11. ^ XML IDREF
  12. ^ A proposed "yield" tag will allow for simple arithmetic calculation.
  13. ^ "Extensible Markup Language (XML) 1.0 (Fourth Edition)". http://www.w3.org/TR/REC-xml/. Retrieved 2007-11-04. 
  14. ^ "Extensible Markup Language (XML) 1.1 (Second Edition)". http://www.w3.org/TR/xml11/. Retrieved 2007-11-04. 
  15. ^ "YAML Ain't Markup Language (YAML) Version 1.1". http://yaml.org/spec/current.html. Retrieved 2007-11-04. 
  16. ^ Extensible Markup Language (XML) 1.1 (Second Edition)
  17. ^ Note, however, that the XML specification does define an "Element Content Model" for XML instance documents that include validity constraints. Validity constraints are user-defined and not mandatory for a well-formed XML instance document. http://www.w3.org/TR/xml11/#sec-element-content. In the case of duplicate Element attribute declarations, the first declaration is binding and later declarations are ignored [2].
  18. ^ Extensible Markup Language (XML) 1.0 (Fourth Edition)
  19. ^ http://www.w3.org/XML/Schema>
  20. ^ The YAML specification identifies an instance document as a "Presentation" or "character stream". [3]
  21. ^ Additional, optional-use, logical structures are enumerated in the YAML types repository."Language-Independent Types for YAML Version 1.1". http://yaml.org/type/index.html. Retrieved 2007-11-04. The tagged types in the YAML types repository are optional and therefore not essential for conformant YAML processors. "The use of these tags is not mandatory."
  22. ^ YAML Ain't Markup Language (YAML) Version 1.1
  23. ^ "Dump" and "Load" operations consist of a few sub-operations, not of all of which need to be exposed to the user or through an API, (see http://yaml.org/spec/current.html#id2504671).
  24. ^ YAML Ain't Markup Language (YAML) Version 1.1
  25. ^ YAML creator Clark Evans inserted this recommendation.

External links