DMLex transformation backend demo

Pretty-print the output JSON
Return XML instead of JSON
Transform!

Results:

You can Download the above examples here.

Syntax of the JSON mapping descriptions

The mapping description as a whole should be an element mapping description (on which see below) for the root of the output DMLex tree (which probably means for the lexicographicResource element).

Element mapping descriptions

An element mapping description (EMD) is a JSON object that describes how certain elements in the input XML document are to be mapped to certain elements of the output DMLex tree. This JSON object should contain the following members:

• inSelector — a string containing an XPath expression which selects the XML elements to which this mapping is to be applied. If the current EMD is the child of another EMD, then the XPath expression is relative to the XML element to which this parent EMD is being applied; otherwise it is relative to the root of the input XML document.
• namespaces — optional; if present, this must be a dictionary that maps any XML namespace prefixes used in inSelector into their corresponding URIs.
• outElement — a string containing the name of the resulting element in the output DMLex tree.
• jsonPlural — the plural of the name given in outElement; if omitted, outElement + "s" will be used. The plural is used in the JSON serialization of a DMLex tree; for example, entry elements are stored in the entries member of a lexicographicResource element.
• children — optional; if present, this must be a list of 0 or more EMDs which are used to generate the children of the current element in the output DMLex tree.
• textVals — optional; if present, this must be a list of 0 or more TMDs (on which see below) which are used to generate the attributes of the current element in the output DMLex tree.
• copyToOutElt — optional; if present, this must be a JSON object whose contents are to be copied without change into (every instance of) the current element in the output DMLex tree. This is intended to be used to provide lists of tags for the lexicographicResource element of the output DMLex tree, but the mechanism is general and works in any EMD.

Text mapping descriptions

A text mapping description (TMD) is a JSON object that describes how certain content from the input XML element is to be used to generate the value of a text attribute of an element in the output DMLex tree. A TMD is always defined in the context of an EMD (within its textVals member), which may therefore be regarded as the parent of that TMD. The JSON object representing a TMD should contain the following members:

• inSelector — optional; a string containing an XPath expression which selects the XML elements to which this mapping is to be applied. If omitted, the XML element to which the parent EMD is being applied will be used instead.
• namespaces — optional; if present, this must be a dictionary that maps any XML namespace prefixes used in inSelector into their corresponding URIs.
• attribute — the name of the attribute (of the XML element selected by inSelector) whose value is to be extracted.
  • To extract the inner text of the element, instead of an attribute, use the pseudo-attribute name "{http://elex.is/wp1/teiLex0Mapper/meta}innerText".
  • To extract the inner text of the element and all of its descendants, use "{http://elex.is/wp1/teiLex0Mapper/meta}innerTextRec".
  • To return a constant value instead of extracting the value of an attribute, use the pseudo-attribute name "{http://elex.is/wp1/teiLex0Mapper/meta}constant".
  • To have the system automatically generate a unique value, use the pseudo-attribute name "{http://elex.is/wp1/teiLex0Mapper/meta}autogenerated".
• regex: a regular expression that is applied to the value of the attribute attribute. If this string does not contain any substring that matches this regular expression, the transformation is aborted (i.e. it is as if it hadn't been selected by the selector at all). If there are several matches, the first one is used. Only the substring that matched the regular expression (rather than the entire value of the attribute attribute) is then used to generate the text attribute for the output DMLex tree. — The regex member is optional. If present, it must use the Python regular expression syntax.
• regexGroup: this member is optional. If present, it must be the name of one of the named groups (?P<name>...) from the regular expression given by the attribute regex. In this case, only the string that matched this named group will be used, rather than the entire substring that matched the regular expression.
• constant: this attribute should be present it attribute was set to "{http://elex.is/wp1/teiLex0Mapper/meta}constant", and should provide the constant value that you want to return as the result of the transformation.
• xlat: this member is optional. If present, it should be a dictionary that will be used to transform the string obtained from the previous steps (attribute lookup, regular expression matching, named group selection). In other words, the string s will be replaced by xlat[s] if s appears as a key in xlat (otherwise, s will remain unchanged, just as if xlat had not been provided at all).
• outElement — a string containing the name under which the the resulting text value should be stored in the output DMLex tree.
• markers — optional; if present, this must be a list of 0 or more MMDs (on which see below) which are used to generate markers associated with the current text value in the output DMLex tree.

Marker mapping description

A marker mapping description (MMD) is a JSON object that specifies how certain subranges of a text value (as obtained by applying a TMD) are to be identified, and stored as a list of {"startIndex": ..., "endIndex": ...} pairs in the output DMLex tree. A MMD is always defined in the context of a TMD (within its markers member), which may therefore be regarded as the parent of that MMD. The JSON object representing a MMD should contain the following members:

• inSelector — optional; a string containing an XPath expression which selects the XML elements to which this mapping is to be applied. If omitted, the XML element to which the parent TMD is being applied will be used instead. Note that specifying an inSelector makes sense only if the parent TMD specified that the text value must be obtained by using innerTextRec instead of by extracting an attribute or using a constant or autogenerated value. In this case, each descendant of the input XML element on which the parent TMD is being applied corresponds to a subrange of the text value that has been extracted from that input XML element, so the inSelector can be used to select one or more of these subranges.
• namespaces — optional; if present, this must be a dictionary that maps any XML namespace prefixes used in inSelector into their corresponding URIs.
• regex — optional; if present, this must be a regular expression which will be applied to each subrange selected by inSelector. Thus each of those subranges can actually give rise to 0 or more smaller subranges (one for each occurrence of the regular expression), and the markers will actually refer to these smaller subranges.
• regexGroup — optional; if present, this must be the name of one of the named groups (?P<name>...) from the regular expression given by the attribute regex. In this case, only the string that matched this named group will be used, rather than the entire contents of the match.
• outElement — a string containing the name under which the resulting list of markers is to be stored in the output DMLex tree. It becomes the sibling of the text value generated by the parent TMD.
• jsonPlural — the plural of the name given in outElement; if omitted, outElement + "s" will be used. The plural is used in the JSON serialization of a DMLex tree; for example, headwordMarker elements are stored in the headwordMarkers member of an exampleTranslation element.

An annotated example

This is an example mapping description interlaced with comments that explain what is going on. The root of our input XML document will be a <Dictionary> element, which we will map into the lexicographicResource element of the output DMLex tree:

{
    "inSelector": "/Dictionary",
    "outElement": "lexicographicResource",

This lexicographicResource element will have a text value called language, whose contents will be extracted from the @sourceLanguage attribute of the <Dictionary> element of the input XML document:

    "textVals": [
        {
            "outElement": "language",
            "attribute": "sourceLanguage"
        }
    ],

The <DictionaryEntry> descendants of the <Dictionary> element will be transformed into entry elements of the output DMLex tree, and will be stored in a list called entries of the lexicographicResource element:

    "children": [
        {
            "inSelector": "DictionaryEntry",
            "outElement": "entry", "jsonPlural": "entries",

Each entry element of the output DMLex tree has an autogenerated id attribute, as well as a headword attribute which is extracted from the inner tex of the <headword> descendant of the <DictionaryEntry> element:

            "textVals": [
                {
                    "outElement": "id",
                    "attribute": "{http://elex.is/wp1/teiLex0Mapper/meta}autogenerated"
                },
                {
                    "outElement": "headword",
                    "inSelector": ".//headword",
                    "attribute": "{http://elex.is/wp1/teiLex0Mapper/meta}innerText"
                },

The entry element also has a text value partOfSpeech, which is obtained by taking the @value attribute of the <PartOfSpeech> descendant of the <DictionaryEntry> element and then passing it through the mapping described by xlat, such that e.g. the string "adjective" becomes "adj", etc.

                {
                    "outElement": "partOfSpeech",
                    "inSelector": ".//PartOfSpeech",
                    "attribute": "value",
                    "xlat": {
                      "adjective": "adj",
                      "adverb": "adv",
                      "adverbpostposition": "x",
                      "cardinal": "num",
                      "conjunction": "cconj",
                      "interjection": "intj",
                      "noun": "noun",
                      "ordinal": "num",
                      "particle": "part",
                      "postposition": "x",
                      "postpositionpreposition": "adp",
                      "preposition": "adp",
                      "pronoun": "pron",
                      "verb": "verb"
                    }
                }
            ],

This is the end of the textVals list for the entry element. An entry element can also have sense elements as children. We see an example of a slightly more advanced XPath expression in inSelector here: if the current <DictionaryEntry> element has any <SenseGrp> descendants, then each of these descendants is transformed into a sense element in the output DMLex tree; otherwise, the <DictionaryEntry> element itself is mapped into a single sense element (which becomes the sole child of the parent entry element; in this case the same <DictionaryEntry> of the input XML document will give rise to both an entry element and a sense element in the output DMLex tree).

            "children": [
                {
                    "outElement": "sense",
                    "inSelector": "(.//SenseGrp)|(self::*[not(descendant::SenseGrp)])",

Now we are in the element mapping specification for sense. We will give each sense an autogenerated id text-value:

                    "textVals": [
                        {
                            "outElement": "id",
                            "attribute": "{http://elex.is/wp1/teiLex0Mapper/meta}autogenerated"
                        }
                    ],

Each <Definition> element that is a descendant of the current <SenseGrp> (or <DictionaryEntry>) will be mapped into a definition child of the current sense element in the output DMLex tree. A definition element has a text attribute called value, obtained as the inner-text of the input <Definition> element and its descendants.

                    "children": [
                        {
                            "outElement": "definition",
                            "inSelector": ".//Definition",
                            "textVals": [
                                {
                                    "outElement": "value",
                                    "attribute": "{http://elex.is/wp1/teiLex0Mapper/meta}innerTextRec"
                                }
                            ]
                        },

A sense element of the output DMLex tree will also have example elements as children, obtained from the <Example> descendant elements in the input XML document. Each example element has a text value called text, obtained as the inner-text of the <Fragment> descendant of the <Example> element.

                        {
                            "outElement":  "example",
                            "inSelector": ".//Example",
                            "textVals": [
                                {
                                    "outElement": "text",
                                    "inSelector": ".//Fragment",
                                    "attribute": "{http://elex.is/wp1/teiLex0Mapper/meta}innerText",

In addition to text, the example element will have a list of markers called vowelPairs, indicating all subranges within text where two adjacent letters are equal vowels. We use a regular expression to identify them; (?i) specifies case-insentive matching. (Note that vowelPairs is not actually a part of DMLex, and is only provided here for illustrative purposes.)

                                    "markers": [
                                        {
                                            "outElement": "vowelPair",
                                            "regex": "(?i)([aeiou])\\1"
                                        }
                                    ]
                                },

Lastly, the example element will have a text-value called xr, obtained from the @href attribute of the <Ptr> child of the <SeeAlso> descendant of <Example. However, we do not use the whole value of the attribute, but only the first group of one or more consecutive lowercase letters. (Note that xr is not actually a part of DMLex, and is only provided here for illustrative purposes.)

                                {
                                    "outElement": "xr",
                                    "inSelector": ".//SeeAlso/Ptr",
                                    "attribute": "{http://www.w3.org/TR/xlink}href",
                                    "regex": "[a-z]+"
                                }
                            ]
                        }
                    ]
                }
            ]
        }
    ],

At the end of the EMD for the lexicographicResource element of the output DMLex tree, we have two arrays, definitionTypeTags and inflectedFormTags, which will be copied without change into the lexicographicResource element of the output DMLex tree.

    "copyToOutElt": {
        "definitionTypeTags": [
            { "tag": "foo", "description": "..." },
            { "tag": "bar", "description": "..." } ],
        "inflectedFormTags": [
            { "tag": "baz", "description": "..." },
            { "tag": "quux", "description": "..." } ]
    }
}

This concludes the example. You can select “KKS (Karelian)” at the top of this page to see it in action.