"Is it YAML?" is a Django application (with tags) for checking whether text is YAML - "a human friendly data serialization standard for all programming languages" - or not. Users type or paste text, then click the "Submit" button. If valid YAML has been entered, the input is shown in canonical form; otherwise, the application presents an error message.
YAML is a text format for exchanging structured data. It exists for cases where XML is too much overhead. I prefer YAML. I find it more robust than XML (where one missing angle bracket could corrupt a whole file), yet easier to type by hand.
I conjured up "Is it YAML?" while writing an experimental specification for YPath - a language for addressing parts of a YAML document, as XPath does for XML. I wanted to write YAML example files, and I needed to check if the example files were good and proper. "Is it YAML?" gave me a test bed where I could copy-and-paste my examples, and see if they really are YAML.
"Is it YAML?" depends on Django (≥ 5.2) and PyYAML (≥ 6.0). Installation is as simple as:
pip install isityaml
Once installed, add "isityaml" to your INSTALLED_APPS
list in settings.py, and include the app URLs under any
path prefix you like:
# project urls.py
from django.urls import include, path
urlpatterns = [
# ... existing routes ...
path("this_is_the_path_to_isityaml/", include("isityaml.urls")),
]
A YAML checker can be as simple as this Django template file:
{% extends "base.html" %}
{% load isityaml_tags %}
{% block main %}
{% isityaml_checker %}
{% endblock %}
Then you can visit the URL where you mounted the app, such as
/this_is_the_path_to_isityaml/. Paste or type YAML into the
text area and
click Submit. Valid input is shown in canonical form;
invalid input shows the parser error and the original text.
Alternately, you can add this to any page with CSRF support:
{% load isityaml_tags %}
{% isityaml_checker %}
By default, the form’s action is {% url "isityaml:index" %}, which resolves
to the "this_is_the_path_to_isityaml/" which you included with isityaml.urls.
After submission, the user lands on a checker page with the results. To post
somewhere else, you can pass an override:
{% isityaml_checker action_url="/tools/yaml-check/" %}
There are a lot of user interface strings, such as headings, labels and button
text, which live in isityaml.copy.DEFAULT_COPY. You can change them in your
settings:
# settings.py
ISITYAML_COPY = {
"textarea_default": "Xin chào thế giới!", # Hello world! in Vietnamese
# ...
}
Or you can provide them as arguments on the tag:
{% isityaml_checker textarea_default="Xin chào thế giới!" %}
You can even call the application using isityaml.yaml_check:
from isityaml.yaml_check import check_yaml, STATE_POST_YES, STATE_POST_NO
result = check_yaml("foo: bar")
if result.yamlstate == STATE_POST_YES:
print(result.yamlcanon) # canonical YAML
elif result.yamlstate == STATE_POST_NO:
print(result.yamlerror)
print(result.yamloriginal)
From the package root (with the package importable, e.g. after
pip install -e .):
PYTHONPATH=. python -m unittest discover -s tests -v
"Is It YAML?" runs on a public facing website. Any internet user can view it, benign or malevolent. So an obvious question is: could this application be hacked? The thought crossed my mind after encountering the presentation "Serialization formats aren't toys", which was delivered by Tom Eastman at PyCon Australia. The talk covered security issues in XML, JSON and YAML. The presentation can be viewed online at the conference's YouTube channel. Let's look at the attack vector mentioned in the video, and see if it could appear here.
"Is It YAML?" uses PyYAML as
the underlying parser. As Tom Eastman explained, if one is not careful with
untrusted input, PyYAML can take it and execute untrusted code. YAML can be used
to represent many things, including Python functions called with specific arguments.
YAML uses tags to indicate the types of data one may encounter in the document, and PyYAML
supports tags of the format !!python/object/apply:module.function
to represent the Python function module.function. The tag is
just the type of the data, and the value of such types is a sequence, representing
the arguments passed to the function to be executed. To use the examples provided at the
presentation, here's some YAML that can list the contents on a directory.
"contents_of_cwd": !!python/object/apply:subprocess.check_output ['ls']Here's some YAML that can then delete the contents of a directory.
"goodbye": !!python/object/apply:os.system ["rm *"]Fortunately, "Is It YAML?" is safe from this attack vector. It scans the input, but only so far as to make sure it is parsable as YAML: sequences that begin with opening square brackets always end with closing square brackets, strings that begin with double quotes always end with them, and so on. It doesn't come anywhere close to constructing the Python objects that are represented in the input YAML. So the app happily parses the last YAML fragment to create the following, and continues along its merry way:
---
!!map {
? !!str "goodbye"
: !!python/object/apply:os.system [
!!str "rm *",
],
}Some years after the presentation, I discovered about Billion laugh attacks. I like Wikipedia's phrasing:
The example attack consists of defining 10 entities, each defined as consisting of 10 of the previous entity, with the document consisting of a single instance of the largest entity, which expands to one billion copies of the first entity. Versions with larger amount of entries also exist.
The point of the billion laughs attack is that in some data representation languages, such as XML, this results in the machine parsing it running out of space; Billion laugh attacks? are a denial of service attack after all. (That's another reason I don't really like XML.) Can you do it with YAML? You could, if you are not careful when writing YAML parsers. Here's some actual YAML that caused a denial of service on Kubernetes:
apiVersion: v1
data:
a: &a ["web","web","web","web","web","web","web","web","web"]
b: &b [*a,*a,*a,*a,*a,*a,*a,*a,*a]
c: &c [*b,*b,*b,*b,*b,*b,*b,*b,*b]
d: &d [*c,*c,*c,*c,*c,*c,*c,*c,*c]
e: &e [*d,*d,*d,*d,*d,*d,*d,*d,*d]
f: &f [*e,*e,*e,*e,*e,*e,*e,*e,*e]
g: &g [*f,*f,*f,*f,*f,*f,*f,*f,*f]
h: &h [*g,*g,*g,*g,*g,*g,*g,*g,*g]
i: &i [*h,*h,*h,*h,*h,*h,*h,*h,*h]
kind: ConfigMap
metadata:
name: yaml-bomb
namespace: default
So is "Is it YAML?" safe for Billion laugh attacks? Go ahead and copy and paste the above YAML into the application. (You know you want to.)
The trick about "Is it YAML?" is that it is interested in the "grammar" of the text it receives,
including whether any aliases received (the *as) have already been declared as an
anchor (the &as). What it is not interested in creating sequence a of
ten "web" string, creating sequence b that is ten copies of a,
creating sequence c that is ten copies of b, and so on to i
that is that is ten copies of h. That's not its job. "Is it YAML?" is interested
in whether braces that are opened are closed later, and other things that parsers do. (I'd also add
that it's a lot better to treat aliases as references to the original data, rather than as
copies of the original data.)
If you wish to host "Is It YAML?", you have to ask yourself the question: is it safe? I believe it is, or at least as safe as the underlying Django code that supports it. It takes textual input, treats it as text, attempts to parse it, and catches all exceptions caused by parsing errors. It never treats the input as executable code. That's what I reckon, and I am standing by it by hosting the app myself on my site.
One of the outstanding issues of PyYAML is that anchor and alias names are limited to ASCII characters. The YAML specification is more generous with the characters it can support for anchors and aliases. The following is legitimate YAML, but PyYAML generally chokes on this:
# Songwriters: (gb) West Richard / Colin Gilbert Angus / Richard (gb) West
# Ebeneezer Goode lyrics © Warner/Chappell Music, Inc
- &😁 "'Eezer Goode 'Eezer Goode He's Ebeneezer Goode"
- *😁
I've overloaded the scanner - the part of the parser concerned with scanning input characters, one at a time - in "Is it YAML?", so that it can read and parse YAML with anchors and aliases made up of emoticons, Greek characters, Russian characters, Vietnamese characters, and the characters of other languages.
Copyright © Peter Murphy 2009–{% now "Y" %}.