Jinja2 Recursive Rendering of a Deep Data Structure to YAML

Internal

• Jinja2
• YAML

Overview

To render a recursive data structure to "flow style" YAML, there's not much to do, Jinja2 renders recursively by default to a flow style. See Render to Flow-Style YAML.

To render a recursive data structure to "block style" YAML, use a recursive macro, as shown below. See Render to Block-Style YAML.

Render to Flow-Style YAML

Assuming the top level structure is a map, the following template and rendering code produce valid flow-style YAML:

The Template

{%- for key, value in m.items() %}
{{ key }}: {{ value }}
{%- endfor %}

The Rendering Code

import yaml
from jinja2 import Environment, FileSystemLoader
from pathlib import Path

tdir = Path('.')
template = Environment(loader=FileSystemLoader(tdir)).get_template('experimental-jinja-template-2.yaml.j2')
yaml_data = """
config:
  color: blue
  size: 1
  enabled: true
  samples:
    - name: venice
      neighborhoods:
        cannaregio: 1
        castello: 2
        giudecca: 3
    - name: genoa
      neighborhoods:
        molo: 1
        foce: 2
        marassi: 3
    - name: florence
      neighborhoods:
        santamaria: 1
        sanmarco: 2
        santacroce: 3
        no-such-neighborhood:
          - a1: v1
            a2: v2
          - b
          - c
"""
m = yaml.safe_load(yaml_data)
rendered_text = template.render(m=m)
print(rendered_text)

m2 = yaml.safe_load(rendered_text)
assert m == m2

The Result

config: {'color': 'blue', 'size': 1, 'enabled': True, 'samples': [{'name': 'venice', 'neighborhoods': {'cannaregio': 1, 'castello': 2, 'giudecca': 3}}, {'name': 'genoa', 'neighborhoods': {'molo': 1, 'foce': 2, 'marassi': 3}}, {'name': 'florence', 'neighborhoods': {'santamaria': 1, 'sanmarco': 2, 'santacroce': 3, 'no-such-neighborhood': [{'a1': 'v1', 'a2': 'v2'}, 'b', 'c']}}]}

Render to Block-Style YAML

The Template

{%- macro recursively_render_data_structure(d, overall_offset, recursion_level_indentation) -%}
{%- if d is boolean %}{{ d | lower }}
{%- elif d is number %}{{ d }}
{%- elif d is string %}"{{ d }}"
{%- elif d is mapping -%}
  {%- for key, value in d.items() %}
{{ overall_offset }}{{ ' ' * recursion_level_indentation }}{{ key }}: {{ recursively_render_data_structure(value, overall_offset, recursion_level_indentation + 2) }}
  {%- endfor %}
{%- elif d is iterable %}
  {%- for i in d %}
{{ overall_offset }}{{ ' ' * recursion_level_indentation }} - {{ recursively_render_data_structure(i, overall_offset, recursion_level_indentation + 4) }}
  {%- endfor %}
{%- else %}ERROR: UNKNOWN VALUE {{ d }}
{%- endif %}
{%- endmacro -%}

{{- recursively_render_data_structure(m, '      ', 0) }}

The Rendering Code

The code includes a test that the rendered structure is equivalent with the input:

import yaml
from jinja2 import Environment, FileSystemLoader
from pathlib import Path

tdir = Path('.')
template = Environment(loader=FileSystemLoader(tdir)).get_template('experimental-jinja-template.yaml.j2')
yaml_data = """
config:
  color: blue
  size: 1
  enabled: true
  samples:
    - name: venice
      neighborhoods:
        cannaregio: 1
        castello: 2
        giudecca: 3
    - name: genoa
      neighborhoods:
        molo: 1
        foce: 2
        marassi: 3
    - name: florence
      neighborhoods:
        santamaria: 1
        sanmarco: 2
        santacroce: 3
        no-such-neighborhood:
          - a1: v1
            a2: v2
          - b
          - c
"""
m = yaml.safe_load(yaml_data)
rendered_text = template.render(m=m)
print(rendered_text)

m2 = yaml.safe_load(rendered_text)
assert m == m2

The Result

config: 
  color: "blue"
  size: 1
  enabled: true
  samples: 
     - 
        name: "venice"
        neighborhoods: 
          cannaregio: 1
          castello: 2
          giudecca: 3
     - 
        name: "genoa"
        neighborhoods: 
          molo: 1
          foce: 2
          marassi: 3
     - 
        name: "florence"
        neighborhoods: 
          santamaria: 1
          sanmarco: 2
          santacroce: 3
          no-such-neighborhood: 
             - 
                a1: "v1"
                a2: "v2"
             - "b"
             - "c"