Bug 1661659 - Reimplement optionally_keyed_by as a validator function rather than in terms of nested Any. r=taskgraph-reviewers,ahal

Using nested Any generates deeply nested non-trivial schemas because we
also end up nesting optionally_keyed_by's, and the addition of one field
in one optionally_keyed_by in bug 1657769 made the whole schema
compilation step an order of magnitude slower.

Even before bug 1657769, the schema compilation step was slow for tests,
and with this change, it's virtually instantaneous.

Differential Revision: https://phabricator.services.mozilla.com/D88543

taskcluster/taskgraph/util/schema.py

@@ -50,15 +50,21 @@ def optionally_keyed_by(*arguments):
     schema = arguments[-1]
     fields = arguments[:-1]
 
-    # build the nestable schema by generating schema = Any(schema,
-    # by-fld1, by-fld2, by-fld3) once for each field.  So we don't allow
-    # infinite nesting, but one level of nesting for each field.
-    for _ in arguments:
-        options = [schema]
-        for field in fields:
-            options.append({'by-' + field: {text_type: schema}})
-        schema = voluptuous.Any(*options)
-    return schema
+    def validator(obj):
+        if isinstance(obj, dict) and len(obj) == 1:
+            k, v = list(obj.items())[0]
+            if k.startswith('by-') and k[len('by-'):] in fields:
+                res = {}
+                for kk, vv in v.items():
+                    try:
+                        res[kk] = validator(vv)
+                    except voluptuous.Invalid as e:
+                        e.prepend([k, kk])
+                        raise
+                return res
+        return Schema(schema)(obj)
+
+    return validator
 
 
 def resolve_keyed_by(item, field, item_name, defer=None, **extra_values):