Ensure Vocabulary JSON-LD is valid

From 26a029d407be480d791972afb5975cf62c9360a6 Mon Sep 17 00:00:00 2001 From: Daniel Baumann Date: Fri, 19 Apr 2024 02:47:55 +0200 Subject: Adding upstream version 124.0.1. Signed-off-by: Daniel Baumann --- .../annotation-vocab/01-validJSON-LD-manual.html | 25 +++ .../02-context-to-triples-manual.html | 26 +++ .../03-graphs-are-isomorphic-manual.html | 26 +++ .../04-graphs-convert-lossless-manual.html | 26 +++ .../05-ontology-parsed-as-valid-manual.html | 13 ++ .../06-ontology-internally-consistent-manual.html | 14 ++ .../web-platform/tests/annotation-vocab/META.yml | 3 + .../tests/annotation-vocab/tools/README.md | 15 ++ .../tests/annotation-vocab/tools/anno.jsonld | 128 +++++++++++ .../tests/annotation-vocab/tools/vocab_tester.py | 240 +++++++++++++++++++++ 10 files changed, 516 insertions(+) create mode 100644 testing/web-platform/tests/annotation-vocab/01-validJSON-LD-manual.html create mode 100644 testing/web-platform/tests/annotation-vocab/02-context-to-triples-manual.html create mode 100644 testing/web-platform/tests/annotation-vocab/03-graphs-are-isomorphic-manual.html create mode 100644 testing/web-platform/tests/annotation-vocab/04-graphs-convert-lossless-manual.html create mode 100644 testing/web-platform/tests/annotation-vocab/05-ontology-parsed-as-valid-manual.html create mode 100644 testing/web-platform/tests/annotation-vocab/06-ontology-internally-consistent-manual.html create mode 100644 testing/web-platform/tests/annotation-vocab/META.yml create mode 100644 testing/web-platform/tests/annotation-vocab/tools/README.md create mode 100644 testing/web-platform/tests/annotation-vocab/tools/anno.jsonld create mode 100644 testing/web-platform/tests/annotation-vocab/tools/vocab_tester.py (limited to 'testing/web-platform/tests/annotation-vocab') diff --git a/testing/web-platform/tests/annotation-vocab/01-validJSON-LD-manual.html b/testing/web-platform/tests/annotation-vocab/01-validJSON-LD-manual.html new file mode 100644 index 0000000000..3d6623a3a5 --- /dev/null +++ b/testing/web-platform/tests/annotation-vocab/01-validJSON-LD-manual.html @@ -0,0 +1,25 @@ + + + +Ensure Vocabulary JSON-LD is valid + + + + +

The Annotation Vocabulary JSON-LD context is defined to be at http://www.w3.org/ns/oa

Retrieve that context document and expand the following example using a JSON-LD validation tool. Does it expand?

+{
+  "@context": "http://www.w3.org/ns/anno.jsonld",
+  "id": "http://example.org/anno4",
+  "type": "Annotation",
+  "body": "http://example.org/description1",
+  "target": {
+    "id": "http://example.com/image1#xywh=100,100,300,300",
+    "type": "Image",
+    "format": "image/jpeg"
+  }
+}
+

+ + diff --git a/testing/web-platform/tests/annotation-vocab/02-context-to-triples-manual.html b/testing/web-platform/tests/annotation-vocab/02-context-to-triples-manual.html new file mode 100644 index 0000000000..bfe030d70e --- /dev/null +++ b/testing/web-platform/tests/annotation-vocab/02-context-to-triples-manual.html @@ -0,0 +1,26 @@ + + + +JSON-LD context document can be used to convert JSON-LD serialized Annotations into RDF triples + + + + +

The Annotation Vocabulary JSON-LD context is defined to be at http://www.w3.org/ns/oa

Use that context document to convert the sample below into RDF triples. Did + the conversion complete?

+{
+  "@context": "http://www.w3.org/ns/anno.jsonld",
+  "id": "http://example.org/anno4",
+  "type": "Annotation",
+  "body": "http://example.org/description1",
+  "target": {
+    "id": "http://example.com/image1#xywh=100,100,300,300",
+    "type": "Image",
+    "format": "image/jpeg"
+  }
+}
+

+ + diff --git a/testing/web-platform/tests/annotation-vocab/03-graphs-are-isomorphic-manual.html b/testing/web-platform/tests/annotation-vocab/03-graphs-are-isomorphic-manual.html new file mode 100644 index 0000000000..ed6869caf9 --- /dev/null +++ b/testing/web-platform/tests/annotation-vocab/03-graphs-are-isomorphic-manual.html @@ -0,0 +1,26 @@ + + + +Graphs produced are isomorphic + + + + +

The Annotation Vocabulary JSON-LD context is defined to be at http://www.w3.org/ns/oa

Use that context to convert the content below into various format RDF + graphs. Are these graphs identical?

+{
+  "@context": "http://www.w3.org/ns/anno.jsonld",
+  "id": "http://example.org/anno4",
+  "type": "Annotation",
+  "body": "http://example.org/description1",
+  "target": {
+    "id": "http://example.com/image1#xywh=100,100,300,300",
+    "type": "Image",
+    "format": "image/jpeg"
+  }
+}
+

+ + diff --git a/testing/web-platform/tests/annotation-vocab/04-graphs-convert-lossless-manual.html b/testing/web-platform/tests/annotation-vocab/04-graphs-convert-lossless-manual.html new file mode 100644 index 0000000000..05e7fb72d2 --- /dev/null +++ b/testing/web-platform/tests/annotation-vocab/04-graphs-convert-lossless-manual.html @@ -0,0 +1,26 @@ + + + +Graphs produced can be converted back into JSON-LD with no loss + + + + +

The Annotation Vocabulary JSON-LD context is defined to be at http://www.w3.org/ns/oa

Using that context, convert the content below into an RDF graph, then + convert that graph back into JSON-LD. Is the content the same?

+{
+  "@context": "http://www.w3.org/ns/anno.jsonld",
+  "id": "http://example.org/anno4",
+  "type": "Annotation",
+  "body": "http://example.org/description1",
+  "target": {
+    "id": "http://example.com/image1#xywh=100,100,300,300",
+    "type": "Image",
+    "format": "image/jpeg"
+  }
+}
+

+ + diff --git a/testing/web-platform/tests/annotation-vocab/05-ontology-parsed-as-valid-manual.html b/testing/web-platform/tests/annotation-vocab/05-ontology-parsed-as-valid-manual.html new file mode 100644 index 0000000000..e1e41e6a48 --- /dev/null +++ b/testing/web-platform/tests/annotation-vocab/05-ontology-parsed-as-valid-manual.html @@ -0,0 +1,13 @@ + + + +Ontology can be parsed without errors + + + + +

The Annotation Vocabulary JSON-LD context is defined to be at http://www.w3.org/ns/oa

One ontology of that context is at at http://www.w3.org/ns/oa.ttl

Parse that ontology into a graph. Did it work?

+ + diff --git a/testing/web-platform/tests/annotation-vocab/06-ontology-internally-consistent-manual.html b/testing/web-platform/tests/annotation-vocab/06-ontology-internally-consistent-manual.html new file mode 100644 index 0000000000..de9b7d069d --- /dev/null +++ b/testing/web-platform/tests/annotation-vocab/06-ontology-internally-consistent-manual.html @@ -0,0 +1,14 @@ + + + +Ontology is internally consistent + + + + +

The Annotation Vocabulary JSON-LD context is defined to be at http://www.w3.org/ns/oa

One ontology of that context is at http://www.w3.org/ns/oa.ttl

Parse that ontology and evaluate its domains and ranges. Are they + consistent with the underlying referenced data types?

+ + diff --git a/testing/web-platform/tests/annotation-vocab/META.yml b/testing/web-platform/tests/annotation-vocab/META.yml new file mode 100644 index 0000000000..f99f921d35 --- /dev/null +++ b/testing/web-platform/tests/annotation-vocab/META.yml @@ -0,0 +1,3 @@ +spec: https://www.w3.org/TR/annotation-vocab/ +suggested_reviewers: + - halindrome diff --git a/testing/web-platform/tests/annotation-vocab/tools/README.md b/testing/web-platform/tests/annotation-vocab/tools/README.md new file mode 100644 index 0000000000..1bc74ec23d --- /dev/null +++ b/testing/web-platform/tests/annotation-vocab/tools/README.md @@ -0,0 +1,15 @@ +Vocabulary Exercising Tools +=========================== + +The Web Annotation Vocabulary specification defines an ontology for +portable annotations. The tools in this directory, along with the +sample files supplied, can be used to exercise the vocabulary +"implementation" against various RDF processing engines. + +vocab-tester.py +=============== + +This python script exercises the vocabulary implementation using rdflib, +rdflib-jsonld and pyld. Note that this means your python environment must +have those additional python modules installed in order to run the tests. + diff --git a/testing/web-platform/tests/annotation-vocab/tools/anno.jsonld b/testing/web-platform/tests/annotation-vocab/tools/anno.jsonld new file mode 100644 index 0000000000..58892919c0 --- /dev/null +++ b/testing/web-platform/tests/annotation-vocab/tools/anno.jsonld @@ -0,0 +1,128 @@ +{ + "@context": { + "oa": "http://www.w3.org/ns/oa#", + "dc": "http://purl.org/dc/elements/1.1/", + "dcterms": "http://purl.org/dc/terms/", + "dctypes": "http://purl.org/dc/dcmitype/", + "foaf": "http://xmlns.com/foaf/0.1/", + "rdf": "http://www.w3.org/1999/02/22-rdf-syntax-ns#", + "rdfs": "http://www.w3.org/2000/01/rdf-schema#", + "skos": "http://www.w3.org/2004/02/skos/core#", + "xsd": "http://www.w3.org/2001/XMLSchema#", + "iana": "http://www.iana.org/assignments/relation/", + "owl": "http://www.w3.org/2002/07/owl#", + "as": "http://www.w3.org/ns/activitystreams#", + "schema": "http://schema.org/", + + "id": {"@type": "@id", "@id": "@id"}, + "type": {"@type": "@id", "@id": "@type"}, + + "Annotation": "oa:Annotation", + "Dataset": "dctypes:Dataset", + "Image": "dctypes:StillImage", + "Video": "dctypes:MovingImage", + "Audio": "dctypes:Sound", + "Text": "dctypes:Text", + "TextualBody": "oa:TextualBody", + "ResourceSelection": "oa:ResourceSelection", + "SpecificResource": "oa:SpecificResource", + "FragmentSelector": "oa:FragmentSelector", + "CssSelector": "oa:CssSelector", + "XPathSelector": "oa:XPathSelector", + "TextQuoteSelector": "oa:TextQuoteSelector", + "TextPositionSelector": "oa:TextPositionSelector", + "DataPositionSelector": "oa:DataPositionSelector", + "SvgSelector": "oa:SvgSelector", + "RangeSelector": "oa:RangeSelector", + "TimeState": "oa:TimeState", + "HttpRequestState": "oa:HttpRequestState", + "CssStylesheet": "oa:CssStyle", + "Choice": "oa:Choice", + "Composite": "oa:Composite", + "List": "oa:List", + "Independents": "oa:Independents", + "Person": "foaf:Person", + "Software": "as:Application", + "Organization": "foaf:Organization", + "AnnotationCollection": "as:OrderedCollection", + "AnnotationPage": "as:OrderedCollectionPage", + "Audience": "schema:Audience", + + "Motivation": "oa:Motivation", + "bookmarking": "oa:bookmarking", + "classifying": "oa:classifying", + "commenting": "oa:commenting", + "describing": "oa:describing", + "editing": "oa:editing", + "highlighting": "oa:highlighting", + "identifying": "oa:identifying", + "linking": "oa:linking", + "moderating": "oa:moderating", + "questioning": "oa:questioning", + "replying": "oa:replying", + "reviewing": "oa:reviewing", + "tagging": "oa:tagging", + + "auto": "oa:autoDirection", + "ltr": "oa:ltrDirection", + "rtl": "oa:rtlDirection", + + "body": {"@type": "@id", "@id": "oa:hasBody"}, + "target": {"@type": "@id", "@id": "oa:hasTarget"}, + "source": {"@type": "@id", "@id": "oa:hasSource"}, + "selector": {"@type": "@id", "@id": "oa:hasSelector"}, + "state": {"@type": "@id", "@id": "oa:hasState"}, + "scope": {"@type": "@id", "@id": "oa:hasScope"}, + "refinedBy": {"@type": "@id", "@id": "oa:refinedBy"}, + "startSelector": {"@type": "@id", "@id": "oa:hasStartSelector"}, + "endSelector": {"@type": "@id", "@id": "oa:hasEndSelector"}, + "renderedVia": {"@type": "@id", "@id": "oa:renderedVia"}, + "creator": {"@type": "@id", "@id": "dcterms:creator"}, + "generator": {"@type": "@id", "@id": "as:generator"}, + "rights": {"@type": "@id", "@id": "dcterms:rights"}, + "homepage": {"@type": "@id", "@id": "foaf:homepage"}, + "via": {"@type": "@id", "@id": "oa:via"}, + "canonical": {"@type": "@id", "@id": "oa:canonical"}, + "stylesheet": {"@type": "@id", "@id": "oa:styledBy"}, + "cached": {"@type": "@id", "@id": "oa:cachedSource"}, + "conformsTo": {"@type": "@id", "@id": "dcterms:conformsTo"}, + "items": {"@type": "@id", "@id": "as:items", "@container": "@list"}, + "partOf": {"@type": "@id", "@id": "as:partOf"}, + "first": {"@type": "@id", "@id": "as:first"}, + "last": {"@type": "@id", "@id": "as:last"}, + "next": {"@type": "@id", "@id": "as:next"}, + "prev": {"@type": "@id", "@id": "as:prev"}, + "audience": {"@type": "@id", "@id": "schema:audience"}, + "motivation": {"@type": "@vocab", "@id": "oa:motivatedBy"}, + "purpose": {"@type": "@vocab", "@id": "oa:hasPurpose"}, + "textDirection": {"@type": "@vocab", "@id": "oa:textDirection"}, + + "accessibility": "schema:accessibilityFeature", + "bodyValue": "oa:bodyValue", + "format": "dc:format", + "language": "dc:language", + "processingLanguage": "oa:processingLanguage", + "value": "rdf:value", + "exact": "oa:exact", + "prefix": "oa:prefix", + "suffix": "oa:suffix", + "styleClass": "oa:styleClass", + "name": "foaf:name", + "email": "foaf:mbox", + "email_sha1": "foaf:mbox_sha1sum", + "nickname": "foaf:nick", + "label": "rdfs:label", + + "created": {"@id": "dcterms:created", "@type": "xsd:dateTime"}, + "modified": {"@id": "dcterms:modified", "@type": "xsd:dateTime"}, + "generated": {"@id": "dcterms:issued", "@type": "xsd:dateTime"}, + "sourceDate": {"@id": "oa:sourceDate", "@type": "xsd:dateTime"}, + "sourceDateStart": {"@id": "oa:sourceDateStart", "@type": "xsd:dateTime"}, + "sourceDateEnd": {"@id": "oa:sourceDateEnd", "@type": "xsd:dateTime"}, + + "start": {"@id": "oa:start", "@type": "xsd:nonNegativeInteger"}, + "end": {"@id": "oa:end", "@type": "xsd:nonNegativeInteger"}, + "total": {"@id": "as:totalItems", "@type": "xsd:nonNegativeInteger"}, + "startIndex": {"@id": "as:startIndex", "@type": "xsd:nonNegativeInteger"} + } +} \ No newline at end of file diff --git a/testing/web-platform/tests/annotation-vocab/tools/vocab_tester.py b/testing/web-platform/tests/annotation-vocab/tools/vocab_tester.py new file mode 100644 index 0000000000..86453a7a29 --- /dev/null +++ b/testing/web-platform/tests/annotation-vocab/tools/vocab_tester.py @@ -0,0 +1,240 @@ + +# Author: Rob Sanderson (azaroth42@gmail.com) +# License: Apache2 +# Last Modified: 2016-09-02 + +import json +from rdflib import ConjunctiveGraph, URIRef +from pyld import jsonld +from pyld.jsonld import compact, expand, frame, from_rdf, to_rdf, JsonLdProcessor +import urllib + +# Stop code from looking up the contexts online for every operation +docCache = {} + +def fetch(url): + fh = urllib.urlopen(url) + data = fh.read() + fh.close() + return data + +def load_document_and_cache(url): + if docCache.has_key(url): + return docCache[url] + + doc = { + 'contextUrl': None, + 'documentUrl': None, + 'document': '' + } + data = fetch(url) + doc['document'] = data; + docCache[url] = doc + return doc + +jsonld.set_document_loader(load_document_and_cache) + +class Validator(object): + + def __init__(self): + + self.rdflib_class_map = { + "Annotation": "oa:Annotation", + "Dataset": "dctypes:Dataset", + "Image": "dctypes:StillImage", + "Video": "dctypes:MovingImage", + "Audio": "dctypes:Sound", + "Text": "dctypes:Text", + "TextualBody": "oa:TextualBody", + "ResourceSelection": "oa:ResourceSelection", + "SpecificResource": "oa:SpecificResource", + "FragmentSelector": "oa:FragmentSelector", + "CssSelector": "oa:CssSelector", + "XPathSelector": "oa:XPathSelector", + "TextQuoteSelector": "oa:TextQuoteSelector", + "TextPositionSelector": "oa:TextPositionSelector", + "DataPositionSelector": "oa:DataPositionSelector", + "SvgSelector": "oa:SvgSelector", + "RangeSelector": "oa:RangeSelector", + "TimeState": "oa:TimeState", + "HttpState": "oa:HttpRequestState", + "CssStylesheet": "oa:CssStyle", + "Choice": "oa:Choice", + "Composite": "oa:Composite", + "List": "oa:List", + "Independents": "oa:Independents", + "Person": "foaf:Person", + "Software": "as:Application", + "Organization": "foaf:Organization", + "AnnotationCollection": "as:OrderedCollection", + "AnnotationPage": "as:OrderedCollectionPage", + "Audience": "schema:Audience" + } + + + def _clean_bnode_ids(self, js): + new = {} + for (k,v) in js.items(): + if k == 'id' and v.startswith("_:"): + continue + elif type(v) == dict: + # recurse + res = self._clean_bnode_ids(v) + new[k] = res + else: + new[k] = v + return new + + def _mk_rdflib_jsonld(self, js): + # rdflib's json-ld implementation sucks + # Pre-process to make it work + # recurse the structure looking for types, and replacing them. + new = {} + for (k,v) in js.items(): + if k == 'type': + if type(v) == list: + nl = [] + for i in v: + if self.rdflib_class_map.has_key(i): + nl.append(self.rdflib_class_map[i]) + new['type'] = nl + else: + if self.rdflib_class_map.has_key(v): + new['type'] = self.rdflib_class_map[v] + elif type(v) == dict: + # recurse + res = self._mk_rdflib_jsonld(v) + new[k] = res + else: + new[k] = v + return new + + def json_to_rdf(self, js, fmt=None): + d2 = self._mk_rdflib_jsonld(js) + js = json.dumps(d2) + g = ConjunctiveGraph() + g.parse(data=js, format='json-ld') + if fmt: + out = g.serialize(format=fmt) + return out + else: + return g + + def rdf_to_jsonld(self, rdf, fmt): + + g = ConjunctiveGraph() + g.parse(data=rdf, format=fmt) + out = g.serialize(format='json-ld') + + j2 = json.loads(out) + j2 = {"@context": context_js, "@graph": j2} + framed = frame(j2, frame_js) + out = compact(framed, context_js) + # recursively clean blank node ids + #out = self._clean_bnode_ids(out) + return out + + def compact_and_clean(self, js): + newjs = compact(js, context_js) + newjs['@context'] = context + if newjs.has_key("@graph"): + for k,v in newjs['@graph'].items(): + newjs[k] = v + del newjs['@graph'] + return newjs + +validator = Validator() + +example = "https://raw.githubusercontent.com/w3c/web-annotation/gh-pages/model/wd2/examples/correct/anno4.json" +example_ttl = "https://raw.githubusercontent.com/w3c/web-annotation/gh-pages/vocab/wd/examples/correct/anno1.ttl" +context = "http://www.w3.org/ns/anno.jsonld" +frameURI = "https://raw.githubusercontent.com/w3c/web-annotation/gh-pages/jsonld/annotation_frame.jsonld" +# ontology = "https://www.w3.org/ns/oa.ttl" +ontology = "https://raw.githubusercontent.com/w3c/web-annotation/gh-pages/vocab/wd/ontology/oa.ttl" + +data = fetch(context) +context_js = json.loads(data) +data = fetch(example) +example_js = json.loads(data) +data = fetch(frameURI) +frame_js = json.loads(data) + +# Test1: JSON-LD context document can be parsed without errors by JSON-LD validators +# Context document is parsable if it can be loaded and used to expand the example +try: + expanded = expand(example_js, context_js) +except: + print("Context is invalid, failed Test 1") + + +# Test2: JSON-LD context document can be used to convert JSON-LD serialized Annotations into RDF triples. +try: + jsonld_nq = to_rdf(example_js, {"base": "http://example.org/", "format": "application/nquads"}) +except: + print("Cannot use context to convert JSON-LD to NQuads") + + +# Test3: Graphs produced are isomorphic +try: + rl_g = validator.json_to_rdf(example_js) + g = ConjunctiveGraph() + js_g = g.parse(data=jsonld_nq, format="nt") + rl_g_nq = rl_g.serialize(format="nquads") + assert(len(rl_g.store) == len(js_g.store)) + assert(rl_g.isomorphic(js_g)) +except: + print("Different triples from two parsers, or non-isomorphic graphs") + + +# Test4: The graphs produced can be converted back into JSON-LD without loss of information +try: + js = validator.rdf_to_jsonld(jsonld_nq, "nt") + js2 = validator.compact_and_clean(js) + assert(js2 == example_js) +except: + print("Failed to recompact parsed data") + raise + + +# Test5: ontology documents can be parsed without errors by validators +try: + g = ConjunctiveGraph().parse(ontology, format="turtle") +except: + raise + + +# Test6: ontology is internally consistent with respect to domains, ranges, etc + +# step 1: find all the classes. +rdftype = URIRef("http://www.w3.org/1999/02/22-rdf-syntax-ns#type") +rdfsdomain = URIRef("http://www.w3.org/2000/01/rdf-schema#domain") +rdfsrange = URIRef("http://www.w3.org/2000/01/rdf-schema#range") +rdfsresource = URIRef("http://www.w3.org/1999/02/22-rdf-syntax-ns#Resource") +rdfssco = URIRef("http://www.w3.org/2000/01/rdf-schema#subClassOf") +asColl = URIRef("http://www.w3.org/ns/activitystreams#OrderedCollection") +skosConcept = URIRef("http://www.w3.org/2004/02/skos/core#Concept") + +otherClasses = [asColl, skosConcept] +classes = list(g.subjects(rdftype, URIRef("http://www.w3.org/2000/01/rdf-schema#Class"))) +props = list(g.subjects(rdftype, URIRef("http://www.w3.org/1999/02/22-rdf-syntax-ns#Property"))) + +for p in props: + domains = list(g.objects(p, rdfsdomain)) + for d in domains: + assert(d in classes) + +for p in props: + ranges = list(g.objects(p, rdfsrange)) + for r in ranges: + if not r in classes and not str(r).startswith("http://www.w3.org/2001/XMLSchema#") and \ + not r == rdfsresource: + print("Found inconsistent property: %s has unknown range" % p) + +for c in classes: + parents = list(g.objects(c, rdfssco)) + for p in parents: + if not p in classes and not p in otherClasses: + print("Found inconsistent class: %s has unknown superClass" % c) + + +print("Done.") -- cgit v1.2.3