diff options
Diffstat (limited to '')
-rw-r--r-- | third_party/heimdal/lib/asn1/README.md | 1327 |
1 files changed, 1327 insertions, 0 deletions
diff --git a/third_party/heimdal/lib/asn1/README.md b/third_party/heimdal/lib/asn1/README.md new file mode 100644 index 0000000..9c4e697 --- /dev/null +++ b/third_party/heimdal/lib/asn1/README.md @@ -0,0 +1,1327 @@ +# Heimdal's ASN.1 Compiler + +This is a new README, and it's not very rich in contents yet. Be sure to check +out the [README on the template backend](/lib/asn1/README-template.md) and the [README +on automatic open type decoding via X.681/X.682/X.683 +annotations](/lib/asn1/README-X681.md). + +## Table of Contents + + 1. [Introduction](#Introduction) + 2. [ASN.1 Support in Heimdal](#asn1-support-in-heimdal) + 3. [News](#News) + 4. [Features](#Features) + 5. [Limitations](#Limitations) + 6. [Compiler Usage](#Compiler-usage) + 7. [APIs Generated by the Compiler](#APIs-generated-by-the-compiler) + 8. [`asn1_print` Usage](#asn1_print-usage) + 9. [Implementation](#implementation) + 10. [Moving From C](#moving-from-c) + +## Introduction + +ASN.1 is a... some would say baroque, perhaps obsolete, archaic even, "syntax" +for expressing data type schemas, and also a set of "encoding rules" (ERs) that +specify many ways to encode values of those types for interchange. + +Some ERs are binary, others are textual. Some binary ERs are tag-length-value +(TLV), others have no need for tagging. Some of the ERs are roundly and +rightly disliked, but then there are XER (XML Encoding Rules) and JER (JSON +Encoding Rules) that really illustrate how the syntax and the encoding rules +really are separate and distinct things. + +ASN.1 is a wheel that everyone loves to reinvent, and often badly. It's worth +knowing a bit about it before reinventing this wheel badly yet again. + +It's also worth pondering that there appears to be ways to map most data +exchange metaschemas and schemas onto others, and therefore too, transliterate +most encodings onto others. + +First, an example of the syntax: + +```ASN.1 +-- This is what a certificate looks like (as in TLS server certificates, or +-- "SSL certs): +Certificate ::= SEQUENCE { + tbsCertificate TBSCertificate, + signatureAlgorithm AlgorithmIdentifier, + signatureValue BIT STRING +} + +-- The main body of a certificate is here though: +TBSCertificate ::= SEQUENCE { + version [0] Version DEFAULT 1, + serialNumber CertificateSerialNumber, + signature AlgorithmIdentifier, + issuer Name, + validity Validity, + subject Name, + subjectPublicKeyInfo SubjectPublicKeyInfo, + issuerUniqueID [1] IMPLICIT BIT STRING OPTIONAL, + subjectUniqueID [2] IMPLICIT BIT STRING OPTIONAL, + extensions [3] EXPLICIT Extensions OPTIONAL +} +``` + +Here we see something akin to a "structure" or "record" with various named +fields of various types. Some of these are optional, which means they can have +no value given in encodings. One is defaulted, which means that if no values +is given in encodings then the default value is intended. + +Those `[0]` things are called tags and are decidedly obsolete, along with all +"tag-length-value" (TLV) or "self-describing" encoding rules. Tags appear as +lexical tokens in ASN.1 only because a) in the early 80s TLV encodings were +thought fantastic, and b) automatic tagging wasn't invented and implemented +until it was too late. New ASN.1 modules should never need to have those tags +appear in the syntax. + +ASN.1 has a lot of competition, and may even be obsolete. Obsolete +technologies take decades to die out because of the need to interoperate with +the installed base. So even if ASN.1 is obsolete, we find ourselves needing to +implement a large subset of it in order to implement certain important network +protocols. + +Encoding rules? There are many: + + - JSON Encoding Rules (JER) ([X.697](https://www.itu.int/rec/T-REC-X/recommendation.asp?lang=en&parent=T-REC-X.697)) + + Use JSON instead of some binary scheme like DER (see below). + + - XML Encoding Rules (XER) ([X.693](https://www.itu.int/rec/T-REC-X/recommendation.asp?lang=en&parent=T-REC-X.693)) + + - Generic String Encoding Rules (GSER) ([RFC3641](https://tools.ietf.org/html/rfc3641)) + + - Basic, Distinguished, and Canonical Encoding Rules (BER, DER, CER) ([X.690](https://www.itu.int/rec/T-REC-X/recommendation.asp?lang=en&parent=T-REC-X.690) + + These are the dreaded tag-length-value encoding rules. They are redundant, + wasteful, and inefficient in spite of being non-textual (i.e., binary)! + + The descriptor "tag-length-value" is due to all values being encoded as some + bytes for a "tag", then some bytes for the length of the encoded value, then + the encoded value itself. The body of a structured type (e.g., + `Certificate`) is itself a concatenation of the TLV encodings of the fields + of that structured type, in order. + + DER and CER are alternative canonical forms of BER. + + - Packed Encoding Rules (PER) ([X.691](https://www.itu.int/rec/T-REC-X/recommendation.asp?lang=en&parent=T-REC-X.691)) and Octet Encoding Rules (OER) ([X.696](https://www.itu.int/rec/T-REC-X/recommendation.asp?lang=en&parent=T-REC-X.696)) + + These are a lot like eXternal Data Representation + ([XDR](https://tools.ietf.org/html/rfc4506.html)), but with 1-octet + alignment instead of 4-octet alignment. + +There is also a meta encoding rule system, the Encoding Control Notation (ECN) +([X.692](https://www.itu.int/rec/T-REC-X/recommendation.asp?lang=en&parent=T-REC-X.692)) +intended to be able to express all sorts of kinds of encodings. + +Heimdal currently only supports DER for encoding, and DER and BER for decoding, +but soon may support JER as well, and can print values as JSON, though not +compliant with JER. + +The syntax itself is specified by +[X.680](https://www.itu.int/rec/T-REC-X/recommendation.asp?lang=en&parent=T-REC-X.680), +with extensions via +[X.681](https://www.itu.int/rec/T-REC-X/recommendation.asp?lang=en&parent=T-REC-X.681), +[X.682](https://www.itu.int/rec/T-REC-X/recommendation.asp?lang=en&parent=T-REC-X.682), +and +[X.683](https://www.itu.int/rec/T-REC-X/recommendation.asp?lang=en&parent=T-REC-X.683),. + +## ASN.1 Support in Heimdal + +Heimdal contains an implementation of: + + - ASN.1 + - PKIX + - Kerberos + - misc. Heimdal-specific protocols related to PKIX and Kerberos, such as: + + - Online certification authority protocols + - Kerberos KDC replication protocols + - Kerberos administration protocols + +PKIX and Kerberos both require ASN.1 and DER support. + +For historical reasons many ASN.1-using projects have used hand-rolled codecs +that have proven difficult to implement, maintain, and extend, and, of course, +buggy. Heimdal has its own ASN.1 module compiler and library in order to avoid +the pitfalls of hand-rolled codecs, and to satisfy Heimdal's internal needs. + +There are other ASN.1 compilers and libraries out there, of course, but it +would prove difficult to switch compilers as generally ASN.1 compilers lack +sufficient control over generated types and APIs for programming languages. + +Heimdal's ASN.1 compiler supports a large subset of X.680, X.681, X.682, and +X.683, as well as a large subset of X.690, with an architecture that should +make it easy to add support for encoding rules other than X.690. + +## News + +In recent times the following features have been added: + + - Feature parity for the "template" backend, even superiority, as the codegen + backend does not yet support automatic open type decoding/encoding. + + - IMPLICIT tagging support is finally complete. + + - Automatic open type traversal, using a subset of X.681/X.682/X.683 for + expressing the requisite metadata. + + - Decoration of ASN.1 types with "hidden" fields (ones that don't get encoded + or decoded) of ASN.1 or C types. + +## Futures + + - JER support? + + - XDR/NDR/OER support? + + - Generate comparators? (lib/hx509 has a half-baked Certificate comparator) + +## Features + + - Most of X.680 is supported. + + - Most of X.690 is supported for decoding, with only DER supported for + encoding. + + - For cryptographic applications there is a `--preserve-binary=TYPE` compiler + option that causes the `TYPE`'s C `struct` to gain a `_save` field where the + original encoding of the `TYPE` is preserved by the decoder. This allows + cryptographic applications to validate signatures, MACs, authenticated + decryption tags, checksums, etc., without having to re-encode the `TYPE` + (which wouldn't even work if the encoding received were BER and BER were + permitted for that `TYPE`). + + - Unconstrained integer types have a large integer representation in C that is + not terribly useful in common cases. Range and member constraints on + integer types cause the compiler to use `int`, `int64_t`, `unsigned int`, + and/or `uint64_t` as appropriate. + + - The Heimdal ASN.1 compiler currently handles a large subset of X.680, and + (in a branch) a small subset of X.681, X.682, and X.683, which manifests as + automatic handling of all open types contained in `SET`/`SEQUENCE` types + that are parameterized with information object sets. This allows all open + types in PKIX certificates, for example, to get decoded automatically no + matter how deeply nested. We use a TCG EK certificate that has eight + certificate extensions, including subject alternative names and subject + directory attributes where the attribute values are not string types, and + all of these things get decoded automatically. + + - The template backend dedups templates to save space. This is an O(N^2) kind + of feature that we need to make optional, but it works. (When we implement + JER this will have the side-effect of printing the wrong type names in some + cases because two or more types have the same templates and get deduped.) + + - There is an _experimental_ ASN.1 module -> JSON feature in the compiler. It + currently dumps type and value definitions, but not class, or object set + definitions. Even for types, it is not complete, and the JSON schema used + is subject to change *WITHOUT NOTICE*. + + Perhaps eventually we can re-write the compiler as a C-coded ASN.1 -> JSON + stage followed by a jq-coded code and template generator state, which would + make it much easier to extend the compiler. + + - We have an `asn1_print` program that can decode DER from any exported types + from any ASN.1 modules committed in Heimdal: + + ```bash + $ ./asn1_print ek.crt Certificate | + jq '.tbsCertificate.extensions[3]._extnValue[]._values' + ``` + + ```JSON + [ + { + "_type": "TPMSpecification", + "family": "2.0", + "level": 0, + "revision": 138 + } + ] + [ + { + "_type": "TPMSecurityAssertions", + "version": 0, + "fieldUpgradable": true, + "ekGenerationType": "ekgt-injected", + "ekGenerationLocation": "tpmManufacturer", + "ekCertificateGenerationLocation": "tpmManufacturer", + "ccInfo": { + "_type": "CommonCriteriaMeasures", + "version": "3.1", + "assurancelevel": "ealevel4", + "evaluationStatus": "evaluationCompleted", + "plus": true, + "strengthOfFunction": null, + "profileOid": null, + "profileUri": null, + "targetOid": null, + "targetUri": null + }, + "fipsLevel": { + "_type": "FIPSLevel", + "version": "140-2", + "level": "sllevel2", + "plus": false + }, + "iso9000Certified": false, + "iso9000Uri": null + } + ] + ``` + + A complete dump of such a certificate: + + ```bash + $ ./asn1_print ek.crt Certificate | jq . + ``` + + ```JSON + { + "_type": "Certificate", + "tbsCertificate": { + "_type": "TBSCertificate", + "_save": "30820376A00302010202146A0597BA71D7E6D3AC0EDC9EDC95A15B998DE40A300D06092A864886F70D01010B05003055310B3009060355040613024348311E301C060355040A131553544D6963726F656C656374726F6E696373204E56312630240603550403131D53544D2054504D20454B20496E7465726D656469617465204341203035301E170D3138313231343030303030305A170D3238313231343030303030305A300030820122300D06092A864886F70D01010105000382010F003082010A0282010100CC14EB27A78CEB0EA486FA2DF7835F5FA8E905B097012B5BDE50380C355B1A2A721BBC3D08DD21796CDB239FA95310651B1B56FD2CFE53C87352EBD996E33256160404CE9302A08066801E786A2F86E181F949966F492A85B58EAA4A6A8CB3697551BB236E87CC7BF8EC1347871C91E15437E8F266BF1EA5EB271FDCF374D8B47DF8BCE89E1FAD61C2A088CB4036B359CB72A294973FEDCCF0C340AFFD14B64F041165581ACA34147C1C75617047058F7ED7D603E032508094FA73E8B9153DA3BF255D2CBBC5DF301BA8F74D198BEBCE86040FC1D2927C7657414490D802F482F3EBF2DE35EE149A1A6DE8D16891FBFBA02A18AFE59F9D6F149744E5F0D559B10203010001A38201A9308201A5301F0603551D230418301680141ADB994AB58BE57A0CC9B900E7851E1A43C0866030420603551D20043B303930370604551D2000302F302D06082B060105050702011621687474703A2F2F7777772E73742E636F6D2F54504D2F7265706F7369746F72792F30590603551D110101FF044F304DA44B304931163014060567810502010C0B69643A353335343444323031173015060567810502020C0C53543333485450484148433031163014060567810502030C0B69643A303034393030303830670603551D090460305E301706056781050210310E300C0C03322E300201000202008A304306056781050212313A30380201000101FFA0030A0101A1030A0100A2030A0100A310300E1603332E310A01040A01020101FFA40F300D16053134302D320A0102010100300E0603551D0F0101FF040403020520300C0603551D130101FF0402300030100603551D250409300706056781050801304A06082B06010505070101043E303C303A06082B06010505073002862E687474703A2F2F7365637572652E676C6F62616C7369676E2E636F6D2F73746D74706D656B696E7430352E637274", + "version": "rfc3280_version_3", + "serialNumber": "6A0597BA71D7E6D3AC0EDC9EDC95A15B998DE40A", + "signature": { + "_type": "AlgorithmIdentifier", + "algorithm": { + "_type": "OBJECT IDENTIFIER", + "oid": "1.2.840.113549.1.1.11", + "components": [ + 1, + 2, + 840, + 113549, + 1, + 1, + 11 + ], + "name": "id-pkcs1-sha256WithRSAEncryption" + }, + "parameters": "0500" + }, + "issuer": { + "_choice": "rdnSequence", + "value": [ + [ + { + "_type": "AttributeTypeAndValue", + "type": { + "_type": "OBJECT IDENTIFIER", + "oid": "2.5.4.6", + "components": [ + 2, + 5, + 4, + 6 + ], + "name": "id-at-countryName" + }, + "value": { + "_choice": "printableString", + "value": "CH" + } + } + ], + [ + { + "_type": "AttributeTypeAndValue", + "type": { + "_type": "OBJECT IDENTIFIER", + "oid": "2.5.4.10", + "components": [ + 2, + 5, + 4, + 10 + ], + "name": "id-at-organizationName" + }, + "value": { + "_choice": "printableString", + "value": "STMicroelectronics NV" + } + } + ], + [ + { + "_type": "AttributeTypeAndValue", + "type": { + "_type": "OBJECT IDENTIFIER", + "oid": "2.5.4.3", + "components": [ + 2, + 5, + 4, + 3 + ], + "name": "id-at-commonName" + }, + "value": { + "_choice": "printableString", + "value": "STM TPM EK Intermediate CA 05" + } + } + ] + ] + }, + "validity": { + "_type": "Validity", + "notBefore": { + "_choice": "utcTime", + "value": "2018-12-14T00:00:00Z" + }, + "notAfter": { + "_choice": "utcTime", + "value": "2028-12-14T00:00:00Z" + } + }, + "subject": { + "_choice": "rdnSequence", + "value": [] + }, + "subjectPublicKeyInfo": { + "_type": "SubjectPublicKeyInfo", + "algorithm": { + "_type": "AlgorithmIdentifier", + "algorithm": { + "_type": "OBJECT IDENTIFIER", + "oid": "1.2.840.113549.1.1.1", + "components": [ + 1, + 2, + 840, + 113549, + 1, + 1, + 1 + ], + "name": "id-pkcs1-rsaEncryption" + }, + "parameters": "0500" + }, + "subjectPublicKey": "2160:3082010A0282010100CC14EB27A78CEB0EA486FA2DF7835F5FA8E905B097012B5BDE50380C355B1A2A721BBC3D08DD21796CDB239FA95310651B1B56FD2CFE53C87352EBD996E33256160404CE9302A08066801E786A2F86E181F949966F492A85B58EAA4A6A8CB3697551BB236E87CC7BF8EC1347871C91E15437E8F266BF1EA5EB271FDCF374D8B47DF8BCE89E1FAD61C2A088CB4036B359CB72A294973FEDCCF0C340AFFD14B64F041165581ACA34147C1C75617047058F7ED7D603E032508094FA73E8B9153DA3BF255D2CBBC5DF301BA8F74D198BEBCE86040FC1D2927C7657414490D802F482F3EBF2DE35EE149A1A6DE8D16891FBFBA02A18AFE59F9D6F149744E5F0D559B10203010001" + }, + "issuerUniqueID": null, + "subjectUniqueID": null, + "extensions": [ + { + "_type": "Extension", + "extnID": { + "_type": "OBJECT IDENTIFIER", + "oid": "2.5.29.35", + "components": [ + 2, + 5, + 29, + 35 + ], + "name": "id-x509-ce-authorityKeyIdentifier" + }, + "critical": false, + "extnValue": "301680141ADB994AB58BE57A0CC9B900E7851E1A43C08660", + "_extnValue_choice": "ext-AuthorityKeyIdentifier", + "_extnValue": { + "_type": "AuthorityKeyIdentifier", + "keyIdentifier": "1ADB994AB58BE57A0CC9B900E7851E1A43C08660", + "authorityCertIssuer": null, + "authorityCertSerialNumber": null + } + }, + { + "_type": "Extension", + "extnID": { + "_type": "OBJECT IDENTIFIER", + "oid": "2.5.29.32", + "components": [ + 2, + 5, + 29, + 32 + ], + "name": "id-x509-ce-certificatePolicies" + }, + "critical": false, + "extnValue": "303930370604551D2000302F302D06082B060105050702011621687474703A2F2F7777772E73742E636F6D2F54504D2F7265706F7369746F72792F", + "_extnValue_choice": "ext-CertificatePolicies", + "_extnValue": [ + { + "_type": "PolicyInformation", + "policyIdentifier": { + "_type": "OBJECT IDENTIFIER", + "oid": "2.5.29.32.0", + "components": [ + 2, + 5, + 29, + 32, + 0 + ], + "name": "id-x509-ce-certificatePolicies-anyPolicy" + }, + "policyQualifiers": [ + { + "_type": "PolicyQualifierInfo", + "policyQualifierId": { + "_type": "OBJECT IDENTIFIER", + "oid": "1.3.6.1.5.5.7.2.1", + "components": [ + 1, + 3, + 6, + 1, + 5, + 5, + 7, + 2, + 1 + ], + "name": "id-pkix-qt-cps" + }, + "qualifier": "1621687474703A2F2F7777772E73742E636F6D2F54504D2F7265706F7369746F72792F" + } + ] + } + ] + }, + { + "_type": "Extension", + "extnID": { + "_type": "OBJECT IDENTIFIER", + "oid": "2.5.29.17", + "components": [ + 2, + 5, + 29, + 17 + ], + "name": "id-x509-ce-subjectAltName" + }, + "critical": true, + "extnValue": "304DA44B304931163014060567810502010C0B69643A353335343444323031173015060567810502020C0C53543333485450484148433031163014060567810502030C0B69643A3030343930303038", + "_extnValue_choice": "ext-SubjectAltName", + "_extnValue": [ + { + "_choice": "directoryName", + "value": { + "_choice": "rdnSequence", + "value": [ + [ + { + "_type": "AttributeTypeAndValue", + "type": { + "_type": "OBJECT IDENTIFIER", + "oid": "2.23.133.2.1", + "components": [ + 2, + 23, + 133, + 2, + 1 + ], + "name": "tcg-at-tpmManufacturer" + }, + "value": { + "_choice": "utf8String", + "value": "id:53544D20" + } + } + ], + [ + { + "_type": "AttributeTypeAndValue", + "type": { + "_type": "OBJECT IDENTIFIER", + "oid": "2.23.133.2.2", + "components": [ + 2, + 23, + 133, + 2, + 2 + ], + "name": "tcg-at-tpmModel" + }, + "value": { + "_choice": "utf8String", + "value": "ST33HTPHAHC0" + } + } + ], + [ + { + "_type": "AttributeTypeAndValue", + "type": { + "_type": "OBJECT IDENTIFIER", + "oid": "2.23.133.2.3", + "components": [ + 2, + 23, + 133, + 2, + 3 + ], + "name": "tcg-at-tpmVersion" + }, + "value": { + "_choice": "utf8String", + "value": "id:00490008" + } + } + ] + ] + } + } + ] + }, + { + "_type": "Extension", + "extnID": { + "_type": "OBJECT IDENTIFIER", + "oid": "2.5.29.9", + "components": [ + 2, + 5, + 29, + 9 + ], + "name": "id-x509-ce-subjectDirectoryAttributes" + }, + "critical": false, + "extnValue": "305E301706056781050210310E300C0C03322E300201000202008A304306056781050212313A30380201000101FFA0030A0101A1030A0100A2030A0100A310300E1603332E310A01040A01020101FFA40F300D16053134302D320A0102010100", + "_extnValue_choice": "ext-SubjectDirectoryAttributes", + "_extnValue": [ + { + "_type": "AttributeSet", + "type": { + "_type": "OBJECT IDENTIFIER", + "oid": "2.23.133.2.16", + "components": [ + 2, + 23, + 133, + 2, + 16 + ], + "name": "tcg-at-tpmSpecification" + }, + "values": [ + "300C0C03322E300201000202008A" + ], + "_values_choice": "at-TPMSpecification", + "_values": [ + { + "_type": "TPMSpecification", + "family": "2.0", + "level": 0, + "revision": 138 + } + ] + }, + { + "_type": "AttributeSet", + "type": { + "_type": "OBJECT IDENTIFIER", + "oid": "2.23.133.2.18", + "components": [ + 2, + 23, + 133, + 2, + 18 + ], + "name": "tcg-at-tpmSecurityAssertions" + }, + "values": [ + "30380201000101FFA0030A0101A1030A0100A2030A0100A310300E1603332E310A01040A01020101FFA40F300D16053134302D320A0102010100" + ], + "_values_choice": "at-TPMSecurityAssertions", + "_values": [ + { + "_type": "TPMSecurityAssertions", + "version": 0, + "fieldUpgradable": true, + "ekGenerationType": "ekgt-injected", + "ekGenerationLocation": "tpmManufacturer", + "ekCertificateGenerationLocation": "tpmManufacturer", + "ccInfo": { + "_type": "CommonCriteriaMeasures", + "version": "3.1", + "assurancelevel": "ealevel4", + "evaluationStatus": "evaluationCompleted", + "plus": true, + "strengthOfFunction": null, + "profileOid": null, + "profileUri": null, + "targetOid": null, + "targetUri": null + }, + "fipsLevel": { + "_type": "FIPSLevel", + "version": "140-2", + "level": "sllevel2", + "plus": false + }, + "iso9000Certified": false, + "iso9000Uri": null + } + ] + } + ] + }, + { + "_type": "Extension", + "extnID": { + "_type": "OBJECT IDENTIFIER", + "oid": "2.5.29.15", + "components": [ + 2, + 5, + 29, + 15 + ], + "name": "id-x509-ce-keyUsage" + }, + "critical": true, + "extnValue": "03020520", + "_extnValue_choice": "ext-KeyUsage", + "_extnValue": [ + "keyEncipherment" + ] + }, + { + "_type": "Extension", + "extnID": { + "_type": "OBJECT IDENTIFIER", + "oid": "2.5.29.19", + "components": [ + 2, + 5, + 29, + 19 + ], + "name": "id-x509-ce-basicConstraints" + }, + "critical": true, + "extnValue": "3000", + "_extnValue_choice": "ext-BasicConstraints", + "_extnValue": { + "_type": "BasicConstraints", + "cA": false, + "pathLenConstraint": null + } + }, + { + "_type": "Extension", + "extnID": { + "_type": "OBJECT IDENTIFIER", + "oid": "2.5.29.37", + "components": [ + 2, + 5, + 29, + 37 + ], + "name": "id-x509-ce-extKeyUsage" + }, + "critical": false, + "extnValue": "300706056781050801", + "_extnValue_choice": "ext-ExtKeyUsage", + "_extnValue": [ + { + "_type": "OBJECT IDENTIFIER", + "oid": "2.23.133.8.1", + "components": [ + 2, + 23, + 133, + 8, + 1 + ], + "name": "tcg-kp-EKCertificate" + } + ] + }, + { + "_type": "Extension", + "extnID": { + "_type": "OBJECT IDENTIFIER", + "oid": "1.3.6.1.5.5.7.1.1", + "components": [ + 1, + 3, + 6, + 1, + 5, + 5, + 7, + 1, + 1 + ], + "name": "id-pkix-pe-authorityInfoAccess" + }, + "critical": false, + "extnValue": "303C303A06082B06010505073002862E687474703A2F2F7365637572652E676C6F62616C7369676E2E636F6D2F73746D74706D656B696E7430352E637274", + "_extnValue_choice": "ext-AuthorityInfoAccess", + "_extnValue": [ + { + "_type": "AccessDescription", + "accessMethod": { + "_type": "OBJECT IDENTIFIER", + "oid": "1.3.6.1.5.5.7.48.2", + "components": [ + 1, + 3, + 6, + 1, + 5, + 5, + 7, + 48, + 2 + ], + "name": "id-pkix-ad-caIssuers" + }, + "accessLocation": { + "_choice": "uniformResourceIdentifier", + "value": "http://secure.globalsign.com/stmtpmekint05.crt" + } + } + ] + } + ] + }, + "signatureAlgorithm": { + "_type": "AlgorithmIdentifier", + "algorithm": { + "_type": "OBJECT IDENTIFIER", + "oid": "1.2.840.113549.1.1.11", + "components": [ + 1, + 2, + 840, + 113549, + 1, + 1, + 11 + ], + "name": "id-pkcs1-sha256WithRSAEncryption" + }, + "parameters": "0500" + }, + "signatureValue": "2048:3D4C381E5B4F1BCBE09C63D52F1F04570CAEA142FD9CD942043B11F8E3BDCF50007AE16CF8869013041E92CDD3280BA4B51FBBD40582ED750219E261A695095674855AACEB520ADAFF9E7E908480A39CDCF900462D9171960FFE55D3AC49E8C981341BBD2EFBCC252A4C18A4F3B7C84CCE42CE70A208C84D2630A7ABFBE72D6271E75B9FF1C971D20EB3DBD763F1E04D834EAA692D2E4001BBF4730A3E3FDA9711AE386524D91C63BE0E516D00D5C6141FCCF6C539F3518E180049865BE16B69CAE1F8CB7FDC474B38F7EE56CBE7D8A89D9BA99B65D5265AEF32AA62426B10E6D75BB8677EC44F755BBC2806FD2B4E04BDF5D44259DBEAA42B6F563DF7AA7506" + } + ``` + + (Notice that OID names look a bit weird. For reasons that may have been + lost to time and may no longer be relevant, these OIDs are defined with + slightly different names in the ASN.1 modules in Heimdal's source tree. + We'll fix this eventually.) + +... + +## Limitations + + - `libasn1`'s and, therefore, `asn1_print`'s JSON support is not X.697 (JER) + compatible. + + - Control over C types generated is very limited, mainly only for integer + types. + + - When using the template backend, `SET { .. }` types are currently not sorted + by tag as they should be, but if the module author sorts them by hand then + DER will be produced. + + - `REAL` is not supported. + + - `EmbeddedPDV` is not supported. + + - `BMPString` is not supported. + + - IA5String is not properly supported -- it's essentially treated as a + `UTF8String` with a different tag. This is true of all the string types. + + - Only types can be imported at this time. Without some rototilling we likely + will not be able to import anything other than types, values, and object + sets. + + - Only simple value syntax is supported. Constructed value syntax (i.e., + values of `SET`, `SEQUENCE`, `SET OF`, and `SEQUENCE OF`), is not supported. + Values of `CHOICE` types are also not supported. + + - There is no way to substitute object sets at run-time. This means that + automatic decoding through open types will spend more CPU cycles than the + application might want, by decoding more types than the application might + care about. The ability to substitute object sets at run-time would require + a change to the APIs generated. + + - ... + +## Compiler Usage + +The various options for the Heimdal ASN.1 compiler are described in its manual +page, which is included below. + +The `--option-file=FILE` option is particularly useful, as it allows additional +compiler options to be read from a file. + +The `--preserve-binary=TYPE-NAME` option is critical for signature validation +as it causes the decoder to save the encoding of the given type so that +signature validation code can easily find the original encoding and thus avoid +having to re-encode or resort to other hacks. E.g., we use this for preserving +the original encoding of the `tbsCertificate` field of `Certificate`. + +The `--sequence=TYPE-NAME` causes the compiler to generate additional utility +functions for adding or removing items from the named type when it is a +`SEQUENCE OF` or `SET OF` type. + +See the manual page `asn1_compile.1`: + +```text +ASN1_COMPILE(1) BSD General Commands Manual ASN1_COMPILE(1) + +NAME + asn1_compile — compile ASN.1 modules + +SYNOPSIS + asn1_compile [--template] [--prefix-enum] [--enum-prefix=PREFIX] + [--encode-rfc1510-bit-string] [--decode-dce-ber] + [--support-ber] [--preserve-binary=TYPE] [--sequence=TYPE] + [--decorate=DECORATION] [--one-code-file] [--gen-name=NAME] + [--option-file=FILE] [--original-order] [--no-parse-units] + [--type-file=C-HEADER-FILE] [--version] [--help] + [FILE.asn1 [NAME]] + +DESCRIPTION + asn1_compile compiles an ASN.1 module into C source code and header + files. + + A fairly large subset of ASN.1 as specified in X.680, and the ASN.1 In‐ + formation Object System as specified in X.681, X.682, and X.683 is sup‐ + ported, with support for the Distinguished Encoding Rules (DER), partial + Basic Encoding Rules (BER) support, and experimental JSON support (encod‐ + ing only at this time). + + See the compiler's README files for details about the C code and inter‐ + faces it generates. + + The Information Object System support includes automatic codec support + for encoding and decoding through “open types” which are also known as + “typed holes”. See RFC 5912 for examples of how to use the ASN.1 Infor‐ + mation Object System via X.681/X.682/X.683 annotations. See the com‐ + piler's README files for more information on ASN.1 Information Object + System support. + + Extensions specific to Heimdal are generally not syntactic in nature but + rather command-line options to this program. For example, one can use + command-line options to: + • enable decoding of BER-encoded values; + • enable RFC1510-style handling of ‘BIT STRING’ types; + • enable saving of as-received encodings of specific types + for the purpose of signature validation; + • generate add/remove utility functions for array types; + • decorate generated ‘struct’ types with fields that are nei‐ + ther encoded nor decoded; + etc. + + ASN.1 x.680 features supported: + • most primitive types (except BMPString and REAL); + • all constructed types, including SET and SET OF; + • explicit and implicit tagging. + + Size and range constraints on the ‘INTEGER’ type cause the compiler to + generate appropriate C types such as ‘int’, ‘unsigned int’, ‘int64_t’, + ‘uint64_t’. Unconstrained ‘INTEGER’ is treated as ‘heim_integer’, which + represents an integer of arbitrary size. + + Caveats and ASN.1 x.680 features not supported: + • JSON encoding support is not quite X.697 (JER) compatible. + Its JSON schema is subject to change without notice. + • Control over C types generated is very limited, mainly only + for integer types. + • When using the template backend, `SET { .. }` types are + currently not sorted by tag as they should be, but if the + module author sorts them by hand then correct DER will be + produced. + • ‘AUTOMATIC TAGS’ is not supported. + • The REAL type is not supported. + • The EmbeddedPDV type is not supported. + • The BMPString type is not supported. + • The IA5String is not properly supported, as it's essen‐ + tially treated as a UTF8String with a different tag. + • All supported non-octet strings are treated as like the + UTF8String type. + • Only types can be imported into ASN.1 modules at this time. + • Only simple value syntax is supported. Constructed value + syntax (i.e., values of SET, SEQUENCE, SET OF, and SEQUENCE + OF types), is not supported. Values of `CHOICE` types are + also not supported. + + Options supported: + + --template + Use the “template” backend instead of the “codegen” backend + (which is the default backend). + + The template backend generates “templates” which are akin to + bytecode, and which are interpreted at run-time. + + The codegen backend generates C code for all functions directly, + with no template interpretation. + + The template backend scales better than the codegen backend be‐ + cause as we add support for more encoding rules and more opera‐ + tions (we may add value comparators) the templates stay mostly + the same, thus scaling linearly with size of module. Whereas the + codegen backend scales linear with the product of module size and + number of encoding rules supported. + + --prefix-enum + This option should be removed because ENUMERATED types should al‐ + ways have their labels prefixed. + + --enum-prefix=PREFIX + This option should be removed because ENUMERATED types should al‐ + ways have their labels prefixed. + + --encode-rfc1510-bit-string + Use RFC1510, non-standard handling of “BIT STRING” types. + + --decode-dce-ber + + --support-ber + + --preserve-binary=TYPE + Generate a field named ‘_save’ in the C struct generated for the + named TYPE. This field is used to preserve the original encoding + of the value of the TYPE. + + This is useful for cryptographic applications so that they can + check signatures of encoded values as-received without having to + re-encode those values. + + For example, the TBSCertificate type should have values preserved + so that Certificate validation can check the signatureValue over + the tbsCertificate's value as-received. + + The alternative of encoding a value to check a signature of it is + brittle. For types where non-canonical encodings (such as BER) + are allowed, this alternative is bound to fail. Thus the point + of this option. + + --sequence=TYPE + Generate add/remove functions for the named ASN.1 TYPE which must + be a ‘SET OF’ or ‘SEQUENCE OF’ type. + + --decorate=ASN1-TYPE:FIELD-ASN1-TYPE:fname[?] + Add to the C struct generated for the given ASN.1 SET, SEQUENCE, + or CHOICE type named ASN1-TYPE a “hidden” field named fname of + the given ASN.1 type FIELD-ASN1-TYPE, but do not encode or decode + it. If the fname ends in a question mark, then treat the field + as OPTIONAL. + + This is useful for adding fields to existing types that can be + used for internal bookkeeping but which do not affect interoper‐ + ability because they are neither encoded nor decoded. For exam‐ + ple, one might decorate a request type with state needed during + processing of the request. + + --decorate=ASN1-TYPE:void*:fname + Add to the C struct generated for the given ASN.1 SET, SEQUENCE, + or CHOICE type named ASN1-TYPE a “hidden” field named fname of + type ‘void *’ (but do not encode or decode it. + + The destructor and copy constructor functions generated by this + compiler for ASN1-TYPE will set this field to the ‘NULL’ pointer. + + --decorate=ASN1-TYPE:FIELD-C-TYPE:fname[?]:[copyfn]:[freefn]:header + Add to the C struct generated for the given ASN.1 SET, SEQUENCE, + or CHOICE type named ASN1-TYPE a “hidden” field named fname of + the given external C type FIELD-C-TYPE, declared in the given + header but do not encode or decode this field. If the fname ends + in a question mark, then treat the field as OPTIONAL. + + The header must include double quotes or angle brackets. The + copyfn must be the name of a copy constructor function that takes + a pointer to a source value of the type, and a pointer to a des‐ + tination value of the type, in that order, and which returns zero + on success or else a system error code on failure. The freefn + must be the name of a destructor function that takes a pointer to + a value of the type and which releases resources referenced by + that value, but does not free the value itself (the run-time al‐ + locates this value as needed from the C heap). The freefn should + also reset the value to a pristine state (such as all zeros). + + If the copyfn and freefn are empty strings, then the decoration + field will neither be copied nor freed by the functions generated + for the TYPE. + + --one-code-file + Generate a single source code file. Otherwise a separate code + file will be generated for every type. + + --gen-name=NAME + Use NAME to form the names of the files generated. + + --option-file=FILE + Take additional command-line options from FILE. + + --original-order + Attempt to preserve the original order of type definition in the + ASN.1 module. By default the compiler generates types in a topo‐ + logical sort order. + + --no-parse-units + Do not generate to-int / from-int functions for enumeration + types. + + --type-file=C-HEADER-FILE + Generate an include of the named header file that might be needed + for common type defintions. + + --version + + --help + +NOTES + Currently only the template backend supports automatic encoding and de‐ + coding of open types via the ASN.1 Information Object System and + X.681/X.682/X.683 annotations. + +HEIMDAL February 22, 2021 HEIMDAL +``` + +## APIs Generated by the Compiler + +Every named type in an ASN.1 module gets a corresponding type in C. +Em-dashes in symbols become underscores. + +Every named type in an ASN.1 module also gets several functions generated +associated with it: + + - `int decode_TypeName(const unsigned char *, size_t, TypeName *, size_t *);` + + Decodes a value of `TypeName` in the given byte array of the given size, + into the given `TypeName` object, and outputs the number of bytes parsed. + + Returns 0 on success, or an error that can be formatted as a string using + the `com_err` library. + + - `int encode_TypeName(unsigned char *, size_t, const TypeName *, size_t *);` + + Encodes the given object of `TypeName` type into the given byte array of the + given size, outputting the number of bytes used. + + NOTE WELL: the `unsigned char *` pointer must point to the _last_ byte of + the buffer! + + Returns 0 on success, or an error that can be formatted as a string using + the `com_err` library, including system errors such as `ENOMEM`. + + - `int length_TypeName(const TypeName *);` + + Returns the number of bytes needed to encode the given object. + + - `void free_TypeName(TypeName *);` + + Releases the memory associated with the content of the given object, but + note that the object itself is _not_ released. + + - `int copy_TypeName(const TypeName *, TypeName *);` + + Copies the content of the given `const` object to the destination, + non-`const` object. + + Returns 0 on success, or an error that can be formatted as a string using + the `com_err` library, including system errors such as `ENOMEM`. + + - `char *print_TypeName(const TypeName *, int);` + + Returns a string (JSON) representation of the given object. The `int` + argument is a bitset of flags: + + - `ASN1_PRINT_INDENT` + + Indent the JSON. If not given the the JSON will have no interstitial + whitespace, including newlines. + +You will want to review the data structures generated. They look like: + +```C + typedef struct TBSCertificate TBSCertificate; + typedef struct AlgorithmIdentifier AlgorithmIdentifier; + typedef struct ... ...; + + /* + Certificate ::= SEQUENCE { + tbsCertificate TBSCertificate, + signatureAlgorithm AlgorithmIdentifier, + signatureValue BIT STRING { + }, + } + */ + + typedef struct Certificate { + TBSCertificate tbsCertificate; + AlgorithmIdentifier signatureAlgorithm; + heim_bit_string signatureValue; + } Certificate; + + /* + TBSCertificate ::= SEQUENCE { + version [0] Version OPTIONAL, + serialNumber CertificateSerialNumber, + signature AlgorithmIdentifier, + issuer Name, + validity Validity, + subject Name, + subjectPublicKeyInfo SubjectPublicKeyInfo, + issuerUniqueID [1] IMPLICIT BIT STRING { + } OPTIONAL, + subjectUniqueID [2] IMPLICIT BIT STRING { + } OPTIONAL, + extensions [3] Extensions OPTIONAL, + } + */ + + typedef struct TBSCertificate { + heim_octet_string _save; + Version *version; + CertificateSerialNumber serialNumber; + AlgorithmIdentifier signature; + Name issuer; + Validity validity; + Name subject; + SubjectPublicKeyInfo subjectPublicKeyInfo; + heim_bit_string *issuerUniqueID; + heim_bit_string *subjectUniqueID; + Extensions *extensions; + } TBSCertificate; +``` + +Note how trivial the mapping onto C is. + +`OPTIONAL` fields become pointer fields, with `NULL` indicating +absence and non-NULL indicating presence. + +And so on. + +## `asn1_print` Usage + +```text +ASN1_PRINT(1) BSD General Commands Manual ASN1_PRINT(1) + +NAME + asn1_print — dump ASN.1 DER encoded values + +SYNOPSIS + asn1_print [-i | --no-indent] [-I | --inner] [-l | --list-types] + [-A | --try-all-types] [-S | --raw-sequence] [-n | --no-print] + [-q | --quiet] [--test-encode] [--test-copy] + [-l -v | --version] [-l -h | --help] [FILE [TypeName...]] + +DESCRIPTION + asn1_print Dumps ASN.1 DER-encoded values. If one or more TypeName argu- + ments are given, then asn1_print will print the value in a JSON-like for- + mat using its knowledge of the ASN.1 modules defining those types, stop- + ping at the first type for which it can successfully decode the value. + If TypeNames are given, they must be the names of ASN.1 types exported by + an ASN.1 modules that are compiled into asn1_print. Use the + --try-all-types option to attempt decoding as all ASN.1 types known to + asn1_print. If neither any TypeName nor --try-all-types are given, then + the value will be parsed and displayed using just the self-describing + nature of DER. + + Options supported: + + -i, --no-indent + Do not indent dump. + + -I, --inner + Try to parse inner structures of OCTET STRING and constructed + values. + + -l, --list-types + List all types known to asn1_print. + + -A, --try-all-types + Attempt to decode the value as any of all types known to + asn1_print. + + -S, --raw-sequence + If a value parses as a given TypeName but any bytes are left + over, try to parse those separately as well until all bytes are + consumed or an error occurs. + + -n, --no-print + For the case where -A or --try-all-types or where a TypeName is + given, do not output a JSON representation of the value, just + attempt to decode it. This is useful for fuzzing. + + -q, --quiet + Similar to -n, --no-print but JSON output will be formatted, just + not output. As with -n, --no-print, this option requires -A -/ + --try-all-types or that a TypeName be given. This is useful for + fuzzing. + + --test-encode + Check that encoding produces the same value as decoding. Useful + for fuzzing. + + --test-copy + Test copy functions. Useful for fuzzing. + + -v, --version + + -h, --help + +HEIMDAL February 22, 2021 HEIMDAL +``` + +## Implementation + +See: + + - `lib/asn1/main.c` for the `main()` function of the compiler + - `lib/asn1/asn1parse.y` for the grammar and most of the parsing into an AST + - `lib/asn1/symbol.h` for the types making up the AST + - `lib/asn1/{hash,symbol}.c` for compiler AST supporting functionality + - `lib/asn1/gen.c` for the C header file generator + - `lib/asn1/gen_template.c` for the template generator + - `lib/asn1/gen_{decode,encode,length,copy,free}.c` for the C code generator + - `lib/asn1/gen_{glue,seq}.c` for misc code generator + - `lib/asn1/template.c` for the template interpreter + - `lib/asn1/der*.c` for primitive type primitives + - `lib/asn1/extra.c` for primitives related to `ANY` + +... + +## Futures + + - Add JER support so we can convert between JER and DER? + + - Add XDR support? There are no ASN.1 Encoding Rules based on XDR, but it is + trivial to construct such for at least that subset of ASN.1 for which the + XDR syntax has equivalent semantics. + + - Add OER support? + + - Add NDR support? There are no ASN.1 Encoding Rules based on NDR, but it is + trivial to construct such for at least that subset of ASN.1 for which the + IDL syntax has equivalent semantics. + + - Perhaps third parties will contribute more control over generated types? + This may require separate publication of the Heimdal ASN.1 compiler from the + rest of Heimdal. + +## Moving From C + + - Generate and output a JSON representation of the compiled ASN.1 module. + + - Code codegen/templategen backends in jq or Haskell or whatever. + + - Code template interpreters in some host language. + + - Eventually rewrite the compiler itself in Rust or whatever. |