diff options
Diffstat (limited to 'security/nss/doc/rst/legacy/an_overview_of_nss_internals/index.rst')
| -rw-r--r-- | security/nss/doc/rst/legacy/an_overview_of_nss_internals/index.rst | 302 |
1 files changed, 302 insertions, 0 deletions
diff --git a/security/nss/doc/rst/legacy/an_overview_of_nss_internals/index.rst b/security/nss/doc/rst/legacy/an_overview_of_nss_internals/index.rst new file mode 100644 index 0000000000..7d705198a8 --- /dev/null +++ b/security/nss/doc/rst/legacy/an_overview_of_nss_internals/index.rst @@ -0,0 +1,302 @@ +.. _mozilla_projects_nss_an_overview_of_nss_internals: + +An overview of NSS Internals +============================ + +.. container:: + + | A High-Level Overview to the Internals of `Network Security Services + (NSS) <https://developer.mozilla.org/en-US/docs/NSS>`__ + | Software developed by the Mozilla.org projects traditionally used its own implementation of + security protocols and cryptographic algorithms, originally called Netscape Security Services, + nowadays called Network Security Services (NSS). NSS is a library written in the C programming + language. It's free and open source software, and many other software projects have decided to + use it. In order to support multiple operating systems (OS), it is based on a cross platform + portability layer, called the Netscape Portable Runtime (NSPR), which provides cross platform + application programming interfaces (APIs) for OS specific APIs like file system access, memory + management, network communication, and multithreaded programming. + | NSS offers lots of functionality; we'll walk through the list of modules, design principles, + and important relevant standards. + | In order to allow interoperability between software and devices that perform cryptographic + operations, NSS conforms to a standard called PKCS#11. (Note that it's important to look at the + number 11, as there are other PKCS standards with different numbers that define quite different + topics.) + | A software or hardware module conforming to the PKCS#11 standard implements an interface of C + calls, which allow querying the characteristics and offered services of the module. Multiple + elements of NSS's own modules have been implemented with this interface, and NSS makes use of + this interface when talking to those modules. This strategy allows NSS to work with many + hardware devices (e.g., to speed up the calculations required for cryptographic operations, or + to access smartcards that securely protect a secret key) and software modules (e.g., to allow + to load such modules as a plugin that provides additional algorithms or stores key or trust + information) that implement the PKCS#11 interface. + | A core element of NSS is FreeBL, a base library providing hash functions, big number + calculations, and cryptographic algorithms. + | Softoken is an NSS module that exposes most FreeBL functionality as a PKCS#11 module. + | Some cryptography uses the same secret key for both encrypting and decrypting, for example + password based encryption (PBE). This is often sufficient if you encrypt data for yourself, but + as soon as you need to exchange signed/encrypted data with communication partners, using public + key encryption simplifies the key management. The environment that describes how to use public + key encryption is called Public Key Infrastructure (PKI). The public keys that are exchanged + between parties are transported using a container; the container is called a certificate, + following standard X.509 version 3. A certificate contains lots of other details; for example, + it contains a signature by a third party that expresses trust in the ownership relationship for + the certificate. The trust assigned by the third party might be restricted to certain uses, + which are listed in certificate extensions that are contained in the certificate. + | Many (if not most) of the operations performed by NSS involve the use of X.509 certificates + (often abbreviated as “cert”, unfortunately making it easy to confuse with the term “computer + emergency response team“). + | When checking whether a certificate is trusted or not, it's necessary to find a relevant trust + anchor (root certificate) that represents the signing capability of a trusted third party, + usually called a Certificate Authority (CA). A trust anchor is just another X.509 certificate + that is already known and has been deliberately marked as trusted by a software vendor, + administrators inside an organizational infrastructure, or the software user. NSS ships a + predefined set of CA certificates. This set, including their trust assignments, is provided by + NSS as a software module, called CKBI (“built-in root certificates”), which also implements the + PKCS#11 interface. On an organizational level the contents of the set are managed according to + the Mozilla CA policy. On a technical level the set is a binary software module. + | A cryptographic transaction, such as encryption or decryption related to a data exchange, + usually involves working with the X.509 certs of your communication partners (peer). It's also + required that you safely keep your own secret keys that belong to your own certificates. You + might want to protect the storage of your secret keys with PBE. You might decide to modify the + default trust provided by NSS. All of this requires storing, looking up, and retrieving data. + NSS simplifies performing these operations by offering storage and management APIs. NSS doesn't + require the programmer to manage individual files containing individual certificates or keys. + Instead, NSS offers to use its own database(s). Once you have imported certificates and keys + into the NSS database, you can easily look them up and use them again. + | Because of NSS's expectation to operate with an NSS database, it's mandatory that you perform + an initialization call, where you tell NSS which database you will be using. In the most simple + scenario, the programmer will provide a directory on your filesystem as a parameter to the init + function, and NSS is designed to do the rest. It will detect and open an existing database, or + it can create a new one. Alternatively, should you decide that you don't want to work with any + persistent recording of certificates, you may initialize NSS in a no-database mode. Usually, + NSS will flush all data to disk as soon as new data has been added to permanent storage. + Storage consists of multiple files: a key database file, which contains your secret keys, and a + certificate database file which contains the public portion of your own certificates, the + certificates of peers or CAs, and a list of trust decisions (such as to not trust a built-in + CA, or to explicitly trust other CAs). Examples for the database files are key3.db and + cert8.db, where the numbers are file version numbers. A third file contains the list of + external PKCS#11 modules that have been registered to be used by NSS. The file could be named + secmod.db, but in newer database generations a file named pkcs11.txt is used. + | Only NSS is allowed to access and manipulate these database files directly; a programmer using + NSS must go through the APIs offered by NSS to manipulate the data stored in these files. The + programmer's task is to initialize NSS with the required parameters (such as a database), and + NSS will then transparently manage the database files. + | Most of the time certificates and keys are supposed to be stored in the NSS database. + Therefore, after initial import or creation, the programmer usually doesn't deal with their raw + bytes. Instead, the programmer will use lookup functions, and NSS will provide an access handle + that will be subsequently used by the application's code. Those handles are reference counted. + NSS will usually create an in-memory (RAM) presentation of certificates, once a certificate has + been received from the network, read from disk, or looked up from the database, and prepare + in-memory data structures that contain the certificate's properties, as well as providing a + handle for the programmer to use. Once the application is done with a handle, it should be + released, allowing NSS to free the associated resources. When working with handles to private + keys it's usually difficult (and undesired) that an application gets access to the raw key + data; therefore it may be difficult to extract such data from NSS. The usual minimum + requirement is that private keys must be wrapped using a protective layer (such as + password-based encryption). The intention is to make it easier to review code for security. The + less code that has access to raw secret keys, the less code that must be reviewed. + | NSS has only limited functionality to look up raw keys. The preferred approach is to use + certificates, and to look up certificates by properties such as the contained subject name + (information that describes the owner of the certificate). For example, while NSS supports + random calculation (creation) of a new public/private key pair, it's difficult to work with + such a raw key pair. The usual approach is to create a certificate signing request (CSR) as + soon as an application is done with the creation step, which will have created a handle to the + key pair, and which can be used for the necessary related operations, like producing a + proof-of-ownership of the private key, which is usually required when submitting the public key + with a CSR to a CA. The usual follow up action is receiving a signed certificate from a CA. + (However, it's also possible to use NSS functionality to create a self-signed certificate, + which, however, usually won't be trusted by other parties.) Once received, it's sufficient to + tell NSS to import such a new certificate into the NSS database, and NSS will automatically + perform a lookup of the embedded public key, be able to find the associated private key, and + subsequently be able to treat it as a personal certificate. (A personal certificate is a + certificate for which the private key is in possession, and which could be used for signing + data or for decrypting data.) A unique nickname can/should be assigned to the certificate at + the time of import, which can later be used to easily identify and retrieve it. + | It's important to note that NSS requires strict cleanup for all handles returned by NSS. The + application should always call the appropriate dereference (destroy) functions once a handle is + no longer needed. This is particularly important for applications that might need to close a + database and reinitialize NSS using a different one, without restarting. Such an operation + might fail at runtime if data elements are still being referenced. + | In addition to the FreeBL, Softoken, and CKBI modules, there is an utility library for general + operations (e.g., encoding/decoding between data formats, a list of standardized object + identifiers (OID)). NSS has an SSL/TLS module that implements the Secure Sockets + Layer/Transport Layer Security network protocols, an S/MIME module that implements CMS + messaging used by secure email and some instant messaging implementations, a DBM library that + implements the classic database storage, and finally a core NSS library for the big set of + “everything else”. Newer generations of the database use the SQLite database to allow + concurrent access by multiple applications. + | All of the above are provided as shared libraries. The CRMF library, which is used to produce + certain kinds of certificate requests, is available as a library for static linking only. + | When dealing with certificates (X.509), file formats such as PKCS#12 (certificates and keys), + PKCS#7 (signed data), and message formats as CMS, we should mention ASN.1, which is a syntax + for storing structured data in a very efficient (small sized) presentation. It was originally + developed for telecommunication systems at times where it was critical to minimize data as much + as possible (although it still makes sense to use that principle today for good performance). + In order to process data available in the ASN.1 format, the usual approach is to parse it and + transfer it to a presentation that requires more space but is easier to work with, such as + (nested) C data structures. Over the time NSS has received three different ASN.1 parser + implementations, each having their own specific properties, advantages and disadvantages, which + is why all of them are still being used (nobody has yet dared to replace the older with the + newer ones because of risks for side effects). When using the ASN.1 parser(s), a template + definition is passed to the parser, which will analyze the ASN.1 data stream accordingly. The + templates are usually closely aligned to definitions found in RFC documents. + | A data block described as DER is usually in ASN.1 format. You must know which data you are + expecting, and use the correct template for parsing, based on the context of your software's + interaction. Data described as PEM is a base64 encoded presentation of DER, usually wrapped + between human readable BEGIN/END lines. NSS prefers the binary presentation, but is often + capable to use base64 or ASCII presentations, especially when importing data from files. A + recent development adds support for loading external PEM files that contain private keys, in a + software library called nss-pem, which is separately available, but should eventually become a + core part of NSS. + | Looking at the code level, NSS deals with blocks of raw data all the time. The common structure + to store such an untyped block is SECItem, which contains a size and an untyped C pointer + variable. + | When dealing with memory, NSS makes use of arenas, which are an attempt to simplify management + with the limited offerings of C (because there are no destructors). The idea is to group + multiple memory allocations in order to simplify cleanup. Performing an operation often + involves allocating many individual data items, and the code might be required to abort a task + at many positions in the logic. An arena is requested once processing of a task starts, and all + memory allocations that are logically associated to that task are requested from the associated + arena. The implementation of arenas makes sure that all individual memory blocks are tracked. + Once a task is done, regardless whether it completed or was aborted, the programmer simply + needs to release the arena, and all individually allocated blocks will be released + automatically. Often freeing is combined with immediately erasing (zeroing, zfree) the memory + associated to the arena, in order to make it more difficult for attackers to extract keys from + a memory dump. + | NSS uses many C data structures. Often NSS has multiple implementations for the same or similar + concepts. For example, there are multiple presentations of certificates, and the NSS internals + (and sometimes even the application using NSS) might have to convert between them. + | Key responsibilites of NSS are verification of signatures and certificates. In order to verify + a digital signature, we have to look at the application data (e.g., a document that was + signed), the signature data block (the digital signature), and a public key (as found in a + certificate that is believed to be the signer, e.g., identified by metadata received together + with the signature). The signature is verified if it can be shown that the signature data block + must have been produced by the owner of the public key (because only that owner has the + associated private key). + | Verifying a certificate (A) requires some additional steps. First, you must identify the + potential signer (B) of a certificate (A). This is done by reading the “issuer name” attribute + of a certificate (A), and trying to find that issuer certificate (B) (by looking for a + certificate that uses that name as its “subject name”). Then you attempt to verify the + signature found in (A) using the public key found in (B). It might be necessary to try multiple + certificates (B1, B2, ...) each having the same subject name. + | After succeeding, it might be necessary to repeat this procedure recursively. The goal is to + eventually find a certificate B (or C or ...) that has an appropriate trust assigned (e.g., + because it can be found in the CKBI module and the user hasn't made any overriding trust + decisions, or it can be found in a NSS database file managed by the user or by the local + environment). + | After having successfully verified the signatures in a (chain of) issuer certificate(s), we're + still not done with verifying the certificate A. In a PKI it's suggested/required to perform + additional checks. For example: Certificates were valid at the time the signature was made, + name in certificates matches the expected signer (check subject name, common name, email, based + on application), the trust restrictions recorded inside the certificate (extensions) permit the + use (e.g., encryption might be allowed, but not signing), and based on environment/application + policy it might be required to perform a revocation check (OCSP or CRL), that asks the + issuer(s) of the certificates whether there have been events that made it necessary to revoke + the trust (revoke the validity of the cert). + | Trust anchors contained in the CKBI module are usually self signed, which is defined as having + identical subject name and issuer name fields. If a self-signed certificate is marked as + explicitly trusted, NSS will skip checking the self-signature for validity. + | NSS has multiple APIs to perform verification of certificates. There is a classic engine that + is very stable and works fine in all simple scenarios, for example if all (B) candidate issuer + certificates have the same subject and issuer names and differ by validity period; however, it + works only in a limited amount of more advanced scenarios. Unfortunately, the world of + certificates has become more complex in the recent past. New Certificate Authorities enter the + global PKI market, and in order to get started with their business, they might make deals with + established CAs and receive so-called cross-signing-certificates. As a result, when searching + for a trust path from (A) to a trusted anchor (root) certificate (Z), the set of candidate + issuer certificates might have different issuer names (referring to the second or higher issuer + level). As a consequence, it will be necessary to try multiple different alternative routes + while searching for (Z), in a recursive manner. Only the newer verification engine (internally + named libPKIX) is capable of doing that properly. + | It's worth mentioning the Extended Validation (EV) principle, which is an effort by software + vendors and CAs to define a stricter set of rules for issuing certificates for web site + certificates. Instead of simply verifying that the requester of a certificate is in control of + an administrative email address at the desired web site's domain, it's required that the CA + performs a verification of real world identity documents (such as a company registration + document with the country's authority), and it's also required that a browser software performs + a revocation check with the CA, prior to granting validity to the certificate. In order to + distinguish an EV certificate, CAs will embed a policy OID in the certificate, and the browser + is expected to verify that a trust chain permits the end entity (EE) certificate to make use of + the policy. Only the APIs of the newer libPKIX engine are capable of performing a policy + verification. + | That's a good opportunity to talk about SSL/TLS connections to servers in general (not just EV, + not just websites). Whenever this document mentions SSL, it refers to either SSL or TLS. (TLS + is a newer version of SSL with enhanced features.) + | When establishing an SSL connection to a server, (at least) a server certificate (and its trust + chain) is exchanged from the server to the client (e.g., the browser), and the client verifies + that the certificate can be verified (including matching the name of the expected destination + server). Another part of the handshake between both parties is a key exchange. Because public + key encryption is more expensive (more calculations required) than symmetric encryption (where + both parties use the same key), a key agreement protocol will be executed, where the public and + private keys are used to proof and verify the exchanged initial information. Once the key + agreement is done, a symmetric encryption will be used (until a potential re-handshake on an + existing channel). The combination of the hash and encryption algorithms used for a SSL + connection is called a cipher suite. + | NSS ships with a set of cipher suites that it supports at a technical level. In addition, NSS + ships with a default policy that defines which cipher suites are enabled by default. An + application is able to modify the policy used at program runtime, by using function calls to + modify the set of enabled cipher suites. + | If a programmer wants to influence how NSS verifies certificates or how NSS verifies the data + presented in a SSL connection handshake, it is possible to register application-defined + callback functions which will be called by NSS at the appropriate point of time, and which can + be used to override the decisions made by NSS. + | If you would like to use NSS as a toolkit that implements SSL, remember that you must init NSS + first. But if you don't care about modifying the default trust permanently (recorded on disk), + you can use the no-database init calls. When creating the network socket for data exchange, + note that you must use the operating system independent APIs provided by NSPR and NSS. It might + be interesting to mention a property of the NSPR file descriptors, which are stacked in layers. + This means you can define multiple layers that are involved in data processing. A file + descriptor has a pointer to the first layer handling the data. That layer has a pointer to a + potential second layer, which might have another pointer to a third layer, etc. Each layer + defines its own functions for the open/close/read/write/poll/select (etc.) functions. When + using an SSL network connection, you'll already have two layers, the basic NSPR layer and an + SSL library layer. The Mozilla applications define a third layer where application specific + processing is performed. You can find more details in the NSPR reference documents. + | NSS occassionally has to create outbound network connections, in addition to the connections + requested by the application. Examples are retrieving OCSP (Online Certificate Status Protocol) + information or downloading a CRL (Certificate Revocation List). However, NSS doesn't have an + implementation to work with network proxies. If you must support proxies in your application, + you are able to register your own implementation of an http request callback interface, and NSS + can use your application code that supports proxies. + | When using hashing, encryption, and decryption functions, it is possible to stream data (as + opposed to operating on a large buffer). Create a context handle while providing all the + parameters required for the operation, then call an “update” function multiple times to pass + subsets of the input to NSS. The data will be processed and either returned directly or sent to + a callback function registered in the context. When done, you call a finalization function that + will flush out any pending data and free the resources. + | This line is a placeholder for future sections that should explain how libpkix works and is + designed. + | If you want to work with NSS, it's often helpful to use the command line utilities that are + provided by the NSS developers. There are tools for managing NSS databases, for dumping or + verifying certificates, for registering PKCS#11 modules with a database, for processing CMS + encrypted/signed messages, etc. + | For example, if you wanted to create your own pair of keys and request a new certificate from a + CA, you could use certutil to create an empty database, then use certutil to operate on your + database and create a certificate request (which involves creating the desired key pair) and + export it to a file, submit the request file to the CA, receive the file from the CA, and + import the certificate into your database. You should assign a good nickname to a certificate + when importing it, making it easier for you to refer to it later. + | It should be noted that the first database format that can be accessed simultaneously by + multiple applications is key4.db/cert9.db – database files with lower numbers will most likely + experience unrecoverable corruption if you access them with multiple applications at the same + time. In other words, if your browser or your server operates on an older NSS database format, + don't use the NSS tools to operate on it while the other software is executing. At the time of + writing NSS and the Mozilla applications still use the older database file format by default, + where each application has its own NSS database. + | If you require a copy of a certificate stored in an NSS database, including its private key, + you can use pk12util to export it to the PKCS#12 file format. If you require it in PEM format, + you could use the openssl pkcs12 command (that's not NSS) to convert the PKCS#12 file to PEM. + | This line is a placeholder for how to prepare a database, how to dump a cert, and how to + convert data. + | You might have been motivated to work with NSS because it is used by the Mozilla applications + such as Firefox, Thunderbird, etc. If you build the Mozilla application, it will automatically + build the NSS library, too. However, if you want to work with the NSS command line tools, you + will have to follow the standalone NSS build instructions, and build NSS outside of the Mozilla + application sources. + | The key database file will contain at least one symmetric key, which NSS will automatically + create on demand, and which will be used to protect your secret (private) keys. The symmetric + key can be protected with PBE by setting a master password on the database. As soon as you set + a master password, an attacker stealing your key database will no longer be able to get access + to your private key, unless the attacker would also succeed in stealing the master password. + | Now you might be interest in how to get the + :ref:`mozilla_projects_nss_nss_sources_building_testing`
\ No newline at end of file |