diff options
Diffstat (limited to 'third_party/python/cbor2/docs')
| -rw-r--r-- | third_party/python/cbor2/docs/conf.py | 33 | ||||
| -rw-r--r-- | third_party/python/cbor2/docs/customizing.rst | 132 | ||||
| -rw-r--r-- | third_party/python/cbor2/docs/index.rst | 15 | ||||
| -rw-r--r-- | third_party/python/cbor2/docs/modules/decoder.rst | 5 | ||||
| -rw-r--r-- | third_party/python/cbor2/docs/modules/encoder.rst | 5 | ||||
| -rw-r--r-- | third_party/python/cbor2/docs/modules/types.rst | 5 | ||||
| -rw-r--r-- | third_party/python/cbor2/docs/usage.rst | 80 | ||||
| -rw-r--r-- | third_party/python/cbor2/docs/versionhistory.rst | 73 |
8 files changed, 348 insertions, 0 deletions
diff --git a/third_party/python/cbor2/docs/conf.py b/third_party/python/cbor2/docs/conf.py new file mode 100644 index 0000000000..0ddeb0719e --- /dev/null +++ b/third_party/python/cbor2/docs/conf.py @@ -0,0 +1,33 @@ +# coding: utf-8 +#!/usr/bin/env python +import pkg_resources + + +extensions = [ + 'sphinx.ext.autodoc', + 'sphinx.ext.intersphinx' +] + +templates_path = ['_templates'] +source_suffix = '.rst' +master_doc = 'index' +project = 'cbor2' +author = u'Alex Grönholm' +copyright = u'2016, ' + author + +v = pkg_resources.get_distribution(project).parsed_version +version = v.base_version +release = v.public + +language = None + +exclude_patterns = ['_build'] +pygments_style = 'sphinx' +highlight_language = 'python' +todo_include_todos = False + +html_theme = 'sphinx_rtd_theme' +html_static_path = ['_static'] +htmlhelp_basename = project.replace('-', '') + 'doc' + +intersphinx_mapping = {'python': ('http://docs.python.org/', None)} diff --git a/third_party/python/cbor2/docs/customizing.rst b/third_party/python/cbor2/docs/customizing.rst new file mode 100644 index 0000000000..bf9b1b4540 --- /dev/null +++ b/third_party/python/cbor2/docs/customizing.rst @@ -0,0 +1,132 @@ +Customizing encoding and decoding +================================= + +Both the encoder and decoder can be customized to support a wider range of types. + +On the encoder side, this is accomplished by passing a callback as the ``default`` constructor +argument. This callback will receive an object that the encoder could not serialize on its own. +The callback should then return a value that the encoder can serialize on its own, although the +return value is allowed to contain objects that also require the encoder to use the callback, as +long as it won't result in an infinite loop. + +On the decoder side, you have two options: ``tag_hook`` and ``object_hook``. The former is called +by the decoder to process any semantic tags that have no predefined decoders. The latter is called +for any newly decoded ``dict`` objects, and is mostly useful for implementing a JSON compatible +custom type serialization scheme. Unless your requirements restrict you to JSON compatible types +only, it is recommended to use ``tag_hook`` for this purpose. + +JSON compatibility +------------------ + +In certain applications, it may be desirable to limit the supported types to the same ones +serializable as JSON: (unicode) string, integer, float, boolean, null, array and object (dict). +This can be done by passing the ``json_compatible`` option to the encoder. When incompatible types +are encountered, a :class:`~cbor2.encoder.CBOREncodeError` is then raised. + +For the decoder, there is no support for detecting incoming incompatible types yet. + +Using the CBOR tags for custom types +------------------------------------ + +The most common way to use ``default`` is to call :meth:`~cbor2.encoder.CBOREncoder.encode` +to add a custom tag in the data stream, with the payload as the value:: + + class Point(object): + def __init__(self, x, y): + self.x = x + self.y = y + + def default_encoder(encoder, value): + # Tag number 4000 was chosen arbitrarily + encoder.encode(CBORTag(4000, [value.x, value.y])) + +The corresponding ``tag_hook`` would be:: + + def tag_hook(decoder, tag, shareable_index=None): + if tag.tag != 4000: + return tag + + # tag.value is now the [x, y] list we serialized before + return Point(*tag.value) + +Using dicts to carry custom types +--------------------------------- + +The same could be done with ``object_hook``, except less efficiently:: + + def default_encoder(encoder, value): + encoder.encode(dict(typename='Point', x=value.x, y=value.y)) + + def object_hook(decoder, value): + if value.get('typename') != 'Point': + return value + + return Point(value['x'], value['y']) + +You should make sure that whatever way you decide to use for telling apart your "specially marked" +dicts from arbitrary data dicts won't mistake on for the other. + +Value sharing with custom types +------------------------------- + +In order to properly encode and decode cyclic references with custom types, some special care has +to be taken. Suppose you have a custom type as below, where every child object contains a reference +to its parent and the parent contains a list of children:: + + from cbor2 import dumps, loads, shareable_encoder, CBORTag + + + class MyType(object): + def __init__(self, parent=None): + self.parent = parent + self.children = [] + if parent: + self.parent.children.append(self) + +This would not normally be serializable, as it would lead to an endless loop (in the worst case) +and raise some exception (in the best case). Now, enter CBOR's extension tags 28 and 29. These tags +make it possible to add special markers into the data stream which can be later referenced and +substituted with the object marked earlier. + +To do this, in ``default`` hooks used with the encoder you will need to use the +:meth:`~cbor2.encoder.shareable_encoder` decorator on your ``default`` hook function. It will +automatically automatically add the object to the shared values registry on the encoder and prevent +it from being serialized twice (instead writing a reference to the data stream):: + + @shareable_encoder + def default_encoder(encoder, value): + # The state has to be serialized separately so that the decoder would have a chance to + # create an empty instance before the shared value references are decoded + serialized_state = encoder.encode_to_bytes(value.__dict__) + encoder.encode(CBORTag(3000, serialized_state)) + +On the decoder side, you will need to initialize an empty instance for shared value lookup before +the object's state (which may contain references to it) is decoded. +This is done with the :meth:`~cbor2.encoder.CBORDecoder.set_shareable` method:: + + def tag_hook(decoder, tag, shareable_index=None): + # Return all other tags as-is + if tag.tag != 3000: + return tag + + # Create a raw instance before initializing its state to make it possible for cyclic + # references to work + instance = MyType.__new__(MyType) + decoder.set_shareable(shareable_index, instance) + + # Separately decode the state of the new object and then apply it + state = decoder.decode_from_bytes(tag.value) + instance.__dict__.update(state) + return instance + +You could then verify that the cyclic references have been restored after deserialization:: + + parent = MyType() + child1 = MyType(parent) + child2 = MyType(parent) + serialized = dumps(parent, default=default_encoder, value_sharing=True) + + new_parent = loads(serialized, tag_hook=tag_hook) + assert new_parent.children[0].parent is new_parent + assert new_parent.children[1].parent is new_parent + diff --git a/third_party/python/cbor2/docs/index.rst b/third_party/python/cbor2/docs/index.rst new file mode 100644 index 0000000000..443c245d86 --- /dev/null +++ b/third_party/python/cbor2/docs/index.rst @@ -0,0 +1,15 @@ +.. include:: ../README.rst + :start-line: 7 + :end-before: Project links + +Table of contents +----------------- + +.. toctree:: + :maxdepth: 2 + + usage + customizing + versionhistory + +* :ref:`API reference <modindex>` diff --git a/third_party/python/cbor2/docs/modules/decoder.rst b/third_party/python/cbor2/docs/modules/decoder.rst new file mode 100644 index 0000000000..c2c58fe9db --- /dev/null +++ b/third_party/python/cbor2/docs/modules/decoder.rst @@ -0,0 +1,5 @@ +:mod:`cbor2.decoder` +==================== + +.. automodule:: cbor2.decoder + :members: diff --git a/third_party/python/cbor2/docs/modules/encoder.rst b/third_party/python/cbor2/docs/modules/encoder.rst new file mode 100644 index 0000000000..c4240eeaad --- /dev/null +++ b/third_party/python/cbor2/docs/modules/encoder.rst @@ -0,0 +1,5 @@ +:mod:`cbor2.encoder` +==================== + +.. automodule:: cbor2.encoder + :members: diff --git a/third_party/python/cbor2/docs/modules/types.rst b/third_party/python/cbor2/docs/modules/types.rst new file mode 100644 index 0000000000..a6dedaa3e4 --- /dev/null +++ b/third_party/python/cbor2/docs/modules/types.rst @@ -0,0 +1,5 @@ +:mod:`cbor2.types` +================== + +.. automodule:: cbor2.types + :members: diff --git a/third_party/python/cbor2/docs/usage.rst b/third_party/python/cbor2/docs/usage.rst new file mode 100644 index 0000000000..54b028ee53 --- /dev/null +++ b/third_party/python/cbor2/docs/usage.rst @@ -0,0 +1,80 @@ +Basic usage +=========== + +Serializing and deserializing with cbor2 is pretty straightforward:: + + from cbor2 import dumps, loads + + # Serialize an object as a bytestring + data = dumps(['hello', 'world']) + + # Deserialize a bytestring + obj = loads(data) + + # Efficiently deserialize from a file + with open('input.cbor', 'rb') as fp: + obj = load(fp) + + # Efficiently serialize an object to a file + with open('output.cbor', 'wb') as fp: + dump(obj, fp) + +Some data types, however, require extra considerations, as detailed below. + +String/bytes handling on Python 2 +--------------------------------- + +The ``str`` type is encoded as binary on Python 2. If you want to encode strings as text on +Python 2, use unicode strings instead. + +Date/time handling +------------------ + +The CBOR specification does not support naïve datetimes (that is, datetimes where ``tzinfo`` is +missing). When the encoder encounters such a datetime, it needs to know which timezone it belongs +to. To this end, you can specify a default timezone by passing a :class:`~datetime.tzinfo` instance +to :func:`~cbor2.encoder.dump`/:func:`~cbor2.encoder.dumps` call as the ``timezone`` argument. +Decoded datetimes are always timezone aware. + +By default, datetimes are serialized in a manner that retains their timezone offsets. You can +optimize the data stream size by passing ``datetime_as_timestamp=False`` to +:func:`~cbor2.encoder.dump`/:func:`~cbor2.encoder.dumps`, but this causes the timezone offset +information to be lost. + +Cyclic (recursive) data structures +---------------------------------- + +If the encoder encounters a shareable object (ie. list or dict) that it has been before, it will +by default raise :exc:`~cbor2.encoder.CBOREncodeError` indicating that a cyclic reference has been +detected and value sharing was not enabled. CBOR has, however, an extension specification that +allows the encoder to reference a previously encoded value without processing it again. This makes +it possible to serialize such cyclic references, but value sharing has to be enabled by passing +``value_sharing=True`` to :func:`~cbor2.encoder.dump`/:func:`~cbor2.encoder.dumps`. + +.. warning:: Support for value sharing is rare in other CBOR implementations, so think carefully + whether you want to enable it. It also causes some line overhead, as all potentially shareable + values must be tagged as such. + +Tag support +----------- + +In addition to all standard CBOR tags, this library supports many extended tags: + +=== ======================================== ==================================================== +Tag Semantics Python type(s) +=== ======================================== ==================================================== +0 Standard date/time string datetime.date / datetime.datetime +1 Epoch-based date/time datetime.date / datetime.datetime +2 Positive bignum int / long +3 Negative bignum int / long +4 Decimal fraction decimal.Decimal +5 Bigfloat decimal.Decimal +28 Mark shared value N/A +29 Reference shared value N/A +30 Rational number fractions.Fraction +35 Regular expression ``_sre.SRE_Pattern`` (result of ``re.compile(...)``) +36 MIME message email.message.Message +37 Binary UUID uuid.UUID +=== ======================================== ==================================================== + +Arbitary tags can be represented with the :class:`~cbor2.types.CBORTag` class. diff --git a/third_party/python/cbor2/docs/versionhistory.rst b/third_party/python/cbor2/docs/versionhistory.rst new file mode 100644 index 0000000000..246e43bb83 --- /dev/null +++ b/third_party/python/cbor2/docs/versionhistory.rst @@ -0,0 +1,73 @@ +Version history +=============== + +This library adheres to `Semantic Versioning <http://semver.org/>`_. + +**4.0.1.** (2017-08-21) + +- Fixed silent truncation of decoded data if there are not enough bytes in the stream for an exact + read (``CBORDecodeError`` is now raised instead) + +**4.0.0** (2017-04-24) + +- **BACKWARD INCOMPATIBLE** Value sharing has been disabled by default, for better compatibility + with other implementations and better performance (since it is rarely needed) +- **BACKWARD INCOMPATIBLE** Replaced the ``semantic_decoders`` decoder option with the ``tag_hook`` + option +- **BACKWARD INCOMPATIBLE** Replaced the ``encoders`` encoder option with the ``default`` option +- **BACKWARD INCOMPATIBLE** Factored out the file object argument (``fp``) from all callbacks +- **BACKWARD INCOMPATIBLE** The encoder no longer supports every imaginable type implementing the + ``Sequence`` or ``Map`` interface, as they turned out to be too broad +- Added the ``object_hook`` option for decoding dicts into complex objects + (intended for situations where JSON compatibility is required and semantic tags cannot be used) +- Added encoding and decoding of simple values (``CBORSimpleValue``) + (contributed by Jerry Lundström) +- Replaced the decoder for bignums with a simpler and faster version (contributed by orent) +- Made all relevant classes and functions available directly in the ``cbor2`` namespace +- Added proper documentation + +**3.0.4** (2016-09-24) + +- Fixed TypeError when trying to encode extension types (regression introduced in 3.0.3) + +**3.0.3** (2016-09-23) + +- No changes, just re-releasing due to git tagging screw-up + +**3.0.2** (2016-09-23) + +- Fixed decoding failure for datetimes with microseconds (tag 0) + +**3.0.1** (2016-08-08) + +- Fixed error in the cyclic structure detection code that could mistake one container for + another, sometimes causing a bogus error about cyclic data structures where there was none + +**3.0.0** (2016-07-03) + +- **BACKWARD INCOMPATIBLE** Encoder callbacks now receive three arguments: the encoder instance, + the value to encode and a file-like object. The callback must must now either write directly to + the file-like object or call another encoder callback instead of returning an iterable. +- **BACKWARD INCOMPATIBLE** Semantic decoder callbacks now receive four arguments: the decoder + instance, the primitive value, a file-like object and the shareable index for the decoded value. + Decoders that support value sharing must now set the raw value at the given index in + ``decoder.shareables``. +- **BACKWARD INCOMPATIBLE** Removed support for iterative encoding (``CBOREncoder.encode()`` is no + longer a generator function and always returns ``None``) +- Significantly improved performance (encoder ~30 % faster, decoder ~60 % faster) +- Fixed serialization round-trip for ``undefined`` (simple type #23) +- Added proper support for value sharing in callbacks + +**2.0.0** (2016-06-11) + +- **BACKWARD INCOMPATIBLE** Deserialize unknown tags as ``CBORTag`` objects so as not to lose + information +- Fixed error messages coming from nested structures + +**1.1.0** (2016-06-10) + +- Fixed deserialization of cyclic structures + +**1.0.0** (2016-06-08) + +- Initial release |