Using talloc in Samba4 ====================== .. contents:: Jelmer Vernooij August 2013 The most current version of this document is available at http://samba.org/ftp/unpacked/talloc/pytalloc_guide.txt pytalloc is a small library that provides glue for wrapping talloc-allocated objects from C in Python objects. What is pytalloc, and what is it not? ------------------------------------- pytalloc is merely a helper library - it provides a convenient base type object for objects that wrap talloc-maintained memory in C. It won't write your bindings for you but it will make it easier to write C bindings that involve talloc, and take away some of the boiler plate. Python 3 -------- pytalloc can be used with Python 3. Usage from Python extension remains the same, but for the C utilities, the library to link to is tagged with Python's PEP3149 ABI tag, for example "pytalloc.cpython34m". To make a build for Python 3, configure with PYTHON=/usr/bin/python3. . =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- pytalloc_Object / pytalloc_BaseObject This is the new base class that all Python objects that wrap talloc pointers derive from. It is itself a subclass of the "Object" type that all objects in Python derive from. Note that you will almost never create objects of the pytalloc_Object type itself, as they are just opaque pointers that can not be accessed from Python. A common pattern is other objects that subclass pytalloc_Object and rely on it for their memory management. Each `pytalloc_Object` wraps two core of information - a talloc context and a pointer. The pointer is the actual data that is wrapped. The talloc context is used for memory management purposes only; when the wrapping Python object goes away, it unlinks the talloc context. The talloc context pointer and the ptr can (and often do) have the same value. Each pytalloc_Object has a custom __repr__ implementation that describes that it is a talloc object and the location of the pointer it is wrapping. it also has a custom __cmp__/__eq__/__neq__ method that compares the pointers the object is wrapping rather than the objects themselves (since there can be multiple objects that wrap the same talloc pointer). It is preferred to use pytalloc_BaseObject as this implementation exposes less in the C ABI and correctly supports pointers in C arrays in the way needed by PIDL. =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- PyTypeObject *pytalloc_GetObjectType(void) Obtain a pointer to the PyTypeObject for `pytalloc_Object`. The reference counter for the object will be NOT incremented, so the caller MUST NOT decrement it when it no longer needs it (eg by using `Py_DECREF`). =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- PyTypeObject *pytalloc_GetBaseObjectType(void) Obtain a pointer to the PyTypeObject for `pytalloc_BaseObject`. The reference counter for the object will be NOT incremented, so the caller MUST NOT decrement it when it no longer needs it (eg by using `Py_DECREF`). =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- int pytalloc_BaseObject_PyType_Ready(PyTypeObject *type); Wrapper for PyType_Ready() that will set the correct values into the PyTypeObject to create a BaseObject =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-==-=-=-=-=-=-=-=-=-=-=- int pytalloc_Check(PyObject *) Check whether a specific object is a talloc Object. Returns non-zero if it is a pytalloc_Object and zero otherwise. =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-==-=-=-=-=-=-=-=-=-=-=- int pytalloc_BaseObject_Check(PyObject *) Check whether a specific object is a talloc BaseObject. Returns non-zero if it is a pytalloc_BaseObject and zero otherwise. =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- int pytalloc_check_type(PyObject *py_obj, type) Check if the object based on `pytalloc_*Object` py_obj. type should be a C type, similar to a type passed to `talloc_get_type`. This can be used as a check before using pytalloc_get_type() or an alternative codepath. Returns non-zero if it is an object of the expected type and zero otherwise. =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- type *pytalloc_get_type(PyObject *py_obj, type) Retrieve the pointer from a `pytalloc_Object` py_obj. type should be a C type, similar to a type passed to `talloc_get_type`. =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- pytalloc_get_ptr(PyObject *py_obj) Retrieve the pointer from a `pytalloc_Object` or `pytalloc_BaseObject` py_obj. There is no type checking - use `pytalloc_get_type` if possible. =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TALLOC_CTX *pytalloc_get_mem_ctx(PyObject *py_obj) Retrieve the talloc context associated with a pytalloc_Object or pytalloc_BaseObject. =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- PyObject *pytalloc_steal_ex(PyTypeObject *py_type, TALLOC_CTX *mem_ctx, void *ptr) Create a new Python wrapping object for a talloc pointer and context, with py_type as associated Python sub type object. This typically used when `mem_ctx` and `ptr` differ, e.g. a pointer to an array element. `pytalloc_get_ptr()` can be used to get the pointer out of the object again. This will *not* increment the reference counter for the talloc context, so the caller should make sure such an increment has happened. When the Python object goes away, it will unreference the talloc context. =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- PyObject *pytalloc_steal(PyTypeObject *py_type, void *ptr) Create a new Python wrapping object for a talloc pointer and context, with py_type as associated Python sub type object. The pointer will also be used as the talloc context. `pytalloc_get_type()` can be used to get the pointer out of the object again. This will *not* increment the reference counter for the talloc context, so the caller should make sure such an increment has happened. When the Python object goes away, it will unreference the talloc context. =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- PyObject *pytalloc_reference_ex(PyTypeObject *py_type, TALLOC_CTX *mem_ctx, void *ptr) Create a new Python wrapping object for a talloc pointer and context, with py_type as associated Python sub type object. This typically used when `mem_ctx` and `ptr` differ, e.g. a pointer to an array element. `pytalloc_get_ptr()` can be used to get the pointer out of the object again. This will increment the reference counter for the talloc context. =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- PyObject *pytalloc_reference(PyTypeObject *py_type, void *talloc_ptr) Create a new Python wrapping object for a talloc pointer, with py_type as associated Python sub type object. The pointer will also be used as the talloc context. `pytalloc_get_type()` can be used to get the pointer out of the object again. This will increment the reference counter for the talloc context. =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- PyObject *pytalloc_new(type, PyTypeObject *typeobj) Create a new, empty pytalloc_Object with the specified Python type object. type should be a C type, similar to talloc_new(). =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- PyObject *pytalloc_GenericObject_steal_ex(void *ptr) Create a new Python wrapping object for a generic talloc pointer, as sub type of `pytalloc_BaseObject`. This typically used when `mem_ctx` and `ptr` differ, e.g. a pointer to an array element. `pytalloc_get_ptr()` can be used to get the pointer out of the object again. This will *not* increment the reference counter for the talloc context, so the caller should make sure such an increment has happened. When the Python object goes away, it will unreference the talloc context. =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- PyObject *pytalloc_GenericObject_steal(void *ptr) Create a new Python wrapping object for a generic talloc pointer, as sub type of `pytalloc_BaseObject`. The pointer will also be used as the talloc context. `pytalloc_get_type()` can be used to get the pointer out of the object again. This will *not* increment the reference counter for the talloc context, so the caller should make sure such an increment has happened. When the Python object goes away, it will unreference the talloc context. =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- PyObject *pytalloc_GenericObject_reference_ex(void *ptr) Create a new Python wrapping object for a generic talloc pointer, as sub type of `pytalloc_BaseObject`. This typically used when `mem_ctx` and `ptr` differ, e.g. a pointer to an array element. `pytalloc_get_ptr()` can be used to get the pointer out of the object again. This will increment the reference counter for the talloc context. =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- PyObject *pytalloc_GenericObject_reference(void *ptr) Create a new Python wrapping object for a generic talloc pointer, as sub type of `pytalloc_BaseObject`. The pointer will also be used as the talloc context. `pytalloc_get_type()` can be used to get the pointer out of the object again. This will increment the reference counter for the talloc context. =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- DEPRECATED! PyObject *pytalloc_CObject_FromTallocPtr(void *); Create a new pytalloc_Object for an abitrary talloc-maintained C pointer. This will use a generic VoidPtr Python type, which just provides an opaque object in Python. The caller is responsible for incrementing the talloc reference count before calling this function - it will dereference the talloc pointer when it is garbage collected. This function is deprecated and only available on Python 2. Use pytalloc_GenericObject_{reference,steal}[_ex]() instead. Debug function for talloc in Python ----------------------------------- The "talloc" module in Python provides a couple of functions that can be used to debug issues with objects wrapped by pytalloc. =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- report_full(obj?) Print a full report on a specific object or on all allocated objects by Python. Same behaviour as the `talloc_report_full()` function that is provided by C talloc. =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- enable_null_tracking() This enables tracking of the NULL memory context without enabling leak reporting on exit. Useful for when you want to do your own leak reporting call via talloc_report_null_full(). This must be done in the top level script, not an imported module. =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- pytalloc_total_blocks(obj?) Return the talloc block count for all allocated objects or a specific object if specified.