Skip to content

Interop module

Reference counting classes using cffi (C Foreign Function Interface for Python) for interoperability.

Implementation of reference counting classes for external resources accessed via interoperability software such as cffi

CffiData: TypeAlias = Any module-attribute

A dummy type to use in type hints for limited documentation purposes

FFI.CData is a type, but it seems it cannot be used in type hinting.

CffiNativeHandle

Bases: NativeHandle

Reference counting wrapper class for CFFI pointers

This class is originally inspired from a class with a similar purpose in C#. See https://github.com/rdotnet/dynamic-interop-dll

Say you have a C API as follows:

  • void* create_some_object();
  • dispose_of_some_object(void* obj);

and accessing it using Python and CFFI. Users would use the calllib function:

from cffi import FFI
ffi = FFI()

# cdef() expects a single string declaring the C types, functions and
# globals needed to use the shared object. It must be in valid C syntax.
ffi.cdef('''
    void* create_some_object();
    dispose_of_some_object(void* obj);
''')
mydll_so = ffi.dlopen('/path/to/mydll.so')
cffi_void_ptr = mydll_so.create_some_object()

at some point when done you need to dispose of it to clear native memory:

mydll_so.dispose_of_some_object(cffi_void_ptr)

In practice in real systems one quickly ends up with cases where it is unclear when to dispose of the object. If you call the dispose_of_some_object function more than once, or too soon, you quickly crash the program, or possibly worse outcomes with numeric non-sense. CffiNativeHandle is designed to alleviate this headache by using native reference counting of handle classes to reliably dispose of objects.

Attributes:

Name Type Description
_handle object

The handle (e.g. cffi pointer) to the native resource.
_type_id Optional[str]

An optional identifier for the type of underlying resource. This can be used to usefully maintain type information about the pointer/handle across an otherwise opaque C API. See package documentation.
_finalizing bool

a flag telling whether this object is in its deletion phase. This has a use in some advanced cases with reverse callback, possibly not relevant in Python.
Source code in refcount/interop.py 
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
class CffiNativeHandle(NativeHandle):
    """Reference counting wrapper class for CFFI pointers

    This class is originally inspired from a class with a similar purpose in C#. See https://github.com/rdotnet/dynamic-interop-dll

    Say you have a C API as follows:

    * `void* create_some_object();`
    * `dispose_of_some_object(void* obj);`

    and accessing it using Python and [CFFI](https://cffi.readthedocs.io).
    Users would use the `calllib` function:

    ```python
    from cffi import FFI
    ffi = FFI()

    # cdef() expects a single string declaring the C types, functions and
    # globals needed to use the shared object. It must be in valid C syntax.
    ffi.cdef('''
        void* create_some_object();
        dispose_of_some_object(void* obj);
    ''')
    mydll_so = ffi.dlopen('/path/to/mydll.so')
    cffi_void_ptr = mydll_so.create_some_object()
    ```

    at some point when done you need to dispose of it to clear native memory:

    ```python
    mydll_so.dispose_of_some_object(cffi_void_ptr)
    ```

    In practice in real systems one quickly ends up with cases
    where it is unclear when to dispose of the object.
    If you call the `dispose_of_some_object` function more
    than once, or too soon, you quickly crash the program, or possibly worse outcomes with numeric non-sense.
    `CffiNativeHandle` is designed to alleviate this headache by
    using native reference counting of `handle` classes to reliably dispose of objects.

    Attributes:
        _handle (object): The handle (e.g. cffi pointer) to the native resource.
        _type_id (Optional[str]): An optional identifier for the type of underlying resource. This can be used to usefully maintain type information about the pointer/handle across an otherwise opaque C API. See package documentation.
        _finalizing (bool): a flag telling whether this object is in its deletion phase. This has a use in some advanced cases with reverse callback, possibly not relevant in Python.
    """


    def __init__(self, handle: "CffiData", type_id: Optional[str] = None, prior_ref_count: int = 0):
        """Initialize a reference counter for a resource handle, with an initial reference count.

        Args:
            handle (object): The handle (e.g. cffi pointer) to the native resource.
            type_id (Optional[str]): An optional identifier for the type of underlying resource. This can be used to usefully maintain type information about the pointer/handle across an otherwise opaque C API. See package documentation.
            prior_ref_count (int): the initial reference count. Default 0 if this NativeHandle is sole responsible for the lifecycle of the resource.
        """
        super(CffiNativeHandle, self).__init__(handle, prior_ref_count)
        # TODO checks on handle
        self._type_id = type_id
        self._finalizing: bool = False
        self._handle: "CffiData" = None
        if handle is None:
            return  # defer setting the handle to the inheritor.
        self._set_handle(handle, prior_ref_count)

    def _is_valid_handle(self, h: "CffiData") -> bool:
        """Checks if the handle is a CFFI CData pointer, acceptable handle for this wrapper.

        Args:
            handle (object): The handle (e.g. cffi pointer) to the native resource.
        """
        return isinstance(h, FFI.CData)

    def __dispose_impl(self, decrement: bool) -> None:
        """An implementation of the dispose method in a 'Dispose' software pattern. Avoids cyclic method calls.

        Args:
            decrement (bool): indicating whether the reference count should be decreased. It should almost always be True except in very unusual use cases (argument is for possible future use).
        """
        if self.disposed:
            return
        if decrement:
            self._ref_count = self._ref_count - 1
        if self._ref_count <= 0:
            if self._release_handle():
                self._handle = None

    @property
    def disposed(self) -> bool:
        """Has the native object and memory already been disposed of.

        Returns:
            (bool): The underlying native handle has been disposed of from this wrapper
        """
        return self._handle is None

    @property
    def is_invalid(self) -> bool:
        """Is the underlying handle valid? In practice synonym with the disposed attribute.

        Returns:
            (bool): True if this handle is valid
        """
        return self._handle is None

    def _release_handle(self) -> bool:
        """Must of overriden. Method disposing of the object pointed to by the CFFI pointer (handle)

        Raises:
            NotImplementedError: thrown if this method is not overriden by inheritors

        Returns:
            bool: Overriding implementation should return True if the release of native resources was successful, False otherwise.
        """
        # See also https://stackoverflow.com/questions/4714136/how-to-implement-virtual-methods-in-python
        # May want to make this abstract using ABC - we'll see.
        raise NotImplementedError("method _release_handle must be overriden by child classes")

    def get_handle(self) -> Union["CffiData", None]:
        """Gets the underlying low-level CFFI handle this object wraps

        Returns:
            (Union[CffiData, None]): CFFI handle or None
        """
        return self._handle

    # TODO?
    # @property.getter
    # def ptr(self):
    #     """ Return the pointer to a cffi object """
    #     return self._handle

    @property
    def type_id(self) -> Optional[str]:
        """Return an optional type identifier for the underlying native type.

        This can be in practice useful to be more transparent about the underlying
        type obtained via a C API with opaque pointers (i.e. void*)

        Returns:
            str: optional type identifier
        """
        return self._type_id

    # @property
    # def obj(self):
    #     """ Return the native object pointed to (cffi object) """
    #     return self._handle

    def __str__(self) -> str:
        """ string representation """
        if self.type_id is None or self.type_id == "":
            return "CFFI pointer handle to a native pointer " + str(self._handle)
        return 'CFFI pointer handle to a native pointer of type id "' + self.type_id + '"'

    @property
    def ptr(self) -> 'CffiData':
        """ Return the pointer (cffi object) """
        return self._handle

    @property
    def obj(self) -> Any:
        """ Return the object pointed to (cffi object) """
        return self._handle[0]


    def __repr__(self) -> str:
        """ string representation """
        return str(self)

    def __del__(self) -> None:
        """ destructor, triggering the release of the underlying handled resource if the reference count is 0 """
        if self._handle is not None:
            # if not self._release_native is None:
            # Protect against accessing properties
            # of partially constructed objects (May not be an issue in Python?)
            self._finalizing = True
            self.release()

    def dispose(self) -> None:
        """Disposing of the object pointed to by the CFFI pointer (handle) if the reference counts allows it."""
        self.__dispose_impl(True)

    def release(self) -> None:
        """Manually decrements the reference counter. Triggers disposal if reference count is down to zero."""
        self.__dispose_impl(True)

disposed: bool property

Has the native object and memory already been disposed of.

Returns:

Type Description
bool

The underlying native handle has been disposed of from this wrapper

is_invalid: bool property

Is the underlying handle valid? In practice synonym with the disposed attribute.

Returns:

Type Description
bool

True if this handle is valid

obj: Any property

Return the object pointed to (cffi object)

ptr: CffiData property

Return the pointer (cffi object)

type_id: Optional[str] property

Return an optional type identifier for the underlying native type.

This can be in practice useful to be more transparent about the underlying type obtained via a C API with opaque pointers (i.e. void*)

Returns:

Name Type Description
str Optional[str]

optional type identifier

__del__()

destructor, triggering the release of the underlying handled resource if the reference count is 0

Source code in refcount/interop.py 
188
189
190
191
192
193
194
195
def __del__(self) -> None:
    """ destructor, triggering the release of the underlying handled resource if the reference count is 0 """
    if self._handle is not None:
        # if not self._release_native is None:
        # Protect against accessing properties
        # of partially constructed objects (May not be an issue in Python?)
        self._finalizing = True
        self.release()

__dispose_impl(decrement)

An implementation of the dispose method in a 'Dispose' software pattern. Avoids cyclic method calls.

Parameters:

Name Type Description Default
decrement bool

indicating whether the reference count should be decreased. It should almost always be True except in very unusual use cases (argument is for possible future use).

 required
Source code in refcount/interop.py 
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
def __dispose_impl(self, decrement: bool) -> None:
    """An implementation of the dispose method in a 'Dispose' software pattern. Avoids cyclic method calls.

    Args:
        decrement (bool): indicating whether the reference count should be decreased. It should almost always be True except in very unusual use cases (argument is for possible future use).
    """
    if self.disposed:
        return
    if decrement:
        self._ref_count = self._ref_count - 1
    if self._ref_count <= 0:
        if self._release_handle():
            self._handle = None

__init__(handle, type_id=None, prior_ref_count=0)

Initialize a reference counter for a resource handle, with an initial reference count.

Parameters:

Name Type Description Default
handle object

The handle (e.g. cffi pointer) to the native resource.

 required
type_id Optional[str]

An optional identifier for the type of underlying resource. This can be used to usefully maintain type information about the pointer/handle across an otherwise opaque C API. See package documentation.

 None
prior_ref_count int

the initial reference count. Default 0 if this NativeHandle is sole responsible for the lifecycle of the resource.

 0
Source code in refcount/interop.py 
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
def __init__(self, handle: "CffiData", type_id: Optional[str] = None, prior_ref_count: int = 0):
    """Initialize a reference counter for a resource handle, with an initial reference count.

    Args:
        handle (object): The handle (e.g. cffi pointer) to the native resource.
        type_id (Optional[str]): An optional identifier for the type of underlying resource. This can be used to usefully maintain type information about the pointer/handle across an otherwise opaque C API. See package documentation.
        prior_ref_count (int): the initial reference count. Default 0 if this NativeHandle is sole responsible for the lifecycle of the resource.
    """
    super(CffiNativeHandle, self).__init__(handle, prior_ref_count)
    # TODO checks on handle
    self._type_id = type_id
    self._finalizing: bool = False
    self._handle: "CffiData" = None
    if handle is None:
        return  # defer setting the handle to the inheritor.
    self._set_handle(handle, prior_ref_count)

__repr__()

string representation

Source code in refcount/interop.py 
184
185
186
def __repr__(self) -> str:
    """ string representation """
    return str(self)

__str__()

string representation

Source code in refcount/interop.py 
167
168
169
170
171
def __str__(self) -> str:
    """ string representation """
    if self.type_id is None or self.type_id == "":
        return "CFFI pointer handle to a native pointer " + str(self._handle)
    return 'CFFI pointer handle to a native pointer of type id "' + self.type_id + '"'

dispose()

Disposing of the object pointed to by the CFFI pointer (handle) if the reference counts allows it.

Source code in refcount/interop.py 
197
198
199
def dispose(self) -> None:
    """Disposing of the object pointed to by the CFFI pointer (handle) if the reference counts allows it."""
    self.__dispose_impl(True)

get_handle()

Gets the underlying low-level CFFI handle this object wraps

Returns:

Type Description
Union[CffiData, None]

CFFI handle or None
Source code in refcount/interop.py 
136
137
138
139
140
141
142
def get_handle(self) -> Union["CffiData", None]:
    """Gets the underlying low-level CFFI handle this object wraps

    Returns:
        (Union[CffiData, None]): CFFI handle or None
    """
    return self._handle

release()

Manually decrements the reference counter. Triggers disposal if reference count is down to zero.

Source code in refcount/interop.py 
201
202
203
def release(self) -> None:
    """Manually decrements the reference counter. Triggers disposal if reference count is down to zero."""
    self.__dispose_impl(True)

CffiWrapperFactory

A class that creates custom python wrappers based on the type identifier of the external pointer being wrapped

Source code in refcount/interop.py 
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
class CffiWrapperFactory:
    """A class that creates custom python wrappers based on the type identifier of the external pointer being wrapped
    """
    def __init__(self, api_type_wrapper:Dict[str,Any], strict_wrapping:bool=False) -> None:
        """_summary_

        Args:
            api_type_wrapper (Dict[str,Any]): dictionary, mapping from type identifiers to callables, class constructors
            strict_wrapping (bool, optional): If true, type identifiers passed at wrapper creation time `create_wrapper` 
                must be known or exceptions are raised. If False, it falls back on creating generic wrappers. Defaults to False.
        """        
        self._strict_wrapping = strict_wrapping
        self._api_type_wrapper = api_type_wrapper

    def create_wrapper(self, obj: Any, type_id: str, release_native: Optional[Callable[["CffiData"], None]]) -> 'CffiNativeHandle':
        """_summary_

        Args:
            obj (Union[CffiData,Any]): An object, which will be wrapped if this is a CFFI pointer, i.e. an instance of `CffiData`
            type_id (Optional[str]): An optional identifier for the type of underlying resource. This can be used to usefully maintain type information about the pointer/handle across an otherwise opaque C API. See package documentation.
            release_native (Callable[[CffiData],None]): function to call on deleting this wrapper. The function should have one argument accepting the object handle.

        Raises:
            ValueError: Missing type_id
            ValueError: If this object is in strict mode, and `type_id` is not known in the mapping
            NotImplementedError: `type_id` is known, but mapping to None (wrapper not yet implemented)
            TypeError: The function to create the wrapper does not accept any argument. 

        Returns:
            CffiNativeHandle: cffi wrapper
        """        
        from inspect import signature
        if type_id is None:
            raise ValueError("Type ID provided cannot be None")
        if type_id not in self._api_type_wrapper.keys():
            if self._strict_wrapping:
                raise ValueError("Type ID {} is unknown".format(type_id))
            else:
                return wrap_cffi_native_handle(obj, type_id, release_native)
        wrapper_type = self._api_type_wrapper[type_id]
        if wrapper_type is None:
            if self._strict_wrapping:
                raise NotImplementedError(
                    "Python object wrapper for foreign type ID {} is not yet implemented".format(
                        wrapper_type
                    )
                )
            else:
                return wrap_cffi_native_handle(obj, type_id, release_native)
        else:
            s = signature(wrapper_type)
            n = len(s.parameters)
            if n == 0:
                raise TypeError("Wrapper constructor must have at least one argument")
            elif n == 1:
                return wrapper_type(obj)
            elif n == 2:
                return wrapper_type(obj, release_native)
            else:
                return wrapper_type(obj, release_native, type_id)

__init__(api_type_wrapper, strict_wrapping=False)

summary

Parameters:

Name Type Description Default
api_type_wrapper Dict[str, Any]

dictionary, mapping from type identifiers to callables, class constructors

 required
strict_wrapping bool

If true, type identifiers passed at wrapper creation time create_wrapper must be known or exceptions are raised. If False, it falls back on creating generic wrappers. Defaults to False.

 False
Source code in refcount/interop.py 
454
455
456
457
458
459
460
461
462
463
def __init__(self, api_type_wrapper:Dict[str,Any], strict_wrapping:bool=False) -> None:
    """_summary_

    Args:
        api_type_wrapper (Dict[str,Any]): dictionary, mapping from type identifiers to callables, class constructors
        strict_wrapping (bool, optional): If true, type identifiers passed at wrapper creation time `create_wrapper` 
            must be known or exceptions are raised. If False, it falls back on creating generic wrappers. Defaults to False.
    """        
    self._strict_wrapping = strict_wrapping
    self._api_type_wrapper = api_type_wrapper

create_wrapper(obj, type_id, release_native)

summary

Parameters:

Name Type Description Default
obj Union[CffiData, Any]

An object, which will be wrapped if this is a CFFI pointer, i.e. an instance of CffiData

 required
type_id Optional[str]

An optional identifier for the type of underlying resource. This can be used to usefully maintain type information about the pointer/handle across an otherwise opaque C API. See package documentation.

 required
release_native Callable[[CffiData], None]

function to call on deleting this wrapper. The function should have one argument accepting the object handle.

 required

Raises:

Type Description
ValueError

Missing type_id
ValueError

If this object is in strict mode, and type_id is not known in the mapping
NotImplementedError

type_id is known, but mapping to None (wrapper not yet implemented)
TypeError

The function to create the wrapper does not accept any argument.

Returns:

Name Type Description
CffiNativeHandle CffiNativeHandle

cffi wrapper
Source code in refcount/interop.py 
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
def create_wrapper(self, obj: Any, type_id: str, release_native: Optional[Callable[["CffiData"], None]]) -> 'CffiNativeHandle':
    """_summary_

    Args:
        obj (Union[CffiData,Any]): An object, which will be wrapped if this is a CFFI pointer, i.e. an instance of `CffiData`
        type_id (Optional[str]): An optional identifier for the type of underlying resource. This can be used to usefully maintain type information about the pointer/handle across an otherwise opaque C API. See package documentation.
        release_native (Callable[[CffiData],None]): function to call on deleting this wrapper. The function should have one argument accepting the object handle.

    Raises:
        ValueError: Missing type_id
        ValueError: If this object is in strict mode, and `type_id` is not known in the mapping
        NotImplementedError: `type_id` is known, but mapping to None (wrapper not yet implemented)
        TypeError: The function to create the wrapper does not accept any argument. 

    Returns:
        CffiNativeHandle: cffi wrapper
    """        
    from inspect import signature
    if type_id is None:
        raise ValueError("Type ID provided cannot be None")
    if type_id not in self._api_type_wrapper.keys():
        if self._strict_wrapping:
            raise ValueError("Type ID {} is unknown".format(type_id))
        else:
            return wrap_cffi_native_handle(obj, type_id, release_native)
    wrapper_type = self._api_type_wrapper[type_id]
    if wrapper_type is None:
        if self._strict_wrapping:
            raise NotImplementedError(
                "Python object wrapper for foreign type ID {} is not yet implemented".format(
                    wrapper_type
                )
            )
        else:
            return wrap_cffi_native_handle(obj, type_id, release_native)
    else:
        s = signature(wrapper_type)
        n = len(s.parameters)
        if n == 0:
            raise TypeError("Wrapper constructor must have at least one argument")
        elif n == 1:
            return wrapper_type(obj)
        elif n == 2:
            return wrapper_type(obj, release_native)
        else:
            return wrapper_type(obj, release_native, type_id)

DeletableCffiNativeHandle

Bases: CffiNativeHandle

Reference counting wrapper class for CFFI pointers

Attributes:

Name Type Description
_handle object

The handle (e.g. cffi pointer) to the native resource.
_type_id Optional[str]

An optional identifier for the type of underlying resource. This can be used to usefully maintain type information about the pointer/handle across an otherwise opaque C API. See package documentation.
_finalizing bool

a flag telling whether this object is in its deletion phase. This has a use in some advanced cases with reverse callback, possibly not relevant in Python.
_release_native Callable[[CffiData], None]

function to call on deleting this wrapper. The function should have one argument accepting the object _handle.
Source code in refcount/interop.py 
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
class DeletableCffiNativeHandle(CffiNativeHandle):
    """Reference counting wrapper class for CFFI pointers

    Attributes:
        _handle (object): The handle (e.g. cffi pointer) to the native resource.
        _type_id (Optional[str]): An optional identifier for the type of underlying resource. This can be used to usefully maintain type information about the pointer/handle across an otherwise opaque C API. See package documentation.
        _finalizing (bool): a flag telling whether this object is in its deletion phase. This has a use in some advanced cases with reverse callback, possibly not relevant in Python.
        _release_native (Callable[[CffiData],None]): function to call on deleting this wrapper. The function should have one argument accepting the object _handle.
    """

    def __init__(
        self,
        handle: "CffiData",
        release_native: Optional[Callable[["CffiData"], None]],
        type_id: Optional[str] = None,
        prior_ref_count: int = 0,
    ):
        """New reference counter for a CFFI resource handle.

        Args:
            handle (CffiData): The handle (expected cffi pointer) to the native resource.
            release_native (Callable[[CffiData],None]): function to call on deleting this wrapper. The function should have one argument accepting the object handle.
            type_id (str, optional): [description]. An optional identifier for the type of underlying resource. This can be used to usefully maintain type information about the pointer/handle across an otherwise opaque C API. See package documentation. Defaults to None.
            prior_ref_count (int, optional): [description]. The initial reference count. Defaults to 0 if this NativeHandle is sole responsible for the lifecycle of the resource.
        """
        super(DeletableCffiNativeHandle, self).__init__(
            handle, type_id, prior_ref_count
        )
        self._release_native = release_native
        self._set_handle(handle, prior_ref_count)

    def _release_handle(self) -> bool:
        """Release the handle, dispose of the native resource.

        Returns:
            bool: Return True if the release of native resources handle was successful, False otherwise.
        """
        if self._handle is None:
            return False
        if self._release_native is None:
            return False
        if self._release_native is not None:
            self._release_native(
                self._handle
            )  # TODO are trapped exceptions acceptable here?
            return True

__init__(handle, release_native, type_id=None, prior_ref_count=0)

New reference counter for a CFFI resource handle.

Parameters:

Name Type Description Default
handle CffiData

The handle (expected cffi pointer) to the native resource.

 required
release_native Callable[[CffiData], None]

function to call on deleting this wrapper. The function should have one argument accepting the object handle.

 required
type_id str

[description]. An optional identifier for the type of underlying resource. This can be used to usefully maintain type information about the pointer/handle across an otherwise opaque C API. See package documentation. Defaults to None.

 None
prior_ref_count int

[description]. The initial reference count. Defaults to 0 if this NativeHandle is sole responsible for the lifecycle of the resource.

 0
Source code in refcount/interop.py 
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
def __init__(
    self,
    handle: "CffiData",
    release_native: Optional[Callable[["CffiData"], None]],
    type_id: Optional[str] = None,
    prior_ref_count: int = 0,
):
    """New reference counter for a CFFI resource handle.

    Args:
        handle (CffiData): The handle (expected cffi pointer) to the native resource.
        release_native (Callable[[CffiData],None]): function to call on deleting this wrapper. The function should have one argument accepting the object handle.
        type_id (str, optional): [description]. An optional identifier for the type of underlying resource. This can be used to usefully maintain type information about the pointer/handle across an otherwise opaque C API. See package documentation. Defaults to None.
        prior_ref_count (int, optional): [description]. The initial reference count. Defaults to 0 if this NativeHandle is sole responsible for the lifecycle of the resource.
    """
    super(DeletableCffiNativeHandle, self).__init__(
        handle, type_id, prior_ref_count
    )
    self._release_native = release_native
    self._set_handle(handle, prior_ref_count)

GenericWrapper

A pass-through wrapper for python objects that are ready for C interop. "bytes" can be passed as C 'char*'

This is mostly a facility to generate glue code more easily

Source code in refcount/interop.py 
391
392
393
394
395
396
397
398
399
400
401
402
class GenericWrapper:
    """A pass-through wrapper for python objects that are ready for C interop. "bytes" can be passed as C 'char*'

    This is mostly a facility to generate glue code more easily 
    """    

    def __init__(self, handle: Any):
        self._handle = handle

    @property
    def ptr(self) -> Any:
        return self._handle

OwningCffiNativeHandle

Bases: CffiNativeHandle

Reference counting wrapper class for CFFI pointers that own and already manage the native memory

Attributes:

Name Type Description
_handle object

The handle (e.g. cffi pointer) to the native resource.
_type_id Optional[str]

An optional identifier for the type of underlying resource. This can be used to usefully maintain type information about the pointer/handle across an otherwise opaque C API. See package documentation.
_finalizing bool

a flag telling whether this object is in its deletion phase. This has a use in some advanced cases with reverse callback, possibly not relevant in Python.
Source code in refcount/interop.py 
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
class OwningCffiNativeHandle(CffiNativeHandle):
    """Reference counting wrapper class for CFFI pointers that own and already manage the native memory

    Attributes:
        _handle (object): The handle (e.g. cffi pointer) to the native resource.
        _type_id (Optional[str]): An optional identifier for the type of underlying resource. This can be used to usefully maintain type information about the pointer/handle across an otherwise opaque C API. See package documentation.
        _finalizing (bool): a flag telling whether this object is in its deletion phase. This has a use in some advanced cases with reverse callback, possibly not relevant in Python.
    """

    # """ a global function that can be called to release an external pointer """
    # release_native = None

    def __init__(
        self,
        handle: "CffiData",
        type_id: str = None,
        prior_ref_count: int = 0,
    ):
        """Reference counting wrapper class for CFFI pointers that own and already manage the native memory

        Args:
            handle (CffiData): The handle (expected cffi pointer) to the native resource.
            type_id (str, optional): [description]. An optional identifier for the type of underlying resource. This can be used to usefully maintain type information about the pointer/handle across an otherwise opaque C API. See package documentation. Defaults to None.
            prior_ref_count (int, optional): [description]. The initial reference count. Defaults to 0 if this NativeHandle is sole responsible for the lifecycle of the resource.
        """
        super(OwningCffiNativeHandle, self).__init__(
            handle, type_id, prior_ref_count
        )
        self._set_handle(handle, prior_ref_count)

    def _release_handle(self) -> bool:
        """Does nothing, as the wrapped cffi pointer is already owning and managing the memory

        Returns:
            bool: Always returns True.
        """
        return True

__init__(handle, type_id=None, prior_ref_count=0)

Reference counting wrapper class for CFFI pointers that own and already manage the native memory

Parameters:

Name Type Description Default
handle CffiData

The handle (expected cffi pointer) to the native resource.

 required
type_id str

[description]. An optional identifier for the type of underlying resource. This can be used to usefully maintain type information about the pointer/handle across an otherwise opaque C API. See package documentation. Defaults to None.

 None
prior_ref_count int

[description]. The initial reference count. Defaults to 0 if this NativeHandle is sole responsible for the lifecycle of the resource.

 0
Source code in refcount/interop.py 
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
def __init__(
    self,
    handle: "CffiData",
    type_id: str = None,
    prior_ref_count: int = 0,
):
    """Reference counting wrapper class for CFFI pointers that own and already manage the native memory

    Args:
        handle (CffiData): The handle (expected cffi pointer) to the native resource.
        type_id (str, optional): [description]. An optional identifier for the type of underlying resource. This can be used to usefully maintain type information about the pointer/handle across an otherwise opaque C API. See package documentation. Defaults to None.
        prior_ref_count (int, optional): [description]. The initial reference count. Defaults to 0 if this NativeHandle is sole responsible for the lifecycle of the resource.
    """
    super(OwningCffiNativeHandle, self).__init__(
        handle, type_id, prior_ref_count
    )
    self._set_handle(handle, prior_ref_count)

cffi_arg_error_external_obj_type(x, expected_type_id)

Build an error message that an unexpected object is in lieu of an expected refcount external ref object.

Parameters:

Name Type Description Default
x object

object passed as an argument to a function but with an unexpected type or type id.

 required
expected_type_id Optional[str]

Expected identifier for the type of underlying resource being wrapped.

 required

Returns (str): the error message

Source code in refcount/interop.py 
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
def cffi_arg_error_external_obj_type(x: Any, expected_type_id: str) -> str:
    """Build an error message that an unexpected object is in lieu of an expected refcount external ref object.

    Args:
        x (object): object passed as an argument to a function but with an unexpected type or type id.
        expected_type_id (Optional[str]): Expected identifier for the type of underlying resource being wrapped.

    Returns (str): the error message
    """
    if x is None:
        return "Expected a 'CffiNativeHandle' but instead got 'None'"
    if not is_cffi_native_handle(x):
        return (
            "Expected a 'CffiNativeHandle' but instead got object of type '{0}'".format(
                str(type(x))
            )
        )
    else:
        return "Expected a 'CffiNativeHandle' with underlying type id '{0}' but instead got one with type id '{1}'".format(
            expected_type_id, x.type_id
        )

is_cffi_native_handle(x, type_id='')

Checks whether an object is a ref counting wrapper around a CFFI pointer

Parameters:

Name Type Description Default
x object

object to test, presumed to be an instance of CffiNativeHandle

 required
type_id Optional[str]

Optional identifier for the type of underlying resource being wrapped.

 ''
Source code in refcount/interop.py 
313
314
315
316
317
318
319
320
321
322
323
324
325
326
def is_cffi_native_handle(x: Any, type_id: str = "") -> bool:
    """Checks whether an object is a ref counting wrapper around a CFFI pointer

    Args:
        x (object): object to test, presumed to be an instance of `CffiNativeHandle`
        type_id (Optional[str]): Optional identifier for the type of underlying resource being wrapped.
    """
    if x is None:
        return False
    if not isinstance(x, CffiNativeHandle):
        return False
    if type_id is None or type_id == "":
        return True
    return x.type_id == type_id

type_error_cffi(x, expected_type)

DEPRECATED Build an error message for situations where a cffi pointer handler is not that, or not of the expected type

Parameters:

Name Type Description Default
x Union[CffiNativeHandle, Any]

actual object that is not of the expected type or underlying type for the external pointer.

 required
expected_type str

underlying type expected for the CFFI pointer handler

 required

Returns:

Name Type Description
str str

error message that the caller can use to report the issue
Source code in refcount/interop.py 
439
440
441
442
443
444
445
446
447
448
449
def type_error_cffi(x:Union[CffiNativeHandle, Any], expected_type:str) -> str:
    """DEPRECATED Build an error message for situations where a cffi pointer handler is not that, or not of the expected type

    Args:
        x (Union[CffiNativeHandle, Any]): actual object that is not of the expected type or underlying type for the external pointer.
        expected_type (str): underlying type expected for the CFFI pointer handler

    Returns:
        str: error message that the caller can use to report the issue
    """
    return cffi_arg_error_external_obj_type(x, expected_type)

unwrap_cffi_native_handle(obj_wrapper, stringent=False)

Unwrap a reference counting wrapper and returns its CFFI pointer if it is found (wrapped or 'raw')

Parameters:

Name Type Description Default
obj_wrapper Any

An object, which will be unwrapped if this is a CFFI pointer, i.e. an instance of CffiData

 required
stringent bool

[description]. if True an error is raised if obj_wrapper is neither None, a CffiNativeHandle nor an CffiData. Defaults to False.

 False

Raises:

Type Description
Exception

A CFFI pointer could not be found in the object.

Returns:

Type Description
Union[CffiData, Any, None]

Union[CffiData,Any,None]: A CFFI pointer if it was found. Returns None or unchanged if not found, and stringent is equal to False. Exception otherwise.
Source code in refcount/interop.py 
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
def unwrap_cffi_native_handle(
    obj_wrapper: Any, stringent: bool = False
) -> Union["CffiData", Any, None]:
    """Unwrap a reference counting wrapper and returns its CFFI pointer if it is found (wrapped or 'raw')

    Args:
        obj_wrapper (Any): An object, which will be unwrapped if this is a CFFI pointer, i.e. an instance of `CffiData`
        stringent (bool, optional): [description]. if True an error is raised if obj_wrapper is neither None, a CffiNativeHandle nor an CffiData. Defaults to False.

    Raises:
        Exception: A CFFI pointer could not be found in the object.

    Returns:
        Union[CffiData,Any,None]: A CFFI pointer if it was found. Returns None or unchanged if not found, and stringent is equal to False. Exception otherwise.
    """
    # 2016-01-28 allowing null pointers, to unlock behavior of EstimateERRISParameters.
    # Reassess approach, even if other C API function will still catch the issue of null ptrs.
    if obj_wrapper is None:
        return None
    if isinstance(obj_wrapper, CffiNativeHandle):
        return obj_wrapper.get_handle()
    if isinstance(obj_wrapper, FFI.CData):
        return obj_wrapper
    else:
        if stringent:
            raise TypeError(
                "Argument is neither a CffiNativeHandle nor a CFFI external pointer"
            )
        else:
            return obj_wrapper

wrap_as_pointer_handle(obj_wrapper, stringent=False)

Wrap an object, if need be, so that its C API pointer appears accessible via a 'ptr' property

Parameters:

Name Type Description Default
obj_wrapper Any

Object to wrap, if necessary

 required
stringent bool

Throws an exception if the input type is unhandled. Defaults to False.

 False

Raises:

Type Description
TypeError

neither a CffiNativeHandle nor a CFFI external pointer, nor bytes

Returns:

Type Description
Union[CffiNativeHandle, OwningCffiNativeHandle, GenericWrapper]

Union[CffiNativeHandle, OwningCffiNativeHandle, GenericWrapper, None]: wrapped object or None
Source code in refcount/interop.py 
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
def wrap_as_pointer_handle(
    obj_wrapper: Any, stringent: bool = False
) -> Union[CffiNativeHandle, OwningCffiNativeHandle, GenericWrapper]:
    """Wrap an object, if need be, so that its C API pointer appears accessible via a 'ptr' property

    Args:
        obj_wrapper (Any): Object to wrap, if necessary
        stringent (bool, optional): Throws an exception if the input type is unhandled. Defaults to False.

    Raises:
        TypeError: neither a CffiNativeHandle nor a CFFI external pointer, nor bytes

    Returns:
        Union[CffiNativeHandle, OwningCffiNativeHandle, GenericWrapper, None]: wrapped object or None
    """    
    # 2016-01-28 allowing null pointers, to unlock behavior of EstimateERRISParameters.
    # Reassess approach, even if other C API function will still catch the issue of null ptrs.
    if obj_wrapper is None:
        return GenericWrapper(None)
        # return GenericWrapper(FFI.NULL)  # Ended with kernel crashes and API call return, but unclear why
    elif isinstance(obj_wrapper, CffiNativeHandle):
        return obj_wrapper
    elif isinstance(obj_wrapper, FFI.CData):
        return OwningCffiNativeHandle(obj_wrapper)
    elif isinstance(obj_wrapper, bytes):
        return GenericWrapper(obj_wrapper)
    else:
        if stringent:
            raise TypeError(
                "Argument is neither a CffiNativeHandle nor a CFFI external pointer, nor bytes"
            )
        else:
            return obj_wrapper

wrap_cffi_native_handle(obj, type_id='', release_native=None)

Create a reference counting wrapper around an object if this object is a CFFI pointer

Parameters:

Name Type Description Default
obj Union[CffiData, Any]

An object, which will be wrapped if this is a CFFI pointer, i.e. an instance of CffiData

 required
release_native Callable[[CffiData], None]

function to call on deleting this wrapper. The function should have one argument accepting the object handle.

 None
type_id Optional[str]

An optional identifier for the type of underlying resource. This can be used to usefully maintain type information about the pointer/handle across an otherwise opaque C API. See package documentation.

 ''
Source code in refcount/interop.py 
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
def wrap_cffi_native_handle(
    obj: Union["CffiData", Any],
    type_id: str = "",
    release_native: Callable[["CffiData"], None] = None,
) -> Union[DeletableCffiNativeHandle, Any]:
    """Create a reference counting wrapper around an object if this object is a CFFI pointer

    Args:
        obj (Union[CffiData,Any]): An object, which will be wrapped if this is a CFFI pointer, i.e. an instance of `CffiData`
        release_native (Callable[[CffiData],None]): function to call on deleting this wrapper. The function should have one argument accepting the object handle.
        type_id (Optional[str]): An optional identifier for the type of underlying resource. This can be used to usefully maintain type information about the pointer/handle across an otherwise opaque C API. See package documentation.
    """
    if isinstance(obj, FFI.CData):
        return DeletableCffiNativeHandle(
            obj, release_native=release_native, type_id=type_id
        )
    else:
        return obj