[ctypes-users] Optional callbacks (passing null for function pointers)

Discussion:

Christian Jacobsen

2007-10-18 16:50:13 UTC

Hi,

I'm trying to use ctypes and have run into an obstacle related to
callbacks and the fact that I want to sometimes specify a callback,
and sometimes specify NULL (the callback is optional). Say I have, for
example, the following C decls:
typedef int(*callback)(void);
void callme(callback cb);
Where 'callme' is a function in a C library that I want to call, which
takes a 'callback' parameter (where the callback takes no parameters
and returns an int). The 'cb' parameter is optional however and can be
specified as NULL.

In Python I have tried to save myself from doing stupid things, so I
have converted the C bits above into a 'cb' CFUNCTYPE for 'callback'
and a 'callme' CFUNCTYPE which sets 'cb' as its only argument.
cb = ctypes.CFUNCTYPE(ctypes.c_int)
callme = ctypes.CFUNCTYPE(None, cb)(('callme', lib), ((1,),))
I can now declare a callback:
@cb
def callback():
return 5
And call 'callme':
callme(callback)
But I cannot :
callme("oops")
or any other silly things like that (which I am nonetheless quite
likely to do by accident)

This is all very good, except that sometimes I do not want to pass in
a callback at all, but would much rather pass NULL (or None). The
interface to the 'callme' function specifies that this is quite
legitimate as the callback is optional. I would have thought I could
do this:
callme(None)
but this results in an ArgumentError:
ArgumentError("argument 1: <type 'exceptions.TypeError'>: expected
CFunctionType instance instead of NoneType",)
This is not unreasonable as it protects me from accidentally passing
in Null instead of a function pointer, but not really what I need in
this case.

Now, I know that I can use ctype.c_void_p instead of my 'cb' CFUNCTYPE
and I can then pass None or 0 to the 'callme' function and things
behave as I expect:
callmeVoidP = ctypes.CFUNCTYPE(None, ctypes.c_void_p)(('callme',
lib), ((1,),))
callmeVoidP(None) # Works ok
Works as expected. I have however now lost some of my safety net as I
can now pass in pretty much anything I like:
callmeVoidP("oops") # Bad!
callmeVoidP(34) # Not good!
callmeVoidP(<insert almost anything here>) # ARGH!

I have tried pretty much everything I can think of to be able use the
CFUNCTYPEs and pass None (or any equivalent that will result in a null
pointer), but I cannot get anything to work. Basically I would like to
be able to pass None where I have a CFUNCTYPE (or perhaps better would
be CFUNCTYPE_OR_NULL) without allowing me to pass absolutely any
garbage (as I can do with c_void_p).

Any pointers (heh) would be greatly appreciated.

Cheers,
Christian

[Runnable sourcecode below for (potential) convenience]
-----------------8<----------- bar.c -----------8<------------------------
include <stdio.h>

typedef int(*callback)(void);

void callme(callback cb)
{
if(cb != 0)
printf("%d", cb());
else
printf("No callback registered");
}
-----------------8<---------- foo.py -----------8<------------------------
import ctypes
import sys

def tryme(thing, j=25):
""" Helper """
sys.stdout.write(thing.ljust(j))
sys.stdout.flush()
try:
eval(thing, globals())
except Exception, (e):
sys.stdout.write(repr(e))
sys.stdout.write("\n")

# gcc -dynamiclib bar.c -o bar.dylib
# gcc -shared bar.c -o bar.so (I think)

lib = ctypes.cdll.LoadLibrary('bar.dylib')

cb = ctypes.CFUNCTYPE(ctypes.c_int)
callme = ctypes.CFUNCTYPE(None, cb)(('callme', lib), ((1,),))
callmeVoidP = ctypes.CFUNCTYPE(None, ctypes.c_void_p)(('callme', lib), ((1,),))

@cb
def callback():
return 5

tryme('callme(callback)')
tryme('callme(None)')
tryme('callme(0)')
tryme('callme("oops")')
tryme('callme(34)')
tryme('callmeVoidP(callback)')
tryme('callmeVoidP(None)')
tryme('callmeVoidP(0)')
# These result in various exciting kinds of badness
tryme('callmeVoidP("oops")')
tryme('callme(34)')
-----------------8<---------- output -----------8<------------------------
callme(callback) 5
callme(None) ArgumentError("argument 1: <type
'exceptions.TypeError'>: expected CFunctionType instance instead of
NoneType",)
callme(0) ArgumentError("argument 1: <type
'exceptions.TypeError'>: expected CFunctionType instance instead of
int",)
callme(34) ArgumentError("argument 1: <type
'exceptions.TypeError'>: expected CFunctionType instance instead of
int",)
callmeVoidP(callback) 5
callmeVoidP(None) No callback registered
callmeVoidP(0) No callback registered
callmeVoidP("oops") Illegal instruction
callmeVoidP(34) Bus error

Thomas Heller

2007-10-18 19:33:42 UTC

Permalink

Post by Christian Jacobsen
Hi,
I'm trying to use ctypes and have run into an obstacle related to
callbacks and the fact that I want to sometimes specify a callback,
and sometimes specify NULL (the callback is optional). Say I have, for
typedef int(*callback)(void);
void callme(callback cb);
Where 'callme' is a function in a C library that I want to call, which
takes a 'callback' parameter (where the callback takes no parameters
and returns an int). The 'cb' parameter is optional however and can be
specified as NULL.

[...]

Post by Christian Jacobsen
I have tried pretty much everything I can think of to be able use the
CFUNCTYPEs and pass None (or any equivalent that will result in a null
pointer), but I cannot get anything to work. Basically I would like to
be able to pass None where I have a CFUNCTYPE (or perhaps better would
be CFUNCTYPE_OR_NULL) without allowing me to pass absolutely any
garbage (as I can do with c_void_p).

Your analysis is correct. IMO it is a bug that one cannot pass None
as a NULL callback function pointer.

However, it is possible to make a workaround.
When a function has an .argtypes attribute, the actual parameter
in the function call is passed to the .from_param() classmethod
of the .argtypes item. In other words, in this code:

CALLBACK_TYPE = CFUNCTYPE(c_int)

callme.argtypes = [CALLBACK_TYPE]

callme(something)

ctypes will internally call CALLBACK_TYPE.from_param(something)
and pass the result to the actual callme function as parameter.
So, the problem boils down to that the CALLBACK_TYPE .from_param
method does not accept 'None'. With a little bit of monkeypatching
it is possible to supply a custom from_param method:

CALLBACK_TYPE = CFUNCTYPE(c_int)

def from_param(cls, obj):
if obj is None:
return None # return a NULL pointer
from ctypes import _CFuncPtr
return _CFuncPtr.from_param(obj)

CALLBACK_TYPE.from_param = classmethod(from_param)

Please look into Lib/ctypes/__init__.py to see why this is needed.
Each call to CFUNCTYPE creates and returns a new type (slightly simplified),
and you have to monkeypatch that type if you want it to accept None.

Thomas

PS: Please submit a bug report to the Python bug tracker.

Christian Jacobsen

2007-10-18 22:55:14 UTC

Permalink

Post by Thomas Heller
Your analysis is correct. IMO it is a bug that one cannot pass None
as a NULL callback function pointer.
However, it is possible to make a workaround

...

Post by Thomas Heller
CALLBACK_TYPE = CFUNCTYPE(c_int)
return None # return a NULL pointer
from ctypes import _CFuncPtr
return _CFuncPtr.from_param(obj)
CALLBACK_TYPE.from_param = classmethod(from_param)

That works, thanks!

However, I incidentally also need to store some function pointers in a
structure (again they can be null). Ie (using same code as previous
email):
class TestStruct(ctypes.Structure):
_fields_ = [('a', cb)]
TestStruct(None) # TypeError('expected CFunctionType instance,
got NoneType',)
TestStruct(callback) # Works fine
The type error originates around line 2346 in _ctypes.c. I guess this
is because the function pointers are CFuncPtrType and not PointerType,
and only things that are PointerType are allowed to be None. That
looks harder to monkeypatch as far as I can ascertain (but I can
probably work around that with c_void_p). I'll add it to the
bugreport.

Post by Thomas Heller
PS: Please submit a bug report to the Python bug tracker.

Will do.

Your help is much appreciated!
Christian

Lenard Lindstrom

2007-10-18 23:31:58 UTC

Permalink

Post by Christian Jacobsen

Post by Thomas Heller
Your analysis is correct. IMO it is a bug that one cannot pass None
as a NULL callback function pointer.
However, it is possible to make a workaround

...

That works, thanks!
However, I incidentally also need to store some function pointers in a
structure (again they can be null). Ie (using same code as previous
_fields_ = [('a', cb)]
TestStruct(None) # TypeError('expected CFunctionType instance,
got NoneType',)
TestStruct(callback) # Works fine
The type error originates around line 2346 in _ctypes.c. I guess this
is because the function pointers are CFuncPtrType and not PointerType,
and only things that are PointerType are allowed to be None. That
looks harder to monkeypatch as far as I can ascertain (but I can
probably work around that with c_void_p).

Cast None to the function pointer type:

TestStruct(cast(None, cb))

To improve performance you can create a null cb instance:

null_cb = cast(None, cb)

Post by Christian Jacobsen

Post by Thomas Heller

null = cast(None, cb)
buffer(null)[:]

'\x00\x00\x00\x00'

--
Lenard Lindstrom
<len-***@telus.net>

Thomas Heller

2007-10-19 05:55:48 UTC

Permalink

Post by Lenard Lindstrom

Post by Christian Jacobsen

Post by Thomas Heller
However, it is possible to make a workaround

...