Python C Extension
Mar 12th, 2021 - written by Kimserey with .
Few weeks ago we looked into CPython, the default implementation of Python. We saw that CPython is implemented in C and provides a way to declare extensions implemented in C to be used from Python. In this post we will look into a simple example of a module implemented in C and see how we can use it from the Python interpretor.
Spam Module
We start first by implementing a simple C function printing Spam!. We do that by creating a spam.c file in the root of our package:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
#include "Python.h"
PyObject *spam(void)
{
PyObject_Print(PyUnicode_FromFormat("Spam!"), stdout, 0);
Py_RETURN_NONE;
}
PyMethodDef methods[] = {
{"spam",
(PyCFunction)spam,
METH_NOARGS,
"Say spam."},
{NULL, NULL, 0, NULL}};
PyModuleDef module = {
PyModuleDef_HEAD_INIT,
"spam",
"Spam module.",
-1,
methods};
PyMODINIT_FUNC PyInit_spam(void)
{
return PyModule_Create(&module);
}
The whole point of the module is to expose our C function spam which prints to stdout “Spam!”:
1
2
3
4
5
PyObject *spam(void)
{
PyObject_Print(PyUnicode_FromFormat("Spam!"), stdout, 0);
Py_RETURN_NONE;
}
But to do so we have to register the module and register the method within the module. Here we see that we include the header Python.h which contains the Python C API. Then looking from bottom to top;
1
PyMODINIT_FUNC PyInit_spam(void)
will be the function that the interpretor will call, it has to follow PyInit_{module name}. Within this method, we call PyModule_Create which takes the module as argument:
1
2
3
4
5
6
PyModuleDef module = {
PyModuleDef_HEAD_INIT,
"spam",
"Spam module.",
-1,
methods};
We define a PyModuleDef which contains the name of the module, the documentation of the module and the method array which the module contains.
1
2
3
4
5
6
PyMethodDef methods[] = {
{"spam",
(PyCFunction)spam,
METH_NOARGS,
"Say spam."},
{NULL, NULL, 0, NULL}};
The method array is an array pointing to our method created. Our function has to be casted to a PyCFunction as the definition of PyMethodDef requires it. Once we are done with the definition of the C modules, we can move on to building and using it.
Use The Extension
To build it we can use setuptools by defining a setup.py which specifies the ext_modules:
1
2
3
4
5
6
7
8
from setuptools import setup, Extension
setup(
name="spam",
version="0.0.1",
description="spam module",
ext_modules= [Extension("spam", sources=["spam.c"])]
)
After that we can build the extension with build_ext and specify --inplace so that the resulting shared library .so file will be at the root, next to spam.c:
1
2
3
4
5
6
7
8
9
❯ python3 setup.py build_ext --inplace
running build_ext
building 'spam' extension
creating build
creating build/temp.macosx-10.9-x86_64-3.8
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -arch x86_64 -g -I/Users/klam/projects/spam/venv/include -I/Library/Frameworks/Python.framework/Versions/3.8/include/python3.8 -c spam.c -o build/temp.macosx-10.9-x86_64-3.8/spam.o
creating build/lib.macosx-10.9-x86_64-3.8
gcc -bundle -undefined dynamic_lookup -arch x86_64 -g build/temp.macosx-10.9-x86_64-3.8/spam.o -o build/lib.macosx-10.9-x86_64-3.8/spam.cpython-38-darwin.so
copying build/lib.macosx-10.9-x86_64-3.8/spam.cpython-38-darwin.so ->
This has generated the following library:
1
spam.cpython-38-darwin.so
We can then use the Python interpretor and load the module and call it:
1
2
3
4
5
6
7
❯ python3
Python 3.8.4 (v3.8.4:dfa645a65e, Jul 13 2020, 10:45:06)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import spam
>>> spam.spam()
'Spam!'
Internally, CPython will load spam via dynamic linking of spam.cpython-38-darwin.so and will call PyMODINIT_FUNC PyInit_spam(void) on import.
And that concludes today’s post! Hope you liked this post and I see you on the next one!