Mar 26th, 2021 - written by Kimserey with .
Few weeks ago we talked about
setup.py. We demonstrated the configuration of a simple
setup.py file in order to package a project. Although
setup.py used to be the default way of packaging applications, Python has progressed to offer a more generic approach described in PEP 517 and PEP 518 with the
pyproject.toml file. In today’s post we will look at the usage of
pyproject.toml in the context of package management.
In our previous post, we looked at how we could use
setup.py to build source distribution and wheels. We used
python3 setup.py sdist bdist_wheel command to do so. This implied that we had
setuptools installed and that prior hand we knew that our build would be using setup tools.
To understand the problem we can think about what
pip does when installing a package from source tree, it will unzip the
sdist then create a wheel with
python3 setup.py bdist_wheel. This command is problematic as from users machine, on installation, it’s not possible to know what tool is required to execute this command, we can’t be sure whether this uses
setuptool, or other build system like
poetry. This has always be a problem hence the reason why
setuptools was installed by default on all virtual environment. On top of that version of the packaging tool had no way to be specified.
There are two problems solved for the build:
The first issue is addressed in
PEP 518 which introduces a setting in
pyproject.toml file use for “Specifying Minimum Build System Requirements for Python Projects”. The
pyproject.toml is a file that would be at the root of the project. This fille will contain requirements for the build under
1 2 [build-system] requires = ["setuptools", "wheel"]
By adding this file, although it is an extra file, it allows
pip to know prior hand the requirements to build the package which is important for
pip install step.
Now that the first issue is addressed, the second issue is addressed by
PEP 517 which specifies a way of creating “A build-system independent format for source trees”. This specification defines a minimal interface for a tool like
pip to interact with the package source tree. From a user perspective, all we need to do is to specify the
pyproject.toml to point to the module containing the definition of
PEP-517, for example for
setuptools it would be:
1 2 3 [build-system] requires = ["setuptools", "wheel"] build-backend = "setuptools.build_meta"
If we look into
setuptools/setuptools/build_meta.py, we will find the definition of the following methods:
build_wheelcreating the wheel,
build_sdistcreating the source distribution,
get_requires_for_build_wheelreturning the list of requirements to build the wheel,
get_requires_for_build_sdistreturning the list of requirements to build the source distribution.
Here we have been using
setuptools but if we were using
flit we could have specified the build system:
1 2 3 4 5 6 7 8 [build-system] requires = ["flit_core >=2,<4"] build-backend = "flit_core.buildapi" [tool.flit.metadata] module = "my_package" author = "Kimserey Lam" author-email = "[email protected]"
If we look into
flit_core/flit_core/buildapi.py we would see the same PEP 517 implementation. Or even with
1 2 3 4 5 6 7 8 9 [build-system] requires = ["poetry_core>=1.0.0"] build-backend = "poetry.core.masonry.api" [tool.poetry] name = "my_package" version = "0.0.1" description = "hello" authors = [ "kimserey <[email protected]>" ]
If we look into
poetry-core/poetry/core/masonry/api.py we would see PEP 517 implementation.
Another detail that come up by comparing the different
pyproject.toml, we can see that different build system uses
pyproject.toml to define extra configuration metadata in their own ways. With
flit for example, the tools allow us to define every metadata that would be defined in
setup.py allowing us to completely get rid of
setup.py. PEP 517 allows the build system to express themselves in complete freedom as shown by the different tomls where different metadata attributes are used.
Lastly the whole point of providing an interface and hooks is to create a unified way of building packages without being tied to any build system implementation and this is brought by
1 pip install pep517
pep517.build, we are able to build all three examples,
poetry with the same following command:
1 python -m pep517.build .
And that concludes today’s post! We now know the purpose of
pyproject.toml. I hope you liked this post and I see you on the next one!