Commit 689df3e5 authored by Baptiste Esteban's avatar Baptiste Esteban

Merge branch 'development/python-doc' into 'next'

Update Python module documentation

See merge request !113
parents ad348e3e edc2923d
Pipeline #27534 failed with stages
in 5 minutes and 15 seconds
Python Module
=============
.. warning:: This module is under development.
.. warning:: This module is under development. The features explained in the following sections are only available in the ``next`` branch of the project repository
and in the ``unstable`` Conan's channel.
.. toctree::
python/first_step.rst
......
Extend the ``pylena`` module
============================
Writing an extension module
===========================
When writing algorithms using the Pylene C++ library, it is possible to expand them in Python. Bellow is a detailed example of how to do it.
While writing an algorithm in C++ is great for performance issue, it is not easy
to prototype algorithms in this language. To solve this problem, Pylene provides
a library, ``Pylene-numpy``, which makes the automatic conversion between
Pylene images and Numpy arrays, thanks to the Pybind11's ``type_caster``.
1. First, write a conanfile as bellow (it is minimal). It is important to set the pylene ``fPIC`` and ``python`` option to True. Else, the ``pylena_numpy`` library will not be available.
In the following, it is explained how to make use of this library to create an
extension module.
Step 1: Setup the project
--------------------------
At first, a ``conanfile.txt`` is created.
.. code-block:: text
[generators]
cmake_find_package
cmake
[requires]
pylene/head@lrde/stable
pylene/head@lrde/unstable
pybind11/2.6.2
[options]
pylene:fPIC=True
pylene:python=True
pylene:fPIC=True # or pylene:shared=True
In this conanfile, two Conan's generators are used: the ``cmake`` and
the ``cmake_find_package`` generators. In the previous section, it is specified
that the ``cmake_find_package`` one is prefered but an issue in the Conan Center
Index makes it unusable for Pybind11 (see `here
<https://github.com/conan-io/conan-center-index/pull/4445>`_ for more details).
For the same reason, the Pybind11 package is not a runtime dependency of the
``Pylene-numpy`` library and should be specified as a dependency of the
extension project. Finally, the `fPIC` option is specified as an option for
Pylene, which enable to have the ``Pylene-numpy`` library during the
dependencies installation.
2. Then, write a CMakeLists.txt. To use the pylena_numpy library, you should use the target ``Pylene::pylena_numpy``.
Then, below is the ``CMakeLists.txt``:
.. code-block:: cmake
cmake_minimum_required(VERSION 3.8.2)
project(pylena_ext)
cmake_minimum_required(VERSION 3.14)
project(pylene_extension)
set(CMAKE_MODULE_PATH "${CMAKE_MODULE_PATH} ${CMAKE_BINARY_DIR}")
if (NOT EXISTS "${CMAKE_CURRENT_BINARY_DIR}/conanbuildinfo.cmake")
message(FATAL_ERROR "Conan dit not run. Please run conan install.")
endif()
include("${CMAKE_CURRENT_BINARY_DIR}/conanbuildinfo.cmake")
set(CMAKE_MODULE_PATH "${CMAKE_MODULE_PATH}" "${CMAKE_BINARY_DIR}" "${CONAN_BUILD_DIRS_PYBIND11}")
find_package(Pylene REQUIRED)
find_package(pybind11 REQUIRED) # Using system pybind11 or conan one (in this case add it in the conanfile.txt)
pybind11_add_module(pylena_ext module.cpp)
target_link_libraries(pylena-ext PRIVATE Pylene::Pylene Pylene::pylena_numpy)
include(pybind11Install)
pybind11_add_module(pylene_extension)
target_sources(pylene_extension PRIVATE pylene_extension.cpp)
target_link_libraries(pylene_extension PUBLIC Pylene::Pylene-numpy)
Step 2: Writing an extension
----------------------------
3. Finally, bellow is an example of source code.
Below is an exemple of an extension module:
.. code-block:: cpp
#include <pln/core/image_cast.hpp> // Should be included in each Compile Unit
#include <pln/core/image_cast.hpp>
#include <mln/core/image/ndimage.hpp>
#include <mln/core/range/foreach.hpp>
mln::ndbuffer_image iota(/* arguments */)
{
mln::ndbuffer_image result;
/* Process iota */
#include <pybind11/pybind11.h>
return result;
}
#include <stdexcept>
PYBIND11_MODULE(pylena_ext, m)
void iota(mln::ndbuffer_image arg_img)
{
m.doc() = "This is an extension of the pylena module";
m.def("iota", &iota, "Return an image filled using the iota function");
auto img = arg_img.cast_to<std::uint8_t, 2>();
if (!img)
throw std::invalid_argument("iota: input image should be a 2D uint8 image");
std::uint8_t i = 0;
mln_foreach(auto p, img->domain())
{
(*img)(p) = i;
i = i == 255 ? 0 : i + 1;
}
}
This will give a python module named ``pylena-ext``.
PYBIND11_MODULE(pylene_extension, m)
{
pln::init_pylena_numpy(m);
m.def("iota", &iota);
}
>>> from pylena_ext import iota
>>> img = iota(...)
>>> type(img)
numpy.ndarray
There are a few important things to note about this code. At first,
**the header** ``<pln/core/image_cast.hpp>``
**must be included in all the compilation unit of the extension module**.
In this header, there is the ``type_caster`` converting Pylene images into Numpy
arrays and inversely. If not included, some undefined behavior may happen.
Then, a ``iota`` function is defined. This function fills inplace a 2D image
with unsigned element on 8 bits. The argument of this function is a
``mln::ndbuffer_image``, the type-erased version of the Pylene images, in which
the type and the dimension of the image are stored dynamically. However, it is
not considered as an image by the library and cannot be manipulated directly: it
has to be casted to a Pylene image, as it is done in the first lines of the
function, thanks to the ``cast_to`` method. This method takes two template
parameters: the type of the image and its dimension. If it matches the ones
stored dynamically, a pointer to the converted image is returned. Else,
``nullptr`` is returned by the method.
Finally, the last lines define the Python module, thanks to Pybind11. It is
important to note that the ``pln::init_pylena_numpy`` is called at the first
line of the module extension.
**It must be called at the first line of the module definition for an extension module.**
Step 3: Using the extension
---------------------------
Finally, below is an example of how to use the simple module developed above.
>>> from pylene_extension import iota
>>> import numpy as np
>>> img = np.zeros((10, 10)).astype(np.float64)
>>> iota(img)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: iota: input image should be a 2D uint8 image
>>> img = img.astype(np.uint8)
>>> iota(img)
>>> img
array([[ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14, 15, 16, 17, 18, 19],
[20, 21, 22, 23, 24, 25, 26, 27, 28, 29],
[30, 31, 32, 33, 34, 35, 36, 37, 38, 39],
[40, 41, 42, 43, 44, 45, 46, 47, 48, 49],
[50, 51, 52, 53, 54, 55, 56, 57, 58, 59],
[60, 61, 62, 63, 64, 65, 66, 67, 68, 69],
[70, 71, 72, 73, 74, 75, 76, 77, 78, 79],
[80, 81, 82, 83, 84, 85, 86, 87, 88, 89],
[90, 91, 92, 93, 94, 95, 96, 97, 98, 99]], dtype=uint8)
\ No newline at end of file
First steps
===========
Introduction
============
The Pylene library makes possible to expose its features in Python. To this aim,
it provides two components :
* The ``Pylene-numpy`` library, which converts Pylene images into Numpy arrays.
* The ``pylena`` module, which binds some Pylene algorithms in Python.
Currently, the ``pylena`` module is under development and is empty. However, it
is possible to create a Python module using `Pybind11
<https://pybind11.readthedocs.io>`_ and the ``Pylene-numpy`` library. This is
described in the next section.
Installation
------------
^^^^^^^^^^^^
.. important:: For now, only the ``pylena_numpy`` library is available. It makes possible to write Python packages using the ``Pylene`` library. In the future,
the ``pylena`` module will be developped, making the ``Pylene`` algorithms available in Python and easily usable.
Before continuing, it is advised to read :ref:doc:`this section</tutorial/installation>` about the Pylene installation.
Using Conan
^^^^^^^^^^^
This subsection describes how to get the ``Pylene-numpy`` library. Two methods
are explained :
.. code-block:: console
* Getting ``Pylene-numpy`` using Conan to use it in a project.
* Building the ``Pylene-numpy`` library from sources.
In any cases, on Linux, make sure to get the Pylene library using the ``Position
Independent Code`` compiler option since Python modules on Linux are dynamic
libraries. On Windows, the standard installation steps will install the Python
components.
Using Conan (preferred)
-----------------------
When writing a `conanfile` to get the `Pylene` library, some ``options`` have to
be added to get the Python components. Below is an exemple of a
`conanfile.txt`, where the `fPIC` option has been added for Pylene.
.. code-block:: text
$ conan install pylene/head@lrde/stable -o pylene:fPIC=True -o pylene:python=True
[generators]
cmake_find_package
For developpers (Build from source)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[requires]
pylene/head@lrde/unstable
[options]
pylene:fPIC=True # Or pylene:shared=True
If the ``pylene:shared`` option is set to ``True``, append the Conan's option
``--build pylene`` because the prebuilt library is linked to the builder Python
library, which should not be the one in the user system.
Building Pylene using CMake
---------------------------
People willing to build the Pylene library, such as Pylene's developpers, should
execute the following command to build the Python components.
.. code-block:: console
$ git clone https://gitlab.lrde.epita.fr/olena/pylene.git
$ cd pylene
$ git checkout next
$ mkdir build && cd build
$ conan install .. --build missing
$ cmake .. -DCMAKE_POSITION_INDEPENDENT_CODE=ON -DPYLENE_BUILD_PYTHON=ON
$ make pylena_numpy
\ No newline at end of file
$ conan install .. -o pylene:fPIC=True
$ cmake .. -DPYLENE_BUILD_PYTHON=ON -DCMAKE_POSITION_INDEPENDENT_CODE=ON
$ make -j4
As explained above, the `fPIC` option has been added to the Conan install
command. Futhermore, two CMake's options are added:
* ``-DPYLENE_BUILD_PYTHON=ON`` to enable the build process of the Python components.
* ``-DCMAKE_POSITION_INDEPENDENT_CODE=ON`` to compile using the `fPIC` flags of the compiler.
If CMake is runned with ``-DPYLENE_BUILD_PYTHON=ON`` but not with
``-DCMAKE_POSITION_INDEPENDENT_CODE=ON``, a warning will be shown during the
build system generation but it will not prevent the build process to run. So
during the build process, a linker error should be raised for the building of
the ``pylena`` module.
\ No newline at end of file
if (NOT CMAKE_POSITION_INDEPENDENT_CODE)
message(WARNING "Building the Python components without POSITION_INDEPENDENT_CODE enabled. \
Build should fail during the linking part of the compilation of the pylena module.")
endif()
find_package(fmt 6.0 REQUIRED)
add_library(Pylene-numpy)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment