Installation#
It is possible to install the main c++ MParT library and its python wrapper via conda-forge:
conda install -c conda-forge mpart
Matlab, and CUDA are currently only supported when compiling from source.
It is also possible to install MParT with Julia bindings using the Julia package manager:
using Pkg
Pkg.add("MParT")
Julia-specific documentation is hosted here.
Compiling from Source#
The MParT source code can be obtained in the MeasureTransport/MParT repository on Github.
MParT uses CMake to handle dependencies and compiler configurations. A basic build of MParT that should work on most operating systems can be obtained with:
cd </path/to/MParT>
mkdir build
cd build
cmake \
-DCMAKE_INSTALL_PREFIX=<your/MParT/install/path> \
-DKokkos_ENABLE_THREADS=ON \
..
make install
This will compile the main c++ mpart library as well as any other language bindings that can be automatically configured. If you are compiling on a multicore machine, you can use make -j N_JOBS install, where N_JOBS is the number of processes the computer can compile with in parallel.
This installation should also automatically install and build Kokkos, Eigen, Cereal, Pybind11, and Catch2, assuming they aren’t installed already. If CMake has trouble finding prior installations of these, then you can configuring CMake using:
cmake \
-DCMAKE_INSTALL_PREFIX=<your/MParT/install/path> \
-DKokkos_ROOT=<your/kokkos/install/root> \
-DEigen3_ROOT=<your/eigen3/install/root> \
-Dcereal_ROOT=<your/cereal/install/root> \
-DKokkos_ENABLE_THREADS=ON \
-DKokkos_ENABLE_SERIAL=ON \
..
Feel free to mix and match previous installations of Eigen, Cereal, Kokkos, Pybind11, and Catch2 with libraries you don’t already have using these X_ROOT flags. Note that Catch2 and Kokkos in this example will need to be compiled with shared libraries. MParT has not been tested with all versions of all dependencies, but it does require CMake version >=3.13. Further, it has been tested with Kokkos 3.7.0, Eigen 3.4.0, Pybind11 2.9.2, Cereal 1.3.2, and Catch2 3.1.0 (there have been some issues encountered when compiling MParT with Catch2 3.0.1).
Tip
If you are using Kokkos <3.7.0, you will need to use the Kokkos_ENABLE_PTHREAD flag instead of Kokkos_ENABLE_THREADS in the CMake configuration.
You can force MParT to use previously installed versions of the dependencies by setting MPART_FETCH_DEPS=OFF. The default value of MPART_FETCH_DEPS=ON will allow MParT to download and locally install any external dependencies using CMake’s FetchContent directive.
Note that if you do not wish to compile bindings for Python, Julia, or Matlab, you can turn off binding compilation by setting the MPART_<language>=OFF variable during CMake configuration. For a default build with only the core c++ library, and without requiring the Cereal library, you can use
cmake \
-DCMAKE_INSTALL_PREFIX=<your/MParT/install/path> \
-DKokkos_ENABLE_THREADS=ON \
-DMPART_PYTHON=OFF \
-DMPART_MATLAB=OFF \
-DMPART_JULIA=OFF \
-DMPART_ARCHIVE=OFF
..
See more details on MParT serialization, powered by the Cereal library, in the serialization section.
MParT is built on Kokkos, which provides a single interface to many different multithreading capabilities like threads, OpenMP, CUDA, and OpenCL. A list of available backends can be found on the Kokkos wiki. The Kokkos_ENABLE_THREADS option in the CMake configuration above can be changed to reflect different choices in device backends. The OSX-provided clang compiler does not support OpenMP, so THREADS is a natural choice for CPU-based multithreading on OSX. However, you may find that OpenMP has slightly better performance with other compilers and operating systems.
Tests#
The command make install will also create a test executable called RunTests in the build directory. The tests can be run with:
./RunTests
Or, with the additional specification of the number of Kokkos threads to use:
./RunTests --kokkos-threads=4
Environment Paths#
The final step is to set the relevant path variables to include the installation of MParT:
export PYTHONPATH=$PYTHONPATH:<your/MParT/install/path>/python
export DYLD_LIBRARY_PATH=$DYLD_LIBRARY_PATH:<your/MParT/install/path>/lib:<your/MParT/install/path>/python
export PYTHONPATH=$PYTHONPATH:<your/MParT/install/path>/python
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:<your/MParT/install/path>/lib:<your/MParT/install/path>/python
Tip
Depending on your python configuration, pybind11 may throw an error during configuration that looks like
CMake Error in bindings/python/CMakeLists.txt:
Imported target "pybind11::module" includes non-existent path
This often results when due to conda environment mismatches, but can typically be circumvented by explicitly setting the path to your python executable. When calling cmake, add -DPYTHON_EXECUTABLE=`which python`.
Tip
On OSX, using MParT with the system version of python might result in an error with something like:
ImportError: dlopen(pympart.so, 2): no suitable image found. Did find:
MParT/python/mpart/pympart.so: mach-o, but wrong architecture
MParT/python/mpart/pympart.so: mach-o, but wrong architecture
You can sometimes force OSX to use the x86_64 version of python using the arch executable. For example, to run a script test.py, you can use
arch -x86_64 /usr/bin/python test.py
Julia Source Installation#
By default, MParT will look for Julia during configuration and will attempt to build the Julia bindings if the Julia CxxWrap package is installed. To install CxxWrap, run the following command in your Julia prompt:
import Pkg; Pkg.add("CxxWrap")
To prevent the Julia bindings from being compiled, even if Julia and CxxWrap are installed, set MPART_JULIA=OFF during the CMake configuration.
Once MParT is installed with Julia bindings (i.e. MPART_JULIA=ON) into /your/MParT/install/path (an equivalent path to CMAKE_INSTALL_PREFIX), you can use MParT in Julia with a few more steps. First, add MParT.jl, which holds the Julia interface for MParT, via using Pkg; Pkg.add("MParT") in the Julia REPL. Then, create a file ~/.julia/artifacts/Overrides.toml with the following lines
[bee5971c-294f-5168-9fcd-9fb3c811d495]
MParT = "/your/MParT/install/path"
Make sure that this file includes a full installation path from root! At this point, you should be able to open up a REPL and type using MParT and get going with any of the provided examples. If you want to develop MParT’s bindings on the Julia-side, then use using Pkg; Pkg.develop("MParT") instead of Pkg.add("MParT") to install the package.
Tip
If you installed Julia with Conda, you may not have a folder at ~/.julia. In this case, you will likely find the artifacts folder in ~/anaconda3/envs/<YOUR ENVIRONMENT>/share/julia/artifacts (or alternatively, ~/miniconda, depending on what version of Conda you installed). If this is the case, then you will need to create a file ~/anaconda3/envs/<YOUR ENVIRONMENT>/share/julia/artifacts/Overrides.toml with the same contents as above.
Compiling with CUDA Support#
Building the Kokkos Dependency#
To support a GPU at the moment, you need a few special requirements. Due to the way that Kokkos handles GPU code, MParT must be compiled using a special wrapper around NVCC that Kokkos provides. Because of this, MParT cannot use an internal build of Kokkos and Kokkos must therefore be compiled (or otherwise installed) manually.
The following cmake command can be used to compile Kokkos with the CUDA backend enabled and with all options required by MParT. Kokkos source code can be obtained from the kokkos/kokkos repository on Github.
cd <path/to/kokkos>
mkdir build
cd build
cmake \
-DCMAKE_INSTALL_PREFIX=</new/kokkos/install/path> \
-DBUILD_SHARED_LIBS=ON \
-DKokkos_ENABLE_SERIAL=OFF \
-DKokkos_ENABLE_THREADS=ON \
-DKokkos_ENABLE_CUDA=ON \
-DKokkos_ENABLE_CUDA_LAMBDA=ON \
-DCMAKE_CXX_STANDARD=17 \
../
Replace the Kokkos_ARCH_VOLTA70 as needed with whatever other arch the compute resource uses that Kokkos supports. If you aren’t sure, try omitting this as Kokkos has some machinery to detect such architecture.
Tip
If Kokkos may not be able to find your GPU information automatically, consider including -DKokkos_ARCH_<ARCH><VERSION>=ON where <ARCH> and <VERSION> are determined by the Kokkos documentation. If Kokkos cannot find CUDA, or you wish to use a particular version, use -DKokkos_CUDA_DIR=/your/cuda/path.
Tip
If you’re getting an error about C++ standards, try using a new version of your compiler; g++, for example, does not support the flag --std=c++17 below version 8. For more details, see this issue in Kokkos.
Installing cublas and cusolver#
MParT uses the CUBLAS and CUSOLVER components of the NVIDIA CUDA Toolkit for GPU-accelerated linear algebra.
NVIDIA’s Cuda installation guide provides detailed instructions on how to install CUDA. For Debian-based x86_64 systems, we have been able to successfully install cuda, cublas, and cusparse for CUDA 11.4 using the command below. Notice the installation of *-dev packages, which are required to obtain the necessary header files. Similar commands may be useful on other systems.
export CUDA_VERSION=11.4
export CUDA_COMPAT_VERSION=470.129.06-1
export CUDA_CUDART_VERSION=11.4.148-1
curl -sL "https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/7fa2af80.pub" | apt-key add -
echo "deb https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/ /" > /etc/apt/sources.list.d/cuda.list
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys A4B469963BF863CC
sudo apt-get -yq update
sudo apt-get -yq install --no-install-recommends \
cuda-compat-${CUDA_VERSION/./-}=${CUDA_COMPAT_VERSION} \
cuda-cudart-${CUDA_VERSION/./-}=${CUDA_CUDART_VERSION} \
libcublas-${CUDA_VERSION/./-} \
libcublas-dev-${CUDA_VERSION/./-} \
libcusolver-${CUDA_VERSION/./-} \
libcusolver-dev-${CUDA_VERSION/./-}
Building MParT#
Using the above documentation on building with an external install of Kokkos, we can then configure MParT from the build directory using the following command:
cd <path/to/MParT>
mkdir build
cd build
cmake \
-DCMAKE_INSTALL_PREFIX=<your/MParT/install/path> \
-DKokkos_ROOT=</new/kokkos/install/path> \
-DCMAKE_CXX_COMPILER=</new/kokkos/install/path>/bin/nvcc_wrapper \
..
Make sure that CMAKE_CXX_COMPILER uses a full path from the root!
Tip
If you’re using a Power8 or Power9 architecture, Eigen may give you trouble when trying to incorporate vectorization using Altivec, specifically when compiling for GPU. In this case, go into CMakeFiles.txt and add add_compile_definition(EIGEN_DONT_VECTORIZE).