.. index::
    double: install; toolchain; OmpSs-2@FPGA

.. _install-ompss2atfpga-toolchain:

Install OmpSs-2\@FPGA toolchain
:::::::::::::::::::::::::::::::

This page should help you install the OmpSs-2\@FPGA toolchain.
However, it is preferable using the pre-build Docker image with the latest stable toolchain.
They are available at `DockerHUB <https://hub.docker.com/r/bscpm/ompss_2_at_fpga>`_.
Moreover, we distribute pre-built SD images for some SoC.
Do not hesitate to contact us at ompss-fpga-support@bsc.es if you need help.

First, it describes the prerequisites to do the toolchain installation.
After that, the following sections explain different approaches to do the installation.


Prerequisites
=============

 - Git Large File Storage (https://git-lfs.github.com/)
 - Python 3.7 or later (https://www.python.org/)
 - Vendor backends:
   - Xilinx Vivado 2021.1 or later (https://www.xilinx.com/products/design-tools/vivado.html)

Git Large File Storage
----------------------

AIT repository uses Git Large File Storage to handle relatively-large files that are frequently updated (i.e. hardware runtime IP files) to avoid increasing the history size unnecessarily.
You must install it so Git is able to download these files.

Follow instructions on their website to install it.

Vendor backends - Xilinx Vivado
-------------------------------

Follow the installation instructions from Xilinx Vitis HLS and Vivado. You will need to enable support for the devices you're working on, as well as install the board files for the given devices.


Stable release
==============

There is a meta-repository that points to latest stable version of all tools: https://github.com/bsc-pm-ompss-at-fpga/ompss-2-at-fpga-releases.
It contains a Makefile which, based on some environment variables, will compile and install the toolchain.
The environment variables are:

* ``TARGET`` [Def: ``native``] Linux architecture that toolchain will target
* ``PREFIX_HOST`` [Def: ``/``] Installation prefix for the host tools (i.e. llvm, ait)
* ``PREFIX_TARGET`` [Def: ``/``] Installation prefix for the target tools (i.e. nanos6, libxdma, libxtasks, ovni)
* ``XTASKS_PLATFORM`` [Def: ``qdma``] Board platform that xtasks backend will target (i.e. zynq, qdma)
* ``XDMA_PLATFORM`` [Def: ``qdma``] Board platform that xdma backend will target (i.e. zynq, qdma)
* ``BUILDCPUS`` [Def: ``nproc``] Number of processes used for building

The following example will cross-build the toolchain for the `aarch64-linux-gnu` architecture and install it in ``/opt/bsc/host-arm64/ompss-2`` and ``/opt/bsc/arm64/ompss-2``:

.. code-block:: text

    git clone --recursive https://github.com/bsc-pm-ompss-at-fpga/ompss-2-at-fpga-releases.git
    cd ompss-2-at-fpga-releases
    export TARGET=aarch64-linux-gnu
    export PREFIX_HOST=/opt/bsc/host-arm64/ompss-2
    export PREFIX_TARGET=/opt/bsc/arm64/ompss-2
    export XTASKS_PLATFORM=zynq
    export XDMA_PLATFORM=zynq
    make


Individual git repositories
===========================

The master branches of all tools should generate a compatible toolchain.
Each package should contain information about how to compile/install itself, look for the README files.
The following points briefly describe each tool and provide a possible build configuration/setup for each one.
We assume that all packages will be installed in a Linux OS in the ``/opt/bsc/arm64/ompss-2`` folder.
Moreover, we assume that the packages will be cross-compiled from an Intel machine to be run on an ARM64 embedded board.

List of tools to install:
 - `AIT <https://github.com/bsc-pm-ompss-at-fpga/ait>`_
 - `Kernel module <https://github.com/bsc-pm-ompss-at-fpga/ompss-at-fpga-kernel-module>`_
 - `xdma <https://github.com/bsc-pm-ompss-at-fpga/xdma>`_
 - `xtasks <https://github.com/bsc-pm-ompss-at-fpga/xtasks>`_
 - `ovni <https://github.com/bsc-pm-ompss-at-fpga/ovni>`_
 - `Nanos6-fpga <https://github.com/bsc-pm-ompss-at-fpga/nanos6-fpga>`_
 - `LLVM <https://github.com/bsc-pm-ompss-at-fpga/llvm>`_

Accelerator Integration Tool (AIT)
----------------------------------

You can install the AIT package through the pip repository ``python3 -m pip install ait-bsc`` or cloning the git repository:

.. code-block:: text

    git clone https://github.com/bsc-pm-ompss-at-fpga/ait
    cd ait
    git lfs install
    git lfs pull
    export AIT_HOME="/path/to/install/ait"
    export DEB_PYTHON_INSTALL_LAYOUT=deb_system
    python3 -m pip install . -t $AIT_HOME

    export PATH=PREFIX/ait/:$PATH
    export PYTHONPATH=$AIT_HOME:$PYTHONPATH

Kernel module
-------------

The driver is only needed to execute the applications.
To compile them, the library must be installed on the host but the kernel module may not be loaded.
Example to cross-compile the driver:

.. code-block:: text

    git clone https://github.com/bsc-pm-ompss-at-fpga/ompss-at-fpga-kernel-module.git
    cd ompss-at-fpga-kernel-module
    export CROSS_COMPILE=aarch64-linux-gnu-
    export KDIR=/home/my_user/kernel-headers
    export ARCH=arm64
    make


XDMA
----

Example to cross-compile the library and install it in the ``/opt/bsc/arm64/ompss-2/libxdma`` folder:

.. code-block:: text

    git clone https://github.com/bsc-pm-ompss-at-fpga/xdma.git
    cd xdma/src/zynq
    export CROSS_COMPILE=aarch64-linux-gnu-
    export KERNEL_MODULE_DIR=/path/to/ompss-at-fpga/kernel/module/src
    make
    make PREFIX=/opt/bsc/arm64/ompss-2/libxdma install


xTasks
------

Example to cross-compile the library and install it in the ``/opt/bsc/arm64/ompss-2/libxtasks`` folder:

.. code-block:: text

    git clone https://github.com/bsc-pm-ompss-at-fpga/xtasks.git
    cd xtasks/src/zynq
    export CROSS_COMPILE=aarch64-linux-gnu-
    export LIBXDMA_DIR=/opt/bsc/arm64/ompss-2/libxdma
    make
    make PREFIX=/opt/bsc/arm64/ompss-2/libxtasks install


ovni
----

Example to cross-compile the library and install it in the ``/opt/bsc/arm64/ompss-2/libovni`` folder:

.. code-block:: text

    git clone https://github.com/bsc-pm-ompss-at-fpga/ovni.git
    mkdir ovni-build
    cd ovni-build
    cmake \
      -DCMAKE_INSTALL_PREFIX=$(PREFIX_TARGET)/libovni \
      -DUSE_MPI=OFF \
      -DCMAKE_C_COMPILER=aarch64-linux-gnu-gcc
    ../ovni
    make
    make install


Nanos6-fpga
-----------

Example to cross-compile the runtime library and install it in the ``/opt/bsc/arm64/ompss-2/nanos6-fpga`` folder:

.. code-block:: text

    git clone https://github.com/bsc-pm-ompss-at-fpga/nanos6-fpga.git
    cd nanos6-fpga
    ./autogen.sh
    mkdir build-fpga-arm64
    cd build-fpga-arm64
    ../configure --prefix=/opt/bsc/arm64/ompss-2/nanos6-fpga --host=aarch64-linux-gnu \
      --enable-fpga \
      --with-xtasks=/opt/bsc/arm64/ompss-2/libxtasks \
      --with-ovni=/opt/bsc/arm64/ompss-2/ovni \
      --disable-discrete-deps \
      --disable-all-instrumentations \
      --enable-stats-instrumentation \
      --enable-verbose-instrumentation \
      --enable-ovni-instrumentation
    make
    make install


LLVM/Clang
----------

Example to build a LLVM/Clang cross-compiler that runs on the host and creates binaries for another platform (ARM64 in the example):

.. code-block:: text

    git clone https://github.com/bsc-pm-ompss-at-fpga/llvm.git
    mkdir build-fpga
    cd build-fpga
    cmake -G Ninja \
      -DCMAKE_INSTALL_PREFIX=/opt/bsc/host-arm64/ompss-2/llvm \
      -DLLVM_TARGETS_TO_BUILD="AArch64" \
      -DCMAKE_BUILD_TYPE=Release \
      -DCLANG_DEFAULT_NANOS6_HOME=/opt/bsc/arm64/ompss-2/nanos6-fpga \
      -DLLVM_USE_SPLIT_DWARF=ON \
      -DLLVM_ENABLE_PROJECTS="clang" \
      -DLLVM_INSTALL_TOOLCHAIN_ONLY=ON \
      -DCMAKE_C_COMPILER=clang \
      -DCMAKE_CXX_COMPILER=clang++ \
      -DLLVM_USE_LINKER=lld \
    ../llvm/llvm
    nina
    ninja install