.. index::
    double: compile; OmpSs-2@FPGA

.. _compile-ompss2atfpga-programs:

Compile OmpSs-2\@FPGA programs
::::::::::::::::::::::::::::::

To compile an OmpSs-2\@FPGA program you should follow the general OmpSs-2 compilation
procedure using the LLVM/Clang compiler. More information is provided in the
OmpSs-2 User Guide (https://pm.bsc.es/ftp/ompss-2/doc/user-guide/llvm/index.html).
The following sections detail the specific options of LLVM/Clang to generate the
binaries, bitstream and boot files.

The entire list of LLVM/Clang options (for the FPGA phase) and AIT arguments are available here:

.. toctree::
   :maxdepth: 1

   llvm-fpga-options
   ait-options

Binaries
========

To compile applications with LLVM/Clang you must add the flag ``-fompss-2`` when using either:
 - ``clang++`` for C++ applications.
 - ``clang`` for C applications.

Bitstream
=========

.. note::
    LLVM/Clang expects the Accelerator Integration Tool (AIT) to
    be available on the PATH, if not the linker will fail.
    Moreover, AIT expects Vivado HLS (or Vitis HLS) and Vivado to be available in the PATH.

.. warning::
    Sourcing the Vivado ``settings.sh`` file may break the
    cross-compilation toolchain. Instead, just add the directory of vivado binaries
    in the PATH.

To generate the bitstream, you should enable the bitstream generation in the
LLVM/Clang compiler (using the ``-fompss-fpga-wrapper-code`` flag) and provide the
FPGA linker (aka AIT) flags with ``-fompss-fpga-ait-flags`` option.
If the FPGA linker flags does not contain the ``-b`` (or ``--board``) and ``-n``
(or ``--name``) options, the linker phase will fail.

For example, to compile the dotproduct application, in debug mode, for the
Alveo U200, with a target frequency of 300Mhz, you can use the following command:

.. code:: bash

    clang -fompss-2 -fompss-fpga-wrapper-code \
      src/dotproduct.c -o dotproduct-d \
      -fompss-fpga-ait-flags "--board=alveo_u200,--clock=300,--name=dotproduct"

.. HW Instrumentation
.. ------------------
.. 
.. You can use the ``--instrument`` (or ``--instrumentation``) option of Mercurium to
.. enable the HW instrumentation generation.
.. The instrumentation can be generated and not used when running the application,
.. but if you generate the bitstream without instrumentation support you will not be
.. able to instrument the executions in the FPGA accelerators.
.. Note that the application binary also has to be compiled with the ``--instrument``
.. (or ``--instrumentation``) option.
.. 
.. For example, the previous compilation command with the instrumentation available
.. will be:
.. 
.. .. code:: bash
.. 
..     arm-linux-gnueabihf-fpgacc --instrument --ompss --bitstream-generation \
..       src/dotproduct.c -o dotproduct-d \
..       --Wf,"--board=zedboard,--clock=100,--name=dotproduct,--hwruntime=som"

Shared memory port
------------------

By default, LLVM/Clang generates an independent port to access the main memory
for each task argument. Moreover, the bit-width of those ports equals to the
argument data type width.
This can result in a huge interconnection network when there are several task
accelerators or they have several non-scalar arguments.

This behavior can be modified to generate unique shared port to access the main
memory between all task arguments. This is achieved with the ``-fompss-fpga-memory-port-width``
option of LLVM/Clang which defines the desired bit-width of the shared port.
The value must be a common multiple of the bit-widths for all task arguments.

.. Also, values above 128 may require properly setting the alignment in FPGA allocator
.. (see option ``fpga-alloc-align`` in :ref:`nanox-fpga-help`).

The usage of the LLVM/Clang variable to generate a 128 bit port in the previous
dotproduct command will be like:

.. code:: bash

    clang -fompss-2 -fompss-fpga-wrapper-code \
      src/dotproduct.c -o dotproduct-d \
      -fompss-fpga-memory-port-width 128 \
      -fompss-fpga-ait-flags "--board=alveo_u200,--clock=300,--name=dotproduct"

Boot Files
==========

Some boards do not support loading the bitstream into the FPGA after the boot,
therefore the boot files should be updated and the board rebooted.
This step is not needed for the z7000 family of devices as the bitstream can
be loaded after boot. AIT supports the generation of boot files for some
boards but the step is disabled by default and should be enabled by hand.

First, you need to set the following environment variables:
 - ``PETALINUX_INSTALL``. Petalinux installation directory.
 - ``PETALINUX_BUILD``. Petalinux project directory.
   See :ref:`build-ultrascale-boot-files` to have more information about how to
   setup a petalinux project build.

Then you can invoke AIT with the same options provided in ``-fompss-fpga-ait-flags`` and
the following new options: ``--from_step=boot --to_step=boot``. Also, you may directly
add the ``--to_step=boot`` option in ``-fompss-fpga-ait-flags`` during the LLVM/Clang launch.