4.4. Ovni FPGA instrumentation

FPGA accelerator instrumentation is provided via ovni. By default, dependency copies and kernel execution are traced when instrumentation is enabled.

4.4.1. Prerequisites

The bitstream needs to be compiled with instrumentation support to create trace accelerator events. Instrumentation can be enabled by adding the -fompss-fpga-instrumentation flag when building the bitstream. More details on building OmpSs@FPGA applications are available in the Compile OmpSs-2@FPGA programs section.

Also, ovni and paraver need to be installed to create and visualize traces. Ovni is automatically installed and updated in cluster installations.

4.4.2. Running the application

Enabling instrumentation

Instrumentation needs to be enabled in nanos6 configuration toml file. The default file is located in the nanos6 installation directory.

$OMPSS_FPGA_HOME/bsc/x86_64/ompss-2/<release>/nanos6/share/nanos6.toml

$OMPSS_FPGA_HOME is defined by the OmpSs@FPGA environment module. The release is the specific release that is being used, 3.2.1 or git, for instance.

This file can be copied to the working directory to edit it and override default nanos6 settings. For instance, to copy the configuration file from the git release, run:

cp $OMPSS_FPGA_HOME/bsc/x86_64/ompss-2/git/nanos6/share/nanos6.toml .

Then, set the instrument entry to ovni:

Then run the application as usual. See Running OmpSs-2@FPGA Programs for more details.

Note

Using an instrumentation-enabled bitstream without enabling instrumentation at the runtime level will result in the application hanging.

After the program finishes, an ovni directory containing ovni traces should be created.

4.4.3. Processing traces

Ovni traces need to be converted to paraver traces to be visualized using paraver. Paraver traces need to be generated from ovni traces for visualization. This is done using ovniemu tool:

ovniemu -x myapp.xtasks.config ovni/

In this example, myapp.xtasks.config is passed to ovniemu (using -x flag) to the tool is able to read accelerator names for properly displaying them.

The output from the emulation process should look like this:

ovniemu: INFO: loaded 16 streams
ovniemu: INFO: sorting looms by name
ovniemu: INFO: loaded 1 looms, 1 processes, 16 threads and 8 cpus
ovniemu: INFO: generated with libovni version 1.10.0 commit unknown
ovniemu: INFO: the following 3 models are enabled:
ovniemu: INFO:    nanos6 1.1.0 '6' (67 events)
ovniemu: INFO:      ovni 1.1.0 'O' (18 events)
ovniemu: INFO:    xtasks 1.0.0 'X' (1 events)
ovniemu: INFO: emulation starts
ovniemu: INFO: loom.fpgan10.770230 burst stats: median/avg/max =  98/102/361 ns
ovniemu: INFO: 100.0% done at avg 1240 kev/s
ovniemu: INFO: processed 1446408 input events in 1.17 s
ovniemu: INFO: writing traces to disk, please wait
ovniemu: INFO: emulation finished ok

In the ovni/ directory, two paraver traces should have been created as well as some paraver config files:

cfg/                     Paraver config files
cpu.pcf                  CPU paraver trace files
cpu.prv
cpu.row
loom.fpgan10.770230/     Ovni trace directory
thread.pcf               Thread trace files
thread.prv
thread.row

FPGA events are emitted to the thread trace. See also Paraver web page for further info on the visualization tool and download links.