7.2. How to keep HLS intermediate files generated by Mercurium?ΒΆ

Mercurium generates an intermediate C++ HLS source file for each FPGA task defined in the source code. Those files are used by AIT to generate the FPGA bitstream, and removed after the invocation. To keep the Mercurium intermediate files, there is the -k option. It will keep the C/C++ intermediate files for the native compiler and the C++ HLS files for AIT. Usage example:

fpgacc --ompss --bitstream-generation -k src/dotproduct.c -o dotproduct \
  --Wf,--board=zedboard

Note

HLS source files are not generated if --bitstream-generation option is not used in Mercurium call

Edit the HLS intermediate files and call AIT

It is not recommended, but under some circumstances one would need to edit the HLS intermediate files before the AIT call. The following steps shows how to do so.

  1. We call Mercurium with the desired options and two extra flags: - -k which keeps the intermediate files generated by the compiler. - --verbose which enables the verbose mode of the compiler. This is needed to gather the AIT command call that Mercurium executes.

    We could add an unsupported argument in the AIT flags to make it abort, as we do not want the bitstream before the modifications. For example, we could add the --abort flag to --Wf option of Mercurium. Example of Mercurium call and generated output:

    [user@machine] $ fpgacc --ompss -k --verbose --bitstream-generation --Wf,-b=zcu102,--hwruntime=som,--abort foo.c -o foo
    Loading compiler phases for profile 'fpgacc'
    Compiler phases for profile 'fpgacc' loaded in 0.02 seconds
    Compiling file 'foo.c'
    gcc -E -D_OPENMP=200805 -D_OMPSS=1 -I/home/user/opt/ompss/git/nanox/include/nanox -include nanos.h -include nanos_omp.h -include nanos-fpga.h -std=gnu99 -D_MCC -D_MERCURIUM -o /tmp/fpgacc_zHMisA foo.c
    File 'foo.c' preprocessed in 0.01 seconds
    Parsing file 'foo.c' ('/tmp/fpgacc_zHMisA')
    File 'foo.c' ('/tmp/fpgacc_zHMisA') parsed in 0.02 seconds
    Nanos++ prerun
    Early compiler phases pipeline executed in 0.00 seconds
    File 'foo.c' ('/tmp/fpgacc_zHMisA') semantically analyzed in 0.01 seconds
    Checking parse tree consistency
    Parse tree consistency verified in 0.00 seconds
    Freeing parse tree
    Parse tree freed in 0.00 seconds
    Checking integrity of nodecl tree
    Nodecl integrity verified in 0.00 seconds
    foo.c:1:13: info: unless 'no_copy_deps' is specified, the default in OmpSs is now 'copy_deps'
    foo.c:1:13: info: this diagnostic is only shown for the first task found
    foo.c:2:13: info: adding task function 'foo' for device 'fpga'
    Nanos++ phase
    foo.c:10:3: info: call to task function 'foo'
    foo.c:2:13: info: task function declared here
    FPGA bitstream generation phase analysis - ON
    Compiler phases pipeline executed in 0.01 seconds
    Prettyprinted into file 'fpgacc_foo.c' in 0.00 seconds
    Performing native compilation of 'fpgacc_foo.c' into 'foo.o'
    gcc -std=gnu99 -c -o foo.o fpgacc_foo.c
    File 'foo.c' ('fpgacc_foo.c') natively compiled in 0.02 seconds
    objdump -w -h foo.o 1> /tmp/fpgacc_d9sKiY
    gcc -o foo -std=gnu99 foo.o -Xlinker --enable-new-dtags -L/home/user/opt/ompss/git/nanox/lib/performance -Xlinker -rpath -Xlinker /home/user/opt/ompss/git/nanox/lib/performance -Xlinker --no-as-needed -lnanox-ompss -lnanox-c -lnanox -lpthread -lrt -lnanox-fpga-api
    Link performed in 0.04 seconds
    ait -b=zcu102 --hwruntime=som --abort --wrapper_version=7 -n=foo
    usage: ait -b BOARD -n NAME
    ait error: unrecognized arguments: --abort. Try 'ait -h' for more information.
    Link fpga failed
    Removing temporary filename '/tmp/fpgacc_d9sKiY'
    Removing temporary filename 'foo.o'
    Removing temporary filename '/tmp/fpgacc_zHMisA'
    
  2. Now the intermediate files are available and we could edit them as desired. In the example, 7288177970:1:foo_hls_automatic_mcxx.cpp is available:

    [user@machine] $ ls -l
    total 44
    -rw-r--r-- 1 user user  3735 Jun  5 10:27 7288177970:1:foo_hls_automatic_mcxx.cpp
    -rwxr-xr-x 1 user user 17568 Jun  5 10:27 foo
    -rw-r--r-- 1 user user   235 Jun  4 15:17 foo.c
    -rw-r--r-- 1 user user 13460 Jun  5 10:27 fpgacc_foo.c
    
  3. After modifying the HLS intermediate files, the AIT call that Mercurium usually performs has to be executed. In the verbose output of the first step, there is one line with the invoked call. This call has to be repeated, removing the extra options added to make AIT abort (if any). In the example, the --abort option has to be removed and the AIT command to invoke will be:

    [user@machine] $ ait -b=zcu102 --hwruntime=som --wrapper_version=7 -n=foo
    Using xilinx backend
    Checking vendor support for selected board
    ...