Tool: mp_c2mpt¶
Overview¶
C to MicroprobeTest (mpt) tool provides a command-line interface (CLI) to generate MPT files (Microprobe test (mpt) format) from C source code by following certain guidelines (C test format). Then, from the MPT format, the user can use existing tools (Command line tools) to produce test cases for different environments and formats .
Note
This tool relies on third-party tools (compilers and object code inspectors). The microbenchmark generation policy implemented in this tool reproduces exactly the code provided by the compiler and the object code inspector. The framework resolves any symbolic references (references to data, or branch targets) and adds necessary code to initialize the required registers so that it does not break the target Application Binary Interface.
Basic usage¶
> mp_objdump2mpt -T TARGET -i OBJDUMP_FILE -O MPT_OUTPUT_FILE
where:
Flag/Argument |
Description |
---|---|
|
Target definition string. Check: Command line target definition scheme |
|
C test file following the format in C test format |
|
Output file name |
Tool work-flow¶
The figure below summarizes the high-level work-flow of the C to MPT command line tool:

Fig. 1 C to MPT high level work-flow¶
From a given C file that complies with the C test format, the process is two-fold (see top and bottom flows in the figure).
The C file is compiled using the local host compiler and executed in the local host. This process is necessary to obtain the variable values at run-time just before the test main function c2mpt_function (check C test format) . Also, the host execution with an exit code of 0 confirms the correctness of the code that is provided.
The C file is compiled using the target host compiler (which can be the same as the local) and then the generated instructions are obtained. The instruction sequence is extended to add the necessary initialization to ensure a correct execution of the code. We call this process as environment construction in the figure. For instance, the stack is defined and the stack pointer is initialized.
Finally, all the information (instructions, variables and their contents) is dumped into a MPT file which can then be used to generate test cases in different formats.
The user can control some features of this work-flow using the following flags:
Flag/Argument |
Description |
---|---|
–no-data-initialization |
Do not run the compiled code locally to get the contents of registered variables before executing the c2mpt_function. Only statically specified variable contents will be dumped in the MPT generated |
|
Store the generated intermediate files permanently; place them in the input source file directory and name them based on the source file |
Note
It is up to the user to control the compiler behavior in order to generate a valid test case. For instance, the compiler might consider a function to be useless and therefore, remove it from the generated code.
C test format¶
The input file has to be compliant with the C test case format in order to ensure the generation of a correct test case. Check the section C test format for details. For your convenience, the tool provides a flag to dump an empty C file template, which you can use to start the specification of the C-based test:
Flag/Argument |
Description |
---|---|
|
Dump a template C file, which can be used afterwards as an input file |
Third-party tool customization¶
As explained in the previous section, this tool relies on third-party tools to generate the final MPT. Three external tools are required: a local host compiler, a target host compiler and the objdump utility. By default, cc and objdump commands are assumed, but in case that the user needs to customize these commands, a set of customization flags are provided:
Flag/Argument |
Description |
---|---|
|
Local C compiler (Default:’cc’) |
|
Local C compiler flags (Default:’-Wall -Werror -m64 -O3 -Wpedantic -pedantic -pedantic-errors -std=c99’) |
|
Target C compiler (Default:’cc’) |
|
Target objdump utility (Default:’objdump’) |
|
Target C compiler flags (Default:’-Wall -Werror -m64 -O3 -Wpedantic -pedantic -pedantic-errors -std=c99’) |
Note
Only GNU GCC tool-chains (gcc, objdump) are currently supported.
Customizing Application Binary Interface parameters¶
The ABI (Application Binary Interface) for a given environment (defined in Linux Foundation - Referenced Specifications ) specifies the semantics of registers and the calling conventions. For instance, usually a register is reserved to point to the stack. So, the code being extracted from the objdump output requires that certain registers contain appropriate values. This tool supports the automatic definition of the stack, the automatic initialization of the stack pointer and the definition of the start symbol.
In case the default values are not appropriate, the user can use the following flags to change the different options:
Flag/Argument |
Description |
---|---|
|
Stack size in bytes (default: 4096) |
|
Stack name (Default: microprobe_stack) |
|
Stack address (Default: allocated in the data area) |
|
End the code with a branch to itself instruction |
Note
The necessary instructions required for initializing the stack pointer will be added at the beginning of the code (on lower addresses) and the start code address will be modified accordingly.
Current limitations¶
Code constraints: Do not call library functions in any of the functions to be included in the final MPT. Notice that this restriction does not affect the initialization functions that are not included in the MPT.
Variable initialization constraints: As explained in previous section, the program is executed locally to obtain the variable values. As a consequence, in the case that the variable type is not a basic type or a fixed width type as specified here and if the local host and the target host endianess and/or struct padding differ, the initial values of the variables will not be correct. A warning is triggered to alert the user about this issue.
Code/Data segment addresses: One should be careful when specifying the addresses for the code and data regions. Sometimes, the user may face some limitations for some ranges that are not valid in some environments. For instance, if you get a segmentation fault when running the test locally and you are sure that the code is correct and gdb is telling you that the fault is generated during the program loading phase, then it is very likely that the code/data segment addresses are conflicting with others or they are out of the valid range for that environment.
Full usage¶
usage: mp_c2mpt.py [-h] [-P SEARCH_PATH [SEARCH_PATH ...]] [-V] [-v] [-d]
[-c CONFIG_FILE [CONFIG_FILE ...]] [-C FORCE_CONFIG_FILE]
[--dump-configuration-file OUTPUT_CONFIG_FILE]
[--dump-full-configuration-file OUTPUT_CONFIG_FILE]
[-A ARCHITECTURE_PATHS] [-M MICROARCHITECTURE_PATHS]
[-E ENVIRONMENT_PATHS] -T TARGET [--list-architectures]
[--list-microarchitectures] [--list-environments]
[--traceback] [--profile PROFILE_OUTPUT]
[--host-c-compiler HOST_C_COMPILER]
[--host-cxx-compiler HOST_CXX_COMPILER]
[--target-c-compiler TARGET_C_COMPILER]
[--target-cxx-compiler TARGET_CXX_COMPILER]
[--target-objdump TARGET_OBJDUMP]
[--host-c-compiler-flags HOST_C_COMPILER_FLAGS]
[--host-cxx-compiler-flags HOST_CXX_COMPILER_FLAGS]
[--target-c-compiler-flags TARGET_C_COMPILER_FLAGS]
[--target-cxx-compiler-flags TARGET_CXX_COMPILER_FLAGS]
[-i INPUT_C_FILE] -O OUTPUT_MPT_FILE [-S]
[-X DEFAULT_CODE_ADDRESS] [-D DEFAULT_DATA_ADDRESS]
[--stack-size STACK_SIZE]
[--host-displacement HOST_DISPLACEMENT]
[--fix-displacement] [--stack-name STACK_NAME]
[--stack-address STACK_ADDRESS] [--no-data-initialization]
[--save-temps] [--dump-c2mpt-template]
[--end-branch-to-itself]
Microprobe C to MPT tool
optional arguments:
-h, --help show this help message and exit
-P SEARCH_PATH [SEARCH_PATH ...], --default_paths SEARCH_PATH [SEARCH_PATH ...]
Default search paths for microprobe target definitions
-V, --version Show Microprobe version and exit
-v, --verbosity Verbosity level (Values: [0,1,2,3,4]). Each time this
argument is specified the verbosity level is
increased. By default, no logging messages are shown.
These are the four levels available:
-v (1): critical messages
-v -v (2): critical and error messages
-v -v -v (3): critical, error and warning messages
-v -v -v -v (4): critical, error, warning and info messages
Specifying more than four verbosity flags, will
default to the maximum of four. If you need extra
information, enable the debug mode (--debug or -d
flags).
-d, --debug Enable debug mode in Microprobe framework. Lots of
output messages will be generated
Configuration arguments:
Command arguments related to configuration file handling
-c CONFIG_FILE [CONFIG_FILE ...], --configuration CONFIG_FILE [CONFIG_FILE ...]
Configuration file. The configuration files will be
readed in order of appearance. Values are reset by the
last configuration file in case of non-list values.
List values will be appended (not reset)
-C FORCE_CONFIG_FILE, --force-configuration FORCE_CONFIG_FILE
Force configuration file. Use this configuration file
as the default start configuration. This disables any
system-wide, or user-provided configuration.
--dump-configuration-file OUTPUT_CONFIG_FILE
Dump a configuration file with the actual
configuration used
--dump-full-configuration-file OUTPUT_CONFIG_FILE
Dump a configuration file with the actual
configuration used plus all the configuration options
not set
Target path arguments:
Command arguments related to target paths
-A ARCHITECTURE_PATHS, --architecture-paths ARCHITECTURE_PATHS
Search path for architecture definitions. Microprobe
will search in these paths for architecture
definitions
-M MICROARCHITECTURE_PATHS, --microarchitecture-paths MICROARCHITECTURE_PATHS
Search path for microarchitecture definitions.
Microprobe will search in these paths for
microarchitecture definitions
-E ENVIRONMENT_PATHS, --environment-paths ENVIRONMENT_PATHS
Search path for environment definitions. Microprobe
will search in these paths for environment definitions
Target arguments:
Command arguments related to target specification and queries
-T TARGET, --target TARGET
Target tuple. Microprobe follows a GCC-like target
definition scheme, where a target is defined by a
tuple as following:
<arch-name>-<uarch-name>-<env-name>
where:
<arch-name>: is the name of the architecture
<uarch-name>: is the name of the microarchitecture
<env-name>: is the name of the environment
One can use --list-* options to get the list of
definitions available in the default search paths or
the paths specified by the different --*-paths options
--list-architectures Generate a list of architectures available in the
defined search paths and exit
--list-microarchitectures
Generate a list of microarchitectures available in the
defined search paths and exit
--list-environments Generate a list of environments available in the
defined search paths and exit
Debug arguments:
Command arguments related to debugging facilities
--traceback show a traceback and starts a python debugger (pdb)
when an error occurs. 'pdb' is an interactive python
shell that facilitates the debugging of errors
--profile PROFILE_OUTPUT
dump profiling information into given file (see
'pstats' module)
Compilation arguments:
Command arguments related to compilation options
--host-c-compiler HOST_C_COMPILER
Local C compiler (Default:'cc')
--host-cxx-compiler HOST_CXX_COMPILER
Local C++ compiler (Default:'c++')
--target-c-compiler TARGET_C_COMPILER
Target C compiler (Default:'cc')
--target-cxx-compiler TARGET_CXX_COMPILER
Target C++ compiler (Default:'c++')
--target-objdump TARGET_OBJDUMP
Target objdump utility (Default:'objdump')
--host-c-compiler-flags HOST_C_COMPILER_FLAGS
Local C compiler flags (Default:'-Wall -Werror -m64
-O3 -pedantic -pedantic-errors -std=c99')
--host-cxx-compiler-flags HOST_CXX_COMPILER_FLAGS
Local C++ compiler flags (Default:'-Wall -Werror -m64
-O3 -pedantic -pedantic-errors -std=c99')
--target-c-compiler-flags TARGET_C_COMPILER_FLAGS
Target C compiler flags (Default:'-Wall -Werror -m64
-O3 -pedantic -pedantic-errors -std=c99')
--target-cxx-compiler-flags TARGET_CXX_COMPILER_FLAGS
Target C++ compiler flags (Default:'-Wall -Werror -m64
-O3 -pedantic -pedantic-errors -std=c99')
C to MPT arguments:
Command arguments related to C to MPT tool
-i INPUT_C_FILE, --input-c-file INPUT_C_FILE
C file to process
-O OUTPUT_MPT_FILE, --output-mpt-file OUTPUT_MPT_FILE
Output file name
-S, --strict Be strict when parsing objdump input, if not set,
silently skip unparsed elements
-X DEFAULT_CODE_ADDRESS, --default-code-address DEFAULT_CODE_ADDRESS
Default code address (default: 0x10030000)
-D DEFAULT_DATA_ADDRESS, --default-data-address DEFAULT_DATA_ADDRESS
Default data address (default: 0x10040000)
--stack-size STACK_SIZE
Stack size in bytes (Default: 4096)
--host-displacement HOST_DISPLACEMENT
Displacement between static objdump code and loaded
image on the host. Default computed automatically
--fix-displacement If data contains addresses (such as pointers) to code
or data regions, the tool will try to fix them adding
the necessary displacement
--stack-name STACK_NAME
Stack name (Default: microprobe_stack)
--stack-address STACK_ADDRESS
Stack address (Default: allocated in the data area)
--no-data-initialization
Do not run the compiled code locally to get the
contents of registered variables before executing the
*c2mpt_function*. Only statically specified variable
contents will be dumped in the MPT generated
--save-temps Store the generated intermediate files permanently;
place them in the input source file directory and name
them based on the source file
--dump-c2mpt-template
Dump a template C file, which can be used afterwards
as an input file
--end-branch-to-itself
A branch to itself instruction will be added at the
end of the test
Environment variables:
MICROPROBETEMPLATES Default path for microprobe templates
MICROPROBEDEBUG If set, enable debug
MICROPROBEDEBUGPASSES If set, enable debug during passes
MICROPROBEASMHEXFMT Assembly hexadecimal format. Options:
'all' -> All immediates in hex format
'address' -> Address immediates in hex format (default)
'none' -> All immediate in integer format
CC Default C compiler
CPP Default C++ compiler
TARGET_OBJDUMP Default target objdump utility
CFLAGS Default C compiler flags
CXFLAGS Default C++ compiler flags
mp_c2mpt.py: INFO: Processing input arguments...
Example outputs¶
Example 1:
Command:
> mp_c2mpt.py -T z14-z14-z64_linux_gcc -i input.c -O output.mpt
Input file input.c
:
Output file output.mpt
:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 | ; Microprobe Test Definition File
[MPT]
mpt_version = 0.5 ; Format version of this MPT file.
[REGISTERS] ; Section to specify the initial register values
; Format: register = value. E.g.:
; Set GR0, GR1 and GR2 register to 0, 1, 2 values respectively
;GR0 = 0x0
[DATA] ; Section to specify the variables
; Data section default address. Variables will be placed from this address
; if their address is not specified
default_address = 0x0000000010040000
; Variable Declaration
; Format: var_name = [ "type", nelems, address, alignment, init_values ]
; where:
; - "type": is a string specifying the type of elements in the variable
; - nelems: is the number of elements in the variable
; - address : is the address of the variable, if set the address will be
; fixed, otherwise, it will be computer by microprobe
; - alignment : alignment requirements of the variable. It should not
; conflict with address if specified. It can be set to None
; - init_values : if it is a single value, all the elements will be
; initialized to that value, if it is an array, elements
; will be initialized to the values specified in a round-
; robin fashion. Two special keywords can be specified:
; RNDFP and RNDINT to initialize the elements to random FP
; and random INT values
;
; Note that variable names ARE NOT case sensitive. I.e. VAR = Var = var
count = ["int64_t", 00000001, 0x0000000010040000, 0x0008, 3405695742]
linkedlist = ["uint8_t", 00000080, 0x0000000010040010, 0x0000, [8, 7, 6, 5, 4, 3, 2, 1, 32, 160, 237, 161, 103, 85, 0, 0, 8, 7, 6, 5, 4, 3, 2, 1, 48, 160, 237, 161, 103, 85, 0, 0, 8, 7, 6, 5, 4, 3, 2, 1, 64, 160, 237, 161, 103, 85, 0, 0, 8, 7, 6, 5, 4, 3, 2, 1, 80, 160, 237, 161, 103, 85, 0, 0, 8, 7, 6, 5, 4, 3, 2, 1, 0, 0, 0, 0, 0, 0, 0, 0]]
microprobe_stack = ["uint8_t", 00004096, None, 0x0010, None]
[CODE] ; Section to specify the code
; Code section default address. Code will be placed from this address
; if the instruction address is not specified
default_address = 0x000000001002fff0
; The code specified after 'instructions' entry (below) is the code that will be
; processed by microprobe. The instruction format is similar to GNU assembler
; format, it also allows the specification of labels (NOT case sensitive) and
; references to the declared variables. It is also possible to specify instruction
; addresses and to do code expansion by referencing other user
; defined entries. Check the example below to see examples of these features.
;
; *****************************************************************************
; ****** Although Microprobe performs some sanity checks, it is the ********
; ****** responsibility of the user to define correct code. ********
; ****** ********
; *****************************************************************************
instructions =
<ELF_ABI_START>:
AUIPC x2, %pcrel_hi(microprobe_stack)
ADDI x2, x2, %pcrel_lo(microprobe_stack_pcrel_1)
JAL x1, c2mpt_function
JAL x1, ELF_ABI_EXIT
0x0000000010030000 <C2MPT_FUNCTION>:
LUI x12, 65600
ADDI x12, x12, 0x0
LD x15, 0x8(x12)
LD x14, 0x50(x12)
BEQ x15, x0, C2MPT_FUNCTION+0X24
LD x13, 0x0(x15)
LD x15, 0x8(x15)
ADD x14, x14, x13
BNE x15, x0, C2MPT_FUNCTION+0X14
SLLI x14, x14, 1
SD x14, 0x50(x12)
JALR x0, 0x0(x1)
0x0000000010030030 <MY_SUBROUTINE>:
SLLI x10, x10, 1
LUI x15, 65600
SD x10, 0x50(x15)
JALR x0, 0x0(x1)
<ELF_ABI_EXIT>:
ADDI x0, x0, 0x0
|