Function call tracing

With system calls information crossing the process to kernel boundary can be viewed: function call names, arguments, and result values. The entire process is treated as a black box.

Most programs use system calls at a low frequency, and watching the calls produces more useful information than watching individual machine instructions. System call information is particularly suitable for filtering on the function call name, argument values or result values. This can help to narrow down the search before going down to the machine instruction level for finer detail. Modern systems provide tools for monitoring system calls in real time.

Library call spoofing is a technique that intercepts calls from some program into system libraries. Library call monitors depend entirely on information that exists within the monitored process address space. If a program does not play by the rules, and the monitoring program isn't designed to control hostile code, then hostile code can bypass library call monitoring mechanisms with relative ease.