CALF - CApio Logging Facility

Calf is a structured, header-only C++17 logging library for the CAPIO ecosystem. It supports indented JSON output and an opt-in streaming protobuf format for lower-overhead trace creation.

Calf is designed to work in two distinct environments:

• STL safe processes: uses std::ofstream for I/O (StlLogger)
• CLI loggers (STL safe): logs to CLI (StdOutLogger)
• NON STL safe processes: uses raw syscalls whenever STL is not safe (SyscallLogger) Both backends share the same JSON structure and produce identical output formats.

The C++ protobuf format works with both StlLogger and SyscallLogger. It uses Google's generated Protobuf C++ classes and writes .pb files matching calf/protobuf/calf_trace.proto. Each file is a valid calf.proto.TraceFile; flat scope-enter, event, and scope-exit records can be streamed and reconstructed through scope_id and parent_scope_id.

CALF Inspector

CALF includes an interactive Inspector for exploring and analysing structured trace logs. Its responsive web interface provides call-tree navigation, lazy trace loading, search, timing details, and aggregated statistics.

See the CALF Inspector documentation for installation, usage and web-server options.

Requirements

• C++17 or later
• CMake 3.16 or later
• Linux for SyscallLogger backend
• Python 3.10 or later for the Python bindings and Inspector

Testing

The GoogleTest C++ suites and Python binding tests run on Linux and macOS. The raw syscall logger suite is built only on Linux because that backend is not supported on macOS.

cmake -S . -B build -DCALF_TESTS=ON
cmake --build build --parallel
ctest --test-dir build --output-on-failure

CALF_TESTS=ON enables CALF_PYTHON_TESTS by default and builds the Python extension needed by those tests. To run only the Python binding test target:

cmake --build build --target calf_python_tests

Output format

Each log scope opened by START_LOG becomes one JSON object in a root array. Inner LOG calls populate the events array. Scopes close with a ts_exit timestamp.

[
  {
    "invoker": "capio_openat",
    "file": "posix/open.cpp",
    "line": 77,
    "ts_enter": 42,
    "args": "dirfd=4, pathname=/tmp/foo, flags=0",
    "events": [
      { "ts": 43, "invoker": "get_file_location", "file": "location.hpp", "line": 96, "args": "path=/tmp/foo" },
      { "ts": 44, "invoker": "get_file_location", "file": "location.hpp", "line": 96, "args": "File found on node host0" }
    ],
    "ts_exit": 45
  }
]

Each thread writes to its own log file under the log directory, named after the thread ID.

Protobuf output

Select protobuf by linking the application target with calf::protobuf. The existing logger include and macro interface remain unchanged:

target_link_libraries(my_server PRIVATE calf::protobuf)

#include <calf/StlLogger.h>

void handle_request(const char *path) {
    START_LOG(calf_current_tid(), "path=%s", path);
    LOG("processing");
}

CMake uses an installed Google Protobuf package when available and otherwise downloads it from source through FetchContent. Without the calf::protobuf link dependency, StlLogger.h and SyscallLogger.h produce JSON. The syscall backend writes serialized protobuf bytes through raw syscalls; constructing the generated messages still uses Google's C++ runtime and its allocations.

Integration

CMake FetchContent

include(FetchContent)

FetchContent_Declare(
        calf
        GIT_REPOSITORY https://github.com/High-Performance-IO/calf.git
        GIT_TAG <commit-sha>   # pin to a specific commit for reproducible builds
        GIT_SHALLOW TRUE
)

set(CALF_LOG ON CACHE BOOL "" FORCE)
set(CALF_BUILD_PYTHON_BINDINGS OFF CACHE BOOL "" FORCE)
set(CALF_TESTS OFF CACHE BOOL "" FORCE)

FetchContent_MakeAvailable(calf)

Then link each target to the appropriate backend:

# Server binary
target_link_libraries(my_server PRIVATE calf::stl)

# POSIX syscall interceptor (.so)
target_link_libraries(my_interceptor PRIVATE calf::syscall)

CMake options

Flag	Default	Description
CALF_LOG	ON	Enable logging macros. When disabled, logging macros are no-ops.
CALF_TESTS	OFF	Build and register the GoogleTest suites with CTest.
CALF_PYTHON_TESTS	Value of CALF_TESTS	Build the Python extension and register its binding tests.
CALF_BUILD_PYTHON_BINDINGS	OFF	Build and install the private calf._py_calf Python extension.
CALF_PROTOBUF	ON	Build the Google Protobuf C++ logger and calf::protobuf target.
CALF_DEFAULT_COMPONENT_NAME	calf	Component directory and CLI header name used by configured targets.
CALF_DEFAULT_LOG_DIR_NAME	./calf_logs	Default log root when CALF_LOG_DIR is unset.

Pass flags during configuration with -D<flag>=<value>. Common examples:

# Build all C++ and Python tests.
cmake -S . -B build -DCALF_TESTS=ON

# Build only the C++ tests.
cmake -S . -B build -DCALF_TESTS=ON -DCALF_PYTHON_TESTS=OFF

# Build the Python bindings without tests.
cmake -S . -B build -DCALF_BUILD_PYTHON_BINDINGS=ON

# Override compiled-in logging defaults.
cmake -S . -B build \
  -DCALF_DEFAULT_COMPONENT_NAME=my_component \
  -DCALF_DEFAULT_LOG_DIR_NAME=/var/log/calf

Usage

Python bindings

The Python package includes bindings for the STL file logger and stdout logger, as well as the CALF Inspector.

Installation

Build and install the package from the repository root:

python -m pip install .

Install a published release from PyPI with:

python -m pip install capio-calf

For development, install it in editable mode:

python -m pip install -e .

The build uses CMake and pybind11 automatically. To build the extension directly with CMake instead:

cmake -S . -B build -DCALF_BUILD_PYTHON_BINDINGS=ON
cmake --build build

STL logger

calf.Logger is an alias for the STL-backed calf.StlLogger. Use it as a context manager so the logging scope is closed deterministically:

import calf

with calf.Logger("processing request") as logger:
    logger.log("reading input")
    logger.log("request complete")
    print(logger.log_file_name)

CALF automatically uses the current Python function name as the invoker. At module scope it uses the script filename, while interactive sessions use python. Pass invoker= to override the detected name. The other optional constructor arguments are file, line, and tid. If tid is omitted, CALF uses the current thread ID. close() can be used instead of a context manager, and get_log_file_name() returns the STL log path.

CALF_LOG_DIR and CALF_LOG_PREFIX configure file output in the same way as the C++ STL logger.

Stdout logger

The stdout logger supports scoped logging and direct messages:

import calf

options = calf.StdoutLogger.get_options()
options.workflow_name = "example"
options.color = calf.CLI_LEVEL_INFO
options.print_header = True
options.use_color = True
calf.StdoutLogger.set_options(options)

with calf.StdoutLogger("starting") as logger:
    logger.log("working")

calf.StdoutLogger.print("finished")

Available colors are CLI_LEVEL_RESET, CLI_LEVEL_STATUS, CLI_LEVEL_INFO, CLI_LEVEL_WARNING, and CLI_LEVEL_ERROR.

Inspector

Installing the package also installs the bundled CALF Inspector:

calf calf_logs

It can also be launched with python -m calf. See the CALF Inspector documentation for all options.

Server / non-interceptor code

Include StlLogger.h. Logging is enabled by default on every thread.

#include <calf/StlLogger.h>

void handle_request(int tid, const char *path) {
    START_LOG(tid, "call(path=%s)", path);

    LOG("processing path=%s", path);

    // ... do work ...

    LOG("done");
} // ts_exit written here when log goes out of scope

POSIX syscall interceptor

Include SyscallLogger.h. Logging starts disabled to avoid re-entrancy during library setup. Call ENABLE_LOGGER() once setup is complete.

If the interceptor uses syscall_no_intercept from the syscall_intercept library, redirect Calf's syscalls before enabling:

#include <calf/SyscallLogger.h>
#include <libsyscall_intercept_hook_point.h>

// Called once during interceptor initialisation.
void setup() {
    SET_CALF_SYSCALL_HANDLER(syscall_no_intercept);
    ENABLE_LOGGER();
}

long hook(long syscall_number, ...) {
    START_LOG(capio_syscall(SYS_gettid), "call()");
    // ...
}

Use DISABLE_LOGGER() around internal operations that must not appear in the log (e.g. Calf's own file I/O):

void internal_op() {
    DISABLE_LOGGER();   // RAII: re-enables on scope exit
    // ... internal syscalls not logged ...
}

Macros

Macro	Description
START_LOG(tid, fmt, ...)	Opens a log scope. Creates a Logger log RAII object on the stack.
LOG(fmt, ...)	Appends an event to the current scope's events array.
ERR_EXIT(fmt, ...)	Logs then terminates (or throws if continue_on_error is true).
ENABLE_LOGGER()	Activates logging on the calling thread (POSIX build only).
DISABLE_LOGGER()	Suspends logging for the current scope via RAII.
DBG(tid, lambda)	Wraps a lambda in a debug-only log scope. Compiled out in release builds.
SET_CALF_SYSCALL_HANDLER	Sets non intercepted syscall function handler. available only in SyscallLogger backend.

Environment variables

Variable	Description	Default
CALF_LOG_DIR	Root directory for all log output.	./calf_logs
CALF_LOG_PREFIX	Filename prefix for per-thread log files.	Backend-specific

Log files are written to $CALF_LOG_DIR/<backend>/<hostname>/<prefix><tid>.log.