NoBug is a debugging library for instrumenting C and C++ programs inspired by ideas originating from Design-by-Contract.
Everyone makes mistakes, but with NoBug you won’t make them twice!
Overview
The following features are provided by NoBug:
-
Three different check levels: from detailed to final no-overhead
-
Scope tags: tell whenever a function or loop is considered to be bug free
-
Precondition, Postcondition and Invariant checks and generic assertions
-
Data structures can be dumped
-
Application activities can be logged
-
Runtime customizable logging via an environment variable
-
Different logging targets to stderr, syslog, debugger, …
-
Annotation of your sourcecode about known bugs, things to do, etc.
-
Tracking resources (files, locks, etc.) used by your program; help in detecting misuse
-
Detecting potential deadlocks
-
Simulate errors by injecting faults
-
Coverage checking to find out if all (instrumentated) errors are properly handled
-
Additionally, the NoBug project is used to maintain a script and some tools to setup testsuites
In contrast to traditional debuggers, NoBug is a non-interactive debugger that is linked to your application doing hard-coded tests in an efficient way without much overhead. Depending on the build level you choose, NoBug features will be an integral part of your application making it possible to gather debugging data for each stage of the application development, including collecting debugging info for finally deployed applications.
NoBug is a (macro-)library, it is not a C/C++ language extension. This means that code must be called at runtime to benefit from the set up contracts. What’s not tested is likely slipping through the net. Being part of the program itself, it is affected by memory corruption. Certain kinds of misuse may introduce new bugs (assert expressions with side effects for example).
Support
When you need help with NoBug, have some ideas about potential features or think you found a bug, then you can contact the NoBug community and developers by sending mail to the NoBug mailinglist. Subscription is available at http://lists.pipapo.org/cgi-bin/mailman/listinfo/nobug.
Building and Installing
Supported Platforms
NoBug has been developed on Linux, using GCC. It should be possible to port it to any other POSIX compliant operating system. Platform/compiler specific things are kept optional. Currently Linux with a GCC that conforms to C99 is supported for both 32 and 64 bit architectures. For non-GCC compilers, some features are not available or degraded.
CPU | OS | Compiler | State |
---|---|---|---|
x86_64 |
Debian |
gcc4 |
supported |
x86 |
other Linux |
gcc4 |
supported |
armel |
maemo5 |
gcc4 |
supported |
x86* |
Mac OS X |
gcc4 |
supported |
x86 |
OpenSolaris |
gcc4 |
supported |
x86 |
OpenSolaris |
suncc |
mostly |
*BSD |
planned |
NoBug has few mandatory dependencies on other software and libraries, some things such as valgrind support are optional and should be automatically detected by ./configure. Nevertheless it requires pkg-config to be installed or you’ll get some weird errors at bootstrapping (autoreconf) already.
Release Tarballs
Releases are available at: http://www.pipapo.org/nobug-releases/
Gpg signed tarballs are being used for distribution. The first step involves checking the signature:
$ gpg nobug-VERSION.tar.gz.gpg
This will produce a nobug-VERSION.tar.gz and report if the signature could be validated.
Since they are built with GNU Autotools, the usual build and install procedure will work:
$ tar xzvf nobug-VERSION.tar.gz $ cd nobug-VERSION $ mkdir -p build $ cd build $ ../configure $ make $ make check # optional, runs the testsuite $ make install # to be done as root, # depending on distribution and setup
Development Version via Git
You can obtain a development version using Git. The Git repository can be cloned from git://git.pipapo.org/nobug.
Clone the Git repository with:
$ git clone git://git.pipapo.org/nobug
After cloning the repository, then bootstrap the Autotools:
$ cd nobug $ autoreconf -i # creates the configure file
Then the usual
$ cd build && ../configure && make && make install
(as above) will work. Careful users may run
$ make check
to run a testsuite before installing.
Keeping Git Up To Date
To update to any new revision, just enter the nobug dir and
$ git pull
After that you can build as above
$ cd build && ../configure && make && make install
This default pull will update from the master branch which is meant to be an on-going stable version (latest release and bugfixes).
Major new releases are assembled in the devel branch, generally this is not considered production ready.
All other branches are volatile and may be deleted or rebased anytime without further notice.
What Is Installed
Currently, NoBug installs the following:
-
A single nobug.h headerfile. Include this in your code.
-
Static libraries. Statically link these to your application:
-
libnobug.a for singlethreaded programs.
-
libnobugmt.a for multithreaded programs.
-
-
Dynamic Libraries. Dynamically link these to your application:
-
libnobug.so for singlethreaded programs.
-
libnobugmt.so for multithreaded programs.
-
associated libtool descriptors (libnobug*.la)
-
-
Pkgconfig control files:
-
nobug.pc for singlethreaded programs.
-
nobugmt.pc for multithreaded programs.
-
-
The nobug_rbdump utility to inspect NoBug ringbuffers.
Generating This Documentation
There are Makefile targets for generating the documentation, either one of the following does what you might expect:
$ make nobug_manual.txt nobug_manual.html nobug_manual.pdf
Alternatively, you can generate all the documentation in one go as follows:
$ make doc
Building the documentation has quite some more dependencies than building NoBug itself. Unless you are a packager you may prefer to refer to the online documentation or the shipped README which is the complete NoBug reference manual in text form. Generating the documentation requires: gawk, asciidoc, graphviz and LaTeX. Check the [header] section of doc/latex.conf for required packages.
Using NoBug
Your application will have to include the header file nobug.h before NoBug can be used. Prior including this, a Build-level has to be choosen, Build-levels are discussed later, c.f., buildlevel. Build-levels are used to define the amount of information NoBug provides to you. Maximum information is generally required while developing an application and the ALPHA build-level is most apropriate during this phase; whereas the released phase of an application will usually only require sparse information, for which the RELEASE build-level has been conceived.
A build-level must always be specified, otherwise the compiler will complain while attempting to compile your application. You can specifiy a build level in one of two ways: use a define statement in one of your modules, or pass the build-level using the -D flag to your compiler. Assuming we’d like to select the ALPHA build-level in your application, then your module would assume the following form:
#define EBUG_ALPHA #include <nobug.h>
Subsequently you’ll have to link the appropriate library to your application.
A number of different libraries are available to link depending on whether you require to statically or dynamically link, or whether your application is multi or single threaded. The link order is important on your link line. Link the NoBug library after your application’s modules. Here’s how to statically link, single-threaded applications:
gcc -o mybinary $(WHATEVER_FLAGS) \ mymodules.o ... -lnobug
However, for more flexibility in selecting a build-level, you might wish to define various targets in your makefile, one for each build-level. In such cases, the -D flag in your makefile is most appropriate; so your link line for an ALPHA build with multi-threaded support would look like the following:
gcc -o mybinary -DEBUG_ALPHA $(WHATEVER_FLAGS) \ mymodules.o -lnobugmt
The NoBug libraries must be initialised before they can be used. To initialise it call the NOBUG_INIT macro, which must be used before any NoBug features can be used or any thread is created. This is discussed in more detail in the multithreading chapter.
So putting all this together, our application using NoBug might look something like the following:
#define EBUG_ALPHA /* If we don't use the -D Flag to cc */ #include <nobug.h> /* Include the NoBug API */ int main() { NOBUG_INIT; /* Initialise NoBug libs */ ... }
Many aspects of NoBug can be configured by overriding macros before nobug.h is included.
Autotools Support
A project using NoBug can use autoconf to check for execinfo and valgrind:
AC_CHECK_HEADER([execinfo.h], AC_DEFINE(HAVE_EXECINFO_H)) PKG_CHECK_MODULES(VALGRIND, [valgrind], [AC_DEFINE(HAVE_VALGRIND_H)])
For Multithreaded programs, you should also check for the availability of pthreads and flavour with the ACX_PTHREAD maxro, see:
When the resulting HAVE_PTHREAD, HAVE_EXECINFO_H and HAVE_VALGRIND_H are defined by the configure script, the relevant features become available.
NOBUG_USE_PTHREAD, NOBUG_USE_VALGRIND and NOBUG_USE_EXECINFO will be defined depeding on the information gathered by configuration. If you do not want to use any of these features in NoBug, you can override these macros by setting them to 0 before including nobug.h.
If NVALGRIND is defined, there will be no support for valgrind.
There are many other macros which can be set and overridden by the user to control behavior. Your help would be appreciated in expanding this documentation if you find some features useful; or simply contact any of the authors.
Here is a rather elaborate snippet how to put this all together into a configure.ac file:
# define autoheader templates for the config macros AH_TEMPLATE(EBUG_ALPHA, [Define to 1 for selecting NoBug ALPHA build level]) AH_TEMPLATE(EBUG_BETA, [Define to 1 for selecting NoBug BETA build level]) AH_TEMPLATE(NDEBUG, [Define to 1 for selecting NoBug RELEASE build level]) # set configure options up AC_ARG_ENABLE(alpha, AC_HELP_STRING([--enable-alpha], [select NoBug ALPHA build level]), nobug_level=alpha AC_DEFINE(EBUG_ALPHA), [ AC_ARG_ENABLE(beta, AC_HELP_STRING([--enable-beta], [select NoBug BETA build level]), nobug_level=beta AC_DEFINE(EBUG_BETA), [ AC_ARG_ENABLE(release, AC_HELP_STRING([--enable-release], [select NoBug RELEASE build level]), nobug_level=release AC_DEFINE(NDEBUG), # default to ALPHA nobug_level=alpha AC_DEFINE(EBUG_ALPHA) )])]) # check for required and optional packages ACX_PTHREAD AC_CHECK_HEADER([execinfo.h], AC_DEFINE(HAVE_EXECINFO_H)) PKG_CHECK_MODULES(VALGRIND, [valgrind], AC_DEFINE(HAVE_VALGRIND_H)) # finally check for nobug itself, multithreaded here PKG_CHECK_MODULES(NOBUGMT_EXAMPLE, [nobugmt >= 201006.1], AC_DEFINE(HAVE_NOBUGMT_H), AC_MSG_ERROR([NoBug pkg-config metadata missing]) )
Additional Tools
Various peripheral tools can be used by NoBug depending on the requirements of the application and the detail desired by the user. Such tools can provide additional, detailed information on the application and its behaviour. However, some applications may not require such detail and the associated overhead in information, and users may decide to omit excess information by excluding such tools.
At the moment NoBug supports the optional inclusion of gdb, valgrind and support for multi-threaded applications and the information that can be provided by these tools. However, support for other tools may be supplied in the future, e.g. the dbx debugger on OpenSolaris.
Link Appropriate Library
Finally, the appropriate library (for either single or multi-threaded applications) is linked to the project.
- libnobug
-
Link-in this library for single threaded applications.
- libnobugmt
-
Link with this library for multi-threaded applications.
NoBug installed static and dynamic libraries. When your application uses multiple dynamic libraries which use NoBug or you build a dynamic library, then you have to link against the dynamic library.
You can use the pkg-config tool to gather information about NoBug in your build system. For example you can query the NoBug version which is installed on your system:
$ pkg-config --modversion nobug 201006.1
Initialization
Global Initialization
Before anything from NoBug can be used, NoBug must be initialised. This is archived by calling the NOBUG_INIT() macro before using any other of the NoBug facilities.
NOBUG_INIT can be called more than once, subsequent calls will be a no-op, thus initialising in main and in libraries won’t interfere with one another.
Care must be taken when one already using NoBug features from dynamic initialized things in C++, one has to ensure that NoBug gets initialized first possibly by pulling up a singleton at very first.
Destroying NoBug
Since NoBug is intended to be available throughout its whole lifetime, destroying it is not to be advised. Nevertheless, there is a destroy function
void nobug_destroy (void);
to shutdown NoBug, and this frees all resources associated with it. This is mostly used in the NoBug testsuite itself to check for leaks, and it might be useful for other programs which employ some kind of leak checker.
Init logging Flags
If you want to use environment variable controlled debuging, then you have to initialize each defined flag with
NOBUG_INIT_FLAG(flagname)
or
NOBUG_INIT_FLAG_LIMIT(flagname, default)
or one of the C++ compatibility macros.
This is documented later in the logging configuration chapter.
Threads
In Multithreaded programs you should assign an identifier to each thread. A thread identifier is a string which will be automatically appended with an underscore and an incrementing integer. It is is created using the following:
NOBUG_THREAD_ID_SET(name)
For example, calling NOBUG_THREAD_ID_SET("worker") will result in a thread identifier worker_1.
If you don’t set an identifier, then NoBug will automatically assign one. This is further documented in the multi threading section of this manual.
Thus the boilerplate code to pulling up NoBug in a multithreaded program looks like (omit NOBUG_THREAD_ID_SET() in single threaded programs):
#include <nobug.h> NOBUG_DEFINE_FLAG(example); ... int main() { NOBUG_INIT(); NOBUG_THREAD_ID_SET("main"); NOBUG_INIT_FLAG(example); ... }
Debugging Information Granularity: The Build Levels
There are three different levels of debugging information available: alpha, beta and release. One of these levels must be specified before compiling, otherwise an error while compiling will occur.
- ALPHA
-
This debugging level is envisaged for the development phase of a project where exhaustive testing and logging are required. All NoBug features are enabled.
- BETA
-
This debugging level is more appropriate for projects beyond the development phase and ready for field-testing with users willing to test the software. The most expensive debugging features are disabled at this level.
- RELEASE
-
This level is for final, end-users. Logging is mostly retained but assertions and other checking is disabled.
A logging level can be selected by either using a define before including nobug.h, or by passing the appropriate level using the -D switch to the compiler:
- ALPHA
-
-DEBUG_ALPHA / #define EBUG_ALPHA
- BETA
-
-DEBUG_BETA / #define EBUG_BETA
- RELEASE
-
-DNDEBUG / #define NDEBUG
If a log level has not been selected, NoBug will abort the compilation with an error.
Logging Concepts
Nearly all NoBug Macros emit some log message. NoBug gives the user fine grained control over these log messages to display only interesting information without loosing details.
Log messages can be routed to various destinations. The following destintaions are available:
- RINGBUFFER
-
The underlying storage backend. Messages are appended to the end of the buffer, overwriting older messages at the front of the buffer. NoBug comes with a highly efficient ringbuffer implementation. This ringbuffer is temporary by default but can be made persistent on disk which can be inspected with the nobug_rbdump tool.
- CONSOLE
-
This is either just stderr, or, if running under a supported debugger, the debuggers facilities to print messages will be used.
- FILE
-
The user can open files for log messages.
- SYSLOG
-
Messages are sent to the standard system logging daemon.
- APPLICATION
-
There are hooks which allow the programmer to catch logmessages and display them in an application which are defined by the application.
Each log message has a priority describing its severity in the same way as syslog messages do.
All non-fatal messages are associated with a programmer defined flag describing the source of the message (subsystem, module, …). This is referred to as channels in other logging systems. Flags make it possible to configure logging in much detail at runtime.
Putting this all together: A user can define which source/flag will be logged at what priority level and to which destination. To make this all easier, NoBug tries to provide reasonable defaults.
Configuration
Each log macro has an explicit or implicit log-level which correspondends to syslog levels. Logging is only emitted when the message is more severe or the same as a defined limit.
INDEX Default levels for logging;loggingleveldefaults; table showing defaults log levels
ALPHA | BETA | RELEASE | |
---|---|---|---|
ringbuffer |
TRACE |
INFO |
NOTICE |
console |
INFO |
NOTICE |
-1 |
file |
TRACE |
NOTICE |
WARNING |
syslog |
-1 |
NOTICE |
WARNING |
application |
INFO |
WARNING |
ERROR |
Depending on the build level, there is a default logging target and a default limit which is selected when the user doesn’t specify one.
- In ALPHA builds
-
NOBUG_LOG_LIMIT_ALPHA is used which defaults to LOG_INFO
- In BETA builds
-
NOBUG_LOG_LIMIT_BETA is used and defaults to LOG_WARNING
- In RELEASE builds
-
NOBUG_LOG_LIMIT_RELEASE is used and defaults to LOG_CRIT
- In ALPHA builds
-
NOBUG_LOG_TARGET_ALPHA defaulting to NOBUG_TARGET_CONSOLE
- In BETA builds
-
NOBUG_LOG_TARGET_BETA defaulting to NOBUG_TARGET_FILE
- In RELEASE builds
-
NOBUG_LOG_TARGET_RELEASE defaulting to NOBUG_TARGET_SYSLOG
You can override all these values with other limits or targets before including nobug.h. As an alternative, NOBUG_LOG_LIMIT and NOBUG_LOG_TARGET can be defined to override all defaults at once.
Log Flags
Flags are used to inform NoBug about subsystems/modules or even finer grained sections of the code. These are referred to as channels in other logging libraries.
Flags are generally used as follows:
-
Declare the flag.
-
Define the flag.
-
Initialise the flag.
NOBUG_DECLARE_FLAG(flagname)
The flag should then be defined most appropriately in some implementation file by using one of the following macros:
NOBUG_DEFINE_FLAG(flagname)
NOBUG_DEFINE_FLAG_LIMIT(flagname, limit)
Moreover, macros are available that accept a parent flag as a parameter, which is then used to initialize the defaults from another flag:
NOBUG_DEFINE_FLAG_PARENT(flagname, parent)
NOBUG_DEFINE_FLAG_PARENT_LIMIT(flagname, parent, limit)
This can be used to create hierarchies of flags.
C++ support, C++ logflags
Additional macros are available for applications written in C++:
NOBUG_CPP_DEFINE_FLAG(name) NOBUG_CPP_DEFINE_FLAG_PARENT(name, parent) NOBUG_CPP_DEFINE_FLAG_LIMIT(name, default) NOBUG_CPP_DEFINE_FLAG_PARENT_LIMIT(name, parent, default)
These macros statically initialize the flags when they are defined, there is no need to call NOBUG_INIT_FLAG() (see below).
Logging Flag Initialization
After a flag has been declared and defined, it has to be initialised:
NOBUG_INIT_FLAG(flagname)
or
NOBUG_INIT_FLAG_LIMIT(flagname, default)
Use either of these macros once at the begining your program for each flag. This macros will parse the $NOBUG_LOG envirionment variable at runtime initializing the given flag dynamically.
For flags defined with NOBUG_DEFINE_FLAG(flagname) the defaults are initialized as in the table above, while NOBUG_DEFINE_FLAG_LIMIT(flagname, level) is used to initialize the default target (depending on build level) to level.
Force declarations only
When NOBUG_DECLARE_ONLY defined to be 1 then all flag definitions here become declarations only. When this is defined to be 0 (which is the default) then all definitions behave as described. This can be used to construct a headerfile which only contains definitions, but, by default, yield only declarations. This provides one convenient single point to maintain flag configurations.
#include <nobug.h> /* if not included from flags.c then declare the flags, else define them */ #ifndef FLAGS_C #define NOBUG_DECLARE_ONLY 1 #endif /* use only DEFINE_FLAG here */ NOBUG_DEFINE_FLAG(example); /* Reset it to 0 to cause no trouble */ #ifndef FLAGS_C #undef NOBUG_DECLARE_ONLY #define NOBUG_DECLARE_ONLY 0 #endif
#define FLAGS_C #include "flags.h" ...
Controlling what gets logged
The NOBUG_INIT_FLAG... series of macros parse the environment variable $NOBUG_LOG. This enviromnet variable is thus used to configure what is logged at runtime. Its syntax is as following:
logdecl_list --> logdecl, any( ',' logdecl_list). logdecl --> flag, opt(limitdecl, any(targetdecl)). flag --> "identifier of a flag". limitdecl --> ':', "LIMITNAME". targetdecl --> '@', "targetname", opt(targetopts). targetopts --> '(', "options for target", ')', opt(targetopts).
Roughly speaking, $NOBUG_LOG contains a comma separated list of declarations for flags, which is the name of the flag followed by a limit which is all written in uppercase letters and preceeded by a colon, followed by target declarations which are the names of targets, introduced by an at sign. Target declarations can have options, which are described in the next section. Limit and target declarations are optional and defaults are selected from the table above. The defaults presented here are currently an approximation of what might be viable. The default values used here may be redefined in a future release.
Targets and Options
The Following options are available:
- @ringbuffer
-
- (file=filename)
-
set filename for the backing ringbuffer
- (size=nnn)
-
set size of the ringbuffer to nnn bytes, rounded-off up to the next page boudary
- (append)
-
don’t erase existing ringbuffer, start where it left off
- (keep)
-
keep file after application ends
- (temp)
-
unlink file instantly at creation
The default ringbuffer is a temporary file in /tmp with the temp option. This means it will not be available and accessible for inspection, but it also won’t leave any stale data behind when the application ends.
- @console
-
- (fd=n)
-
redirect console output to fd n
When running under a debugger, NoBug tries to use debugger facilities to print console messages.
- @file
-
- (name=filename)
-
log to filename
- (append)
-
append to (existing) logfile
- @syslog
-
- (ident=name)
-
global prefix for syslog
- (cons)
-
log to system console if syslog is down
- (pid)
-
include the process identifier in the log
- (perror)
-
log to stderr as well (Not available on all systems)
# set limit to the default target and default limit # (see table above) NOBUG_LOG='flag,other' # set the limit of the default target to DEBUG NOBUG_LOG='flag:DEBUG' # set console and syslog limits for flag to DEBUG NOBUG_LOG='flag:DEBUG@console@syslog' # trace 'other' to a persistent ringbuffer NOBUG_LOG=\ 'other:TRACE@ringbuffer(file=log.rb)(size=8192)(keep)'
Predefined Flags
There are some debugging flags which are predefined by NoBug.
The flag NOBUG_ON is always enabled at LOG_DEBUG level. This is static and can not be changed.
The flag NOBUG_ANN is used for the source annotations. This is static and can not be changed. It differs from NOBUG_ON as in never logging to syslog and only define a LOG_WARNING limit for the application callback.
Actions on NoBug itself will be logged under the nobug flag itself. To see whats going on you can enable it with NOBUG_LOG=nobug:TRACE. This is particulary useful to check if NOBUG_INIT_FLAG() is called on all flags.
#include <nobug.h> NOBUG_DEFINE_FLAG (test); int main() { NOBUG_INIT; NOBUG_INIT_FLAG (test); TRACE (test, "Logging enabled"); TRACE (NOBUG_ON, "Always on"); return 0; }
$ cc -DEBUG_ALPHA -lnobug example.c $ ./a.out 0000000003: TRACE: example.c:11: -: main: Always on $ NOBUG_LOG=test:TRACE ./a.out 0000000003: TRACE: example.c:10: -: main: Logging enabled 0000000004: TRACE: example.c:11: -: main: Always on
Log Line Format
Logging follows a rather rigid format and is not configurable. This is intentional to make it easier to compare and process logs afterwards. NoBug is designed with the goal to effectively do very fine grained logging and then analyze this logs later, either manually or with further tools.
A typical log looks like:
0000000003: TRACE: test.c:10: thread_1: main: message 0000000003! TRACE: test.c:10: thread_1: main: continued 0000000005: NOTICE: test.c:15: thread_1: main: next
The components are delimited by ": " or in one case by "! " are as following:
-
Sequence Number
Each logging event increments a counter, for preformance (and privacy) reasons we do not use timestamps here. The sequence number is represented by ten decimal digits, overflows are not specially handled. There is a special case that when the final message (see below) spans more log lines, then the delimiter after the sequence number is a "! " and not ": ". The sequence numbers in a log can be sparse because not every event is send to the log, the left out lines are still available in the ringbuffer. -
NoBug Facility
Describes what NoBug macro produced the output. This can be a simple annotation or logging as well as assertions or one of the more complex NoBug facilities. -
File and Linenumber
The filename is only printed without it’s path. This is important for some reasons. First it makes the log vastly more readable and second depending on source location and build system and configuration thereof paths are not easily reproducible, this makes it hard to use simple tools like grep or diff on logs generated on different systems. For some messages the filename and/or the linenumber is not available or meaningless, a "-" is used then. -
Thread Identifier
Shows which thread generated this message. The single threaded version of NoBug uses always "-". -
Function Name
Gives the function in which the logging occured. For a lot macros it is possible to pass a context (file, line and function name) around to make it possible to delegate validation and logging to some special functions while still being able to present the context where the call originated. NoBug only uses the function name as identifier here no pretty printed prototype or demangled C++ method. The reason for this is that neither is well specified an may yield to non unique results which may even interfere with the logging format syntax and thus make it very unreliable to write tools to process logs. Some contexts dont provide a function name, a "-" is used then. -
Message
The rest of the log line is the user-defined and optional message part. There are no constraints on the format and one should constrain himself to print only meaningful textual information. Embedded newlines will always be translated into a new logging line as shown in the example above.
Macros Overview
The NoBug interface is almost completely implemented using preprocessor macros. This is required because NoBug uses the __FILE__, __LINE__ and __func__ macros to log information on the current file, line number and function. Moreover, all the flat namespace uppercase identifiers make it easy to recognise the macros in source code.
All macros are available without condition with a NOBUG_... prefix. Many macros (the common cases) are also available without this prefix as a convenience, however macros without this prefix must not have been previously defined. When NOBUG_DISABLE_SHORTNAMES is defined before including nobug.h, then only the NOBUG_ prefixed macros are available and the short forms will never be defined.
A set of macros are provided by NoBug that are postfixed by ..._IF. These macros have the following form:
-
..._IF(when, ...)
They perform the desired action only if when is true. For example:
-
REQUIRE_IF(foo != NULL, foo->something == constrained)
The assertion will only be performed if foo is non NULL.
NoBug also contains a facility to pass the source context (file, line, function) around, this can be used to write functions which handle things where one is more interested in the context of the caller rather than the location where the macro appears.
These macros are postfixed with ..._CTX and take an extra context parameter (usually at last but before the logging format specifier and any variable argument list). The context parameter must be of type const struct nobug_context.
When the _CTX context form is used together with the conditional _IF form then the suffix of the macros is always ..._IF_CTX.
Macros that can accept a context have no short form and must always be prefixed with NOBUG_....
Parameters types
We use names for parameters which describe their type. These names are orthogonal through all macro definitions.
when |
Assertion is only performed if expression when is true at runtime |
expr |
Test without side effects |
flag |
Destination flag for control logging at runtime |
type |
Data type to be checked as a single identifier name |
pointer |
Pointer to type |
lvl |
Log level |
depth |
Depth for invariants and dumps |
context |
Source context of type struct nobug_context |
... |
printf-like format string followed by its arguments |
Source Contexts
NOBUG_CONTEXT NOBUG_CONTEXT_NOFUNC
NoBug passes information about the source location of a given statement in const struct nobug_context structures. These can be generated with NOBUG_CONTEXT or NOBUG_CONTEXT_NOFUNC. The later one doesn’t define a function name and must be used when the function context is not available like in static initialization etc..
Macros for Assertions
The assertion set of macros provide a convenient status check on whether to continue running, or abort execution as some condition was not fulfilled. Assertion failures are fatal and must abort the application immediately by calling the NOBUG_ABORT macro which in turn may call a user defined abort hook.
CHECK(expr, ...) CHECK_IF(when, expr, ...)
This assertion is never optimized out. Its main purpose is in implementing test suites where one would like to assert tests independent of the build level.
REQUIRE(expr, ...) REQUIRE_IF(when, expr, ...) NOBUG_REQUIRE_CTX(expr, context,...) NOBUG_REQUIRE_IF_CTX(when, expr, context, ...)
Precondition (input) check. Use these macros to validate the input a function receives. The checks are enabled in ALPHA and BETA builds, but have not effect in RELEASE builds.
ENSURE(expr, ...) ENSURE_IF(when, expr, ...) NOBUG_ENSURE_CTX(expr, context, ...) NOBUG_ENSURE_IF_CTX(when, expr, context, ...)
Postcondition (progress/output) check. Use these macros to validate the data a function produces (example: return value). ENSURE is enabled unconditionally in ALPHA builds and have no effect in BETA builds for scopes which are tagged as CHECKED.
The ENSURE_IF variants are enabled in ALPHA and BETA builds.
In RELEASE builds these checks are optimized out.
ASSERT(expr, ...) ASSERT_IF(when, expr, ...) NOBUG_ASSERT_CTX(expr, context, ...) NOBUG_ASSERT_IF_CTX(when, expr, context, ...)
Generic check. Use these macros when you want to validate something which doesn’t fall into one of the above categories. Generally to be avoided! The checks are enabled in ALPHA and BETA builds and have no effect in RELEASE builds.
assert(expr)
NoBug overrides the standard assert macro, using NOBUG_ASSERT. This is just a compatibility feature, its use is not suggested.
INVARIANT(type, pointer, depth) INVARIANT_IF(when,type, pointer, depth) INVARIANT_ASSERT(expr, ...)
Checking invariants. You can provide more complex checking functions which test the validity of datastructures. Invariants are only enabled in ALPHA builds for scopes which are not tagged as CHECKED and otherwise optimized out.
TODO: describe how to create invariant checks
Macros That Log Information
Logging targets a flag (except for ECHO) and is done at a log-level related to syslog levels.
ECHO(...)
Never optimized out, logs at LOG_NOTICE level. Its main purpose is for implementing testsuites where one would like to print and log messages independent of the build level
ALERT(flag, ...) ALERT_IF(when, flag, ...) NOBUG_ALERT_CTX(flag, context, ...) NOBUG_ALERT_IF_CTX(when, flag, context, ...)
This is the most critical condition that might be registered by an application. Situations might arise when the application encounters such a serious error that can only be adequately treated by, for example, safely shutting down the application.
CRITICAL(flag, ...) CRITICAL_IF(when, flag, ...) NOBUG_CRITICAL_CTX(flag, context, ...) NOBUG_CRITICAL_IF_CTX(when, flag, context, ...)
An error which can not be handled occured but the application does not need to be shutdowen, perhaps waiting for an operator to fix the cause.
ERROR(flag, ...) ERROR_IF(when, flag, ...) NOBUG_ERROR_CTX(flag, context, ...) NOBUG_ERROR_IF_CTX(when, flag, context, ...)
This is used when an application registers an error and appropriate action will have to be taken.
WARN(flag, ...) WARN_IF(when, flag, ...) NOBUG_WARN_CTX(flag, context, ...) NOBUG_WARN_IF_CTX(when, flag, context, ...)
When an application encounters a rare and unexpected situation, these macros can be used.
INFO(flag, ...) INFO_IF(when, flag, ...) NOBUG_INFO_CTX(flag, context, ...) NOBUG_INFO_IF_CTX(when, flag, context, ...)
It may be benificial to output information at various locations throughout the code, e.g., messages on programm progress.
NOTICE(flag, ...) NOTICE_IF(when, flag, ...) NOBUG_NOTICE_CTX(flag, context, ...) NOBUG_NOTICE_IF_CTX(when, flag, context, ...)
Same as the INFO() macros, except more verbose.
TRACE(flag, ...) TRACE_IF(when, flag, ...) NOBUG_TRACE_CTX(flag, context, ...) NOBUG_TRACE_IF_CTX(when, flag, context, ...)
The same as the NOTICE() macros, except very fine-grained information. a common use case is to put just TRACE(debugflag) just at the begin of every non-trivial function. This allows to watch fine grained application progress in the log.
TRACE corresponds to LOG_DEBUG, because using DEBUG could be ambiguous. |
NOBUG_LOG_CTX(flag, lvl, context, ...) NOBUG_LOG_IF_CTX(when, flag, lvl, context, ...)
Generic logging macro which takes the level explicitly. Avoid this, unless you implement your own logging facilities.
there is no logging macro for LOG_EMERG, this is only used by the assertions as a fatal message |
NOBUG_LOG_BASELIMIT_ALPHA NOBUG_LOG_BASELIMIT_BETA NOBUG_LOG_BASELIMIT_RELEASE NOBUG_LOG_BASELIMIT
Anything more detailed than this base limits will be optimized out. This is used to reduce the logging overhead for RELEASE builds. By default the limit is set to LOG_DEBUG for ALPHA and BETA builds, so all logging is retained and LOG_NOTICE in RELEASE builds to log the application progress only coarsely.
These macros can be defined before including nobug.h to some other log level (as defined in syslog.h).
Dumping Data Structures
One can write functions to dump complex data structures using the NoBug facilities. This is done by writing a custom function for each data structure to be dumped, which may recursively call other dumping functions. There are macros that can log within such a dumper function and to initialise a dump of a given data structure.
A dump function has the prototype:
void nobug_NAME_dump (const struct NAME* POINTER, const int DEPTH, const struct nobug_context context, void* EXTRA);
where NAME is the identifier for what you want to dump, POINTER is a pointer to the data to be dumped, DEPTH is an integer which will be decremented when recursing into the data structure dumper (your dump function does that, see below) to limit the recursion depth, context is a source context generated by nobug when you call DUMP() and EXTRA is a pointer transparently passed around that you can use to store some additional state. The context variable must be named context because the DUMP_LOG() macro relies on this.
DUMP(flag, type, pointer, depth, extra) DUMP_IF(when, flag, type, pointer, depth, extra)
These macros call a data structure dump of the object (pointer) in question. DUMP has only effect in ALPHA and BETA builds, DUMP_IF is also enabled for RELEASE builds.
extra is a void* which is transparently passed around and can be used to pass a particular state around. NoBug does not alter this.
DUMP_LOG(...) DUMP_LOG_IF(when, ...)
Any output from DUMP handlers should be done by these macros.
#define NOBUG_DUMP_LEVEL ...
Dumping is by default done at LOG_DEBUG level, this can be overridden by defining NOBUG_DUMP_LEVEL to some other logging level.
struct STRUCTNAME { int INTEGER_MEMBER; char * STRING_MEMBER; struct STRUCTNAME* next; }
Thereafter, define a funcion as follows:
void nobug_STRUCTNAME_dump (const struct STRUCTNAME* self, const int depth, const struct nobug_context context, void* extra) { /* check for self != NULL and that the depth limit was not exceeded in recursive data structures */ if (self && depth) { /* you may or may not do something with the extra parameter here, extra is transparently passed around */ (void) extra; /* use DUMP_LOG not LOG to print the data */ DUMP_LOG("STRUCTNAME %p: int is %d, string is %s", self, self->INTEGER_MEMBER, self->STRING_MEMBER); /* now recurse while decrementing depth */ nobug_STRUCTNAME_dump (self->next, depth-1, context, extra); } }
Now you can use the DUMP() macros within the code
example() { struct STRUCTNAME foo; init(&foo); /* extra can be anything, NULL is suggested when you don't use it */ DUMP (my_flag, STRUCTNAME, &foo, 2, NULL); }
Source Code Annotations
Obsolete, buggy or precarious code can be marked in the code itself in comments at the location where the code occurs. However these code locations can be easily overlooked, especially in large projects. NoBug provides macros to tag code with such annotations. This provides an additional instrument to alert or remind the programmer that there is still dubious code embedded in the application. This tagging scheme not only informs the programmer at compile time that there is code in the application that needs attending to, but long after compilitation a test team might become aware of dubious code due to runtime messages.
DEPRECATED(...)
Use this macro to identify code that is depreciated and should not be used in future or productive code.
UNIMPLEMENTED(...)
Use this macro to identify code that is functionally not yet complete and should not be used. This is convenient for code under development or being reviewed.
FIXME(...)
Use this macro to mark a known and unfixed bug.
TODO(...)
Enhancement or non-critical bug to be done soon.
PLANNED(...)
Future enhancement, optimization to similar which has no side effect on the current program.
NOTREACHED(...)
Code which must never be reached.
ELSE_NOTREACHED(...)
This macro is the same as else NOTREACHED(), but completely optimized out in release builds.
Annotation Semantics
The action that should be taken when an annotated source line is reached depends on the build level.
ALPHA | BETA | RELEASE | |
---|---|---|---|
DEPRECATED |
log |
nothing |
won’t compile |
UNIMPLEMENTED |
abort |
abort |
won’t compile |
FIXME |
log |
won’t compile |
won’t compile |
TODO |
log |
log |
won’t compile |
PLANNED |
log |
nothing |
nothing |
NOTREACHED |
abort |
abort |
removed |
Scope Checks
The programmer can tag any scope as UNCHECKED or CHECKED. In ALPHA and BETA builds, a global UNCHECKED is implied. In RELEASE builds, UNCHECKED scopes are not allowed.
int myfunc() { /* the outer scope of this function is not validated yet*/ UNCHECKED; if (...) { /* everything in this scope is considered ok */ CHECKED; ... } return ...; }
ALPHA | BETA | RELEASE | |
---|---|---|---|
UNCHECKED |
Preconditions, Postconditions, Invariants |
Preconditions, Postconditions |
compiling will abort |
CHECKED |
Preconditions, Postconditions |
Preconditions |
Fault injection
NoBug has some macros which can be used to simulate erroneous behaviour, in other words, we can inject errors, or faults, into the code using NoBug.
INJECT_GOODBAD(expr, good, bad)
Substitutes to an expression and returns good when expr is false and bad when expr is true. In BETA and RELEASE builds good is always returned.
INJECT_FAULT(expr, bad)
Substitutes to a statement which executes bad when expr is true. This is only available in ALPHA builds and is optimitsed out in BETA and RELEASE builds.
Fault coverage checking
Fault coverage checking is an experimental feature! |
NoBug can automatically inject faults at instrumented points and permute through all potential error paths of an application by restarting it with the state from a former run. It can be used to provide information on whether a particular error is adequately treated in an application. Fault coverage checking is only available in ALPHA builds; it is optimized out in all other builds.
This kind of testing is very expensive on performance. |
How Does Automatic Fault Injection Work?
First, NoBug checks if the NOBUG_COVERAGE environment variable is set. If it is set, it must contain a space, comma or a semicolon separated list of filenames which are the logs from a previous run. These logs are then parsed storing the old state in a lookup tree.
The application will then proceed to run as usual. When an instrumented coverage point is hit, its status is checked against all states that have been recorded previously. Any failure point that is encountered for a first time causes a fault to be injected; the last seen but previously failed point will now be passed, all other fault injection points behave as in their previous run. This ensures that each successive run of the application changes only one injection point and, thus, permutes through all possible code paths.
Fault injection points are identified by a 64bit hash over the backtrace (return addresses on the stack) leading to it. This means each unique way to reach an injection point is recorded. Parameters and threads are intentionally not considered in this calculation.
How to Invoke Fault-Coverage Checking
Each fault-injection point emits a logging message about its identity and state, this logging uses the normal NoBug logging facilities. Thus, one can control logging with the NOBUG_LOG environment variable. Furthermore, fault coverage checking is only active when the NOBUG_COVERAGE environment variable has been set to point to log files which are the results from a previous run. NoBug comes with a tests/coverage.sh example script which permutes through all possible error paths.
Coverage checking Macros
COVERAGE_FAULT(flag, ...)
Injects the statement at ... when simulating a failure. Only active in ALPHA builds.
COVERAGE_GOODBAD(flag, good, bad)
Substitutes to an expression and injects bad when simulating a failure and good otherwise. Only active in ALPHA builds.
#define NOBUG_COVERAGE_LEVEL ...
Define the logging level in which fault-coverage logging is emitted.
Supressing log output with this level will not supress fault injection, actually the opposite is true since every newly seen failure path gets injected. This might be changed in future releases. |
NOBUG_COVERAGE_DISABLE NOBUG_COVERAGE_ENABLE
Sometimes fault injection yields false positives, errors which may never happen in real life (and are possibly enforced with an ENSURE afterwards). For this cases coverage fault injection can be disabled temporarly and reenabled later. Disabling/enabling may nest and must properly match.
Resource Tracking
With little effort, NoBug can watch all kinds of resources a program uses. This becomes useful for resources which are distributed over a multithreaded program. Resource tracking includes logging actions on resource and checking locking policies over acquired resources. Resource logging is active in ALPHA and BETA builds when NOBUG_RESOURCE_LOGGING is defined to 1 (the default). The resource tracker which supervises locking policies is only enabled in ALPHA builds.
Concepts
Resources are an abstract entity for NoBug, which has little knowledge about the kinds of resources; it only keeps records of resources and the code using these resources and ensures basic constraints. More detailed checks on resource usage have to be done with other NoBug facilities.
Resources are identified by an arbitrary identifier which is just a pointer. Additionally a name, the type and the source locations which registered the resource are stored.
Code which requiring to use a resource, calls an ENTER macro, supplying an identifier and state. The state can be altered. Thereafter a LEAVE macro is used when the the code is finished with the resources.
When a resource is used, one has to pass one of these states:
- NOBUG_RESOURCE_WAITING
-
For resources which might need to be blocked (locks), enter with a WAITING state first, as soon at the resource is acquired, change the state to one of the following.
- NOBUG_RESOURCE_EXCLUSIVE
-
Acquired the resource exclusively. The resource must not be acquired again, not even from the same thread.
- NOBUG_RESOURCE_RECURSIVE
-
The resource might be entered multiple times from the same thread with this state.
- NOBUG_RESOURCE_SHARED
-
The resource might be entered multiple times from any thread with this state.
Notes
The Resource Tracker relies on proper announce/forget and enter/leave are properly paired. The programmer should ensure that this is correctly done, otherwise the results are unpredictable.
Resource tracking macros
Unless the user defines NOBUG_RESOURCE_LOGGING to 0 each of the above macros will emit a log message at NOBUG_RESOURCE_LOG_LEVEL which defaults to LOG_DEBUG.
RESOURCE_HANDLE(name) RESOURCE_HANDLE_INIT(name) RESOURCE_USER(name) RESOURCE_USER_INIT(name)
Define and initialize handles for to track resources.
- name
-
identifer to be used for the handle
There are two kinds of handles. Resource themself are abstracted with a RESOURCE_HANDLE while uses of a resources are tracked by RESOURCE_USER handle. These macros takes care that the declaration is optimized out in the same manner as the rest of the resource tracker would be disabled. You can still instantiate handles as struct nobug_resource_record* or struct nobug_resource_user* in structures which must have a constant size unconditional of the build level. The two *_INIT macros can be used to initialize resource handles and are optimized out when the resource tracker gets disabled.
RESOURCE_ANNOUNCE(flag, type, name, identifier, handle){} NOBUG_RESOURCE_ANNOUNCE_RAW(flagptr, type, name, ptr, handle){} NOBUG_RESOURCE_ANNOUNCE_RAW_CTX(flagptr, type, name, ptr, handle, context){}
Publishes resources.
- flag
-
the NoBug flag name which turns logging on for this macro
- type
-
a string which should denote the domain of the resource, examples are "file", "mutex", "lock", "database" and so on
- name
-
the actual name of a named resource this as string. type and name must be available through the entire lifetime of the resource, using literal strings is recommended
- identifier
-
a pointer which must be unique for this resource, any kind of pointer will suffice, it is only used for identification. In multithreaded applications the thread identifier becomes an additional identifier
- handle
-
a NOBUG_RESOURCE_HANDLE which will be initialized to point to the newly created resource.
Resources must be unique, it is a fatal error when a resource it tried to be announced more than one time.
RESOURCE_ANNOUNCE() acts like the head of a C loop statement, it ties to the following (block-) statement which will be handled atomic. This statement must not be left by break, return or any other kind of jump.
RESOURCE_FORGET(flag, handle){} NOBUG_RESOURCE_FORGET_RAW(flagptr, handle){} NOBUG_RESOURCE_FORGET_RAW_CTX(flagptr, handle, context){}
Removes resources that have become unavailable from the registry.
- flag
-
the NoBug flag which turns logging on for this macro
- handle
-
the NOBUG_RESOURCE_HANDLE used to track this resource
The resource must still exist and no users must be attached to it, else a fatal error is raised.
RESOURCE_FORGET() acts like the head of a C loop statement, it ties to the following (block-) statement which will be handled atomic. This statement must not be left by break, return or any other kind of jump.
RESOURCE_RESETALL(flag) NOBUG_RESOURCE_RESETALL_RAW(flagptr) NOBUG_RESOURCE_RESETALL_RAW_CTX(flagptr, context)
Sometimes the resource tracker can give false positives when it finds a locking order violation while the programmer knows that this will never happen in the real program, because for example this is only used at initialization or shutdown and never overlaps. This macro can then be used to reset all whats learnt about all resources and start over.
- flag
-
the NoBug flag which turns logging on for this macro
RESOURCE_RESET(flag, handle) NOBUG_RESOURCE_RESET_RAW(flagptr, handle) NOBUG_RESOURCE_RESET_RAW_CTX(flagptr, handle, context)
Sometimes the resource tracker can give false positives when it finds a locking order violation while the programmer knows that this will never happen in the real program, because for example this is only used at initialization or shutdown and never overlaps. This macro can then be used to reset all whats learnt about a single resources and start over.
- flag
-
the NoBug flag which turns logging on for this macro
- handle
-
the NOBUG_RESOURCE_HANDLE used to track this resource
RESOURCE_ENTER(flag, announced, user, state, handle){} NOBUG_RESOURCE_ENTER_CTX(flag, resource, user, state, handle, context){}
Acquire a resource.
- flag
-
nobug flag which turns logging on for this macro
- announced
-
the handle set by RESOURCE_ANNOUNCE
- user
-
a literal string defining the purpose
- state
-
the state to enter
- handle
-
a NOBUG_RESOURCE_HANDLE which will be initialized to the entering node
RESOURCE_ENTER() acts like the head of a C loop statement, it ties to the following (block-) statement which will be handled atomic. This statement must not be left by break, return or any other kind of jump.
RESOURCE_WAIT(flag, resource, user, handle){} NOBUG_RESOURCE_WAIT_CTX(flag, resource, user, handle, context){}
This is just an alias for
RESOURCE_ENTER(flag, resource, user, NOBUG_RESOURCE_WAITING, handle)
RESOURCE_WAIT(flag, resource, user, handle); if (lock_my_resource() == ERROR) NOBUG_RESOURCE_LEAVE(flag, handle); else RESOURCE_STATE(flag, NOBUG_RESOURCE_EXCLUSIVE, handle);
RESOURCE_TRY(flag, resource, user, handle){} NOBUG_RESOURCE_TRY_CTX(flag, resource, user, handle, context){}
This is just an alias for
RESOURCE_ENTER(flag, resource, user, NOBUG_RESOURCE_TRYING, handle)
Trying on a resource is similar to waiting but will not trigger a deadlock check. This can be used when a deadlock is expected at runtime and one handles this otherwise (by a timed wait or something like that).
RESOURCE_STATE(flag, entered, state){} NOBUG_RESOURCE_STATE_CTX(flag, state, entered, context){} NOBUG_RESOURCE_STATE_RAW(flagptr, state, entered){} NOBUG_RESOURCE_STATE_RAW_CTX(flagptr, state, entered, context){}
Changes resource’s state.
- flag
-
is nobug flag which turns logging on for this macro
- state
-
the new state Note that only certain state transitions are allowed, see discussion/diagram above
- entered
-
the handle set by RESOURCE_ENTER
RESOURCE_STATE() acts like the head of a C loop statement, it ties to the following (block-) statement which will be handled atomic. This statement must not be left by break, return or any other kind of jump.
RESOURCE_LEAVE(flag, handle){} NOBUG_RESOURCE_LEAVE_RAW(flagptr, handle){} NOBUG_RESOURCE_LEAVE_RAW_CTX(flagptr, handle, context){}
Disconnect from a resource identified with its handle.
- flag
-
nobug flag which turns logging on for this macro
- handle
-
the handle you got while entering the resource
RESOURCE_LEAVE() acts like the head of a C loop statement, it ties to the following (block-) statement which will be handled atomic. This statement must not be left by break, return or any other kind of jump.
NOBUG_RESOURCE_LEAVE(flag, handle) { unlock_my_resource(); }
RESOURCE_ASSERT_STATE(resource, state) RESOURCE_ASSERT_STATE_IF(when, resource, state) NOBUG_RESOURCE_ASSERT_STATE_CTX(resource, state, context) NOBUG_RESOURCE_ASSERT_STATE_IF_CTX(when, resource, state, context)
Assert that we have a resource in a given state. For multithreaded programms the topmost state of the calling thread is checked, for non threadeded programs the most recent state on resource is used.
- when
-
Condition which must be true for testing the assertion
- resource
-
Resource handle
- state
-
The expected state
NOBUG_RESOURCE_DUMP(flag, handle) NOBUG_RESOURCE_DUMP_IF(when, flag, handle)
Dump the state of a single resource.
- when
-
Condition which must be true to dump the resource
- flag
-
Nobug flag for the log channel
- handle
-
handle of the resource to be dumped
NOBUG_RESOURCE_DUMPALL(flag) NOBUG_RESOURCE_DUMPALL_IF(when, flag)
Dump the state of all resources.
- when
-
Condition which must be true to dump the resources
- flag
-
Nobug flag for the log channel
NOBUG_RESOURCE_LIST(flag) NOBUG_RESOURCE_LIST_IF(when, flag)
List all registered resources.
- when
-
Condition which must be true to list the resources
- flag
-
Nobug flag for the log channel
NOBUG_DEFINE_FLAG_LIMIT(test, LOG_DEBUG); void example() { /* define a mutex and announce it */ pthread_mutex_t my_mutex = PTHREAD_MUTEX_INITIALIZER; RESOURCE_HANDLE(resource); /* 'example' is just a pointer to this function which suffices as unique identifier */ RESOURCE_ANNOUNCE(test, "mutex", "my_mutex", example, resource); /* the following would be done in a different thread in a real program */ /* define a handle */ RESOURCE_HANDLE(enter); /* announce that we want to use the resource &enter also suffices as unique pointer, which is all we need as identifer here */ RESOURCE_WAIT(flag, resource, &enter, enter) { /* lock() might block */ pthread_mutex_lock (&my_mutex); /* assume no errors, got it, change the state */ RESOURCE_STATE(test, NOBUG_RESOURCE_EXCLUSIVE, enter); } /*************************************** ** program does something useful here ** ***************************************/ /* we don't need it anymore */ RESOURCE_LEAVE(test, enter) /* << no semicolon! */ pthread_mutex_unlock (&my_mutex); /* back in the main thread */ /* remove the resource from the public registry */ RESOURCE_FORGET(test, resource); }
Deadlock Detection
The Resource Tracker is able to detect potential deadlocks. This is done by learning the relations between locks (precedence) and watching the order in which resources are acquired. It has some heuristics to detect certain patterns which are deadlock free. A possible deadlock results in a log message and a fatal abort. Note that only waiting on resources can lead to a deadlock. Deadlock detection is implemented in the Resource Tracker and active in ALPHA builds and optimized out on any other build level.
For details about the deadlock detection algorithm see Appendix: Resource Tracking Alorithm.
Callbacks
NoBug provides callbacks, applications can use these to present logging information in some custom way or hook some special processing in. The callbacks are initialized to NULL and never modified by NoBug, it is the sole responsibility of the user to manage them.
There are certain constraints what and what not can be done in callbacks documented below which must be followed. |
typedef void (*nobug_logging_cb)(const struct nobug_flag* flag, int priority, const char *log, void* data)
- flag
-
Flag structure which defines the logging configuration for this event
- priority
-
Log level of the current event
- log
-
Pointing to the current log line in the ringbuffer or NULL
- data
-
Global pointer defined by the user, passed arround (see below)
typedef void (*nobug_abort_cb)(void* data)
- data
-
Global data defined by the user, passed arround (see below)
void* nobug_callback_data
This global variable is initialized to NULL and will never be touched by NoBug. One can use it to pass extra data to the callback functions.
nobug_logging_cb nobug_logging_callback
This callback gets called when something gets logged. NoBug will still hold its mutexes when calling this hook, calling NoBug logging or resource tracking functions from here recursively will deadlock and must be avoided. The log parameter points to the logging message in the ringbuffer. Unlike other logging targets it is not automatically limited to the log level configured in the flag but called unconditionally. The callback should implement its own limiting.
When one wants to do complex calls which may include recursion into logging and resource tracking functions, the intended way is to pass contextual information possibly including a copy of the log parameter in NOBUG_THREAD_DATA to the postlogging callback (see below). Other internal NoBug facilties, like the ringbuffer etc, are protected by the mutexes and may be accessed from this function.
nobug_logging_cb nobug_postlogging_callback
This callback gets called after something got logged. The log parameter is always NULL and all NoBug mutexes are released. This means that this function may call any complex things, including calling logging and resource tracking, but may not call internal NoBug facilities. Contextual created in the nobug_logging_callback and stored in NOBUG_THREAD_DATA can be retrieved here and may need to be cleaned up here.
nobug_abort_cb nobug_abort_callback
This callback gets called when the application shall be terminated due an error. It can be used to hook exceptions or similar things in. When it returns, abort() is called.
Errors detected by NoBug are always fatal. If one handles and possible throws an exception here, the application must shut down as soon as possible. Most causes for aborts are optimitzed out in RELEASE builds. |
Tool Macros
NOBUG_FLAG_RAW(ptr)
Using this macro one can pass a direct pointer to a flag where a name would be expected. This is sometimes convinient when flag pointers are passed around in management strutures and one wants to tie logging to dynamic targets.
NOBUG_DEFINE_FLAG(myflag); ... struct nobug_flag* ptr = &NOBUG_FLAG(myflag); TRACE(NOBUG_FLAG_RAW(ptr), "Passed flag by pointer")
BACKTRACE NOBUG_BACKTRACE_CTX(context)
The backtrace macro logs a stacktrace using the NoBug facilities. This is automatically called when NoBug finds an error and is due to abort. But one might call it manually too.
NOBUG_ABORT_
This is the default implementation for aborting the program, it first syncs all ringbuffers to disk, then calls the abort callback if defined and then abort().
NOBUG_ABORT
If not overridden, evaluates to NOBUG_ABORT_. One can override this before including nobug.h to customize abortion behaviour. This will be local to the translation unit then.
NOBUG_ALPHA_COMMA(something) NOBUG_ALPHA_COMMA_NULL
Sometimes it is useful to have initializer code only in ALPHA builds, for example when you conditionally include resource handles only in ALPHA versions. An initializer can then use this macros to append a comma and something else only in ALPHA builds as in:
struct foo = {"foo", "bar" NOBUG_ALPHA_COMMA_NULL };
Becomes the following in ALPHA builds
struct foo = {"foo", "bar", NULL};
and
struct foo = {"foo", "bar"};
in BETA and RELEASE builds.
NOBUG_IF_ALPHA(...) NOBUG_IF_NOT_ALPHA(...) NOBUG_IF_BETA(...) NOBUG_IF_NOT_BETA(...) NOBUG_IF_RELEASE(...) NOBUG_IF_NOT_RELEASE(...)
This macros allow one to conditionally include the code in (…) only if the criteria on the build level is met. If not, nothing gets substituted. Mostly used internally, but can also be used for custom things.
Multithreading
It is important that NoBug protects certain operations with locks in multithreaded programs. You have to ensure that HAVE_PTHREAD_H is defined by the configuration system and use the libnobugmt library for linking. It is particular important that libraries using NoBug are compiled with HAVE_PTHREAD_H enabled when they are intended to be used in multithreaded programs.
When Multithreading is used, log messages contain a identifier of the originating thread. This identifier should be set by
NOBUG_THREAD_ID_SET(name)
- name
-
New name for the thread
Nobug will assemble a unique identifier by appending a underscore and a number to name, for example NOBUG_THREAD_ID_SET("gui") will result in a identifier like gui_5. When you don’t set a thread identifier, then NoBug assigns one automatically with the name thread preprended if needed. Thread identifiers may be reset with a new call to this macro.
NOBUG_THREAD_ID_GET
Will return a const char* of the thread id in multithreaded programs and a pointer to a literal empty string in singlethreaded programs.
NOBUG_THREAD_DATA
Evaluates to a variable of type void* which can be used to store thread local information. This is useable for callbacks which may prepare context information to be reused later.
This macro is also available in singlethreaded programs, refering to a single global variable.
Nobug initializes this variable to NULL and then touches it never again.
Dumping Persistent Ringbuffers
NoBug installs the nobug_rbdump tool for dumping the content of a persistent ringbuffer. It is invoked with the filename of the ringbuffer, the content is then printed to stdout.
$ NOBUG_LOG='test:TRACE@ringbuffer(file=test.rb)(keep)' ./a.out 0000000004: TRACE: example.c:11: -: main: Always on $ nobug_rbdump test.rb 0000000003: TRACE: example.c:10: -: main: Logging 0000000003! TRACE: example.c:10: -: main: enabled
Best Practices
this section is very work in progress |
-
Development
-
Write a testsuite, build your program with -O0 -g -DEBUG_ALPHA and run the testsuite under valgrind control. Hack until the program meets the specifications defined by the testsuite.
-
-
Beta Test
-
Build with desired optimization level and -g -DEBUG_BETA and give the program to your beta testers.
-
-
Release
-
Build it with optimization and without -g -DEBUG_*
-
-
Add REQUIRE checks on your interfaces (incoming parameters). Especially if an argument might not cover the whole range of the underlying type.
-
Don’t waste your and your CPU’s time with unnecessary checks. The testsuite should validate your program. NoBug aids in debugging. You can add Postconditions (ENSURE) and Invariants when you have a bug somewhere and want to nail it down.
-
Added checks don’t need to be removed.
-
When you use the CHECKED/UNCHECKED features then don’t forget C scoping rules, tag things as CHECKED from the leaves to the root.
-
TRACE(flagname) at the begin of every nontrivial function will easily log the progress of your application.
-
Trying a RELEASE build will abort on certain conditions (known BUG, TODO’s, UNCHECKED code), you can use this to find these spots.
Appendix
The Resource Tracking Algorithm
Each resource registers a global resource_record.
Every new locking path discovered is stored as resource_node structures which refer to the associated resource_record.
Threads keep a trail of resource_user structures for each resource entered. This resource_user struct refer to the resource_nodes and thus indirectly to the associated resource_record.
The deadlock checker uses this information to test if the acqusition of a new resource would yield a potential deadlock.
Entering Resources
In multithreaded programs, whenever a thread wants to wait for a resource_record the deadlock checker jumps in.
The deadlock checking algorithm is anticipatory as it will find and abort on conditions which may lead to a potential deadlock by violating the locking order learned earlier.
Each thread holds a stack (list) of each resource_user it created. Leaving a resource will remove it from this stacklist.
Each resource_record stores the trail which other resource_records are already entered. This relations are implemented with the resource_node helper structure.
First we find out if there is already a node from the to be acquired resource back to the topmost node of the current threads user stack.
struct nobug_resource_user* user = NULL; struct nobug_resource_node* node = NULL; if (!llist_is_empty (&tls->res_stack)) { user = LLIST_TO_STRUCTP (llist_tail (&tls->res_stack), struct nobug_resource_user, res_stack); struct nobug_resource_node templ = { ... user->current->resource, ... }; node = (struct nobug_resource_node*) llist_ufind (&resource->nodes, &templ.node, nobug_resource_node_resource_cmpfn, NULL); } ...
Deadlock checking is only done when the node is entered in WAITING state and only available in multithreaded programs.
if (state == NOBUG_RESOURCE_WAITING) { #if NOBUG_USE_PTHREAD ...
If node was found above, then this locking path is already validated and no deadlock can happen, else, if this stack already holds a resource (user is set) we have to go on with checking.
if (!node && user) { ...
If not then its checked that the resource to be entered is not on any parent trail of the current topmost resource, if it is then this could be a deadlock which needs to be further investigated.
LLIST_FOREACH (&user->current->resource->nodes, n) { for (struct nobug_resource_node* itr = ((struct nobug_resource_node*)n)->parent; itr; itr = itr->parent) { if (itr->resource == resource) { ...
if the resource was on the trail, we search if there is a common ancestor before the resource on the trail and the threads current chain, if yes then this ancestor protects against deadlocks and we can continue.
for (struct nobug_resource_node* itr2 = itr->parent; itr2; itr2 = itr2->parent) { LLIST_FOREACH_REV (&tls->res_stack, p) { struct nobug_resource_user* user = LLIST_TO_STRUCTP (p, struct nobug_resource_user, res_stack); if (user->current->resource == itr2->resource) goto done; } ...
If no ancestor found, we finally abort with a potential deadlock condition.
nobug_resource_error = "possible deadlock detected"; return NULL;
Leaving Resources
store the tail and next aside, we need it later
#if NOBUG_USE_PTHREAD struct nobug_resource_user* tail = LLIST_TO_STRUCTP (llist_tail (&user->thread->res_stack), struct nobug_resource_user, res_stack); struct nobug_resource_user* next = LLIST_TO_STRUCTP (llist_next (&user->res_stack), struct nobug_resource_user, res_stack); ...
remove user struct from thread stack The res_stack is now like it is supposed to look like with the user removed. We now need to fix the node tree up to match this list.
llist_unlink_fast_ (&user->res_stack); ...
When the the user node was not the tail or only node of the thread stack, we have to check (and possibly construct) a new node chain for it. No valdation of this chain needs to be done, since it was already validated when entering the resources first.
if (user != tail && !llist_is_empty (&user->thread->res_stack)) { struct nobug_resource_user* parent = NULL; if (llist_head (&user->thread->res_stack)!= &next->res_stack) { parent = LLIST_TO_STRUCTP (llist_prev (&next->res_stack), struct nobug_resource_user, res_stack); } ...
iterate over all users following the removed node, finding nodes pointing to this users or create new nodes.
LLIST_FORRANGE (&next->res_stack, &user->thread->res_stack, n) { struct nobug_resource_user* cur = LLIST_TO_STRUCTP (n, struct nobug_resource_user, res_stack); struct nobug_resource_record* resource = cur->current->resource; ...
find the node pointing back to parent, create a new one if not found, rinse repeat
struct nobug_resource_node templ = { ... parent?parent->current:NULL, ... }; struct nobug_resource_node* node = (struct nobug_resource_node*) llist_ufind (&resource->nodes, &templ.node, nobug_resource_node_parent_cmpfn, NULL); if (!node) { node = nobug_resource_node_new (resource, parent?parent->current:NULL); if (!node) { nobug_resource_error = "internal allocation error"; return 0; } } parent = cur;
Reference Index
- Aborting
-
abort the program
- callback for aborting
-
hook to handle a termination
- abort callback prototype
-
type of a abort callback function
- ALERT
-
about to die
- Annotation Semantics
-
actions taken on annotations
- ASSERT
-
generic assertion
- assert
-
C standard assertion
- Macros for Assertions
-
assert the state of the application
- Backtraces
-
generate a backtrace
- Build Levels
-
selecting the build level
- passing data to callbacks
-
data to be passed to callbacks
- Callbacks
-
function hooks to catch nobug actions
- CHECK
-
unnconditional assertion for test suites
- CHECKED, Scope
-
tag scope as reviewed
- Disabling and enabling fault-coverage checks
-
handle false positives
- COVERAGE_FAULT
-
coverage fault injection, statement
- COVERAGE_GOODBAD
-
coverage fault injection, expression
- COVERAGE_LEVEL
-
coverage fault injection log level
- C++ support, C++ logflags
-
C\++ support for log flags
- CRITICAL
-
can not continue
- The Resource Tracking Algorithm
-
how resources are tracked
- DECLARE_FLAG
-
declaring a flag
- DEFINE_FLAG
-
defining a flag
- DEFINE_FLAG_LIMIT
-
defining a flag w/ log limit
- DEFINE_FLAG_PARENT
-
defining a flag hierarchy
- DEFINE_FLAG_PARENT_LIMIT
-
defining a flag hierarchy, w/ log limit
- DEPRECATED
-
to be discarded in future
- DISABLE_SHORTNAMES
-
require NOBUG_ prefix everywhere
- DUMP
-
dumping datastructures
- DUMP_LEVEL
-
logging level at which DUMPs are done
- DUMP_LOG
-
logging helper for dumping
- Dumping Data Structures
-
dump complete datastructures
- ECHO
-
unconditional logging for tests
- ELSE_NOTREACHED
-
alternative never taken
- ENSURE
-
postconditions (computation outcomes)
- ERROR
-
something gone wrong
- FIXME
-
known bug
- INFO
-
progress message
- Logging Flag Initialization
-
initialize log flags from environment
- Initialization example
-
initialize NoBug, example
- INJECT_FAULT
-
fault injection statement
- INJECT_GOODBAD
-
fault injection expression
- Setting the logging level for fault injections
-
log level for fault injection
- INVARIANT
-
validate invariant state
- The following default limits are
-
default logging limits
- LOG
-
generic logging
- LOG_BASELIMIT
-
minimum compliled-in logging limit
- LOG_EMERG
-
no logging macro for emergency logging
- Log Flags
-
define hierarchies for logging output
- Log Line Format
-
format of a log line
- Macros That Log Information
-
generate logging messages
- callback when logging
-
hook when something get logged
- logging callback prototype
-
type of a logging callback function
- Logging Concepts
-
destinations for logging output
- Targets and Options
-
configure what gets logged
- Macros Overview
-
concepts and parameters for macros
- Support
-
how to reach the NoBug developers
- Multithreading
-
using NoBug in multithreaded programs
- NOBUG_ALPHA_COMMA
-
append something after a comma in ALPHA builds
- NOBUG_ANN
-
log flag for annotations
- Source Contexts
-
pass information about the source location
- Force declarations only
-
force flag declarations
- Controlling what gets logged
-
environment variable for loging control
- NOBUG_FLAG_RAW
-
pass direct flag pointer
- NOBUG_IF_*
-
include code conditionally on build level
- NOBUG_ON
-
log flag which is always enabled
- Entering Resources
-
deadlock check on enter
- Leaving Resources
-
fix resource lists
- nobug
-
log flag used to show nobug actions
- NOTICE
-
detailed progress message
- NOTREACHED
-
code path never taken
- PLANNED
-
ideas for future
- callback after logging
-
hook after something get logged
- Dumping Persistent Ringbuffers
-
dumping persistent ringbuffers
- REQUIRE
-
preconditions (input parameters)
- RESOURCE_ANNOUNCE
-
publish new resources
- RESOURCE_ASSERT_STATE
-
assert the state of a resource
- RESOURCE_DUMP
-
dump the state of a single resource
- RESOURCE_DUMPALL
-
dump the state of all resources
- RESOURCE_ENTER
-
claim a resource
- RESOURCE_FORGET
-
remove resources
- RESOURCE_HANDLE
-
define resource handles
- RESOURCE_LEAVE
-
relinquish a claimed resource
- RESOURCE_LIST
-
enumerate all registered resources
- RESOURCE_LOG_LEVEL
-
select the log level for resource logging
- RESOURCE_LOGGING
-
switch resource logging on and off
- RESOURCE_RESET
-
reset a single resource to a pristine state
- RESOURCE_RESETALL
-
reset the resource tracker to a pristine state
- RESOURCE_STATE
-
change the state of a resource
- RESOURCE_TRY
-
wait for a resource to become available
- RESOURCE_WAIT
-
wait for a resource to become available
- The default targets are
-
default logging targets
- NOBUG_THREAD_DATA
-
thread local data for application use
- NOBUG_THREAD_ID_GET
-
query thread id
- NOBUG_THREAD_ID_SET
-
set or reset thread id
- TODO
-
things to be done
- TRACE
-
debugging level message
- UNCHECKED, Scope
-
tag scope as unreviewed
- UNIMPLEMENTED
-
not yet implemented
- WARN
-
unexpected fixable error
License
NoBug Copyright (C) 2006, 2007, 2008, 2009, 2010 Christian Thäter <ct@pipapo.org>
This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
NoBug is released under the "GNU General Public License version 2 or any later" to protect its freedom. If one wants to use NoBug in a propietary program, please contact the main author for acknowledging relicensing terms.
For BSD license style Free Software, this means you can not distribute binaries linking NoBug without making its source available. To make this compatible, it is suggested that you dual-license your software with your prefered BSD like license and the GPL. As long as it uses NoBug, the GPL will take over and you have to make the source available, while one can ship a BSD or LGPL Licensed headerfile which defines all NoBug macros as empty macros and remove libnobug from the linking, then NoBug isn’t used anymore and you may apply BSD license terms for resulting binaries.
Improvements and patches must be licensed as "GPL v2 or any later" to be acceptable. Further a contributor must either assign his copyright to the main NoBug author or agree with the possibility that NoBug can be relicensed for propietary use:
Independent of the GPL license as stated above, The main author of NoBug explicitly reserve the right to relicense NoBug under different, even propietary terms. Any contributor agrees to such a possiblility by sending his contribution to be included into the official releases.
This agreement is bilateral, every contributor who worked on a substantial part of NoBug has the right to relicense it after negotiation with the NoBug main author. Exact terms of such relicensing are worked out on a per case base.
The intention is that anyone who worked on NoBug should be able to benefit from his work. This means one should be able to use it at his workplace, to gain a job or as well as relicense it for a customer. Unlike other projects which simply ask for transfering the copyright to the main author, NoBug tries to make it possible to retain the copyright by anyone who helped the project.
This additional agreement has no impact on the GPL, it’s sole purpose is to define relicensing policies between the NoBug main author and contributors. When you recieve NoBug it will be licensed under GPL unless you personally acknowledged other terms with the NoBug main author (or any other main contributor).
If anyone feels he is not credited in the AUTHORS file or in any copyright notice, please contact the main author for inclusion.
Credits
- Design, implementation
-
Christian Thaeter <ct@pipapo.org>
- Testsuite
-
Christian Thaeter <ct@pipapo.org>
- Documentation
-
Christian Thaeter <ct@pipapo.org>
Benny Lyons <benny.lyons@uniserv.com>
François KUBLER <ih8tehuman@free.fr>
Andreas Hermann Braml <andreas@braml.org>
- Autoconf build system improvements
-
Peter Simons <simons@cryp.to>
Luca Barbato <lu_zero@gentoo.org>
Michael Ploujnikov <ploujj@gmail.com>
- Ringbuffer dump tool
-
Simeon Voelkel <simeon_voelkel@arcor.de>
- Logo
-
Pablo Lizardo <pablo_lizardo@yahoo.com.ar>