What is Undefined Behavior?

(https://stackoverflow.com/questions/2397984/undefined-unspecified-and-implementation-defined-behavior)

Undefined behavior is one of those aspects of the C and C++ language that can be surprising to programmers coming from other languages (other languages try to hide it better). Basically, it is possible to write C++ programs that do not behave in a predictable way, even though many C++ compilers will not report any errors in the program!

Let's look at a classic example:

#include <iostream>
int main() {
char* p = "hello!\n"; // yes I know, deprecated conversion
p[0] = 'y';
p[5] = 'w';
std::cout << p;
}

The variable p points to the string literal "hello!\n", and the two assignments below try to modify that string literal. What does this program do?

According to section 2.14.5 paragraph 11 of the C++ standard, it invokes undefined behavior:

The effect of attempting to modify a string literal is undefined.

I can hear people screaming "But wait, I can compile this no problem and get the output yellow" or "What do you mean undefined, string literals are stored in read-only memory, so the first assignment attempt results in a core dump". This is exactly the problem with undefined behavior. Basically, the standard allows anything to happen once you invoke undefined behavior (even nasal demons). If there is a "correct" behavior according to your mental model of the language, that model is simply wrong; The C++ standard has the only vote, period.

Other examples of undefined behavior include accessing an array beyond its bounds, dereferencing the null pointer, accessing objects after their lifetime ended or writing allegedly clever expressions like i++ + ++i.

Section 1.9 of the C++ standard also mentions undefined behavior's two less dangerous brothers, unspecified behavior and implementation-defined behavior:

The semantic descriptions in this International Standard define a parameterized nondeterministic abstract machine.

Certain aspects and operations of the abstract machine are described in
this International Standard as implementation-defined (for example, sizeof(int)). These constitute the parameters of the abstract machine. Each implementation shall include documentation describing its characteristics and behavior in these respects.

Certain other aspects and operations of the abstract machine are described in this International Standard as unspecified (for example, order of evaluation of arguments to a function). Where possible, this International Standard defines a set of allowable behaviors. These define the nondeterministic aspects of the abstract machine.

Certain other operations are described in this International Standard as undefined (for example, the effect of dereferencing the null pointer). [ Note: this International Standard imposes no requirements on the behavior of programs that contain undefined behavior. —end note ]


Specifically, section 1.3.24 states:

Permissible undefined behavior ranges from ignoring the situation completely with unpredictable results, to behaving during translation or program execution in a documented manner characteristic of the environment (with or without the issuance of a diagnostic message), to terminating a translation or execution (with the issuance of a diagnostic message).

--(continued)

What are the different stages of compilation?

The compilation of a C++ program involves three steps:

Preprocessing: the preprocessor takes a C++ source code file and deals with the #includes, #defines and other preprocessor directives. The output of this step is a "pure" C++ file without pre-processor directives.

Compilation: the compiler takes the pre-processor's output and produces an object file from it.

Linking: the linker takes the object files produced by the compiler and produces either a library or an executable file.

Preprocessing

The preprocessor handles the preprocessor directives, like #include and #define. It is agnostic of the syntax of C++, which is why it must be used with care.

It works on one C++ source file at a time by replacing #include directives with the content of the respective files (which is usually just declarations), doing replacement of macros (#define), and selecting different portions of text depending of #if, #ifdef and #ifndef directives.

The preprocessor works on a stream of preprocessing tokens. Macro substitution is defined as replacing tokens with other tokens (the operator ## enables merging two tokens when it makes sense).

After all this, the preprocessor produces a single output that is a stream of tokens resulting from the transformations described above. It also adds some special markers that tell the compiler where each line came from so that it can use those to produce sensible error messages.

Some errors can be produced at this stage with clever use of the #if and #error directives.

Compilation

The compilation step is performed on each output of the preprocessor. The compiler parses the pure C++ source code (now without any preprocessor directives) and converts it into assembly code. Then invokes underlying back-end(assembler in toolchain) that assembles that code into machine code producing actual binary file in some format(ELF, COFF, a.out, ...). This object file contains the compiled code (in binary form) of the symbols defined in the input. Symbols in object files are referred to by name.

Object files can refer to symbols that are not defined. This is the case when you use a declaration, and don't provide a definition for it. The compiler doesn't mind this, and will happily produce the object file as long as the source code is well-formed.

Compilers usually let you stop compilation at this point. This is very useful because with it you can compile each source code file separately. The advantage this provides is that you don't need to recompile everything if you only change a single file.

The produced object files can be put in special archives called static libraries, for easier reusing later on.

It's at this stage that "regular" compiler errors, like syntax errors or failed overload resolution errors, are reported.

Linking

The linker is what produces the final compilation output from the object files the compiler produced. This output can be either a shared (or dynamic) library (and while the name is similar, they haven't got much in common with static libraries mentioned earlier) or an executable.

It links all the object files by replacing the references to undefined symbols with the correct addresses. Each of these symbols can be defined in other object files or in libraries. If they are defined in libraries other than the standard library, you need to tell the linker about them.

At this stage the most common errors are missing definitions or duplicate definitions. The former means that either the definitions don't exist (i.e. they are not written), or that the object files or libraries where they reside were not given to the linker. The latter is obvious: the same symbol was defined in two different object files or libraries.

What are circular dependencies and how to break them?

Imagine you are writing a compiler. And you see code like this.

// file: A.h
class A {
B _b;
};

// file: B.h
class B {
A _a;
};

// file main.cc
#include "A.h"
#include "B.h"
int main(...) {
A a;
}

When you are compiling the .cc file (remember that the .cc and not the .h is the unit of compilation as mentioned here), you need to allocate space for object A. So, well, how much space then? Enough to store B! What's the size of B then? Enough to store A! Oops.

Clearly a circular reference that you must break.

You can break it by allowing the compiler to instead reserve as much space as it knows about upfront - pointers and references, for example, will always be 32 or 64 bits (depending on the architecture) and so if you replaced (either one) by a pointer or reference, things would be great. Let's say we replace in A:

// file: A.h
class A {
// both these are fine, so are various const versions of the same.
B& _b_ref;
B* _b_ptr;
};

Now things are better. Somewhat. main() still says:

// file: main.cc
#include "A.h" //Problem here

#include, for all extents and purposes (if you take the preprocessor out) just copies the file into the .cc. So really, the .cc looks like:

// file: partially_pre_processed_main.cc
class A {
B& _b_ref;
B* _b_ptr;
};
#include "B.h"
int main (...) {
A a;
}

You can see why the compiler can't deal with this - it has no idea what B is - it has never even seen the symbol before.

So let's tell the compiler about B. This is known as a forward declaration, and is discussed further in this answer.

// main.cc
class B;
#include "A.h"
#include "B.h"
int main (...) {
A a;
}

This works. It is not great. But at this point you should have an understanding of the circular reference problem and what we did to "fix" it, albeit the fix is bad.

The reason this fix is bad is because the next person to #include "A.h" will have to declare B before they can use it and will get a terrible #include error. So let's move the declaration into A.h itself.

// file: A.h
class B;
class A {
B* _b; // or any of the other variants.
};

And in B.h, at this point, you can just #include "A.h" directly.

// file: B.h
#include "A.h"
class B {
// note that this is cool because the compiler knows by this time
// how much space A will need.
A _a;
}

How can I use RAII to manage resources programmed using a C-style API?

There is an easy way to use RAII to manage resources from a C-style interface: the standard library's smart pointers, which come in two flavors:
std::unique_ptr for resources with a single owner and the team of std::shared_ptr and std::weak_ptr for shared resources. If you're having trouble deciding which your resource is, this FAQ should help you decide. Accessing the raw pointer managed by a smart pointer is as easy as calling its get member function.

If you want simple, scope-based resource management, std::unique_ptr is an excellent tool for the job. It's designed for minimal overhead, and is easy to set up to use custom destruction logic. So easy, in fact, that you can do it when you declare the resource variable:

#include <memory> // allow use of smart pointers

struct CStyleResource; // c-style resource

// resource lifetime management functions
CStyleResource* acquireResource(const char *, char*, int);
void releaseResource(CStyleResource* resource);

// my code:
std::unique_ptr<CStyleResource, decltype(&releaseResource)>
resource{acquireResource("name", nullptr, 0), releaseResource};

acquireResource executes where you call it, at the start of the variable's lifetime. releaseResource will execute at the end of the variable's lifetime, usually when it goes out of scope.

You can do much the same with std::shared_ptr, if you require that brand of resource lifetime instead:

// my code:
std::shared_ptr<CStyleResource>
resource{acquireResource("name", nullptr, 0), releaseResource};

Now, both of these are all well and good, but the standard library has std::make_unique and std::make_shared and one of the reasons is further exception-safety.

GotW#56 mentions that the evaluation of arguments to a function are unordered, which means if you have a function that takes your shiny new std::unique_ptr type and some resource that might throw on construction, supplying that resource to a function call like this:

func(
std::unique_ptr<CStyleResource, decltype(&releaseResource)>{
acquireResource("name", nullptr, 0),
releaseResource},
ThrowsOnConstruction{});

means that the instructions might be ordered like this:

1. call acquireResource
2. construct ThrowsOnConstruction
3. construct std::unique_ptr from resource pointer

Now our precious C interface resource won't be cleaned up properly if step 2 throws because the unique_ptr hasn't been constructed yet..

Again as mentioned in GotW #56, there's actually a relatively simple way to deal with the exception safety problem. Unlike expression evaluations in function arguments, function evaluations can't be interleaved. So if we acquire a resource and give it to a unique_ptr inside a function, we'll be guaranteed no tricky business will happen to leak our resource when ThrowsOnConstruction throws on construction. We can't use std::make_unique, because it returns a std::unique_ptr with a default deleter, and we want our own custom flavor of deleter. We also want to specify our resource acquisition function, since it can't be deduced from the type without additional code. Implementing such a thing is simple enough with the power of templates.

#include <memory> // smart pointers
#include <utility> // std::forward

template <
typename T,
typename Deletion,
typename Acquisition,
typename...Args>
std::unique_ptr<T, Deletion> make_c_handler(
Acquisition acquisition,
Deletion deletion,
Args&&...args){
return {acquisition(std::forward<Args>(args)...), deletion};
}

You can use it like this:

auto resource = make_c_handler<CStyleResource>(
acquireResource, releaseResource, "name", nullptr, 0);

and call func worry-free, like this:

func(
make_c_handler<CStyleResource>(
acquireResource, releaseResource, "name", nullptr, 0),
ThrowsOnConstruction{});

The compiler can't take the construction of ThrowsOnConstruction and stick it between the call to acquireResource and the construction of the unique_ptr, so you're good.