I watched some time ago a presentation by James Gosling and one of the things he said was that he created Java because he spent so much time debugging memory bugs that he wanted to make sure he will not do that again. I programmed in Java, maybe more than I programmed in C++ and for a moment I really felt that. In that moment I remembered how it is to write code and not think about freeing things and not think about who destroys who, and what to delete in your destructor and all that: it’s so much fun. Sure it’s fun when I want control and I get that and I do a really good trick to save memory but most of the time I don’t care about memory. Most of that code seems so repetitive. So boring!

But the problem remains. It needs to be done when writing C++ and I think about ways to do this well a lot. Recently something obvious occurred to me. In all these years I programmed in C++ doing games and in games there is not a single framework for doing things. Everybody has his own engine or framework and it occurred to me that nobody uses this to their advantage. I mean if I write a framework then I can provide some useful stuff to the end-user but at the same time I get to impose some rules on that user. I get to say how my engine is used and if I get to say that, then maybe there is a way to simplify the memory management, at least in the context of my engine. And there is! Apple does this already in Objective-C with Cocoa. I don’t understand why these things are not more common. Because of the nature of game engines I would expect to see this kind of memory management in every engine. It’s like a kind of bait: if you want to use this wonderful engine then abide by these rules and voilà your management for the memory is so much simple.

And also in a game engine you could have a lot of fun stuff especially for games. For example tagged allocations. I mean if I allocate everything with the tag “Level1″ attached, then when I want to free Level1 I can just say something like ResMgr->Free(“Level1″) and not care about all the things I’ve created for “Level1″ because my resource manager knows that. What I mean is that it’s not helping anyone if the code in a game engine is generic. It should be specific, it should take into account the domain.

Anyway, for some time, I just thought that it wouldn’t be possible to do this so I didn’t bothered but in the past month I did a build system at work using CMake and I learned a lot of things. CMake is truly an incredible tool! So today I gave it a shot. It was easy but I learned a couple of things in the process.

cmake_minimum_required (VERSION 2.8)
PROJECT (kernel)

INCLUDE_DIRECTORIES (include)

SET (CMAKE_CXX_FLAGS "-m32 -ffreestanding -fno-rtti -fno-exceptions -nostdlib -nostdinc -fno-stack-protector")
SET (CMAKE_C_FLAGS "-m32 -ffreestanding -fno-exceptions -nostdlib -nostdinc -fno-stack-protector")
SET (CMAKE_EXE_LINKER_FLAGS "-m elf_i386 -T${CMAKE_CURRENT_LIST_DIR}/link.ld")
SET (CMAKE_ASM_NASM_OBJECT_FORMAT "elf")

ENABLE_LANGUAGE (ASM_NASM)

FILE (GLOB_RECURSE kernel_C_SOURCES "kernel/*.c")
FILE (GLOB_RECURSE kernel_CPP_SOURCES "kernel/*.cpp")
FILE (GLOB_RECURSE kernel_ASM_SOURCES "arch/x86/asm/*.nasm")
FILE (GLOB_RECURSE kernel_ARCH_C_SOURCES "arch/x86/*.c")
FILE (GLOB_RECURSE kernel_ARCH_CPP_SOURCES "arch/x86/*cpp")

ADD_LIBRARY (kernel.img STATIC
			${kernel_C_SOURCES} 
			${kernel_CPP_SOURCES} 
			${kernel_ARCH_C_SOURCES} 
			${kernel_ARCH_CPP_SOURCES} 
			${kernel_ASM_SOURCES})

ADD_CUSTOM_COMMAND (TARGET kernel.img
	POST_BUILD
	COMMAND "/usr/bin/ld -m elf_i386 -T${CMAKE_CURRENT_LIST_DIR}/link.ld -o phOS.img ${CMAKE_BINARY_DIR}/libkernel.img.a")

As you can see the file came out pretty small. The first thing that I’ve learned was about compiling asm files. CMake has now support for that. All you have to do is ENABLE_LANGUAGE (ASM_NASM) and when you feed it *.nasm and *.asm files it will do the job. The thing that I was stuck with was the format of the outputed binary. By default CMake gave nasm a -f elf64 option (considering that I’m on a x86_64 platform). It took me a while to discover that you can change that by setting CMAKE_ASM_NASM_OBJECT_FORMAT variable.

The second major problem was with linking. I want to issue the link command myself and I tried to influence CMake but it came out wrong. As far as I understand CMake generates a link script and on linux it uses the compiler to link object files instead of LD (in the end LD is used probably but trough the compiler). In the end I solved this by making a static library and then adding a POST_BUILD command that relinked that library in something that I wanted. After the build finishes the custom command is executed and I obtain the binary that I want.

Now I get all the good things that come with CMake, IDE support being the most wanted feature on my list.

Let’s say you’re developing a 2D game engine. Your game engine will be a library that you and possibly other users will use. The engine provides many abstractions and one of them is loading and drawing an image on screen. Now your engine supports many image formats by using several libraries like libpng, libjpeg etc. To abstract the various image formats you have a class called Image with the interface declared in Image.h.

#include <png.h>
#include <jpeg.h>

class Image
{
...
}

Your API interface has a sprite class that you can use to draw images on the screen in Engine/Sprite.h that looks something like this.

#include <Image.h>

class Sprite
{
public:
    Sprite(std::string& filename);
    Render();
private:
    Image* img;
}

What is wrong with the code above? The Sprite class should abstract away all the details concerning third party image libraries. But because Image is included in the header and in Image.h I include png.h and jpeg.h, it’s probable that some structures or symbols will be included in my code. This means that the client of the API must have access to the include files of libpng nd/or libjpeg. This is useless because the user of the library should not be concerned with other dependencies, because that’s the purpose of the engine: to hide complexity and to provide a nice interface that does things in a friendlier way. Even if it is not a problem including some more include paths in the build, it’s still wrong because you actually signal to the client of your API that is ok to directly use libpng and libjpeg and this defeats the entire purpose of the API.

The solution to this problem is called forward declaration.

class Image;  // forward declaration

class Sprite
{
...
}

Now you’ll have to include the header file in the corresponding Sprite.cpp file but everything from Image.h will be included only in the engine and not in the clients code.

So what are good rules of thumb of forward declaration?

1. You can’t forward declare the classes that you inherit from and if you think a little about this, it’ll make sense.
2. Things that come from the standard library can be ignored. Don’t bother to forward declare those because they are already available to every unit that is compiled.
3. You can forward declare everything except for things mentioned at 1 and 2 (optional): classes, structs, typedefs.

Number 3 can bring some problems especially if you use a C library. C libraries tend to have a lot of defined types. One such example is the FreeType library. FT_Face is a defined type. So if you declare one in your *.h file as a member, how can you forward declare this? You have to forward declare everything starting from the struct, and typedefs.

struct FT_FaceRec_;  // first forward declare the struct
typedef struct FT_FaceRec_*  FT_Face;  // then forward declare the typedef because the typedef depends on the struct

So in order to forward declare FT_Face you have to search for the definition of the type and then forward declare the initial type (struct FT_FaceRec_) and then the new type (FT_Face) because the typedef declaration depends on the declaration of the struct.

This pretty much covers everything one needs to forward declare efficiently and I hope that the reasons to do this are compelling enough and are actually more important than compilation speed. It’s still good when something is done for the wrong reasons but the bad thing is that is easy to be inconsistent if those reasons are not strong enough to go all the way.

The author just told me that if I can replicate that problem in an unit test case then he would take a look at it. So I was stuck because the problem was in the implementation of a hash_map and I was using several instances of hash_map in my application but only this one crashed. This was a hash_map holding textures so I was assuming that maybe something was wrong with the implementation of the texture class. To make the matters worse when a colleague of mine moved some lines up and down or commented them the problem suddenly vanished. Because we were pressured by time we let this one go, thinking that if it has something to do with the implementation of the hash_map somebody else will bump into it and we’ll try to solve it then.

Then after a couple of months another colleague found something unusual in the implementation of the same texture module, a class for loading a picture format. Some struct was guarded by a #pragma pack(push, 1) but there was no #pragma pack(pop) at the end of the struct. As you probably know this directive is telling the compiler to let the struct unaligned (the default behavior is to align the struct at say 4 bytes – platform dependent – because of the faster access). This is useful if you want to read a struct from a file in one go just mapping the bytes on the structure. Is especially useful when reading headers of files and things like that so it is no surprise that it was used for texture loading.

Read more about pragmas at http://msdn.microsoft.com/en-us/library/d9x1s805.aspx

Anyway we figured that this was the original cause for the hash_map crash. Probably the implementation of the hash_map did some assumptions about the alignment of structures.

The thing that stuck with me from all this situation is that at the moment I was trying to figure out a way to prevent this. What do I or anyone else could have done to prevent these kind of problems or to, at least, minimize the effects?

Last week I had a discussion at work about the usefulness of forward declaring classes in C++ and in the discussion we kept enumerating good reasons for doing this. Everybody will tell you about compilation speed but I jumped ahead and said something like “well it’s good to do this because you only include what you really need and in this way you can prevent all sorts of bad things”. And everybody kept waiting looking at me: “like what?”. And I start thinking at things and then the initial story pops into my head. If we have kept the includes at a minimum that problem couldn’t have manifested itself because probably the problem was that the header file was included in other header files and the pragma option applied to several other structures that we were using or even structures in EASTL.

So this was my answer. We could have kept the effects at a minimum if we respected a simple coding style rule: Forward declare everything you can!

BTW we’ve started doing this after we’ve committed ourselves to use Google C++ Coding Style.

The other solution to these kinds of problems is to have a static analyzer checking your code periodically but you don’t know if the static analyzer will check for these kinds of issues, so respecting strong coding guidelines is better.

It looks that I’m out of luck with Make in this regard because all the tutorials are either basic or advanced discussing how to use Make in large projects for example. Let’s see why the basic tutorials are useless and which parts should be covered in order to remedy their shortcomings. The basic tutorials tell you that Make, when invoked, looks for a Makefile file in the current directory. This Makefile contains rules that tell Make what commands to issue. The rules in the Makefile have the following form:

target: dependencies
[tab] system command


You have a special target called all that can be used as a default target. When you invoke make, if you don’t specify the target, this target and the corresponding rules will be executed.

Knowing this the tutorial shows how to compile a file (or two):

all: test

test: test.o sort.o
	gcc test.o sort.o -o test

test.o: test.c
	gcc -c test.c

sort.o: sort.c
	gcc -c sort.c

The all target is dependent on test and the test target is dependent on test.o and test.o target is dependent on test.c and test.c isn’t a target, so the command associated with this rule is executed producing test.o object, and going back we see that test is also dependent on sort.o and sort.o is dependent on sort.c, so sort.c is compiled and then the command for test is executed now that all its dependencies are satisfied and we end up with the compiled and linked program named test.

What I’m trying to say is that all the dependencies are organized into a tree of dependencies and the rules executed from leafs to root like in the figure below.

Make script dependency tree

Now you can write a Makefile to compile some source files but usually you want to use a build system to compile a small project and you want to use it to simplify your life. What you learned so far doesn’t simplify shit because you can do the same, involving the same amount of work, with bash scripts. Let me elaborate!

Let’s say that you will add 10 more files to the project. For each file you add 3 lines (2 lines of code and a empty line) of repetitive commands and you modify the test target to contain the new dependencies. That means 30 lines more of text adding to a total of 40 lines. I bet you can do better using a Bash script! Imagine what happens when you delete a file or when you rename a file. Every time you modify something you have to edit the Makefile. This is the reason I think many basic Make tutorials are not so good. Because they teach you some basic stuff about the system but those things are not enough to use the system in such a way that simplifies things for you.

Letting make Deduce the Recipes

 
Let’s see what info is missing that we can use to have a simple, more generic Makefile that adapts more easily to simple modifications. The first important bit of info that will help us a lot can be found in the GNU Make manual and it concerns a special feature of make that lets make deduce predefined recipes for you. For example make knows that a .o file is produced from a .c or .cpp file. So if you don’t specify an explicit recipe for a target like this, make will execute a predefined one for you if it finds a corresponding .c or .cpp file for the intended target.

all: test

test: test.o sort.o
	gcc test.o sort.o -o test

Using this info we can modify the Makefile to look like above. When a new file is added we just add it in two places. And even for 30 more files we can keep the build file at only 4 lines. But still can we do better? Of course!

Special make variables

 
First, what do better in our example means? When we add a file, modify a file or delete a file we must modify the same information in two places. We can avoid this by placing the list of files in a variable but it pays to learn about several special variables that can come in handy at times even if for this example they are not the only answer.

• $^ – will expand to the list of prerequisites (dependencies)
• $< – will expand to the first prerequisite (dependency)
• $@ – will expand to the target

The explanations above are very simple and you can use the variables like this for simple examples but watch out for unexpected behaviour and if something unusual happens, go read the manual. Also in the manual you’ll find several others useful special make variables. Using these we can transform our Makefile:

SOURCES=test.o sort.o
all: test

test: $(SOURCES)
	gcc $^ -o $@

So now if something in our project changes we have to make the change to a single line and not worry about the rest.

Configuring the compiler

 
Previously when we discussed the automatic recipes I’ve left out an important bit of info. When the compiler is invoked you’ll probably see a simple command like cc -c -o file.o file.c but usually we want to pass configuration options to both gcc or g++. We can do that by putting those configure options in the environment variables CFLAGS (for gcc), CXXFLAGS (for g++) or CPPFLAGS (for the preprocessor) or setting them in our Makefile like in the example below:

SOURCES=test.o sort.o
CFLAGS=-O3
CXXFLAGS=-O3
all: test

test: $(SOURCES)
	gcc $^ -o $@

These variables can be configured when you issue the make command:
CFLAGS=-O3 make

Conclusions

 
This is, in my opinion, everything one needs to start out with make and use it to save time. And for simple projects we just move one Makefile around modify the SOURCES line and we’re compiling things. But by using a final tip we can have a generic Makefile that compiles all .c or .cpp files in the current directory. This is achieved by using two make functions: wildcard and patsubst.

SOURCES=$(patsubst %.c,%.o,$(wildcard *.c))
CFLAGS=-O3
CXXFLAGS=-O3
all: test

test: $(SOURCES)
	gcc $^ -o $@

Wildcard makes a list of all .c files in the current directory and then patsubst replaces the extension from .c to .o.

You now have a generic file that can be used as a starting point for any small project. For bigger projects I strongly recommend reading the GNU make manual. It’s up to you to add a clean target or any other targets that your project may need but you’ll find this info elsewhere.

For example you hold two cards, you know the flop and you want to see how you stand against an opponent. In order to do this you must list all the other possible hands and all the combinations of 2 cards that represents the turn and the river cards. You compare all the possible hands and see in what percent of the cases you win and in what percent you lose and you know what are your chances. This is only an example but in order to achieve this you must write code to generate all combinations.

When I have to generate all possible solutions, the first (and only) algorithm that comes to mind is Backtracking. Recursive backtracking is really easy to implement so I’ve implemented one and it worked OK for combinations of 2 but for 3 i got heap overflow exceptions. The solution now is to implement an iterative backtracking algorithm. I could’ve modified the recursive one and transform it into an “ugly” iterative version but I said to myself that first I should check out some books or papers about backtracking. If I need it and I’m not remembering it so well, at least I got to try to understand it right this time.

So I opened Knuth – The art of computer programming – Volume 4A and I started with generating n-tuples and on the second page it struck me. In several lines of pseudocode it was this simple algorithm to generate all n-tuples, that you can use to generate all permutations and with some modifications you can also use it to generate all combinations. And above all it is not backtracking.

The idea is super simple. You start with a number expressed in n digits and you always add one to it and print the solutions. So you start with 00 and you add 1 and the number becomes 01 and you print the solution. Then you add 1 and the number becomes 02 and so on. When the number is 09 and you add one to it, it will become 10 and so on until 99.

When you want to generate combinations you must impose the condition that the digits are always in ascending or descending order. Also if you have more than 10 symbols to combine, you just use a different radix for each digit: in my case radix 52 because in a pack of cards you have 52 different cards. And if for each digit you have a different set of symbols you can use an array with different radices for each digit.

So simple and elegant!

Edit: An implementation of the idea above can be found at CardListGenerator.java from the jpokerlib project.

ld: i386 architecture of input file `arch/monitor.o' is incompatible with i386:x86-64 output

After some goggling I found out how to force LD into linking the binaries for an i386 architecture by giving it the -m elf_i386 argument but after that it started complaining about the binaries outputted by GCC not being compatible with the i386 output.

ld: i386:x86-64 architecture of input file `kernel/main.o' is incompatible with i386 output

That is because on x86-64 GCC outputs 64 bit binaries. In order to tell GCC to output x86-32 binaries you have to use -m 32 flag. After this it worked perfectly!

Coders at work cover

I’ve just finished reading Coders at work by Peter Seibel and I’ll do a little review here to note some things that for sure I’ll want to remember and maybe these things are interesting for others too. The book is an interview collection with several prominent figures from the field of “computer science” and if one thinks that you have things to learn from interviews then this book is a must read. Personally I’m interested in the context that other professionals work and the way that that context influenced them and the book does a pretty good job at answering this. Sure it could be better but you can’t find so many books with the same profile so I’m happy with less than nothing at all.

One interesting point that I found in the book is that all the interviewees said that the qualities needed to become a good programmer/craftsman/coder/engineer (the term is not agreed upon ) remained pretty much unchanged but right now we have some people doing something that looks like programming but is not really programming according to some of the people interviewed in the book. I’m referring here to people that are just using APIs and are fitting pieces of code together all day long without being interested in the art of programming or the history of this field.

There are a lot of lessons in the book for technical leaders and project managers that I found very interesting.

One of the things I’ve been pushing is code reading. I think that is the most useful thing that a community of programmers can do for each other—spend time on a regular basis reading each other’s code. There’s a tendency in project management just to let the programmers go off independently and then we have the big merge and if it builds then we ship it and we’re done and we forget about it. One of the consequences of that is that if you have weak or confused programmers you’re not aware of their actual situation until much too late. And so the risks to the project, that you’re going to have to build with stuff that’s bad and the delays that that causes, that’s unacceptable. The other thing is that you may have brilliant programmers on the project who are not adequately mentoring the other people on the team. Code reading solves both of those problems. — Douglas Crockford

Douglas Crockford talks a lot about how he hires people and what are the things you should look for when managing people. The advices from him are very good and all the people that have a management job usually touch this subject some way or another. Of course there are others in the book that touch these kinds of problems. Frances Allen makes some very interesting remarks about the appearance of the waterfall method of managing software projects. It clearly shows that in this respect the domain was in a pioneering state and nobody knew exactly what was the right way to do things. It’s interesting that even today the processes are not so well established and at an instinctual level we are still looking for Brooks silver bullet.

Then there’s—I don’t know if I read it somewhere or if I invented it myself—Joe’s Law of Debugging, which is that all errors will be plus/minus three statements of the place you last changed the program. — Joe Armstrong

The book contains some interesting things about debugging like the quote from Joe Armstrong. Also Bernie Cosell discusses some interesting stories about the bugs although he admits that his weapon of choice for squashing bugs is to rewrite the code that seems fishy.

At the end there is the cherry on the cake: Donald Knuth. His interview is the most interesting for me. I think he really shows that programming is an art, a science and a craft all at the same time and this is the real beauty of it. It really shows that he sees this from a lot of perspectives and that makes his point of view so rich and complete.

So one has to give a good implementation for three stacks into a single array.

Well I thought about an esoteric solution that involved some bit operations. The idea was to divide each element in three. So if the vector was of type integer you would divide the integer into [10 10 12] bits = [32] bits. But in this stack you can hold numbers between ([512, 511]) big if you consider the sign. You can argue that since nothing is specified about what numbers you have to hold in the stacks this is a good solution. But anyway this is not very good because the counter argument can be that the size of an element is given by the type of elements that the array can hold.

So a better solution is needed. I saw on some blog that somebody else considered to divide the vector intro three equal parts putting the first stack at the beginning the second stack at the end of the array growing downward and the third stack at one third of the array growing upward. This is still a bad solution because the second and the third stack can have at most two thirds of the vector in the eventuality that one of them is not used but the first stack only has one third of the vector.

I thought that you can start a stack somewhere in the vector but somehow be fair to the other stacks and following this train of thought I found the solution. You just start in the middle with the third stack and grow it in both directions alternatively. You hold the stack pointer and when you do a pop on the third stack you have to find the previous element in the stack to update the stack pointer. If the index of the element being popped is smaller than the middle of the vector then the next element is situated at 2*m-idx, m being the index of the middle element and idx being the index of the element being popped and 2*m-idx+1 if the index is bigger. And for the push the formula for finding the next index is somewhat similar.

Anyway this is just reinventing the wheel because the solution to this particular problem is all over the Internet but is fun in spite of this.


CONFIG_DEBUG_INFO
CONFIG_DEBUG_SLAB
CONFIG_DEBUG_PAGE_ALLOC
CONFIG_DEBUG_SPINLOCK
CONFIG_DEBUG_SPINLOCK_SLEEP
CONFIG_INIT_DEBUG
CONFIG_DEBUG_INFO
CONFIG_MAGIC_SYSRQ
CONFIG_DEBUG_STACKOVERFLOW
CONFIG_DEBUG_STACK_USAGE
CONFIG_KALLSYMS
CONFIG_IKCONFIG
CONFIG_IKCONFIG_PROC
CONFIG_ACPI_DEBUG
CONFIG_DEBUG_DRIVER
CONFIG_SCSI_CONSTANTS
CONFIG_INPUT_EVDEBUG
CONFIG_PROFILING


I just compiled linux 2.6.37-rc4 and tried to compile a test driver. The compilation failed because it didn’t found ioctl while initializing struct file_operations and after a little searching on the net and in the sources I’ve found that ioctl was replaced by a version that is not protected by BKL and another version protected (?) by the BKL to keep compatibility (unlocked_ioctl, compat_ioctl in linux/fs.h).

The second thing that is missing is the init_MUTEX macro that was removed from linux/semaphore.h. The macro just did a sema_init (&sem, 1), so a simple replace did the job.