Last time we found out that linking ate up most of the time to recompile a simple vibe.d project. By using ld.gold instead of ld.bfd we were able to reduce the turnaround time to about 2s, which is already nice to work with.

This time we’ll reduce the compilation time by 60%.

You can get a nice overview of what the compiler is doing by running dmd in verbose mode (-v switch).

# compile command
dmd -v -c -of.dub/build/application-debug-linux.posix-x86_64-dmd_2065-84BE4C49CAEF5E71588D56CC0DC1CAB2/vibed-turnaround.o -debug -g -w -version=VibeDefaultMain -version=VibeLibeventDriver -version=Have_vibed_turnaround -version=Have_vibe_d -version=Have_libevent -version=Have_openssl -Isource/ -I../../../.dub/packages/vibe-d-0.7.20/source/ -I../../../.dub/packages/libevent-master -I../../../.dub/packages/openssl-master -Jviews source/app.d ../../../.dub/packages/vibe-d-0.7.20/source/vibe/appmain.d

This shows that dmd imports 247 modules and generates code for 862 functions. All of this only for our basic vibe.d app as all dependencies are precompiled by dub to static libraries. To get even more information on compile times we can use a nice tool by Vladimir Panteleev called DBuildStat. It currently only supports a single source file but I created a fake source that imports source/app.d and vibe.d’s appmain.d to workaround this issue.

echo 'import app, vibe.appmain;' > stat.d
# run dbuildstat on stat.d
dbuildstat --iterations=5 -c -of.dub/build/application-debug-linux.posix-x86_64-dmd_2065-84BE4C49CAEF5E71588D56CC0DC1CAB2/vibed-turnaround.o -debug -g -w -version=VibeDefaultMain -version=VibeLibeventDriver -version=Have_vibed_turnaround -version=Have_vibe_d -version=Have_libevent -version=Have_openssl -Isource/ -I../../../.dub/packages/vibe-d-0.7.20/source/ -I../../../.dub/packages/libevent-master -I../../../.dub/packages/openssl-master -Jviews stat.d

After quite a while you get the result in a stat.dbuildstat file. DBuildStat comes with 3 more binaries to view the results, printtimes, makedot, and makesvg.

I converted the resulting timings into the graph below but only included the upper quartile sorted by compilation time. Take the numbers with a salt of grain because they are based on separate compilation. Usually you’ll compile multiple modules (e.g. a library) at once which allows the compiler to reuse module passes when something is imported multiple times.

Still this graph shows something interesting, a lot of time is spent to import modules that arent’s used by our application. Most prominently, almost all openssl headers are imported.

There are a couple of reasons why the compiler processes all of the SSL implementation. Foremost the DMD frontend does most things eagerly. Eventually the compiler will be improved leading to faster compile times (see Issues 13255 and 13242).

Another reason is that vibe.d is imported as complete package. Replacing

import vibe.d;

in source/app.d with

import vibe.core.log, vibe.http.server;

already reduces the compile time a little (from 0.87s to 0.77s).

A third possibility to reduce compile times is to hide implementation details. This can be achieved very effectively by using scoped imports. When an import appears globally the compiler has to eagerly load the imported module to correctly resolve symbols.

import deimos.openssl.ssl;

Socket createSocket()
{
    return new Socket(SSL_new(sslctx));
}

If the import appears in a function body instead, the compiler only has to load that module when generating code for the function (e.g. for template functions or during inlining).

Socket createSocket()
{
    import deimos.openssl.ssl;
    return new Socket(SSL_new(sslctx));
}

This is also a common issue (13253) in phobos and there is a lot of ongoing work by Ilya Yaroshenko to improve this.

Other ways to hide implementations are to use separate .di header files which only contain declarations and splitting implementations into an interfaces and a class.

As an outcome of this article we’ve hidden the openssl implementation in vibe.d behind 2 SSL interfaces (see #757). Client code will now use the factory functions createSSLContext or createSSLStream and no longer sees the openssl implementation. In fact it would be possible to replace openssl with a different SSL implementation without changing vibe.d’s ABI, so an SSL implementation could even be loaded at runtime.

By only importing actually needed vibe.d modules and by hiding the openssl implementation we were able to reduce the compilation time for our basic vibe.d by 60%.

Next time we’ll look a bit closer at dub.