One of our most frequent feature requests for CLion is the ability to use project or build systems other than CMake. There are many good reasons for needing to use something different – but if you could only pick one we still feel CMake is the best choice. That’s not because we think it is perfect (although it’s worth noting that some of the issues that people have with it may be addressed by focusing on “Modern CMake”). Instead it’s because CMake is in very widespread use – especially in the open-source world, where it has become the de-facto standard.
CMake also has the advantage that it is a meta-build system. It can generate build files for other major build systems. In fact that is really the only way works, although CLion uses it in a way that mostly hides that from you by default (generating Makefiles behind the scenes).
While this is a great starting point, we do want to support other build systems and project formats and work has been progressing over the last couple of releases towards fully decoupling CMake. The initial results of that ongoing work have been: support for Cargo (for Rust projects) in 2018.1, a third-party plug-in for Bazel, and, in 2018.2, Gradle C++ projects and the ability to open Compilation Database JSON files. This is just the start, but is also a lot more useful already than you may realise. To find out why we need to look at what a Compilation Database actually is.
What is a Compilation Database?
In the most general terms a Compilation Database is simply the collection of all information needed to compile a source file, or set of source files – on a file-by-file basis. That includes the filename, all flags and other arguments to be passed to the compiler and the directory to invoke the compiler from (this affects relative paths).
More practically a Compilation Database is most often serialised as a JSON file and comes out of the Clang tooling world. It has become an important mechanism for interoperating between different tools that need to know how code should be compiled – even if not for compilation itself.
Compilation DB JSON files can be generated by an increasing number of build systems and other tools.
What does this mean for CLion users?
Now that CLion can open Compilation DB JSON files it means it can effectively understand projects from many more build systems.
This gives CLion full code-completion, static analysis, navigation and even refactoring on such projects. For 2018.2 we don’t yet have build (except for individual files) or debug capabilities – but these are planned for an upcoming release.
To see just how useful this is, we’ll work through a couple of examples.
Compilation Database Generator
Before we get to the examples themselves, we’ll need to install a tool that can generate Compilation DB JSON files from Makefiles. The tool we’ll use for this tutorial is compiledb. It’s written in Python, so the simplest way to get it is to use pip.
$ pip install compiledb
(you may need to do this with sudo)
Example 1: OpenSSL
OpenSSL is a large, mature, well-known and widely used, open source code base. It’s written mostly in C and has an incredibly complex build system using generated make files.
If you want to follow along you can find the source hosted on GitHub. Taking a fresh clone of this repository you’ll notice it has no Makefiles yet. Like many large, complex, make-based projects you must first run a configuration step that generates the Makefile. Full instructions are in the text file INSTALL in the root of the repository. For Unix-based OSs (including Linux and Mac OS/X/ macOS), the first step is to execute the configure step in a shell:
$ ./config
(Note there are some prerequisites, such as having Perl 5 with core modules – see the INSTALL file for more details).
This runs briefly and, hopefully, completes with no errors, leaving a Makefile in the same directory. If you open the Makefile you’ll see it is not really for human reading – and certainly not for modifying. But now we can use it to give CLion insight into how the codebase should be understood.
To do that we now need to generate a Compilation DB JSON file:
$ compiledb -n make
(passing -n stops make from performing the actual build. If we omit it it will build the whole of OpenSSL at the same time).
This should tell us that it is “Writing compilation database with [some number of] entries to compile_commands.json”.
We can now open the directory in CLion.
![Open directory]()
CLion will detect the compile_commands.json file and look in there for its project information. You may see it think about it for a few seconds, then show the “Compiler Info” tab on the Build tool window, telling you that everything finished ok (there were any errors or warnings you’ll see them in the “Sync” tab).
![Compiler Info tab in Build tool window]()
We can now explore the codebase in CLion. What better way to start than to do a “Search Everywhere” (hit the Shift key twice) and type aria.c to bring up one of the source files. Assuming indexing has finished (which shouldn’t take long) we should instantly find this file.
If we open that and browse through you’ll see full syntax highlighting. Half of this file is #ifdefed out. The second half should lack highlighting and code insight features as OPENSSL_SMALL_FOOTPRINT is not defined (if you used the default configuration).
Find one of the functions (in the enabled first half), e.g. aria_encrypt. If you “Find Usages” on this it will find all the usages in other files in the project.
![Find Usages]()
So now we can finally find our way around large Makefile codebases like OpenSSL.
But because OpenSSL’s make files are generated, we shouldn’t modify them ourselves. Let’s look at a simpler example to see what happens when we do.
Example 2: Modifying Makefiles
For this example we’re going to consider a very simple Makefile. It doesn’t matter what the code does. What’s important is that it has two or more source files, but is simple enough we can easily see what the Makefile is doing. Here’s a starting point:
appname := question
CXX := clang++
CXXFLAGS := -std=c++11
srcfiles := main.cpp foo.cpp
objects := $(patsubst %.cpp, %.o, $(srcfiles))
all: $(appname)
$(appname): $(objects)
$(CXX) $(CXXFLAGS) $(LDFLAGS) -o $(appname) $(objects) $(LDLIBS)
clean:
rm -f $(objects)
Notice here we have two source files. To keep things even simpler we’ll leave out discussion of how to automatically extract header file dependencies.
Now, as the project grows you’ll probably want to add additional source files – or make other changes to this make file. If we do that it’s going to get out of sync with the Compilation DB that CLion is working off.
We can, of course, drop to the terminal and invoke compiledb again every time this happens. But that’s tedious and can be error prone. In any case we’re programmers. We like to automate things!
That’s where CLion’s File Watchers come in. File Watchers are a powerful feature in CLion that allows us to trigger external tools or scripts in response to changes in specific files.
Before we get started we’ll need to make sure that CLion is aware of Makefiles. We can do that by adding the Makefile support plug-in (Preferences->Plugins->Browse Repositories, “Makefile support”)
![Browsing plugin repositories]()
Once installed (you’ll need to restart CLion) if you open the Makefile you’ll see it is now syntax highlighted. But that’s not why we needed the plugin.
With the Makefile open, go to preferences again, and this time find “File Watchers” (under “Tools” – or just start typing “Watch”). Click the + at the bottom left and select “” from the list of templates.
Give the new File Watcher a name, like “Makefile watcher”, and from the “File type” dropdown select “GNU Makefile” (start typing “make” on the field to find it quickly). That’s what we needed the plugin for.
Fill out the rest of the fields as below (or set “Show console” to your choice):
![Creating a new File Watcher]()
After you click OK, CLion will start monitoring the Makefile and, if it changes – whether from inside the IDE or externally – it will trigger compiledb to run again, regenerating the Compilation DB. To verify this, open the compile_commands.json file side-by-side with the Makefile. Add or remove source files from the srcfiles variable and watch as compile_commands.json is regenerated automatically.
At time of writing there is still a limitation as the Compilation DB itself is not automatically reloaded by CLion. You’ll have to ask CLion to reload it (Tools->Compilation Database>Reload Compilation Database Project). So this is not quite seamless yet. Please upvote this ticket if this is something you’d like to have.
Something to build on
We’re not stopping here. We’ll continue working towards supporting more build systems and project formats in more integrated ways. But as of now there are some very useful and valuable options you can take advantage of today.
Nowadays Clang is definitely the most accurate solution in terms of correct C++ code parsing and resolve. However in its current state it can’t support all the code insight features we’d like to provide in CLion. So we’ve decided to keep both language engines working at the same time.
