LLVM Project News and Details from the Trenches

Monday, November 18, 2013

Google Summer of Code: C++ Modernizer Improvements

Google Summer of Code (GSoC) offers students stipends to participate in open source projects during the summer. This year, I was accepted to work on the Clang C++ Modernizer, a project formerly known as the C++11 Migrator, driven by a team at Intel. The goals of the tool are to modernize C++ code by using the new features of new C++ standards in order to improve maintainability, readability and compile time and runtime performance. The project was featured in the April blog post “Status of the C++11 Migrator” and has been evolving since, both in terms of architecture and features.

This article presents the improvements made to the tool in the last few months, which include my work from this summer for GSoC. For a complete overview of the tool and how to install it, please visit the documentation: http://clang.llvm.org/extra/clang-modernize.html#getting-started. For a demonstration of the tool you can take a look at the Going Native 2013 talk given by Chandler Carruth: The Care and Feeding of C++'s Dragons. clang-modernize is featured starting at ~33min.

Tuesday, November 5, 2013

The clang-cl /fallback mode

There has been a lot of work lately towards bringing an LLVM toolchain to the Windows platform (see A path forward for an LLVM toolchain on Windows). One result of that work is a new driver mode for Clang: clang-cl. This mode makes Clang understand the same kind of command-line arguments as Visual Studio's compiler, cl.exe. For example, a typical command to compile a file into an executable with Clang might be "clang hello.cc -o hello", whereas with cl.exe, one would use "cl.exe hello.cc /Fehello". Now one can use the latter syntax with Clang by substituting "cl.exe" with "clang-cl". This makes it easy to use Clang for existing projects in Visual Studio.

For the most part, clang-cl accepts exactly the same arguments as cl.exe. However, it also accepts some Clang-specific options. One such option that was added recently is the /fallback flag. The purpose of this flag is to make it easy to use clang-cl even in projects where Clang cannot yet compile all of the code. This post gives an example of how /fallback can be used.

Friday, October 11, 2013

New libunwind implementation in libc++abi

Today we are happy to announce that the LLVM family of C++ support libraries is more complete than ever. To enable clang++ to port to platforms that do not have a system unwinder, Apple has contributed their unwinder implementation to LLVM. It is now dual licensed under LLVM and MIT license which is the same license as the rest of LLVM’s runtime support libraries.

The new unwinder has been added to the libcxxabi.llvm.org project.  The unwinder has two levels of API.  The high level APIs are the _Unwind_* functions which the cxa_* exception functions in libcxxabi require.  The low level APIs are the unw_* functions which are an interface defined by the the old HP libunwind project (which shares no code with this libunwind).

Most architectures now use "zero cost" exceptions for C++.   The zero cost means there are no extra instructions executed if no exceptions are thrown.  But if an exception is thrown, the runtime must consult side tables and figure out how to restore registers and "unwind" from the current stack frame to the catch clause.  That ability to modify the stack frames and cause the thread to resume in a catch clause with all registers restored properly is the main purpose of libunwind.

The new unwinder sources are known to build on Darwin, but will need some #defines configured to build on other platforms (patches welcome!).   The low level libunwind API was designed to work either in-process (aka local) or to operate on another process (aka remote), but only the local path has been completed, as that is all that C++ exceptions require.

-Nick Kledzik

Tuesday, October 8, 2013

OpenMP* project

I am extremely glad to announce that Intel has decided to provide a copy of our Intel® open-source OpenMP* runtime as an LLVM sub-project (and the LLVM project has been kind enough to accept our contribution!). This gives the community a fully LLVM license compatible version of the OpenMP runtime for use in  OpenMP development projects. The complete source code is now available at openmp.llvm.org

We open-sourced the Intel OpenMP runtime code to support the development of a full LLVM-based implementation of the OpenMP specification. Intel’s compiler team in Moscow has made outstanding progress in implementing the Clang changes to support the OpenMP language extensions (you can see their work at http://clang-omp.github.io), and now we’ve reached a milestone where we can create an LLVM sub-project for some of the other components that are needed to build a complete OpenMP system.

Personally, I am very happy (and proud) to be associated with LLVM, and I look forward to a long and productive collaboration.

-- Jim Cownie

Monday, September 23, 2013

libc++ and C++1Y

I am proud to announce that as of September 21, libc++ has a full implementation of the C++1Y standard library.

In April of 2013, the C++ standards committee approved a draft version of the next revision of the C++ language, called "C++1Y". (C++11 was called "C++0x" while in development). We hope that this draft standard will be approved next year, and become C++14.

The draft standard includes changes to both the language and the standard library. These changes are designed to correct some mistakes that have been discovered in the C++11 standard and to add new features.

Both clang and libc++ are being enhanced to support this new standard, and they each have a web page to show their current progress: clang C++1Y Status and libc++ C++1Y status.

The C++ standards committee is meeting this week (September 23-28) to consider comments and defect reports in the C++1Y draft standard, and there will certainly be changes made.

We could have waited until the final version of the C++1Y standard was approved; that would certainly have been less work. The draft standard will to be revised in September (and probably in February), so the libc++ implementation will have to change to track the standard.

However, by implementing the draft standard has several advantages.
  • We gain experience in working with the new standard.
  • We found several places where the changes to the draft standard "exposed" other places were improvements can be made. For example, while implementing the new optional feature made it clear that interface of the the comparison functions plus, less (and others) could be enhanced. The result of that was the new paper n3749.
  • By implementating the standard, we can identify problems/inconsistencies in the draft standard. For example, when we implemented the new library feature dynarray, we discovered that the interfaces for the allocator-based constructors for the container were wrong; in particular there were cases where it was not possible for the compiler to determine the correct constructor to call. This was raised as LWG Issue 2255, and will have to be resolved before the standard becomes final.

Thursday, September 5, 2013

A path forward for an LLVM toolchain on Windows

Over the past several months, contributors from Google and elsewhere in the community have begun actively working on bringing an LLVM toolchain to Windows in a way that would support and enhance a fully native development experience. This toolchain works with Visual Studio and the Windows development process that Windows developers are already using, making LLVM tools and technologies available to them. We want to cross the streams (in the Ghostbuster proton-pack sense) of the Visual Studio IDE and the LLVM-based C++ tools to enable developers to build C++ software better.

To that end, we’ve been driving many of the efforts around compatibility with Visual Studio and native Windows C++ code in Clang and LLD (the LLVM linker). Today, as announced at my GoingNative 2013 talk, we are able to build a trivial C++ application that in turn links against native C++ libraries and uses them in real, interesting ways. This is a huge milestone for the project, and something we’re really excited to be talking more about. Lots of folks in the LLVM project came together to get the open source project here. Thanks to every single one of you.

Going forward, we would really like to see an increased level of involvement from Windows developers. We’re launching an alpha-build website as part of llvm.org where you can get fresh builds of Clang, various Clang-based-tools, LLD, and the rest of the LLVM toolchain for Windows on a regular basis. These installable packages should make it dramatically easier to test and experiment with this new set of tools in your development environment. Please feel free to try it out, file bugs, and even submit patches to help move the toolchain forward on this new platform. Keep in mind, we are following the tried-and-true open source release mantra of releasing early and frequently. These are alpha-quality releases intended at developers interested in helping us track down and understand bugs.

There is still a lot of exciting work left to make LLVM and the C++ toolchain built on top of LLVM fully support the Windows platform. Come join the fun, because this is one place where patches are truly welcome.

-Chandler Carruth

Wednesday, September 4, 2013

Clang Warnings

Clang has two types of diagnostics, errors and warnings.  Errors arise when the code does not conform to the language.  Such things as missing semi-colons and mismatched braces prevent compilation and will cause Clang to emit an error message.

On the other hand, warnings are emitted on questionable constructs on language conforming code.  Over time, certain patterns have been determined to have a strong likelihood of being a programming mistake.  Some examples of these include: order of operations confusion, mistaking similarly named language features, and easily made typos that still result in valid code.

Although warnings may have false positives, the utility of finding bugs early usually outweigh their downsides.  Keep reading for a demonstration of Clang's warnings, as well as a comparison to GCC's warnings.

Friday, August 2, 2013

Object Caching with the Kaleidoscope Example Program

In previous posts I described the process of porting the LLVM Kaleidoscope tutorial program to use MCJIT as its execution engine and introduced a lazy compilation implementation with the MCJIT engine.  The lazy implementation produced similar, and in some cases better, performance when compared with an implementation based on the older JIT execution engine, but it used more memory.

In this post, I’m going to extend the new implementation to use MCJIT’s object caching interface.  This will give our interpreter a way to store pre-compiled versions of previously used function and retrieve them for execution in later runs of the program.

Monday, July 29, 2013

Kaleidoscope Performance with MCJIT

In a previous post I described the process of porting the LLVM Kaleidoscope tutorial program to use MCJIT as its execution engine.  After navigating through a serious of road blocks we ended up with an implementation that was working as expected.

So it works, but the next question is, “Is it any good?”

A lot of people considering the transition from the older JIT execution engine to MCJIT have concerns about the possible performance implications, particularly related to the fact that MCJIT doesn’t support lazy compilation.  The older JIT engine will generate code for functions in an LLVM module one function at a time, delaying compilation of each function until it is about to be executed.  The MCJIT engine operates on entire modules, generating code for all functions in a module at once.  In the previous post, we modified the Kaleidoscope interpreter to create multiple modules as needed, but we’re still compiling the entire current module when a function is executed.

So what does that look like in terms of performance?

Monday, July 22, 2013

Using MCJIT with the Kaleidoscope Tutorial

You may have noticed that there are two different JIT execution engines in the LLVM project.  The older implementation (llvm::JIT) is a sort of ad hoc implementation that brings together various pieces of the LLVM code generation and adds its own glue to get dynamically generated code into memory one function at a time.  The newer implementation (llvm::MCJIT) is heavily based on the core MC library and emits complete object files into memory then prepares them for execution.

MCJIT has several advantages, including broader platform support and better tool integration.  However, because it is designed to compile entire modules into object images the MCJIT engine doesn’t directly support some key features of the older JIT implementation, such as lazy compilation.  By lazy compilation, I mean deferring compilation of individual functions until just before the function is going to be executed.

At this point you may find yourself saying, “Wait a minute?  Are you saying MCJIT doesn’t do ‘just-in-time’ compilation?!?”  Well…sort of.  It’s more of a dynamic code emitter than a true just-in-time compiler.  That said we’d like it to become a long term replacement for the old JIT so that we can reap the benefits of ongoing development in core MC code generation.

So the question becomes, can we make MCJIT do what the older JIT engine does?  The current answer is, “I hope so.”  As a means of exploring this question, I decided to try to convert the Kaleidoscope tutorial to use MCJIT.

Friday, June 28, 2013

LLDB 3.3 and beyond

The LLVM project debugger (LLDB) has seen a recent upswing of activity around the LLVM 3.3 release.  While the debugger has long been the default tool with Xcode, its potential beyond Darwin has had room to grow.  Especially within the last year, the development community has grown beyond its roots with OS/X and iOS to include substantial contributions for Linux, Windows, and FreeBSD. In addition, experimental packages are available for a growing number of distributions including Debian, Ubuntu and Arch.

Wednesday, June 19, 2013

LLVM 3.3 Released!

LLVM 3.3 is now available! For details, you can read the Release Notes.

LLVM 3.3 is a big release: it adds new targets for the AArch64 and AMD R600 GPU architectures, adds support for IBM's z/Architecture S390 systems, and major enhancements for the PowerPC backend (including support for PowerPC 2.04/2.05/2.06 instructions, and an integrated assembler) and MIPS targets.

Performance of code generated by LLVM 3.3 is substantially improved: the auto-vectorizer produces much better code in many cases and is on by default at -O3, a new SLP vectorizer is available, and many general improvements landed in this release. Independent evaluations show that LLVM 3.3's performance exceeds that of LLVM 3.2 and of its primary competition on many benchmarks.

3.3 is also a major milestone for the Clang frontend: it is now fully C++'11 feature complete. At this point, Clang is the only compiler to support the full C++'11 standard, including important C++'11 library features like std::regex. Clang now supports Unicode characters in identifiers, the Clang Static Analyzer supports several new checkers and can perform interprocedural analysis across C++ constructor/destructor boundaries, and Clang even has a nice "C++'11 Migrator" tool to help upgrade code to use C++'11 features and a "Clang Format" tool that plugs into vim and emacs (among others) to auto-format your code.

LLVM 3.3 is the result of an incredible number of people working together over the last six months, but this release would not be possible without our volunteer release team! Thanks to Bill Wendling for shepherding the release, and to Ben Pope, Dimitry Andric, Nikola Smiljanic, Renato Golin, Duncan Sands, Arnaud A. de Grandmaison, Sebastian Dre├čler, Sylvestre Ledru, Pawel Worach, Tom Stellard, Kevin Kim, and Erik Verbruggen for all of their contributions pulling the release together.

If you have questions or comments about this release, please contact the LLVMdev mailing list! Onward to LLVM 3.4!

Tuesday, May 28, 2013

LLVM 3.3 Vectorization Improvements

I would like to give a brief update regarding vectorization in LLVM. When LLVM 3.2 was released, it featured a new experimental loop vectorizer that was disabled by default. Since LLVM 3.2 was released, we have continued to work hard on improving vectorization, and we have some news to share. First, the loop vectorizer has new features and is now enabled by default on -O3. Second, we have a new SLP vectorizer. And finally, we have new clang command line flags to control the vectorizers.

Monday, May 6, 2013

EuroLLVM 2013, Paris, France

Two days after the end of EuroLLVM 2013, I finally got the energy to write a piece about it. It was a lot of hard work by an amazing team of volunteer organizers lead by Tobias Grosser, Duncan Sands, Sylvestre Ledru and Arnaud de Grandmaison, plus the usual suspects of the previous events, and in the end there was very little that had gone wrong, even in the slightest.

This was our biggest event yet, with 187 attendees, 12 talks, 2 tutorials, 7 lightning talks and 10 posters! The posters, slides and videos are available on the EuroLLVM 2013 website, as well as some idea on the abstracts, location (ENS, in Paris) and the great dinner cruise on Monday.

You can also find on the site the results of our questionnaire, distilled and anonymized.

Sunday, April 21, 2013

Clang support for C++11 and beyond

As of r179861, Clang implements the entirety of the C++11 language standard. The following features have been implemented since the release of Clang 3.2, along with our plans for "C++1y".

Thursday, April 18, 2013

Euro LLVM 2013 in Paris

In two weeks, the 2013 edition of the Euro LLVM conference will start in historic center of Paris, France.  The schedule has just been published.  Most of different aspects of the LLVM infrastructure will present at the event.  See below for the full details!

Monday, April 15, 2013

Status of the C++11 Migrator

Since the design document for cpp11-migrate, the C++11 migrator tool, was first proposed in early December 2012 development has been making steady progress. In this article I'll talk about what's been implemented in cpp11-migrate so far, what's coming up, and how you can get involved.

The purpose of the C++11 Migrator is to do source-to-source translation to migrate existing C++ code to use C++11 features to enhance maintainability, readability, runtime performance, and compile-time performance. Development is still early and transforms fall mostly into the first two categories. The migrator is based on Clang's LibTooling and the AST Matching library.

Most of the development so far has been carried out by a small core group at Intel. Our focus so far has been to set up project infrastructure and testing, implement a few basic transforms, and make sure those transforms work well. Our aim is to make this tool useful to the community so we're always listening for transform ideas and feedback.

Static analysis tools: using Clang in CppDepend

Static analysis is a method of computer program debugging that is done by examining the code without executing the program. The process provides an understanding of the code structure, can help to ensure that the code adheres to industry standards, and can find bugs not easy to detect.

To develop a C / C++ static analysis tool, a parser is needed to parse the source code. C++ is a very powerful language but its syntax is a little bit complicated, what makes the parser not easy to develop.

When we began the development of CppDepend about four years ago we needed a reliable C / C++ parser.  At that time, Clang was an option but was not widely used and we didn’t know if it would ultimately develop into a fully-featured compiler frontend.

Last year, for the major release of CppDepend 3.0, we re-evaluated our C / C++ parser with a goal of getting more reliable results.  We checked Clang to see where its evolution went and were very surprised that it now implements virtually all C++'11 features and became very popular.  Clang now provide solid infrastructure to write tools that need syntactic and semantic information about a program.

Wednesday, April 10, 2013

LLVM Recipient of the 2012 ACM System Software Award

The ACM just announced that the LLVM project is the recipient of the 2012 ACM System Software Award. This award recognizes a "software system that has had a lasting influence, reflected in contributions to concepts, in commercial acceptance, or both." Many important software systems are previous recipients of this award, including Eclipse, VMware, Eiffel, Make, Java, Apache, TCP/IP, PostScript, SMALLTALK, TeX, and UNIX (among others).

This is fantastic recognition for the impact LLVM has had on the compiler and languages industry, and is recognition that all LLVM Developers should feel proud of.

Wednesday, April 3, 2013

LLVM Debian/Ubuntu nightly packages

In order to facilitate testing and to improve the deployment of the LLVM toolchain, we are happy to publish LLVM Debian/Ubuntu nightly packages. Read on for information about how it works and what we're building.

Monday, April 1, 2013

Testing libc++ with -fsanitize=undefined

[This article is re-posted in a slightly expanded form from Marshall's blog]

After my last article, Testing libc++ with Address Sanitizer, I thought "what other tests can I run?"

Address Sanitizer (ASan) is not the only "sanitizer" that clang offers. There are "Thread Sanitizer" (TSan), "Undefined Behavior Sanitizer" (UBSan), and others. There's an integer overflow sanitizer which is called IOC coming in the 3.3 release of clang. The documentation for UBSan can be found on the LLVM site.

I have been looking at the results of running the libc++ test suite with UBSan enabled. Even if you're not interested in libc++ specifically, this post can be a useful introduction to useful Clang bug detectors, and shows several classes of problems they can find.

Thursday, March 28, 2013

Testing libc++ with Address Sanitizer

[This article is re-posted in a slightly expanded form from Marshall's blog]
I've been running the libc++ tests off and on for a while. It's a quite extensive test suite, but I wondered if there were any bugs that the test suite was not uncovering. In the upcoming clang 3.3, there is a new feature named Address Sanitizer which inserts a bunch of runtime checks into your executable to see if there are any "out of bounds" reads and writes to memory.
In the back of my head, I've always thought that it would be nice to be able to say that libc++ was "ASan clean" (i.e, passed all of the test suite when running with Address Sanitizer).
So I decided to do that.

Wednesday, March 20, 2013

Instruction Relationship Framework in LLVM

The article provides an overview of the new Relationship framework of TableGen. This TableGen feature is used to describe user defined relationships between instructions. It was added to LLVM in October 2012.