LLVM Project Blog

LLVM Project News and Details from the Trenches

Monday, January 26, 2015

LLVM Weekly - #56, Jan 26th 2015

Welcome to the fifty-sixth issue of LLVM Weekly, a weekly newsletter (published every Monday) covering developments in LLVM, Clang, and related projects. LLVM Weekly is brought to you by Alex Bradbury. Subscribe to future issues at http://llvmweekly.org and pass it on to anyone else you think may be interested. Please send any tips or feedback to asb@asbradbury.org, or or @llvmweekly or @asbradbury on Twitter.

I'll be talking the lowRISC project to produce a fully open-source SoC at FOSDEM this coming weekend. Do come and see my main track talk and read my speaker interview for more background. There is of course an LLVM toolchain devroom on the Sunday.

The canonical home for this issue can be found here at llvmweekly.org.

Tuesday, January 20, 2015

LLDB is Coming to Windows

We've spoken in the past about teaching Clang to fully support Windows and be compatible with MSVC.  Until now, a big missing piece in this story has been debugging the clang-generated executables.  Over the past 6 months, we've started working on making LLDB work well on Windows and support debugging both regular Windows programs and those produced by Clang.

Why not use an existing debugger such as GDB, Visual Studio's, or WinDBG?  There are a lot of factors in making this kind of decision.  For example, while GDB understands the DWARF debug information produced by Clang on Windows, it doesn't understand the Microsoft C++ ABI or debug information format.  On the other hand, neither Visual Studio nor WinDBG understand the DWARF debug information produced by Clang.  With LLDB, we can teach it to support both of these formats, making it usable with a wider range of programs.  There are also other reasons why we're really excited to work on LLDB for Windows, such as the tight integration with Clang which lets it support all of the same C++ features in its expression parser that Clang supports in your source code.  We're also looking to continue adding new functionality to the debugging experience going forward, and having an open source debugger that is part of the larger LLVM project makes this really easy.

The past few months have been spent porting LLDB's core codebase to Windows.  We've been fixing POSIX assumptions, enhancing the OS abstraction layer, and removing platform specific system calls from generic code.  Sometimes we have needed to take on significant refactorings to build abstractions where they are necessary to support platform specific differences.  We have also worked to port the test infrastructure to Windows and set up build bots to ensure things stay green.

This preliminary bootstraping work is mostly complete, and you can use LLDB to debug simple executables generated with Clang on Windows today.  Note the use of the word "simple".  At last check, approximately 50% of LLDB's tests fail on Windows.  Our baseline, however, which is a single 32-bit executable (i.e. no shared libraries), single-threaded application built and linked with Clang and LLD using DWARF debug information, works today.  We've tested all of the fundamental functionality such as:
  1. Various methods of setting breakpoints (address, source file+line, symbol name, etc)
  2. Stopping at and continuing from breakpoints
  3. Process inspection while stopped, such as stack unwinding, frame setting, memory examination, local variables, expression evaluation, stepping, etc  (one notable exception to this is that step-over doesn't yet work well in the presence of limited symbol information).
Of course, there is still more to be done.  Here are some of the areas we're planning to work on next:
  1. Fixing low hanging fruit by improving the pass-rate of the test suite.
  2. Better support for debugging multi-threaded applications.
  3. Support for debugging crash dumps.
  4. Support for debugging x64 binaries.
  5. Enabling stepping through shared libraries.
  6. Understanding PDB (for debugging system libraries, and executables generated with MSVC).  Although the exact format of PDB is undocumented, Microsoft still provides a rich API for querying PDB in the form of the DIA SDK.
  7. Adding debugging commands familiar to users of WinDBG (e.g. !handle, !peb, etc)
  8. Remote debugging
  9. Symbol server support
  10. Visual Studio integration
If you're using Clang on Windows, we would encourage you to build LLDB (it should be in the Windows LLVM installer soon) and let us know your thoughts by posting them to lldb-dev.  Make sure you file bugs against LLDB if you notice anything wrong, and we would love for you to dive into the code and help out.  If you see something wrong, dig in and try to fix it, and post your patch to lldb-commits.

Monday, January 19, 2015

LLVM Weekly - #55, Jan 19th 2015

Welcome to the fifty-fifth issue of LLVM Weekly, a weekly newsletter (published every Monday) covering developments in LLVM, Clang, and related projects. LLVM Weekly is brought to you by Alex Bradbury. Subscribe to future issues at http://llvmweekly.org and pass it on to anyone else you think may be interested. Please send any tips or feedback to asb@asbradbury.org, or @llvmweekly or @asbradbury on Twitter.

It seems to have been a very busy week in the world of LLVM, particularly with regards to discussion on the mailing list. Due to travel etc and the volume of traffic, I haven't been able to do much summarisation of mailing list discussion I'm afraid.

The canonical home for this issue can be found here at llvmweekly.org.

Monday, January 12, 2015

LLVM Weekly - #54, Jan 12th 2015

Welcome to the fifty-fourth issue of LLVM Weekly, a weekly newsletter (published every Monday) covering developments in LLVM, Clang, and related projects. LLVM Weekly is brought to you by Alex Bradbury. Subscribe to future issues at http://llvmweekly.org and pass it on to anyone else you think may be interested. Please send any tips or feedback to asb@asbradbury.org, or @llvmweekly or @asbradbury on Twitter.

As you receive this week's issue, I should be on my way to California where I'll be presenting lowRISC at the RISC-V workshop in Monterey and having a few mother meetings. I'm in SF Fri-Sun and somewhat free on the Saturday if anyone wants to meet and chat LLVM or lowRISC/RISC-V.

The canonical home for this issue can be found here at llvmweekly.org.

Monday, January 5, 2015

Using clang for Chrome production builds on Linux

Chrome 38 was released early October 2014. It is the first release where the Linux binaries shipped to users are built by clang. Previously, this was done by gcc 4.6. As you can read in the announcement email, the switch happened without many issues. Performance stayed roughly the same, binary size decreased by about 8%. In this post I'd like to discuss the motivation for this switch.

Motivation

There are two reasons for the switch.

1. Many Chromium developers already used clang on Linux. We've supported opting in to clang for since before clang supported C++ – because of this, we have a process in place for shipping new clang binaries to all developers and bots every few weeks. Because of clang's good diagnostics (some of which we added due to bugs in Chromium we thought the compiler should catch), speed, and because of our Chromium-specific clang plugin, many Chromium developers switched to clang over the years. Making clang the default compiler removes a stumbling block for people new to the project.

2. We want to use modern C++ features in Chromium. This requires a recent toolchain – we figured we needed at least gcc 4.8. For Chrome for Android and Chrome for Chrome OS, we updated our gcc compilers to 4.8 (and then 4.9) – easy since these ports use a non-system gcc already. Chrome for Mac has been using Chromium's clang since Chrome 15 and was already in a good state. Chrome for iOS uses Xcode 5's clang, which is also new enough. Chrome for Windows uses Visual Studio 2013 Update 4. On Linux, switching to clang was the easiest way forward.

Keeping up with C++'s evolution in a large, multi-platform project

C++ had been static for many years. C++11 is the first real update to the C++ language since the original C++ standard (approved on July 27 1998). C++98 predated the founding of Google, YouTube, Facebook, Twitter, the releases of Mac OS X and Windows XP, and x86 SSE instructions. The time between the two standards saw the rise and fall of the iPod, several waves of social networks, and the smartphone explosion.

The time between C++11 and C++14 was three years, and the next major iteration of the language is speculated to be finished in 2017, three years from C++14. This is a dramatic change, and it has repercussions on how to build and ship C++ programs. It took us 3+ years to get to a state where we can use C++11 in Chromium; C++14 will hopefully take us less long. (If you're targeting fewer platforms, you'll have an easier time.)

There are two parts to C++11: New language features, and new library features. The language features just require a modern compiler at build time on build machines, the library features need a new standard library at runtime on the user's machine.

Deploying a new compiler is conceptually relatively simple. If your developers are on Ubuntu LTS releases and you make them use the newest LTS release, they get new compilers every two years – so just using the default system compiler means you're up to two years behind. There needs to be some process to relatively transparently deploy new toolchains to your developers – an "evergreen compiler". We now have this in place for Chromium – on Linux, by using clang. (We still accept patches to keep Chromium buildable with gccs >= 4.8 for people who prefer compiling locally over using precompiled binaries, and we still use gcc as the target compiler for Chrome for Android and Chrome OS.)

The library situation is slightly more tricky: On Linux and Mac OS X, programs are usually linked against the system C++ library. Chrome wants to support Mac OS X 10.6 a bit longer (our users seem to love this OS X release), and the only C++ library this ships with is libstdc++ 4.2 – which doesn't have any C++11 bits. Similarly, Ubuntu Precise only has libstdc++ 4.6. It seems that with C++ updating more often, products will have to either stop supporting older OS versions (even if they still have many users on these old versions), adopt new C++ features very slowly, or ship with a bundled C++ standard library. The latter implies that system libraries shouldn't have a C++ interface for ABI reasons – luckily, this is mostly already the case.

To make things slightly more complicated, gcc and libstdc++ expect to be updated at the same time. gcc 4.8 links to libstdc++ 4.8, so upgrading gcc 4.8 while still linking to Precise's libstdc++ 4.6 isn't easy. clang explicitly supports building with older libstdc++ versions.

For Chromium, we opted to enable C++11 language features now, and then allow C++11 library features later once we have figured out the story there. This allows us to incrementally adopt C++11 features in Chromium, but it's not without risks: vector<int> v0{42} for example means something different with an old C++ library and a new C++ library that has a vector constructor taking an initializer_list. We disallow using uniform initialization for now because of this.

Since bundling a C++ library seems to become more common with this new C++ update cadence, it would be nice if compiler drivers helped with this. Just statically linking libstdc++ / libc++ isn't enough if you're shipping a product consisting of several executables or shared libraries – they need to dynamically link to a shared C++ library with the right rpaths, the C++ library probably needs mangled symbol names that don't conflict with the system C++ library which might be loaded into the same process due to other system libraries using it internally (for example, maybe using an inline namespace with an application-specific name), etc.

Future directions

As mentioned above, we're trying to figure out the C++ library situation. The tricky cases are Chrome for Android (which currently uses STLport) and Chrome for Mac. We're hoping to switch Chrome for Android to libc++ (while still using gcc as compiler). On Mac, we'll likely bundle libc++ with Chrome too.

We're working on making clang usable for compiling Chrome for Windows. The main motivations for this are using AddressSanitizer, providing a compiler with great diagnostics for developers, and getting our tooling infrastructure working on Windows (used for example automated large-scale cross-OS refactoring and for building our code search index – try clicking a few class names; at the moment only code built on Linux is hyperlinked). We won't use clang as a production compiler on Windows unless it produces a chrome binary that's competitive with Visual Studio's on both binary size and performance. (From an open-source perspective, it is nice being able to use an open-source compiler to compile an open-source program.)

You can reach us at clang@chromium.org

LLVM Weekly - #53, Jan 5th 2015

Welcome to the fifty-third issue of LLVM Weekly, a weekly newsletter (published every Monday) covering developments in LLVM, Clang, and related projects. LLVM Weekly is brought to you by Alex Bradbury. Subscribe to future issues at http://llvmweekly.org and pass it on to anyone else you think may be interested. Please send any tips or feedback to asb@asbradbury.org, or @llvmweekly or @asbradbury on Twitter.

I'm going to be in California next week for the RISC-V workshop. I'm arriving at SFO on Monday 12th and leaving on Sunday the 18th. Do let me know if you want to meet and talk lowRISC/RISC-V or LLVM, and we'll see what we can do.

The canonical home for this issue can be found here at llvmweekly.org.

Monday, December 29, 2014

LLVM Weekly - #52, Dec 29th 2014

Welcome to the fifty-second issue of LLVM Weekly, a weekly newsletter (published every Monday) covering developments in LLVM, Clang, and related projects. LLVM Weekly is brought to you by Alex Bradbury. Subscribe to future issues at http://llvmweekly.org and pass it on to anyone else you think may be interested. Please send any tips or feedback to asb@asbradbury.org, or @llvmweekly or @asbradbury on Twitter.

This issue marks the end of one full year of LLVM Weekly. It's a little shorter than usual as the frenetic pace of LLVM/Clang development has slowed over the holiday period. Surprising even to me is that we managed to make it full all 52 weeks with an issue every Monday as promised. This requires a non-trivial amount of time each week (2-3+ hours), but I am intending to keep it going into 2015. I'd like to give a big thank you to everyone who's said hi at a conference, sent in corrections or tips on content, or just sent a random thank you. It's been very helpful in motivation. I don't currently intend to change anything about the structure or content of each issue for next year, but if you have any ideas then please let me know.

I can't make it to 31C3 due to the awkward timing of the event, but do let me know if there are any LLVM/Clang related talks worth sharing. There was a talk about Code Pointer Integrity which has previously been covered in LLVM Weekly and is working towards upstreaming. The video is here. If you're interested in lowRISC and at 31C3, Bunnie is leading a discussion about it at 2pm on Monday (today).

The canonical home for this issue can be found here at llvmweekly.org.