Right now we are clearly in a distributed trend. Data is increasingly stored in the cloud and accessed from a variety of devices. People shift from phones to laptops to desktops and expect to see the same list of contacts, the same e-mail, the same calendar. The cloud sites are an extreme version of centralization: millions of users store their data in the same place. What will the computing world look like when and if it oscillates back to a more distributed system?

One possibility is that people will increasingly acquire their own data storage which will be accessible over the net. They’ll keep small cheap redundant servers to hold their data. They’ll have one server at home and one at the office, and they will automatically sync up. Access will be very fast most of the time, and will be possible at over times. The servers will be updated automatically and so forth, and they will (somehow) be easy to administer. The advantage will be fast access to data most of the time and actual control over your data. If you want to delete something, it’s gone, and not available for resurrection.

That particular vision is easy for me to think of because it’s similar to our ideas when I co-founded a company, Zembu, back in 1998. I don’t know how compelling it is. I suspect that going back to a distributed environment will require some cost advantage, and I’m not sure I see that here. Much of cloud computing these days tends to be free, in the sense that advertising pays the bills. Few people will spend money to avoid ads. Few is more than zero, but it’s not enough to build a business on.

During any predominant paradigm it’s difficult to see what the next paradigm will be. History suggests that we will oscillate back, that the cloud will condense at some point. But history is not always right. It seems inherently unlikely to me that data will increasingly be centralized. But I don’t know what the alternative will look like.

Unfortunately, I’ve started to notice that as programs get smarter and as interface designers get more clever, machines are becoming more like people. Interfaces for web sites and phones are increasingly adjusting based on your past interactions. In many ways this is good, as over time the interaction gets smoother and easier. However, it means that there is a lack of consistency: an input today does not produce the same effect as the same input did yesterday. It also means that there is an increase in ambiguity: it’s difficult to tell the difference between working correctly and being slightly broken.

In effect, the computing world is becoming increasingly tuned for people who prefer dealing with people rather than people who prefer dealing with machines. On average this is of course a good thing, as most of the population seems to find it frustrating to deal with machines. But it’s somewhat ironic considering that the programmers doing most of the work tend to be people who prefer dealing with machines.

I don’t want to give up the advantages I get when things go well, so I guess I’m stuck in an increasingly inconsistent and ambiguous world.

I just completed the implementation of this in gccgo. It turned out to be fairly complex, so I’m writing some notes here on how it works.

The language requires that panic runs the deferred functions before unwinding the stack. This means that if the deferred function calls runtime.Callers (which doesn’t work in gccgo, but never mind, it will eventually) it gets a full backtrace of where the call to panic occurred. If the language did not work that way, it would be difficult to use recover as a general error handling mechanism, because there would be no good way to dump a stack trace. Building up a stack trace through each deferred function call would be inefficient.

The language also requires that recover only return a value when it is called directly from a function run by a defer statement. Otherwise it would be difficult for a deferred function to call a function which uses panic and recover for error handling; the recover might pick up the panic for its caller, which would be confusing.

As a general gccgo principle I wanted to avoid requiring new gcc backend features. That raised some difficulty in implementing these Go language requirements. How can the recover function know whether it is being invoked directly by a function started by defer? In 6g, walking up the stack is efficient. The panic function can record its stack position, and the recover function can verify that it is at the correct distance below. In gccgo, there is no mechanism for reliably walking up the stack other than exception stack unwinding, which does not provide a helpful API. Even if it did, gccgo’s split stack code can introduce random additional stack frames which are painful to account for. And there is no good way for panic to mark the stack in gccgo.

What I did instead was have the defer statement check whether the function is it deferring might call recover (e.g., it definitely calls recover, or it is a function pointer so we don’t know). In that case, the defer statement arranges to have the deferred thunk record the return address of the deferred function at the top of the defer stack. This value is obtained via gcc’s address-of-label extension, so no new feature was required. This gives us a value which a function which calls recover can check, because a function can always reliably determine its own return address via gcc’s __builtin_return_address function.

However, if the stack is split, then __builtin_return_address will return the address of the stack splitting cleanup code rather than the real caller. To avoid that problem, a function which calls recover is split into two parts. The first part is a small thunk which is marked to not permit its stack to be split. This thunk gets its return address and checks whether it is being invoked directly from defer. It passes this as a new boolean parameter to the real function, which does permit a split stack. The real function checks the new parameter before calling recover; if it is false, it just produces a nil rather than calling recover. The real function is marked uninlinable, to ensure that it is not inlined into its only call site, which could blow out the stack.

That is sufficient to let us know whether recover should return a panic value if there is one, at the cost of having an extra thunk for every function which calls recover. Now we can look at the panic function. It walks up the defer stack, calling functions as it goes. When a function sucessfully calls recover, the panic stack is marked. This stops the calls to the deferred functions, and starts a stack unwind phase. The stack unwinding is done exactly the way that g++ handles exceptions. The g++ exception mechanism is general and cross-language, so this part was relatively easy. This means that every function that calls recover has an exception handler. The exception handlers are all the same: if this is the function in which recover returned a value, then simply return from the current function, effectively stopping the stack unwind. If this is not the function in which recover returned a value, then resume the stack unwinding, just as though the exception were rethrown in C++.

This system is somewhat baroque but it appears to be working. Everything is reasonably efficient except for a call to recover which does not return nil; that is as expensive as a C++ exception. Perhaps I will think of ways to simplify it over time.

There has been a long effort to prepare for this, by moving GCC’s code base from C to the common subset of C and C++. While people naturally think of C++ as an extension to C, they are different languages and there is a lot of C code which is not valid C++. In the GCC code base, one of the biggest issues was that enums are more restricted by the type system in C++ than they are in C. Another was that in C++ you may not use the same name as a typedef and a struct tag, except for the special case of making the struct tag be a typedef for the struct itself.

Gabriel Dos Reis did the first substantial work on moving the GCC code base to the common subset, and many other people contributed. I think it’s fair to say that I did the lion’s share of the work, starting with my surprise presentation on the advantages of C++ at the 2008 GCC Summit. I did a lot of work to improve the -Wc++-compat warning option to warn about C code which was not in the C/C++ common subset, and I did a lot of work to make GCC code compile with that option without warnings.

C++ will not magically make the GCC code base better. However, I believe that it will give us some useful tools to incrementally improve the code base over time, making it easier to read, easier to modify, and more efficient. I say this not based on theory, but on my experiences with gold and with the gccgo frontend. I’ve already started writing some draft C++ coding conventions which I hope we can use to guide our efforts.

On the other hand, the GCC project has some problems. The major individual contributors to GCC are hired to work on it. That means that they have a lot of time and resources to use to improve the compiler, which is good. However, it also has some negative effects. It’s difficult for new volunteers to join the community. It’s hard for them to learn the code base and it’s hard for them to keep up with the pace of change. It’s also hard for them to learn the conventions of how the project works, and they get little help in getting their patches in. Also, the people who work on GCC have learned the intricacies of the code base over time. They do not rely on the internal documentation. The effect is that the internal documentation for some parts of the code base is quite weak, and none of the main contributors are motivated to fix it.

Another, separate, problem is that there is no single person or group with a clear ability and willingness to decide on the direction of the project. In the past the direction has been set at different times by people like Richard Stallman, Richard Kenner, Jeff Law, and Richard Henderson. None of them are playing that role today. The effect is that nobody can see whether significant new features should or should not go into the project, which leads to a tendency for inconclusive discussions and unreviewed patches. People hoping to contribute are left with no clear path forward. (I should mention that groups like the GCC Steering Committee and the GCC Release Managers are effective but do not take on this role, which is more that of an architect.)

A third problem is that GCC has no public relations activity. The project web page tells you what GCC is but says nothing about how it compares to other compilers or how it has improved over time. There are some common criticisms of GCC, such as the belief that it is measurably worse than proprietary compilers, or that it is stagnating, which the project makes no attempt to discuss or dispute.

None of these issues are critical. As I said, GCC is highly successful. But they are areas where I think GCC could improve. Unfortunately, pointing out these issues is insufficient; it’s necessary for peole to step up to take on these roles. The various companies which pay people to work on GCC are generally less interested in these aspects of the project, which makes it that much harder to find people to work on them.

These ideas are significantly different from destructors, which are associated with a type, and are executed when an object of that type goes out of lexical scope or is explicitly deleted. Destructors are primarily used to release resources acquired by an object of the type. This is a less important concept in a garbage collected language like Go.

The absence of destructors means that Go does not support the RAII pattern, in which an object is used to acquire a mutex or some other resource for the scope of a lexical block. Implementing this in Go requires two statements: one to acquire the mutex, and a defer statement to release the mutex on function exit. Because deferred functions are run on function exit, the mapping is not exact; you can not use this technique to acquire a lock in a loop. In fact, acquiring a mutex in a loop and correctly releasing it when a panic occurs is rather difficult in Go; fortunately it is easy to handle correctly by moving the body of the loop to a separate function. In any case, Go discourages this type of programming. Mutexes are available in Go, but channels are the preferred mechanism for synchronization.

Are defer statements and finalizers sufficient replacement for destructors in a garbage collected language? They are for me. When I write C++ my destructors are almost entirely concerned with releasing memory. In fact, in the gold linker I often deliberately omitted destructors, because many of the data structures live for the life the program; in such a case, destructors serve only to slow down program exit. I would be interested to hear of a pattern of programming which relies on destructors for cases other than releasing memory or RAII.

I’ve recently been adding Go support to SWIG. SWIG can be used to build interfaces between various different languages and C++. SWIG is a nice program in many ways: it provides a lot of power and flexibility in defining the interface. Internally, though, big chunks are written in C++ but it uses a totally different container model. I assume this is for historical reasons dating back to a beginning in C or possibly even Python. In C++, though, it’s a mess. There is no type checking: it’s all void * pointers. Practically every type (strings, lists, hash tables) converts to everything else implicitly. It brings all the weaknesses of a pure dynamically typed language into C++, without providing any of the benefits.

Sometimes one encounters a programmer who carries a container model from program to program, rewriting it into different languages as needed. The result is a program written for one person. Don’t do that. Use a language in the idioms of the language; don’t try to make it look like a different language. Use the containers which the language provides.

Object oriented programming had a few different ideas, all of which centered around the notion of separating data into relatively small and independent units. Methods are defined independently of specific data, and a given method may be applied to different data units. Data units are grouped by sets of methods, and the method sets can be put into inheritance and abstraction hierarchies.

Object oriented approaches were greatly oversold, and it became clear over time that while they did have many good ideas, they were not the solution to the problems of programming. One of the problems of the object oriented approach is that it encourages you to focus on the relationship between sets of methods that apply to data, rather than focusing on the structure of the data itself. In C++ or Java terms, it’s easy to spend more time thinking about the inheritance relationships between classes than about the individual objects.

Over time, C++ shifted toward a somewhat different style of programming found in the Standard Template Library. This style is based on templates, and focuses on containers and algorithms. I’ve seen people advocate this approach to the point of saying that a good program should have no non-abstract loops: that everything should be done via an appropriate algorithm.

The issue I find with these approaches in practice is that they tend to lead to too much abstraction. Fundamentally all programs deal with data. The goal of a program is to manipulate the data in some way. The goal of abstraction and class inheritance is to permit you to write less code. The goal of data encapsulation is to make your program more maintainable over time. Abstraction beyond those points may make your program better by some artificial index, but it is not really helping you get things done.

If you find yourself asking yourself whether to introduce a new level in your class hierarchy, or you find yourself writing a new template which will only be instantiated once, then you may be falling victim to the abstraction penalty. Shake it off and return to focusing on what your program really needs to do.

The success of Facebook belies that lesson. It seems clear that many people conduct a large portion of their Internet life entirely within Facebook. Facebook provides various different forms of communication—e-mail, chat, photo sharing, etc.&mdashbut it all exists only within Facebook. Sure, you can point offsite, but it’s hardly the main form of communication. Facebook is a newer and larger version of the walled gardens of yesterday.

I don’t myself find Facebook to be interesting or useful. I do have an account, but I only go to the Facebook site when I get some message from it. Typically it’s because somebody wrote something on my “wall;” why they didn’t just send me an e-mail message, I don’t know.

Widespread use of Facebook does not necessarily translate to revenue, but all the rumors these days are that they are making plenty of money. The introduction of games which people are willing to pay small amounts of money for was a brilliant idea in this regard—at least, I assume that Facebook gets a small cut.

So my lesson about walled gardens was wrong. Walled gardens works fine if they are compelling enough. That would be fine if Facebook takes its position as a platform seriously enough, if they work to provide neutral access to everybody. So far they do. Their history of trying different approaches is not too encouraging in this regard, but the economics support it. I just wish that I liked the site more.

What I think most about was Blake Stowell’s question to me as I was leaving. Blake Stowell was at the time SCO’s Director of Public Relations. He asked what I would do if I owned some proprietary code that somebody else had copied, implying that SCO’s behaviour was not merely legally justified but was even morally justified. It was a long time ago, but the impression that I recall was that he sincerely thought that SCO was doing something which reasonable people would consider to be OK, and wanted to see whether I agreed. My answer at the time was not very good.

It’s a question which I now think brings us to the heart of copyright laws. If I write something myself, what rights do I have to prevent other people from making derivative works? If I buy the rights to something that somebody else wrote, do I have the same rights with regard to derivative works? Does it matter how those derivative works are being used? I’m raising these rhetorical questions not as a matter of law—the law is what it is—but as a matter of what we, as a society, ideally want to permit. SCO was acting as the copyright equivalent of a patent troll: they acquired the rights to something which they did not create, and attempted to gain revenue from other people using the same ideas. Should we permit that?

In considering issues like this, it’s very important to not mix up copyright with real property. It’s natural to start thinking that something that I write is like something that I own. If I own a car, it’s not OK for somebody to drive it without my permission. The issues with code are far less clear. Copyright is a balance between the rights of the authors and the rights of everybody else. Copyright does not last forever, unlike my ownership of the car. There are various exceptions to copyright, such as fair use. Even if we take SCO’s very best case, they were talking about a tiny percentage of code being copied from an earlier version of Unix into the Linux kernel. Did that give them the right to charge people for using the Linux kernel? If the code was removed from the Linux kernel, as did in fact happen later, would they still have the right to charge based on an expansive notion of derived work?

Intuitively I think that while the original author has considerable rights to control code that she or he writes, those rights tend to decrease with time and distance. It’s not obvious to me that control over an author’s work is something that can be sold or inherited. It also makes a difference whether the work is used in its entirety or whether a portion of the work is excerpted. It also makes a difference whether the work is used by itself or in combination with work by other authors.

Unfortunately these issues are all fuzzy. For law to be useful, the issues have to be spelled out, which is hard, and tends to give too much weight to the author at the expense of the rest of society. And the rules appropriate for books and music may not be appropriate for code.

A sane copyright law has to make it clear that legal assertions like the ones that SCO made claiming to own Linux are unsupportable. The remedy for minor code copying is to remove the copied code; it is not to grant ownership rights to the larger package into which the code was copied.

SCO’s actions were not justified. The fact that they appear to be failing is a triumph of justice.