Airs - Ian Lance Taylor

It is the common wisdom these days that future processors will have increasingly many cores but will not run at significantly higher clock rates. The constant drive for increased performance will come by letting the processor do more at one time, rather than by making it run faster. Along the same lines is the increased use of hyperthreading, in which a single processor has multiple threads of control, thus permitting one thread of control to absorb the memory latency of another. I see the problems with this line of development.

The first is that the multiple cores have to share a lot of ancillary hardware. That is fine for a few cores, but as you get more the level of contention increases. When they all share a memory bus, you don’t have to get too high before contention gets unbearable. The natural step is to partition memory in some way. Eventually you have to partition everything. Then you have several different computers on a chip–they will have very fast interconnects, but the interconnects will require programming, they won’t be shared memory. I don’t see how to avoid this endpoint. I’m sure a lot of people are thinking about it, but I don’t know what they’ve concluded.

That endpoint leads into the second problem, which is that it’s difficult to program these machines. It’s not like programming a traditional single computer. It’s like programming a set of computers with a fast communication channel. Programming a set of computers is fundamentally different from programming a single computer. But if this future is realized, the only way to continue to get performance gains is to program a set of computers.

In other words, we’re coming to the parallel programming model, only we’re doing it by the back door. We’ll be doing parallel programming on a single computer.

Researchers have been looking at parallel programming for a long time. It was what I studied in my one year in graduate school, back in 1986. In all that time, I think they’ve come up with one really significant result: people find parallel programming difficult.

Although our brains are inherently parallel, our thought processes are single-threaded. We find it difficult to think about several things at once. Cordwainer Smith has a nice short story about a person who is able to think on three different levels, but certainly one of the lessons is that it is very difficult indeed. The effort of parallel programming is to figure out how to write the program without having to think of several things at once.

There have been various attempts to let people write programs sequentially, but to extract the available parallelism. These attempts have generally not succeeded, because the available parallelism is too small to make a significant different in performance.

It will be interesting to see what happens when computer processors hit the limits of people’s programming ability in this way. Perhaps we will work out new ways to write code.

Blog Action Day is asking blog authors to write about the environment. Since I’m a fan of communal action, it seems appropriate for me to participate, even though I have no idea what good it might do.

This is hardly an original observation, but surely the strangest thing about the environment today is that it has become a partisan political issue. It’s not like the political parties fall naturally on either side of the environmental position (as they do in Kim Stanley Robinson’s “The Memory of Whiteness” which had Red Mars and Green Mars political parties, later reprised in his Martian trilogy). For some reason Democrats have become associated with environmental support, and Republicans have become associated with its absence. This makes no sense.

Only a fool would argue that environmental degradation can not occur, when we so many historical examples. While I suppose it’s possible to argue in a principled manner that environmental degradation does not matter, I hold no brief for that position. Our children deserve to live in a world at least as good as ours, and that means one with a livable environment. In any case, few people actually make that argument, except perhaps the Christian millenialists (as in James Watt’s comment “I do not know how many future generations we can count of before the Lord returns.”)

One can of course validly discuss whether environmental degradation is happening today, and one can validly discuss what to do about it. I think the first question has been long settled. The only people who still claim that the Earth is not warming up, or that human activity has nothing to do with it, simply aren’t paying attention.

What to do about it is much less clear. But there is one argument which absolutely does not hold water: the claim that it would be too expensive to do anything about it. That is a complete misunderstanding of how a modern economy works. Money spent on improving the environment is not money buried in holes; it is money spent on productive activity, money which employs people and spurs investment. It is certainly true that spending money to improve the environment would cause money to stop going to some people and start going to some other people. But that happens all the time as the economy and technology changes.

So why are Republicans opposed to working to improve the environment? I really and truly don’t know. One could argue that it is because they are being sponsored by the people who have money now, and therefore might stand to lose it; however, the truth is, the same is true of the Democrats.

We may have to solve this mystery before we are able to do anything effective to help the environment.

By the way, I should add that I’ve seen the argument that we are destroying the planet. That of course is not true. But we are in the process of destroying the habitats of literally billions of people. Those people aren’t going to quietly accept it, which means that we are heading toward massive warfare. Let’s try to avoid that if possible.

Various people have argued that religion is hardwired into the human psyche, that most of us can’t help but believe in something larger than ourselves. Sometimes people go a bit farther and argue that some sort of religion is actually necessary for a healthy society. (Sorry I don’t have useful references here, I’ve forgotten them.)

I’m an atheist myself, but I don’t really consider myself to be a counter-example. I consider myself to be a fairly religious atheist: for me, atheism is a matter of faith. The people who are counter-arguments to the argument are the dedicated agnostics: the ones who say that they simply don’t know, and moreover have no plans to think about it. If religion is necessary for society, then those people are trouble.

These ideas date back at least to Jean-Jacques Rousseau, who said “no state has ever been founded without religion at its base.” Rousseau is most famous for writing about the social contract which forms the basis for a legitimate government, an idea that was of course also important to Hobbes.

So, is it true that we need to believe in something? I think it’s pretty clear that we tend to believe in something. I would describe this as an inborn tendency to see every action as caused by an agent. That is a specialization of our strong mental habit of looking for patterns. Seeing the world in terms of agents gives us a focused way to look for patterns. This is particularly important for children, who must learn enormous amounts about how the world works. It’s much safer, in general, to assume there is a pattern or an agent when there isn’t one, than to assume that there isn’t one when there really is.

Given this natural tendency, when we see events larger than ourselves, such as the weather, we tend to naturally look for larger agents who cause it. Thus we get thunder gods and the like. As people understand their world better, they transfer the agency to the entity which created the world and set it in motion, and thus we get omnipotent and omniscient creator gods.

Anyhow, other people have told that story better than I. The question I want to ask is: what happens if we deny that? What if we set aside god, and say that religion is simply irrelevant?

One reaction is to say that if there is no creator, then there is no design for life, and there is no purpose in it. Can we find a purpose within ourselves? I think that we can, and I think that many people do; people can lead a good life without being given a goal. Can we find a purpose for society? I believe that the goal of society is simply to provide a general happiness for its members, so, again, for me the answer is yes (note that many societies do have different goals).

Another reaction is to say that ethics depends on religion. With no god, we can not rely on any clear guidance as to what is good and what is evil. My answer to this is a bit subtle, and may indeed be wrong. I think that we have evolved such that certain types of behaviour are natural for us. That is, I agree with Socrates that we have a natural sense of the good. This leads to an ethical system which may not be truly universal, but may in general apply to humanity.

For example, my understanding is that in all human societies the murder of another member of society is forbidden. Murder of members of other societies may be permitted, and some societies practice ritual sacrifice, but murder is forbidden. Similarly, young children are always to be protected, and it requires special explanations to harm the children of non-members of the society (e.g., the other children are subhuman, etc.). Newborn babies, on the other hand, have no such special rights, and some cultures practice routine infanticide (some would argue that our culture does as well); however, while killing your own baby is acceptable in some cultures, killing another person’s baby is always wrong.

I believe that these sorts of rules are universal in the sense that that all societies subscribe to them. That said, some societies do become decadent, and permit any rule to be broken. Even then, though, breaking the rules is, in fact, considered to be breaking the rules. It is not considered to be normal even by members of the decadent societies.

Now, obviously, these sorts of evolved guidelines to human behaviour do not truly constrain our behaviour: there are many murderers. And they are very incomplete, in that many ethical issues have completely different answers in different cultures.

So I’m going to argue that we don’t need religion to build a society. But I’m going to do it by saying that we are born with the parts of religion that we need: enough shared ideas to hold a society together. The rest of the ideas come from growing up within the society.

This is obviously an error-prone sort of argument. It’s easy to fall into the trap of thinking that what is true today is always true. And it’s easy to fall into the trap of sociobiology and evolutionary psychology, to think that if you can find a nice explanation of some sort of behaviour that you have found the true explanation. So I don’t know how confident I am that what I am saying is right.

If I am right, though, it is interesting to consider what may happen as we encounter nonhuman intelligences. I’m not thinking so much about aliens from the stars, although that would be nice, as I am about robot intelligence, or the farfetched idea of copying human minds into computers. Without the shared evolution of culture, what will those minds be like? Why should they see things as we do?

I think of a programming as a craft, like making furniture. Despite the names that people use, it’s not science, it’s not engineering, and it’s not art.

Since programming is a craft, it doesn’t advance the way that science and engineering do. Instead, we see waves of fashion, and we see adustments due to changes in the underlying technology. Examples of programming fashion are ideas like object oriented programming or agile programming. Examples of changes in technology are systems like Ajax.

I wish we did a better job of teaching this to new programmers. If people learning how to program understand that it is a craft, they approach it differently. There are some basic ideas that must be mastered, like O(n) notation and the basic meaning of language constructs. Once you understand that, the way to learn programming is not to read books or take classes. It is to write programs, and to study the master programmers, ideally both at the same time.

Fortunately a good number of master programmers are easy to study, because their work is available in the form of source code. This is like studying a master furniture maker by examining their furniture. Actually, the situation with programming is even better, because you can take apart the program without having to buy it.

I first learned how to program by disassembling the BASIC in the ROM of my TRS-80. I had pages and pages of notes on the disassembled Z-80 code. Later somebody else actually made a book along the lines of my notes–I wish I had thought of that. This BASIC was licensed from Microsoft, and thus in some sense I learned programming by studying the work of Bill Gates.

It had some nice tricks. For code like “if (f) then a; else b;”, where b could be done in a single-byte instruction, the a sequence would end with a load immediate of a single byte to a scratch register. The conditional would branch into the middle of that load immediate. The single byte would be the instruction which needed to be executed. This would save a branch around that single byte instruction, thus improving performance and, more importantly, saving program space.

These days there are fortunately books of high quality source code for people to read, and lots and lots of source code of various quality available on the web. Reading some source code should be the the second or third step for anybody who wants to learn how to program. They should have courses on it in university–sort of a great books track.

One of the things I try to teach my daughter is how to live in historical time. By this I mean something in between deep time and the present. I try to regularly explain what is new in our environment and what is old. “When your mother and I were young, nobody had cell phones.” “When your grandfather was young, there were hardly any televisions.” “When your great-grandfather was young, there were no refrigerators, and hardly any cars.” “All the roads you see were built since your great-grandfather was born.”

We carry this back to trains, ships, horse transport, etc. And, of course, medicine. The change in ordinary human lives since her great-grandfather was born in the 1890s is simply staggering. It took me many years to learn to see the present as a snapshot of the ongoing change and development of our world. I’m hoping to give my daughter the ability to see that at a younger age.

Despite all the changes in historical time, human nature hasn’t really changed. Our ethical rules have changed, but our ways of thinking have not. For example, historical time doesn’t seem to come very naturally to people. How many events in history seemed to change everything, only to be forgotten? How many people think about the Lisbon Earthquake today? How many people think the future will be pretty much like the present? Everybody who thought that in the past turned out to be right in general themes, but completely wrong in detail.

On this theme I recently read Bruce Sterling’s story “Dinner in Audoghast.” It’s basically a nice retelling of Ozymandias, done before the fall rather than after.