Archive for Random

Complexity

I’ve noticed that I have a bias toward complexity. I tend to try to look at problems from all the angles I can think of. Most problems are complex from at least some points of view; otherwise, they would not be problems at all. Looking at problems from all angles encourages a focus on their complex aspects rather than their simple ones.

I do profoundly believe H.L. Mencken’s classic quote: “For every complex problem, there is a solution that is simple, neat, and wrong.” For any problem, it’s very tempting to shear away the complexity and push forward with a solution that solves the simpler part. Unfortunately solving the simpler part of the problem tends to make the complex part both worse and harder to solve later.

This is a common problem in programming. I think it is the primary reason that programs become harder to maintain over time. It is very hard to keep a focus on solving complex problems rather than ignoring the complexity in order to solve a simpler problem. This is also a common problem in politics, and I’m sure we can all find our favorite examples there.

At the same time, I’ve come to realize that focusing too much on complexity is a different and more subtle trap. It encourages you to focus on the specific issues rather than the general ones. Sometimes a complex problem can be solved by solving a different, larger, problem. Sometimes you need to take a step back. The trick is to not ignore the complexity, but to be open to the possibility that it simple doesn’t matter.

When I was younger, I always felt that Alexander cheated when he simply cut the Gordian Knot. He didn’t actually solve the problem, and indeed I think the analogy is often used poorly, to suggest that it’s somehow smarter to not solve the problem. However, I’ve slowly come to realize that there is a good use of the analogy. I don’t know whether Alexander’s solution was a good one; he didn’t live long enough to prove it one way or another. But there are times when that is the right approach.

I apologize for the abstract nature of this post. I’ve been thinking of this in the context of working on Go. I think that working on the project has helped me a lot in terms of looking for the larger simplicity without ignoring the real complexity.

Comments (1)

Halloween decline

Every year we have fewer trick or treaters come by on Halloween. This year we only had nine people, three of whom were teenagers who knocked on every door on the street at 9:30.

This year I think I finally figured out what is happening. There are some streets in the city which put on a Halloween extravaganza. All the houses give out treats, they have decorations, and all the kids invite their friends. More and more kids must be going to those streets, leaving fewer and fewer more the ordinary quiet streets like ours.

It’s a nice example of Winner Takes All. Admittedly it only works because parents all feel obligated to walk around with their children these days, which means that driving to another part of the city is a real possibility. When I was a kid we went out in groups, but no parents came along.

Comments (2)

Water

Today is Blog Action Day and the topic for the year is water. Access to fresh water is an increasing problem around the world as the combination of increasing population and climate change leave more people without it. Many people get their fresh water from glacial runoff or from aquifers, and the one is melting away and the other is being drained.

Of course there is no shortage of water in the world. There isn’t even a shortage of fresh water. The problem is one of getting the fresh water where the people are; moving people somewhere else is mostly infeasible.

Jerry Pournelle had a series of short stories in the ’70s which involved moving icebergs to get fresh water for irrigation. I always thought that was a great idea. You would have these huge icebergs slowly moving across the oceans. Unfortunately, I don’t think it pans out. Icebergs are really heavy to move around the ocean, and moving them outside of the ocean is impossible. Any place you can move an iceberg, you have plenty of water; it’s just not fresh. The energy it takes to move an iceberg would be more efficiently used doing desalination. Pournelle’s stories weren’t completely crazy when it wrote them, but desalination technology has come a long way, and the physics of moving an iceberg is never going to change.

The successful use of desalination in places like the United Arab Emirates and Israel shows that the problem of access to fresh water is one of energy and infrastructure. It’s not really a typical scarce resource problem. The problem for the world is the large number of poor people with decreasing fresh water supplies. Transporting water is expensive and requires a lot of infrastructure work, particularly if you have to do it uphill. Poor people have no money to spend on doing it. Nobody is likely to do it for them.

The most efficient way to get people water is likely going to be to significantly increase reuse, and to shift agriculture to areas where water can be provided more cheaply. Increasing reuse also requires infrastructure, but it’s cheaper to build than long pipelines. Shifting agriculture has its own set of issues, but it’s likely to be easier to move food than to move people.

The last thing I have to say about water concerns privatization. There is no substitute for water, and it is needed daily, so people will pay any price to get it. Creating alternative water distribution mechanisms is extremely expensive, and to my knowledge has never been done. Putting a private company in charge of water distribution means granting a monopoly for a price inelastic good to a profit seeking entity. There is an enormous incentive to jack up prices as high as the market will bear, which is to say almost all the money available. There is little incentive to invest heavily in new infrastructure, since the company is already absorbing all the available money. That is admittedly the extreme version of what can happen, but really it can not end well, and in general, when it has been done, it has not ended well. Privatization can work well when it creates a competitive market. Water distribution is never a competitive market. Of course there are roles for private companies in the water system, but only in some sort of public/private partnership.

Comments (7)

The Facebook Movie

I really enjoyed The Social Network, although I doubt it had much connection to the real people involved. I don’t doubt the events as such, but I doubt the interpretation.

I noticed one thing in particular. One of the motivations of the Mark Zuckerberg character was that he really wanted to get into a Finals Club. But the Finals Clubs are essentially an archaic holdout of aristocratic WASP privilege. I didn’t attend Harvard, but I did attend an Ivy League school with similar organizations. They were not taken particularly seriously by anybody I knew, except perhaps by those whose parents had belonged to one (and those people were pretty much all accepted). This is true even though I knew people who were accepted.

It’s very difficult to imagine somebody obsessing about getting into one. It’s particularly difficult to imagine that from somebody writing web programs. The path to fame and riches in the computer world is through what you do, not who you know. The Finals Clubs are naturally inclined toward the people who are going to become lawyers and politicians. Why would somebody like Zuckerberg care about a Finals Club?

Of course people’s motivations are mysterious. Zuckerberg did go to Phillips Exeter, an elite private prep school, and he could have absorbed some sort of Finals Club ethos there. Still, Zuckerberg has said publically that he was not interested in joining a Finals Club, and I’m inclined to take him at his word.

Comments (1)

School Testing

I always did extremely well on standardized tests in school, which convinces me that the standardized tests are almost completely useless for assessing knowledge or intelligence or the skills of one’s teacher. Success at standardized tests does not require general intelligence, it requires the particular ability to think like the people who create the tests. Where my friends would sometimes get hung up on issues like trying to figure out the right answer to a question, my approach was instead to figure out which answer the test writers were looking for. This approach only works for multiple choice tests, but in that limited arena it was highly effective.

I still recall my moment of realization in high school. There was a pair of math tests which were the leadin to selecting the members of the U.S. team for the International Math Olympiad. The first test was multiple choice, I think 100 questions, and I did very well as usual. I placed third in the state or something, at any rate good enough to go on to the next level. The second test was a series of five or so math problems. You had three or four hours to solve them. The questions were essays, and so were the answers—no multiple choice. I scored a zero. I could understand the questions, but I had no idea whatsoever how to actually solve them. A good friend of mine, on the other hand, did know how to solve them, and in fact went on to be a member of the U.S. team.

My conclusion is that standardized tests tell you something, but they don’t tell you what you really want to know: a good a student is operating in the real world. The big school reforms which are based on using standardized tests to assess students, like Bush’s No Child Left Behind or Obama’s Race to the Top, are thus based on an invalid premise. And when you start to judge teachers based on how well their students do on standardized tests, you are creating a perverse incentive: you are rewarding them if they produce students who do well on tests rather than producing students who do well in the real world.

I do agree that assessing students is important: you need to know how well your student body is doing. And standardized tests are the best mechanism I know for doing so: they are easy to hand out, easy to grade, and if they are carefully written (which is very hard) they can results which can be compared across socio-economic gaps. But you must always be deeply aware of the shortcomings of these tests. They are telling you something, but they are not telling you everything or even most of what is important.

When tests are used to assess the quality of teachers, you are falling deep into the measurement problem: you are judging people based not on what they are achieving, but based on what you can measure.

And all that said, I do agree that it is important to assess teachers, because there are bad teachers out there. I had some of them myself. There has to be some way to get them out of schools and into some place where they can stop holding children back. It’s just that standardized testing is not that way. Just because there is something we need to do, and there is something else that we can measure, we should not start to think that the thing we can measure will tell us what we need to do.

Comments (12)

« Previous Page« Previous entries « Previous Page · Next Page » Next entries »Next Page »