Voluntary Foreclosure

The otherwise completely forgettable movie Larry Crowne had one scene I found quite interesting. The eponymous protagonist, played by Tom Hanks in his blandest mode, is presented as an all-around good guy. He is gentlemanly, helpful, considerate, and in fact has no flaws except for the rather minor one which starts the little action there is in the movie: he did not go to college. In college he studies, among other things, economics, in a class taught by Lt. Sulu.

Studying economics leads Larry Crowne to the one interesting scene: he abandons his mortgage and returns his house to the bank. The bank representative warns him that this will hurt his credit, but he assures her that it is the right thing for him to do, and that he has the right to do it, provided he vacates the house in 30 days.

The U.S. is a country which famously agreed to pay all its debts when it was founded—the newly created federal government assumed the debts that the various states had accumulated during and before the War of Independence. In general American culture assumes that you are required to pay all your debts. Declaring bankruptcy is a sign of personal failure.

Apparently the financial crisis, and the stagnation of middle class income compared with the increasing price of housing, has now made it acceptable for a mainstream movie character to renege on his debt. This is of course a calculation that businesses make routinely, but not individuals. Admittedly voluntary foreclosure is not quite as bad as bankruptcy. After all, the bank does get the house, which is worth something although it is clearly understood that the bank does not want it.

Given the current main news story, I just have to hope that this acceptance of foreclosure does not carry over to thinking it is OK for the entire country to go into default.

Comments (2)

CVS SSH

Sorry for the long posting hiatus. I have enough readers now that it’s hard to write the usual random nonsense.

I was recently reminded of an old problem using CVS over SSH, which was an interesting example of various different instances of reasonable behaviour adding up to a bug. It’s possible that this bug has been fixed, but I’ll assume that it hasn’t. The bug is that “cvs diff 2>&1 | less” will sometimes drop data, leaving you looking at an incomplete diff.

When CVS invokes SSH, it sets up file descriptors 0 (standard input) and 1 (standard output), but not 2 (standard error). Thus SSH inherits file descriptor 2 from CVS. This means that any SSH errors get reported to the standard error passed to CVS, which is what you want to have happen. However, when using 2>&1, this means that SSH’s file descriptor 2 will be the same as CVS’s file descriptors 1 and 2.

SSH puts its file descriptors 0, 1, and 2 into nonblocking mode, so that it can use select to send data back and forth without blocking. This means that SSH puts CVS’s file descriptor 2 into nonblocking mode. When using 2>&1, file descriptors 1 and 2 are the same, so this puts CVS’s file descriptor 1 into nonblocking mode.

CVS uses stdio to output data to standard output. When writing to a pipe, the buffer can fill up. CVS naturally never puts the descriptor into nonblocking mode, but when SSH has done it indirectly, and the buffer fills, the data being written will be discarded. This is what causes the bug: the discarded data is never seen by the user.

So what’s the fix? It’s reasonable for SSH to put its descriptors into nonblocking mode. It’s reasonable for CVS to pass its file descriptor 2 to SSH. It’s reasonable for CVS to use stdio to output data. It’s reasonable for stdio to not specially handle a nonblocking file descriptor—any program which wants to use a nonblocking descriptor needs to handle I/O retries itself. It’s reasonable for 2>&1 to mean that file descriptors 1 and 2 refer to the same underlying pipe. It’s reasonable for the user to use 2>&1 when piping cvs diff output to less.

I think the only remaining link in the sequence leading to the bug is that when SSH sets its file descriptor 2 to nonblocking mode, this affects the file in CVS. This is a consequence of the Unix file model, in which file descriptors refer to underlying files. A file descriptor has only one flag: whether it is closed if the exec system cal is run. All the other information is attached to the underlying file. Using 2>&1 means that two file descriptors point to the same file. Forking and execing SSH does not change this–in fact, it adds two more file descriptors, in the SSH process, which point to the same file. Any change in the flags associated with that file is seen by all the associated file descriptors.

This separation of file descriptor and file is what makes 2>&1 work. It’s also what makes >> work; >> opens a file in append mode, and the append flag is inherited by other processes which refer to that file. In any case, what really counts here is not the exec, but the fork; forking a process should not change the flags associated with a file. Further, I’m sure there programs which depend on the fact that changing the flags on a file after a fork affects the file as seen by the parent process.

It’s possible to imagine that file descriptors point to a new shared structure which then points to the underlying file. The file position and some flags would stay with the underlying flie. The new shared structure would just hold some flags which need not always be shared: O_NONBLOCK, O_ASYNC, etc. Calling fork would not create a new shared structure, but calling exec would, copying the existing structure. That would let some flags not be copied across exec process boundaries.

However, that would be a significant change to the Unix file model, a model which has lasted for decades and is not clearly broken. Absent that change, we are left with a complex bug, in which all choices are reasonable.

The workaround for the problem is to invoke SSH with 2>/dev/null, and assume that SSH never writes anything useful to standard error. The 2>/dev/null disassociates SSH file descriptor 2 from CVS file descriptor 2, so CVS file descriptor 1 is not accidentally set into nonblocking mode.

Comments (7)

Executable stack

The gcc compiler implements an extension to C: nested functions. A trivial example:


int f() {
int i = 2;
int g(int j) { return i + j; }
return g(3);
}

The function f will return 5. Note in particular that the nested function g refers to the variable i defined in the enclosing function.

You can mostly treat nested functions as ordinary functions. In particular, you can take the address of a nested function, and you can pass the resulting function pointer to another function, that function can make a call through the function pointer to the nested function, and the nested function will correctly refer to variables in its caller’s stack frame. I’m not here going to go into the details of how this is implemented. What I will say is that gcc currently implements this by writing instructions to the stack and using a pointer to those instructions. This requires that the stack be executable.

This approach was implemented many years ago, before computers were routinely attacked. In the hostile Internet environment of today, an area of memory that is both writable and executable is dangerous, because it gives an attacker space to create brand new instructions to execute. Since the stack must be writable, this means that we want to make the stack non-executable if possible. Since very few programs use nested functions, this is normally possible. But we don’t want to break those few programs either.

This is how the GNU tools do it on ELF systems such as GNU/Linux. The compiler adds a new section to all code that it compiles. The section is named .note.GNU-stack. It is empty and not allocated, which means that it takes up no space at runtime. If the code being compiled does not require an executable stack—the normal case—the compiler doesn’t set any flags for the section. If the code does require an executable stack, the compiler sets the SHF_EXECINSTR flag.

When the linker links a program, it checks each input object for a .note.GNU-stack section. If there is no such section, the linker assumes that the object must be old, and therefore may require an executable stack. If there is such a section, the linker checks the section flags to see whether the code requires an executable stack. The linker discards the .note.GNU-stack sections, and creates a PT_GNU_STACK segment in the output executable. The PT_GNU_STACK segment is empty and is not part of any PT_LOAD segment. The segment flags PF_R and PF_W are always set. If the linker has determined that the program requires an executable stack, it also sets the PF_X flag.

When the Linux kernel starts a program, it looks for a PT_GNU_STACK segment. If it does not find one, it sets the stack to be executable (if appropriate for the architecture). If it does find a PT_GNU_STACK segment, it marks the stack as executable if the segment flags call for it. (It’s possible to override this and force the kernel to never use an executable stack.) Similarly, the dynamic linker looks for a PT_GNU_STACK in any executable or shared library that it loads, and changes the stack to be executable if any of them require it.

When this all works smoothly, most programs wind up with a non-executable stack, which is what we want. The most common reason that this fails these days is that part of the program is written in assembler, and the assembler code does not create a .note.GNU_stack section. If you write assembler code for GNU/Linux, you must always be careful to add the appropriate line to your file. For most targets, the line you want is:

.section .note.GNU-stack,"",@progbits

There are some linker options to control this. The -z execstack option tells the linker to mark the program as requiring an executable stack, regardless of the input files. The -z noexecstack option marks it as not requiring an executable stack. The gold linker has a --warn-execstack option which will cause the linker to warn about any object which is missing a .note.GNU-stack option or which has an executable .note.GNU-stack option.

The execstack program may also be used to query whether a program requires an executable stack, and to change its setting.

These days we could probably change the default: we could probably say that if an object file does not have a .note.GNU-stack section, then it does not require an executable stack. That would avoid the problem of files written in assembler which do not create the section. It’s possible that this would cause some programs to incorrectly get a non-executable stack, but I think that would be quite unlikely in practice. An advantage of changing the default would be that the compiler would not have to create an empty .note.GNU-stack section in all object files.

By the way, there is one thing you can do with a normal function that you can not do with a nested function: if the nested function refers to any variables in the enclosing function, you can not return a pointer to the nested function to the caller. If you do, the variable will disappear, so the variable reference in the nested function will be dangling reference. It’s worth noting here that the Go language supports nested function literals which may refer to variables in the enclosing function, and when using Go this works correctly. The compiler creates variables on the heap if necessary, so they do not disappear until the garbage collector determines that nothing refers to them any more.

Finally, I’ll mention that there are some plans to implement a different scheme for nested functions in C, one which does not require any memory to be both writable and executable, but these plans have not yet been implemented. I’ll leave the implementation as an exercise for the reader.

Comments (5)

Thor

My expectations going to see the Thor movie were fairly low, so I was not disappointed by the result. The movie was not dumb, it was entirely watchable, it just could have been a lot better. My daughter was excited about the idea of a Thor movie because she thought it would be from the Norse myths she has heard over many years, and that could have been an interesting if perhaps somewhat depressing movie. She is not interested in the movie based on the comic book, in which they don’t even get Thor’s hair color right, nor Sif’s.

The comic book Thor is a hard character to make a good movie from. This movie does do a good job of showing Thor as he starts out: arrogant and thoughtless. His misbehaviour is believable, and so is Odin’s anger, and Thor’s banishment. That’s all from the original Lee and Kirby stories, albeit told out of order. But Lee always had a gift for making the core of the character make sense, at least in melodramatic terms. In the comics, Odin doesn’t just banish Thor to Midgard, he takes away his memory, puts him in the form of a man who is physically weak, and makes him a doctor, someone who often succeeds but also often inevitably fails to save his patients. It makes sense that that combination would teach humility. In the movie Thor is banished, but retains his memory and physical form if not his powers. It’s not clear why he learns anything from this. The key scene, which is meant to show that he is worthy of regaining his power, doesn’t show him as having learned anything. He is brave and self-sacrificing, but it’s easy to believe that he was that before his banishment. The movie wants to show that he has learned that people can get hurt, and that that matters, but there isn’t any clear reason why he should know that at the end of the movie when he didn’t know it at the start.

I was kind of disappointed by the vision of Asgard, too. The picture from afar was pretty but didn’t look like anything other than a picture. The rooms in the houses were huge and the actors playing gods tended to look lost in them. Having Kenneth Branagh direct was an interesting move but he was not in his element handling special effects.

Some short notes on other movies I’ve seen recently.

Sucker Punch. Very weak. If you’re going to try to hide your simple-minded story ideas behind good visuals, you need better visuals. And you need to make it more impressive as you go, not less. And if you’re going to dress your women like that, you need to do it with a sense of humor, something this movie completely lacked. It’s impossible to imagine Superman without a sense of humor, so I have to hope that rumors that Zack Snyder will direct the next Superman movie will turn out to be false.

The Adjustment Bureau. A solid movie all the way around.

Limitless. I liked this. They didn’t bother to fill in all the plot holes, but I thought that worked given the style of the movie and the protagonist.

Source Code. The title makes no sense but this is a coherent science fiction movie, a very rare thing indeed. And nicely acted by all. The second coherent science fiction movie by Duncan Jones, who directed (and, in that case, wrote) Moon.

Battle Los Angeles. Would have a been a nice movie if they hadn’t had to suddenly decide to wrap up the whole war.

Jane Eyre. A well done adaption, with a Jane who is both plausibly plain and plausible attractive to her Mr. Rochester.

Something Borrowed. All romantic comedies are inherently predictable, but this movie really stood out in its unrelenting predictability. Also it seems odd to make a movie in which the two main characters are so very ordinary and remarkably without interesting characteristics: just like real life, and very much unlike a movie. Still it had a few good moments.

Comments

A Third Way

I have long felt that there is a space in U.S. politics for a party which holds traditionally right-wing views on social issues but traditionally left-wing views on economic issues. Many voters in the U.S. vote against their economic interests in support of their social views. The limousine liberal is a cliché, a wealthy person who votes in favor of higher taxes on the wealthy. Conversely there are many poor people who vote for Republican candidates because they oppose abortion, accepting the fact that those same candidates will vote to cut services their supporters rely on every day, such as food stamps and WIC. So why not a candidate who stands against abortion, against teaching evolution in schools, against gay marriage, but in favor of governmental support for the poor? Some conservative Democrats do take that position, but they are a small minority within the party.

I read recently that France’s National Front has moved into exactly this space. Marine le Pen the traditionally right-wing, racist and anti-Semitic party to be more of an economically left-wing, socially anti-immigrant party. I disagree with her positions in many ways, but I wonder if any U.S. politicians will see an inspiration there.

I suppose the flip side would be socially left-wing and economically right-wing, but many Democrats are in that space already. It pretty much describes Bill Clinton, for example, and Barack Obama is not far off either. The Republican party has been steadily shifting rightward economically–Obama is well to the right of Nixon on economic issues, for example. I don’t know if the Republicans are leading the Democrats to the right, or if the Democrats moving right are pushing the Republicans into ever more extreme positions in order to differentiate themselves.

Comments

« Previous Page« Previous entries « Previous Page · Next Page » Next entries »Next Page »