Absence of Evidence is Evidence of Absence

• if the evidence had a chance to show up, i.e., if someone looked for it.
  
• if evidence is not understood as proof but as an indication. Of course, absence of proof is not proof of absence.

I recently found a really nice proof of this statement using conditional probabilities. I found the symbols and the layout to be not optimal. Therefore, I created my own version of the proof:


What I like especially about this proof is that it first explicitly defines what evidence is: Some observation E is evidence for a hypothesis X if X being true is more likely when given that E is observed than when given that E is not observed, i.e., P(X|E) > P(X|not E). If this is given, observing E will increase the likelihood of X being true. Using Bayes' Theorem, you can precisely calculate how much it is increased. Assume you assigned some a priori probability P(X) to X. The a posteriori probability, i.e., the probability that X is true given that you observed E, is
P(X|E) = P(E|X)*P(X) / ( P(E|X)*P(X) + P(E|not x)*P(not X) ).
It is easy to see that the "shift" from a priori to a posteriori probability will be the higher the bigger P(E|X) / P(E|not X) is. If it is likely to observe E given that X is true, say P(E|X) = 80%, and at the same time unlikely to observe E given that X is false, say P(E|not X) = 0.1%, observing E is strong evidence. On the other hand, if P(E|X) is only slightly greater than P(E|not X), say 80% vs. 75%, E is weak evidence.

Knowing what strong and weak evidence is and knowing that absence of evidence is evidence of absence helps you evaluating lots of things. I find these facts to be actually quite intuitive but it helps if you can be sure of them on a very formal basis.

For example, if your friend John tells you about that new alternative medical treatment that has helped him getting rid of his cold for just 50€. In that case, the hypothesis X is that the treatment is efficacious and the evidence E is that John recovered from his cold. Well, you know that John would probably have recovered anyway, so P(E|not X) is nearly 1. Therefore, however high P(E|X) is, E can only be very weak evidence. This, of course, matches the intuition that you cannot rely on an individual case to judge such a treatment.

Some time later you read about a double-blind, placebo-controlled clinical trial in which they tested the treatment John was talking about. It showed no statistical significant effect, i.e., the result they got did not contradict the null hypothesis that the treatment has no other effect than a placebo. (The result would have contradicted the null hypothesis if, assuming the null hypothesis being true, it was less likely than the level of significance they used -- usually 5%.) This absence of evidence of the treatment's efficacy is of course no proof that it is inefficacious. But it is evidence for that. Again, this matches the intuition. They looked for efficacy, they found no efficacy, so there being no efficacy is more likely now than before they looked.

What about if John tells you about that new lifestyle drug that turns your skin green (hypothesis X). He took it and his skin turned green (evidence E). Your intuition tells you that this should be strong evidence, and it is. Assuming that John tells the truth, you can be pretty sure about the efficacy of that drug because the probability P(E|not X) that he would turn green without the drug being efficacious is practically zero.

Fullscreen editing in TextMate à la WriteRoom

I always wanted distraction free text editing like supported by WriteRoom – but using a powerful text editor I am used to, e.g. TextMate.

As you can see in the screenshots, I managed to get very close to this goal. I use BackDrop to hide the desktop, and I use a patched version of Megazoomer to hide the Mac OS X menu bar, the dock, and the current window's title bar. The patch was necessary because Megazoomer's normal behaviour is to maximize both width and height of the current window and maximizing the width of a text editor can be problematic. Therefore, I created a version of megazoomer that maximizes only the window's height. Thus, you can also put multiple editing windows beside each other as you can see in the second screenshot. I also changed the shortcut to megazoom a window to Cmd+Ctrl+Alt+M to make it more TextMate compatible. You can find my patched version of Megazoomer here (installation instructions included).

Agile Language Engineering Tutorial

This week, Guido Wachsmuth and I gave a tutorial at the University of Agder on using the tools MMUnit, EProvide, and TEF for agile language engineering. We described how to generate a prototype of a DSL with a test-first approach to metamodel development and with an operational semantics approach for describing the DSL's semantics.

As an example DSL, we used a very simple language for describing the behaviour of a telephone remote control system. The idea is that you can call the system with your mobile phone and use your phone's DTMF tones to control something remotely in your house. The DSL looks like this:

main {
 say "main menu";
 listen {
   1: "control lights" -> lights;
 }
}

lights {
 say "your lights are";
 say lights_status();
 listen {
   1: "switch your lights" -> switch_lights;
   0: "return to main menu" -> main;
 }
}

switch_lights {
 lights_switch();
 say "your lights are now";
 say lights_status();
 goto main;
}

You can find the slides of the tutorial here and here. There was also a handout that explains step-by-step how to create the DSL prototype. In this folder, there are some code snippets you may find useful if you want to go through this tutorial.
If you are lazy you can also just download the completed project or watch this screencast showing the prototype in action.

EProvide 2.1.0

Yesterday, we released a new version of EProvide.

(EProvide is an Eclipse EMF-based tool with which you can describe the operational semantics of your DSL using different description languages. The idea behind EProvide is to metamodel not only the static structure of your DSL but also the runtime states. Thus, the runtime state of a DSL program is encoded in a model that changes step-wise over time. With EProvide, these step-wise changes are described with a model transformation that transforms one runtime state to its successor. Based on such a description and on an editor for your DSL, EProvide can execute your models in an animated way so that you can observe and debug model execution.)

The new version is much better integrated into the Eclipse debug framework than its predecessor: you can use Eclipse's standard GUI elements for controlling model execution (suspend, resume, step, drop to frame = reset initial state, terminate). Also, we added a new step-back button to the debug view that allows you to step back in the execution history of your model execution.

You can see the new features demonstrated in this screencast.

Time Machine freezes computer

I let Time Machine backup into a sparse bundle on a windows share. This worked a little slow but otherwise fine until yesterday when Time Machine froze my MacBook every time it ran. To get it back running, I had to turn off the MacBook by holding down the power key for a few seconds. After restart, Mac OS told me the reason: a kernel panic with this backtrace:

panic(cpu 0 caller 0x00321C20): "hfs_UNswap_BTNode: invalid forward link (0xABAC0006)\n"@/SourceCache/xnu/xnu-1228.5.20/bsd/hfs/hfs_endian.c:254
Backtrace, Format - Frame : Return Address (4 potential args on stack)
0x3cb4b768 : 0x12b0fa (0x4592a4 0x3cb4b79c 0x133243 0x0)

After some web search [1], I found out that the reason was probably a defect sparse bundle caused by turning off my computer during backups, which I sometimes do. So the solution was to repair the sparse bundle:

1. I mounted the windows share on which my Time Machine sparse bundle lies.
   Then, I executed the following commands in the terminal:
   
2. Make the sparse bundle available as a device (in my case, it was /dev/disk1, but hdiutil tells you, which device it assigns the sparse bundle to):
   hdiutil attach -nomount -readwrite /Volumes/windows_share/my.sparsebundle
   
3. Repair the sparse bundle (this takes some time):
   fsck_hfs -f /dev/disk1
   

Are writing code and writing text so different?

It seems to me that programming and paper writing modify your brain in incompatible ways. Whenever I have been heavily in programming for a couple of days, I have problems switching to paper writing -- and vice versa. Maybe it is because in paper writing, you are forced to structure your thoughts in a sequential order, whereas in programming, knowing your code structure requires you to keep track of many things in parallel: call and inheritance hierarchies, operation parameters, names of variable, operations, and classes, ...

Eclipse Ganymede OutOfMemoryError

The new version of Eclipse is out now: Ganymede. I installed it and most things seem to work fine. The only problem I had was getting an OutOfMemoryError when starting a second Eclipse instance with the Eclipse plugins I develop. This error occurs in the second Eclipse instance after activating some of the plugins by GUI interaction. Normally, such an error can be solved by increasing the maximum heap size the JVM can use with the VM argument -Xmx. The default size is 128m and usually -Xmx256m is enough for my purposes.

However, in this case, increasing the heap size did not solve the problem. The error that occurred was not the typical OutOfMemoryError. The total heap size was big enough but the PermGen region was too small. This was indicated by the error's message:
java.lang.OutOfMemoryError: PermGen space

To solve this error, I had to increase this region's size with this VM argument (the default size is 64m):
-XX:MaxPermSize=128m