Reporting Tales

Nuclear power is safe*

Yesterday, once more nuclear power has been proven to be to dangerous to be handled by humans. At the site of nuclear plant of Tricastrine, about 360kg Uranium leaked into the local water system.

So far, that's nothing big, such things happen from time to time. Eventually, we get used to it. But it shows, that not only the plants itself are a can of worms, but all activities around the nuclear power production chain can have devastating long term effects.

The radioactive half-life time of Uranium-235 is 704 million years. So in only 352 million years, the radioactive pollution in that area will be down to half of the level it is today. Great, so your grand-grand-grand-(repeat 10 million times)-grand-kids will be lucky enough to live in a somewhat healthy area again.

But this incident must not be seen as a singular incident. Nuclear waste is generally disposed in rock salt and old salt mines. The salt is supposed to keep the waste dry and to prevent any contamination of the ground water. So far the theory. Newer research showed, that this disposal schema is not safe at all. Nuclear waste still generates heat as the nuclear processes never stop. Research now has shown that salt rock deforms when being exposed to constant temperatures of 100 degree celsius and more. Although the experiments were conducted in a laboratory only, the result indicates a great risk of storing large amounts of nuclear waste in such salt mines for a couple of thousand years. As a result, the only safe way to store nuclear waste is to monitor it actively for the next ... 350 million years.

(Home work exercise: Calculate the full costs of 1.70 MWh nuclear power under the unrealistic assumption that energy companies are paying for *all* clean-up costs for as long as the nuclear waste is dangerous to humans. 1.70 MWh equals one barrel oil.)

During the last years, nuclear power plants have been proven less safe than advertised. In 2006 at Ringhals nuclear power plant in Sweden, a fire shut down the plant and a faulty emergency power aggregate almost lead to a melt-down. The emergency power is required to safely shut down the plant in case of black-outs. Without a safe shutdown, a melt-down is the guaranteed result.

The very same security architecture was installed in the German nuclear plants in Brunsbüttel and Krummel (link in German), which also had to be shut down after a fire and short-circuits in the emergency power system.

And just in June this year, a Europe wide nuclear alert was issued after a accident in a Slowenian nuclear plant.

Despite the fact that Germany has a rather strict monitoring of the security of nuclear power plants, the list of incidents and accidents in German nuclear power plants is still impressive.

How many Chernobyl-style disasters have to happen, before we realize that the risk and long term costs do not justify the short-term profit of nuclear power.

Germany, Sweden and Spain agreed to stop using nuclear power within the next years.

Italy had been a nuclear free zone for the last 20-years, but due to the current power-holder Silvio Berlusconi Italy will most likely return to use nuclear power in 2013.

So the question is: What do you value higher: Short-term solutions to well-known energy problems which everyone chose to ignore for the last decades or long-term safety and a healthy environment.

Independence Day

While the former colonies celebrated their successful rebellion to evade British taxes on tea and other goods, I created my own Declaration of Independence (from old and obsolete rules that burden the pioneer spirit and now just exist for historical reasons).

As the name suggests, the "extensions-reportdesigner-parser" project is a Classic-Engine extension that allows the engine to use *.report files as produced by the current Report-Designer as ordinary report-definition files.

On the quick-win side, this immediately removes the need to convert reports into the extended-XML format during the publish process. As long term gain, this module acts as a compatibility layer as soon as we start the report-designer redesign.

There are several reasons why it is a good idea to separate the report-definition fileformat from the internal model representation:

The report-designer's XML file references classnames and therefore makes it hard to do refactorings without affecting the names put in there. Now we can start to move classes and functionality around without having to worry how our actions may break existing reports.
The report-designer's internal model is a incomplete (and in some cases incompatible) copy of the reporting-engine's report-model. This results in subtle bugs and diverging feature sets, which in return leads to frustration and ugly hacks.
As the designer maintains a own copy of the reporting-engine's model, changes in the reporting engine's processing (notably new features or extensions of existing features) need a long time until they become available in the designer.
And of course: Maintaining a own model is expensive. The time we have to spend to keep the models in sync, is time we cannot spend to make the designer better.

This module now is the living proof, that there is no need for a separate report-designer model and therefore no need to have a *.report format in the future.

With 0.8.11, we can safely map everything that is inside the report-designer's file format into the native elements of the reporting-engine. The charting in the designer is covered by the legacy-charting sub-project, the design-time properties in the *.report-files (like the horizontal and vertical rulers) are mapped into Element-attributes, and for each data-set offered by the designer, we have a sane equivalent. (And in case the dataset-specification itself was a bit insane in itself, as it happens with the XML-datasources, we now have legacy-projects for these parts as well. No matter how strange the code or behavior is, for the sake of full backward compatibility we'll keep that code alive.)

During the next two weeks, we will concentrate on making the Report-Designer 2.0 ready for a bright future. At the end, we will end up with a designer that is a natural extension of the reporting engine core. In the process to that glorious goal, we will happily remove the code that simply duplicates the engine's behavior. Add a bit of reducing information redundancy and let us reduce the magic of reflection to a minimum and we shall be in a good shape for the next years to come.

Taking small steps to cross the tab

After a long silence, let's have something positive today: Pentaho Reporting now officially talks to Mondrian and any other OLAP4J datasources.

We now ship with two flavors of MDX access. The existing MDX capabilities are covered by the BandedMDXDataFactory, while the crosstabbing functionality will rely on a new DenormalizedMDXDataFactory.

The BandedMDXDataFactory takes a two dimensional MDX-Query-Result and maps the multidimensional dataset into a flat table. The approach is reasonable if you want to access the cube row-by-row, but it fails badly as soon as your query has more than two dimensions or if your query-result displays a ragged hierarchy. The report-designer used this mode for a very long time to provide at least some access to Mondrian-DataSources. The banded mode is still great if you need banded reporting over MDX datasources.

However, with Version 0.8.11 of the reporting engine, we finally have to provide real crosstabbing capabilities.

At that point, the pre-chewed data provided by the BandedMDXDataFactory is totally unsuitable for anything sophisticated. You cannot reconstruct a cow from a steak. In the same way we cannot use the banded data to reconstruct the axis and hierarchy information provided by the real MDX-ResultSet. At the same time, the complex (and in some points ambiguous) nature of the data-processing that happens inside the BandedMDXDataFactory makes it next to impossible to use plain queries as source for a crosstabbed report.

The goals for our crosstab-implementation are straightforward:

It has to work on existing data-sources using only TableModels as input (Don't over-architect)
The internal data-source structures must be simple so that any source-system is capable of providing the data in the correct format. (Don't exclude anyone.)
Provide only simple aggregation as built-in functions (Don't copy Mondrian.)
Make sure that functions and expressions work exactly like in relational reports. (Don't be special.)

The new denormalized MDX-DataFactory provides a streaming view over the MDX-Cells. Any datasource can provide a similar view by simply joining the fact-table with all dimensions (and by sorting them according to the desired axis structure). The denormalized view now makes it possible to treat MDX-Columns and Rows (and any of the other 253 possible axises) as relational groupings, which just happen to be displayed in a non-banded manor.

Now with the data-problem solved, displaying the data will be quite easy, even for huge result-sets.

Not likely to break (Rank 10)

Pentaho Reporting has been ranked #10 in Enerjy's analysis of open source projects that are not likely to break.

Enerjy performs some static code analysis on the source code and computes several metrics on how well the software is maintained. Thanks to our paranoid programming, which assumes that the users of the code (in most cases: I) are way to stupid to code correctly all the time. Therefore we write the code in a way that breaks early and hard - including strict assertations, strong typing and explict checks for Null-References as soon as we receive parameters on public or protected methods instead of happily assuming that the humans never produce bugs and accepting everything first (hoping that things will continue to go well) and acting surprised if - no: when - later in the process the assumptions have been proven wrong.

Always protect yourself when coding - you never know what diseases you might catch otherwise.

Why OpenSource is the thing that changes the world

During a TED conference back in 2005, Yochai Benkler gave a great talk about why OpenSource and the whole social production schemas we saw emerging during the last years are a revolution as big as the transition of agricultural societies into the industrial age.

We in the OpenSource crowd of course know that our path is leading to a bright future. But if you go out and randomly select one of the many OpenSource community members to explain, why this thing is THE BIG THING, you probably end up drowned in words but you wont be smarter than before. Until now, I haven't found a explanation as clear and simple as the one given in this 17 minutes talk.

(Source: TED recorded conference talks)