Reporting Tales - performance tag

Spotlight on caching: LibLoader

Sat, 02 Jun 2007 07:36:00 GMT

When the Classic-Engine was started, we did not care much about resource-loading. Resource-loading patterns is what happens to other people, not us. When a resource was needed, we simple wrote some code to load the resource in place. We lived our happy life, until, at one day, we wanted to support image-loading.

Image-loading is easy, as long as you just use the AWT-Toolkit and its built in capabilities. But with Pixie (our WMF-renderer) and the many other image libraries out there, we hit the wall the first time. Not the resource-loading code that has been scattered all around the code backfired at us. Either we copy the new image-handling code to every sinlgle occurence of the old code, or we would create some callable library code.

Luckily we chose the library path and created an image-factory. Our XML parser code was the same story – first a funny collection of random code, and in the next moment a nice little library. Then came the Drawable-factory, so we now have three different resource-loader implementations.

With the dawn of the Flow-Engine, the numbers started to explode. DataSource-definitions, stylesheets, subreports – suddenly every piece was (potentially) loadable.

The walls started to move so that they could hit us from a better angle ..

On a parallel thread of events, deep inside the Pentaho plattform, a new potential source of problems was hatched by the Pentaho-engineers. Smart as they are (as long as they dont fall into the '.., but it works' pattern of creating horrible hacks, of course), they created their Platform to be independent of the underlying storage system. So in the Platform, you store your reports and their resources either in a 'SolutionRepository', which can be anything, from a filesystem based solution to a relational database).

And actually, that was the point, where it made click. What we need is simply a common, reusable and (hopefully) well-designed library, that helps us with locating and loading resources. I'm bad at creating fancy names, so I simply called it 'LibLoader'.

LibLoader is here to solve all resource loading problems, we encountered in the past time:

resource loading is slow, so it adds caching to the IO-layer
resource creation is slow, so it caches the actual parsing or resource-interpretation step
resource loading from different sources like filesystems to db-repositories to network storages is awfully complicated, so we address this as well by creating a common resource naming schema.
And last but not least: It must be lightweight and should not reinvent the wheel.

To add effective caching, we have to solve a couple of problems.

1.Make your cacheable resources identifiable

First, we have to create some sort of naming schema. For us, there are only two systems that seem to be important: Hierarchical storages, where your entries form a tree with parent child relations (like in filesystems), and flat storages, where entries have no relation between each other (like in database storages). Names must have at least some minimal interoperationality – so it must be possible to go from one naming system (like URLs) to another system (like the infamous solution repository). And finally: for hierarchical names, there must be some way to construct names using relative paths (as this is crucial inside the reporting engine).

2.Make it possible to detect changes in the underlying resources

Thats quite easy: Every storage system has such a facility. Filesystems call it 'last modified timestamp', CVS called it version, as does the Solution-repository. All we have to do is to map it into a global scope. For that, we simply let the storage system implement a service interface. So we pushed the whole problem down to the low-level layer. Problem solved :)

3.Avoid unnecessary work

Parsing itself is also an expensive operation. XML processing is no fast job – and even if you use streaming parsers like SAX you waste a lot of time doing all the string processing and construction of Attribute-Collections.

So if your resource can be stored safely (so it is protected from changes or it is immutable), LibLoader can also cache the parsing result for optimal performance.

4.Make it easy to use

For historical reasons, Classic-Engine supports multiple report-definition descriptions. Our policy on parsing resources is simple: The user of our code should not care about where it came from, the only thing he should have to care about is the result. In Classic-Engine, and now in LibLoader, we implemented a resource-loading multiplexer. Sounds complicated, but its meaning is simple: For any resource that should be loaded, the library tries all known resource-handlers to interpret the raw-data. So if there is at least one implementation that handles the given data and is able to produce the requested resource-type from it, the resource will be loaded.

Now, if a new resource-type should be added, the implementor only has to care about the acutal loading, caching and dependency management comes at no additional costs.

The Art of Caching: How to be lazy in a good way

Wed, 02 May 2007 19:00:00 GMT

We all agree that avoiding work is a 'Good Thing'(tm). However, in most cases we can't simply use the direct approach and tell the caller (in a more or less friendly way) to get lost. And if we move to common programming languages, most of them even lack the 'Tell to get lost' operation.

So we have to find other ways.

The most common way of avoiding work, is caching. Caching is the process of storing results of an expensive computation for later reuse. Therefore, implementing caches trades memory for computation time.

Caching can be done on all levels of a computation. First, we can cache the final result of an operation. Where this is not possible, we can fall back to cache intermediate results. And if even this path is blocked, we can try to collect hints to make later computations more efficient and cache those hints instead.

How can you tell, whether caching makes sense in your code?

Repeated computations: Caching only makes sense, if the operation in question is repeatedly called with the same parameters. The more the operation is repeated, the more we'll gain through the caching. Adding caching to a one-time operation is a waste of time.
Expensive computations: The more expensive an computation is, the more caching is justified. Even an operation that is executed only twice in your programms lifetime may be a good target for caching, if it's execution expensive enough.

But of course, not everything is cachable by default. There are a couple of prequesites that must be fulfilled before you can think about implementing the cache:

Deterministic operations without side-effects: If your application's architecture uses a clean model and its operations are kept free of any hidden side-effects, then you should not have trouble adding caches. But if you write spaghetti-code and heavily rely on a global state - then adding caches will create a uncontrollable monster with funny, but hard to track state-bugs. Good luck in that case - you will need it.
Separatable units: The operation in question must be a separate unit with one entry and one exit-point. It should be a function in the mathematical sense: The result of the function should only depend on the given input parameters. The function's call must be repeatable - the same input parameters must always yield the same result. Therefore the function must not modify the global state.
Clean inputs: The operation must have identifiable boundaries: The input parameters must be identifiable (which can be hard if you have to deal with a deeply nested object-structure). It must be possible to compare the parameters with as little costs as possible. The fewer parameters the operation has, and the easier it is to compare them, the more efficient the caching will be.
Clean output: The computed result must be an atomic unit. The result that is stored in the cache must retain its state and must not be modifiable from outside. Ideally, the result is an immutable object. If that's not possible, the result should allow to create an independent copy of itself (Java, for instance, offers the clone() method for that purpose).

In object oriented systems, caching can be added easily. As classes are treated as black-boxes (we know which operations the object offers, but we dont need to know how these operations are implemented), we can rely on the polymorphism mechanism to substitute the original implementation with the cache. One of the easiest way to implement caching is to implement the cache as proxy for the original implementation.

Lets look at some code to make this clearer.

public interface ExpensiveOperationModule
{
  public Object compute (Object parameter);
}

public class ExpensiveOperationModuleImpl 
   implements ExpensiveOperationModule
{
   public Object compute (Object parameter)
   {
      final long startTime = System.currentTimeInMillis();
      while ((startTime + 100000) < System.currentTimeInMillis())
      {
        // this loop simulates a long running computation.
        // sit back and wait while we wait for the computation 
        // to be finished.
        // This loop will take 100 seconds to complete and 
        // will bring your CPU usage to 100% to show you how 
        // expensive it is.
      }
      return String.valueOf (parameter);
   }
}


// lets run it
ExpensiveOperationModule module = 
    new ExpensiveOperationModuleImpl();
System.out.println ("Start");
module.compute (new Integer (10));
System.out.println ("One");
module.compute (new Integer (10));
System.out.println ("Two");

As you can see in the implementation, the computation is a long running task, that takes a long time to execute. Each operation takes painfull 100 seconds. Let's cache that.

public class CachingExpensiveOperationModule 
     implements ExpensiveOperationModule
{
   private ExpensiveOperationModule parent;
   private Object lastParameter;
   private Object lastResult;
   
   public CachingExpensiveOperationModule
          (ExpensiveOperationModule parent)
   {
     this.parent = parent;
   }

   public Object compute (Object parameter)
   {
     if (lastParameter != null)
     {
       if (lastParameter.equals(parameter))
       {
         return lastResult;
       }
     }
     lastResult = parent.compute(parameter);
     lastParameter = parameter;
     return lastResult;
   }
}

// lets run this one
ExpensiveOperationModule module = 
      new CachingExpensiveOperationModule 
          (new ExpensiveOperationModuleImpl());
System.out.println ("Start");
module.compute (new Integer (10));
System.out.println ("One");
module.compute (new Integer (10));
System.out.println ("Two");

Except for the first line, the main method has not changed. But with the caching in place, the second call to 'compute()' returns immediately, as the result had been computed previously. Of course, this example is very simple and does not work efficiently in case a complexer call-sequence is used (for example: alternating parameters: compute(10), compute(20), compute(10) ..).

In most cases, caching can be seen as a simple Key => Value lookup, where the input parameters of the computations act as key and the computation result forms the value. All serious programming languages offer a Map or Dictionary collection type, which maps a Key into a Value. When using Java, the built-in HashMap is almost always a good Map implementation that offers good performance on lookups.

But a word of warning: When implementing cache-lookup strategies, make sure that these lookups are as inexpensive as possible. Caching makes no sense if the cache-lookup becomes more expensive than the operation that should be cached. A huge and complicated key will be a lot more expensive to compare (and therefore to lookup) than a simple key.

The Quest for Performance

Fri, 27 Apr 2007 17:35:00 GMT

The new layouter implementation once again sent me on the quest for more performance.

I'm sure we all know the three basic principles of performant job execution:

(1) Cache: The best job is the job, where you don't have to work at all.
(2) Algorithms: If you can't aviod work, then do it as quick as possible, using the best algorithms available.
(3) Resources: Dont make a mess, while doing your job. You'll have to clean up afterwards - and that costs time too.

After applying these priciples, the Layouter implementation is now three times faster than before (400 seconds pagination time vs. 120 seconds pagination time with the new layouter for 200.000 rows). However, this is still bad compared to the 75 seconds of the old layouter. But that lazy beast had mastered the art of caching - it avoided almost every real work by using its extensive caches.

A new layout algorithm for the Classic-Engine

Sat, 14 Apr 2007 18:51:00 GMT

When you're afraid of touching your old code, it is a good time for a rewrite. The code that I was afraid most was burried deep inside the Layout-Controller.

The development of layout-engine of the Classic-Engine was purely driven by need with an almost obscene lack of planing. However, it worked surprising well - the engine was swift and fast. Even when the layout-manager implementation was almost already out of control (better dont touch it, but still working), we were able to get more performance out of it.

Stephane Grenier, founder and author of LandLordMax, was truly amazed about the performance gains we achieved in the last year. To quote his comment on his Blog

In regards to performance, as far as I understood the biggest gains were in upgrading to 0.8.7-10 I even posted a message to the forum about this. Based on the performance differences, and the fact that version 0.8 (based on information from their head developer) will eventually be able to print combined reports (which is great if you want to print multiple invoices directly within the invoice list) we definitely upgraded to the latest 0.8 version.

But in JFreeReport 0.8.8, the subreporting caused some major changes in the layouting system. If you add new features, you always pay a price for it.

Fixing this innocent little bug #1681595 is more difficult than expected. A quick and dirty fix would tear down the remaining architecture - at the end, we may have implemented a fix for one bug, but laid the fundament for a whole new series of bug. No! Heading down this path would have been stupid and a receipt for doom and desperation.

But: We already stole the subreport-implementation from the Flow-Engine. And once you've started stealing, you can't stop halfways. LibLayout, the layouting system of that engine looks nice too ..

During the last week, I've downported and modified the rendering pipeline of LibLayout. The Classic-Engine has no need for CSS-StyleSheets and it does not (and cannot [for compatibility reasons]) make use of the DOM-oriented architecture of LibLayout. So all we need is the last stage of the rendering process: The part where abstract elements turn into content.

So far, the extracted renderer is working and produces reports with the new layouting system. With this layouter, a many of the most hated restrictions of the old engine will be history:

Bands can span multiple pages.
Text-Elements can produce formatted text. A single text element can use several different fonts and font-styles.
Text-Elements can contain inline-images.
Dynamic-Height Text-Elements will be considerable faster.
The layouter now has documented rules how the layout is computed.

For me, the last point is the most important one. It took me several days to understand how the old layouting system really works. It is easy, if all elements are static. But as soon as one element uses relative positioning (percentages instead of absolute values), things became very messy. With the new system in place, the report layout finally behaves deterministic.

But you alway pay a price ..

As the layouter now uses different principles to perform its job, the internal API massively changed. There old OutputTargets are almost gone - right now they are dummy classes to maintain some backward compatiblity. All of the old layouter implementation classes are gone and only a few will continue to exist as dummy or backward-compatibility classes.

There will be border-cases, where the new layouting engine does not exactly behave like the old one. Everyone who relied on the old behaviour, that bands will not span multiple pages, will now have to change the report definitions.

Functions which modify the page-footer or a repeating group-footer during the page-finished event will produce different results now. In the old days, such changes never changed the page-break position. Now, this can happen. If the page-footer grows, it may start to move the last band to the next page. However, it is still better not to change the footer-structure during this even.

Well, the fact, that - right now - this two-weeks old implementation is slower than the old one should surprise no one. Once we have the same insane amount of caching and heuristics in place, we shall come back to our old performance high.

One final word: Once this implementation is complete, the next release of the classic-engine will be called 'JFreeReport-0.8.9'. One step closer to a '1.0' version.