OSGi Alliance Blog

Why Services are Important

2008-08-25T16:27:00.003Z

Many people choose OSGi because it provides them with far superior class loading functions than plain Java. Some even say that OSGi is class loaders on steroids. However, viewing OSGi from this perspective is like building a car that looks like a horse carriage with engine. Taking full advantage of OSGi requires the use of services, however, the service model is much more enigmatic for most people than class loading. A good friend of mine, Niclas Nilsson, who visited me last week told me a story about how people view software languages in a very subjective way. If the languages they are familiar with do not have a specific feature, then they have a tendency to view this feature as unimportant and consider them (unnecessary) syntactic sugar. For example, few Basic programmers missed objects in the nineties.
I think services are a bit like this. There are millions very experienced Java programmers that build impressive libraries and applications using class loaders as extension mechanisms. There is a wealth of knowledge and experience. In contrast, OSGi services implement a concept where one cannot find a comparable mechanism in plain Java, nor in other languages/environments.

Why are services then so important if so many applications can be built without them? Well, services are the best known way to decouple software components from each other. And, I assume that you are aware of the many advantages that decoupling gives you in almost any aspect of software development.

One of the most important aspects of services is that they significantly minimize class loading problems because they work with instances of objects, not with class names. Instances that are created by the provider, not the consumer. The reduction of the complexity is quite surprising. Simple example is Hibernate; most of the class loading problems in Hibernate are caused by the fact that the configuration is done in XML files. Working with Class instances and instances of those classes make most problems go away.

What kind of problems do you have with the class names? Well, first you have to configure it. This means that if you want to change the implementation, you must not only edit some (hard to read) XML or properties file, you must also make sure that the right JAR files are on your class path. In a service based system, you only install the proper bundle. Look at the 'interesting' world of logging in Java. Most logging subsystems handle their extension needs with class names. This requires having property or XML files in the right place and requires you to have the right logger implementation on your JAR path. In an OSGi world, the only thing you have to do is install an implementation of the log service and everybody uses that log service. As Richard Hall always says: 'The set of installed bundles is your configuration.'

Not only do services minimize configuration, they also significantly reduce the number of shared packages. In Class.forName based system you need to be able to see all possible implementation classes. In a service based system, you only need to see the package in which the service is defined. And the best package is one that is not shared.

To be honest, after working for many years with OSGi I really had to get used to the idea that one could also share packages with implementation code; before R4 OSGi was only intended to share service packages. It seemed such a bad idea.

Sharing implementation classes is almost always bad because it makes bundles extremely dependent on each other. The reason we added all those powerful class loading mechanisms was not for them to be used in new designs, their primary purpose was to ease conversion of existing applications.

And last not least: versioning. Class.forName must fundamental flaw is that the only parameter is a string for the class name. In complex systems there will be multiple classes with the same name. This confusion is completely absent in a service based system because service objects are already correctly wired up.

However, I guess there is no escape to Niclas' story. If you have never built a real system with services it is hard to see the advantages and it is easy consider them as an extra without a lot of value. But, maybe this blog can give you the push to start using services in real designs and find out how surprisingly powerful this very simple concept is. Let me know!

Peter Kriens

Classy Solutions to Tricky Proxies

2008-08-05T16:58:00.005Z

Though the quest for reusable systems is the guiding principle in my life, I only recently start to understand the reason for some of the mechanisms in Spring. For example, I start to see that the whole Aspect Oriented Programming (AOP) business is not about aspect oriented programming, but it is about hammering an existing code base into its intended slot. Unfortunately, some of the slots are square while the code bases are quite round. Therefore, proxying is an important aspect of developing a system by making a round object look square, or vice versa.

The bad news is that OSGi is not making this easier. The OSGi framework was designed with green field applications in mind that would follow proper modularity rules and communicate through services. In a perfect OSGi world, bundles are highly cohesive and are coupled through their services. In the real world, class loaders are (ab)used for configuration and ad-hoc plugin systems. Some of those problems are quite complex.

Today, I was asked about the following problem.

A company is using Spring. One of their bundles referred to a service that another bundle had registered. Spring has decided that services are too jumpy, so they proxy the service object for their clients and add some mixins in the proxy that handle the situations when a service goes away (Spring also adds some other mixins).

Creating a proxy is class loader magic. You take a set of interfaces (the mixins) and generate the byte codes for a class that implements all these interfaces. The implementation of the interface methods forwards the call to a handler object using the object, the method that was called and the arguments.

If you generate the byte codes for a class, you need to define this class in a class loader and this is where the modularity comes into play. In a traditional Java application you can just take the application class loader and define this new class in this class loader. The application class loader usually has full visibility. In a modular system, however, visibility is restricted to what you need to see. In OSGi, the imported packages are all a bundle can see (at least, when you want to work modular and not use dynamic imports buddy class loading).

The catch is that the proxy requires access to all the mixin classes. However, some of these mixin classes come from one bundle, and some come from another bundle. There is likely no class loader that can see both bundle class spaces unless that bundle specifically imports the correct interfaces. However, if a client bundle gets one of his objects proxified, then he has no knowledge of the mixins (that is the whole idea). It would be fundamentally wrong to import the interface classes while he is oblivious of them. However, the bundle that generates the proxy might know the mixin classes but it is unlikely it knows the classes of the target object. If the proxy generator was implemented as a service (as it should be) then there would even be 3 bundles involved: the proxy generator, the bundle needing the proxy, and the client bundle.

Spring solved this problem with a special class loader that chained the class loader from the client bundle together with the class loader of the Spring bundle. They made the assumption that the Spring bundle would have the proper imports for the mixin classes it would provide and the client bundle would have knowledge of the proxied object's classes.

So all should work fine? Well obviously not or I would not write this blog.

The generated proxy class can see the Spring bundle with the interface classes and it can see the all the classes that the client bundle can see. In the following picture, the client bundle imports the service interface from package p, but it does not import package q which is used in package p. If you create a proxy for a class in package p, it will need access to package q, and the client bundle can not provide this.

For example, assume the following case:


  javax.swing.event.DocumentEvent de =
          (DocumentEvent) Proxy.newProxyInstance(getClass().getClassLoader(),
          new Class<>[]{DocumentEvent.class}, this );

The DocumentEvent is from the javax.swing.event package. However, it uses classes from the javax.swing.text package. Obviously the bnd tool correctly calculates the import for the DocumentEvent class. However, when the generated proxy class is instantiated, it will need access to all the classes that the DocumentEvent refers to: the super class, classes used as parameters in method calls, exceptions thrown, etc. These auxiliary classes are crucial for the DocumentEvent class, but they are irrelevant for the client bundle, unless these classes are actually used in the client bundle, where they would be picked up by bnd.

So, if the client bundle does not import javax.swing.text then you will get a ClassDefNotFoundError when you try to instantiate the proxy class. This error gets generated when the VM sees the reference to class in the javax.swing.text package and tries to load it through the proxy class' class loader (that is why this is an error, it happens deep down in the VM).

To be specific, this is exactly as it should be in a proper modular system. The client should not be required to import classes that it does not need. The onus is on the proxy generating code to do it right.

Fortunately, the solution is not that hard but it highlights how easy it is to make the wrong assumption, believe me, these Spring guys are quite clever.

Just take a step back and look at the problem.

Do we need visibility on the bundle level here? Look at the information given to the proxy generator: a set of mixin classes. It does not really matter from what bundle these interfaces classes come from; the class loader that we need must be able to see each of the interface classes class loaders. By definition, those class loaders can see the right class space.

So instead of chaining the bundle class loaders, the problem can be solved by having a class loader that searches each of the class loaders of the used mixin classes.

For the Advanced

The sketched model is too simplistic because there is the cost of a class loader per generated proxy. Even though enterprise developers seem to live in a wonderful world where memory is abundant, as an old embedded geezer such abuse of class loaders worries me. Though it is possible to use a single combined class loader by keep adding the required class loaders but this raises the question of life cycle. Such an über class loader would pin an awful lot of bundle class loaders in memory and it will be very hard to not run into class space consistency problems because over time this class loader will see all class spaces. That is, this approach brings us back to all the problems of today's application servers.

I have not tried this, but the solution is likely to make the proxy generator service based. Not only will this make it easier to plugin different generators, it also means that the generator is bundle aware. It can then create a combining class loader for each bundle that uses the Proxy Generator service. This class loader will then be associated with the class space of that bundle and should therefore not be polluted with classes from other class spaces. In the Spring case, this would be the client bundle, not the spring bundle. The reason is, is that the spring bundle could be used from different class spaces.

However, the proxy generator must track any bundle that it depends on. That is, it must find out which bundle's export the interface classes and track their life cycle. If any of these bundles stop, it must clear the client's combining class loader and create a new one when needed. This will allow the classes from the stopped bundle to be garbage collected.

Peter Kriens

For the hard core aficionados, some code that demonstrates the problem:

First a base class that acts as activator. This base class calls a createProxy method in the start method. This is then later extended with a proxy generation method that fails and one that works ok.


package aQute.bugs.proxyloaderror;
import java.lang.reflect.*;
import javax.swing.event.*;
import org.osgi.framework.*;

public abstract class ProxyVisibility implements BundleActivator,
        InvocationHandler {

    public void start(BundleContext context) throws Exception {
        try {
            DocumentEvent de = createProxy(this, DocumentEvent.class);
            System.out.println("Succesfully created proxy " + de.getClass());
        } catch (Throwable e) {
            System.out.println("Failed to create proxy" + e);
        }
    }

    public void stop(BundleContext context) throws Exception {
        // TODO Auto-generated method stub

    }

    public Object invoke(Object proxy, Method method, Object[] args)
            throws Throwable {
        return null;
    }

    abstract protected  T createProxy(InvocationHandler handler,
            Class primary, Class... remainder);

}

The following class uses the base activator but uses only the client bundle's class loader. Trying to create the proxy will therefore fail because this class loader can not see the javax.swing.text package.


package aQute.bugs.proxyloaderror;
import java.lang.reflect.*;

public class ProxyLoadError extends ProxyVisibility {

    @SuppressWarnings("unchecked")
    public  T createProxy(InvocationHandler h, 
        Class primary, Class... others) {

        Class parms[] = new Class[1 + others.length];
        parms[0] = primary;
        System.arraycopy(others, 0, parms, 1, others.length);

        return (T) Proxy.newProxyInstance(getClass().getClassLoader(), parms,
                h);
    }
}

And at last, the solution. The following class extends the base activator and creates a proxy that uses a Combined Class Loader. This loader traverses all the class loaders of the mixin classes.


package aQute.bugs.proxyloaderror;
import java.lang.reflect.*;
import java.lang.reflect.Proxy;
import java.net.*;
import java.util.*;

public class ProxyLoadOk extends ProxyVisibility {
    
    @SuppressWarnings("unchecked")
    public  T createProxy(InvocationHandler h, Class primary,
            Class... others) {
        CombinedLoader loader = new CombinedLoader();

        Class parms[] = new Class[1 + others.length];
        parms[0] = primary;
        System.arraycopy(others, 0, parms, 1, others.length);
        for (Class c : parms) {
            loader.addLoader(c);
        }

        return (T) Proxy.newProxyInstance(loader, parms, h);
    }
    
    static class CombinedLoader extends ClassLoader {
        Set loaders = new HashSet();

        public void addLoader(ClassLoader loader) {
            loaders.add(loader);
        }

        public void addLoader(Class clazz) {
            addLoader(clazz.getClass().getClassLoader());
        }

        public Class findClass(String name) throws ClassNotFoundException {
            for (ClassLoader loader : loaders) {
                try {
                    return loader.loadClass(name);
                } catch (ClassNotFoundException cnfe) {
                    // Try next
                }
            }
            throw new ClassNotFoundException(name);
        }

        public URL getResource(String name) {
            for (ClassLoader loader : loaders) {
                URL url = loader.getResource(name);
                if (url != null)
                    return url;
            }
            return null;
        }
    }

}

Sun Hires Richard Hall

2008-08-04T12:06:00.004Z

It took sometime but yesterday it was official: Sun has hired Richard Hall in the Glassfish team! This is good news for Richard, who was looking for just a position like that, Sun, who can clearly use the expertise on OSGi that Richard as one of the foremost OSGi experts provides, Apache Felix because this means that Richard will be able to spend more time on this project, and also good news for the OSGi Alliance that will have one of its key expert group members represent Sun. Rarely is a change positive in so many ways.

I look forward to continue working with Richard in his new role. I hope that Richard's role will mean that Glassfish will become one of the key players in the next OSGi release, they clearly have the experience to match. Though it will be interesting to find out what happens in JSR 277 because Richard is currently an independent member. Will he continue to participate (though participate is a big word with the activity in the last year) or will he be able to actively work in that group (and get the long overdue EDR2 out)? That is still a question mark.

I wish Sun and Richard a very happy and fruitful cooperation and hope to be able to work with Richard for a long time.

Peter Kriens

OSGi Application Grouping

2008-07-17T07:17:00.002Z

Several people in and outside the OSGi want application grouping. They want to be able to deploy a set of bundles as an "application". These "applications" should have some scoping on class loading and service issues. I am opposed to this but, as Eric Newcomer pointed out to me last week in Dublin, I never really made my case and shown a solution to some of the valid requirements. So, here it is ...

Back to basics. In the beginning, OSGi was about providing functionality to a device. Adding bundles would add more functions and removing bundles would get rid of those functions. Key was that these bundles would collaborate, they would not just be started and running in some silo container; ignorant of their siblings around them. However, these bundles would not have a priori knowledge of each other because the business idea was that different service providers would supply them independently.

The only way you can make this work is if you have a communication model that allows you to very loosely couple the bundles. This was the service model. The service model is very simple: the producer registers some object under an interface and the consumer gets it through the interface. The implementation is safely hidden, the consumer has no clue. Unfortunately, this service model required rules about class loading because those service interfaces involve classes and you better make sure that the producer and the consumer both share the same class loader for that interface/class. It was never the original intention to share implementation classes between bundles because bundles were supposed to communicate through services with a minimal interface to promote loose coupling. Sharing implementation classes was more or less an unintended side effect. Prior to R4 we could not even support multiple versions of the same exported package because that need is not very acute in a service oriented world program.

In the meantime at plain Java, the poor man's extension model became Class.forName. Where the OSGi model offers the services to the consumer, in the poor man's model the consumer creates the services through dynamic class loading, albeit with the indirection of some XML or other text file. This, alas, requires implementation visibility into the provider bundle.

The popularity of this Class.forName extension model makes the OSGi class loaders on steroids the most attractive aspect of OSGi for many people. However, having to export implementation classes from a bundle obviously opens huge holes in the modularity story, suddenly you are exporting implementation classes to the world. This makes the coupling a lot less loose.

My personal feeling is that application models are only trying to patch up these modularity holes with an emergency bandage. This is like drinking heavily and then taking aspirine against the headache. Maybe less drinking would be healthier?

This said, there is of course a concept of applicationness, otherwise not so many people would demand it. However, I do feel that the bundle is the right granularity for application granularity. A good bundle provides one or more functions and expresses dependencies on other bundles through services. That is all, no other primitives are needed (the services imply one or more Java packages). If I want to run an application I pick a bundle that provides me with the right functionality. Systems like OBR can then be used to find bundles that provide the needed services.

Dynamic dependency resolving is needed to create flexible systems. One of the key problems with an application model is that it fixes the application to a specific set of bundles; making it applicable to only one environment and thereby limiting severely the overall reusability.

One of the key issues is of course testing. Many companies want to be able to send a set of bundles to QA and be sure that that is exactly the set that will be deployed on the target system. I do understand this issue. However, I think that the best solution to this requirement is to run these bundles in a framework, or even a VM, of their own. In certain cases, it is even wise to just wrap up all the bundles into one so you are sure they can not be separated. From a modularity perspective, if these bundles are so closely coupled together, why separate them?

Fortunately, the EGs have come up with a model that seems to fit the needs to scope a set of bundles: nested frameworks. They are currently investigating a solution where you can create new frameworks inside an existing framework with relatively little cost. This solution seems to be very unintrusive to the spec because it adds a feature but does not (I hope) influence the existing features, at least not in a major way. Nested framework allow a managing framework to create nested frameworks for each "application". Services can be shared between frameworks by using the existing notification mechanisms and registering the service in alternative frameworks. Even packages could be shared with the

To conclude. I think the demand for an application model is largely driven by problems that are caused by the Class.forName extension model and tight coupling that caused developers to share implementation classes. Even after ten years I think that the bundle-service model of OSGi is the cleanest solutions to software development that I know. I do hope that we will not dilute this model by patching OSGi to support the lesser implementation class sharing model. Then again, with nested frameworks we all can seem to get what we want!

Peter Kriens

QVO VADIMVS?

2008-07-02T06:04:00.005Z

We are seeing more and more outlines appearing for the next OSGi release. One of the major issues is legacy code. Not only inside the OSGi, but if you go to the web you see a lot of people struggling to get old code to work inside OSGi frameworks. Obviously, we want to mitigate the issues around legacy code as much as possible, the more people that use OSGi the better. However, lately I have some (personal, this absolutely is not an OSGi standpoint!) musings about how to attack the issue of legacy code.

A short story to illustrate my musings. In the eighties, I worked on a page-makeup terminal for the newspaper industry. Petr van Blokland, a graphic designer turned computer specialist, introduced me to the layout grid. This grid had columns and between the columns there was a small gutter. Text and pictures were placed on this grid, usually encompassing multiple columns and gutters. Like:

The OSGi always reminds me of this grid. Why? Because they both restrict you severely but in return they provide simplicity. Instead of having infinite freedom to do whatever you feel like, you must obey some pretty basic rules, which some people find upsetting. But what you get back is that the elements work together as a whole, instead of fighting with each other.

Layouts done with this grid almost invariably look good with no effort (try working with the average layout manager in Swing or SWT!). The advantage is that elements always line up and there is always the same space between elements. Without a grid, it is very hard to avoid unwanted visual effects.

Genuine OSGi bundles almost invariably collaborate with each other without much effort (anybody saw the combination Eclipse and Spring coming?) because modules are self-contained and can only export packages and communicate via services instead of the myriad of ways people have devised in Java.

Interestingly, both are achieved by restricting ones freedom, the opposite of providing more features. But neither OSGi nor this grid is simplistic. A simplistic grid would be a square 8x8 grid, and they just do not work. A simplistic OSGi would be some Class.forName based system without handling versions and dependencies. Both OSGi and the grid seem to be in a sweet spot: simple but not simplistic, providing maximum bang for the buck.

However, legacy code seems to be forcing us to add more and more mechanisms to the OSGi specification. Unfortunately, these mechanisms are often also then used for new OSGi applications because the legacy concepts they represent feel familiar to people. See how many people use Require-Bundle and fragments.

If we add all these freedoms to the next generation, will we not pollute the original model and become in the end much less attractive? Or, if we do not make it easier to use legacy code, will people turn away because they feel affronted that their direct needs are not addressed? Should we leave these issues to framework implementations making legacy code not really portable?

The current popularity of OSGi seems to allow the OSGi to make a stand. What do you think?

Peter Kriens

JSR 277 and Import-Package

2008-06-17T14:43:00.005Z

It looks like Sun is not convinced that Import-Package is a good idea. On the JSR 277 mailing list there was an interesting discussion about the dependency model: Does Import-Package have value over Require-Bundle (to use the OSGi terms)?

Require-Bundle has an intuitive appeal that is hard to deny. These are the artifacts on the class path during compile time and there is a clear advantage to have these same artifacts during deployment. Import-Package is cumbersome to maintain by hand and basically requires tools like bnd. Besides, the whole world seems to use the Require-Bundle model (maven, ivy, netbeans, ...). So, who would ever want to use Import-Package?

Well, have you ever had chewing gum in your hair?

The harder you try to remove it, the more entangled it gets? This is what Require-Bundle gives you. Import-Package is more like Play-Doh. It is still better not to have it in your hair, but you can ask any kindergarten teacher what she prefers!

So, now we have the right mental image, what is the technical reason for this stickiness? I see two reasons:

Require-Bundle gives you much more than you actually use because your class files just do not refer to it, a key advantage of a static typed language. Now before you jump to the conclusion you do not care because it is free, realize that there is rent to pay for these unused parts. These unused parts have their own dependencies that need to be satisfied; so they've has just become your dependencies. Visualize the chewing gum in that strand of hair? Dependencies are annoying, and unnecessary dependencies are worse, but these kinds of tied dependencies can be a death knell if they turn out not to be compatible with yours when your code is combined with other bundles to create a system.

You depend on the whims of the bundle's author. He usually has no clue what parts of his bundle are used and is already busy working on the next version. He will make decisions based on his needs, not yours. It is therefore likely that the constituents in the bundle will change over time. How can you be sure that this migration happens in a way that is compatible with your bundles? Not only can the next version bring quite unnecessary and unexpected new dependencies, it might actually remove the actual packages you depend on. Unfortunately, your bundle resolves because the bundle you require is present, but its changes were just not compatible with your expectations so you get a Class Not Found Exception in the middle of shutting down this nuclear reactor.

The underlying reason for this problem is lack of cohesion. Cohesion is the measure of how much the code in a JAR is related. For example, utility packages or bundles often have quite low cohesion. For example, the Apache Derby SQL database contains a very interesting file system abstraction that is highly usable without Derby. That is, the cohesion between the core SQL database code and this abstraction is very low. The core Derby uses it, but it is not pertinent to being a SQL database. This is common practice because we programmers really dislike to get lots of puny little artifacts.

It is a fact of life that the cohesion in most JARs is quite low while the cohesion in packages is very high. As an example, the Apache Derby file abstraction is nicely packaged in a specification package and an implementation package. Why not extract it one day? Well, I am fine with this improvement and it would not affect any of my bundles at all. Import-Package does not care what bundle provides the package. If I had not warned them so often, I would feel sorry for all my colleagues that just had their bundles broken by a lone programmer somewhere in the world ...

Two very practical use cases that illustrate these problems. Eclipse, who heavily relies on Require-Bundle, needed to support SWT (the graphics library) on the embedded platform and on the SE platform. The embedded platform needed to be a subset of the SE platform. Unfortunately, all users of SWT had been using Require-Bundle. This made the simple solution, refactoring the SWT bundle in two bundles, impossible because it would break each and every Eclipse plugin. Bundles that used Import-Package would have been oblivious of this change.

The other use case is described in a recent blog: Catch-22 Logging with OSGI Frameworks . It is a rather long story but it boils down to that he could not combine two libraries due to unnecessary constraints caused by Require-Bundle with respect to logging. If you have the masochistic desire for the same sensation as having chewing gum in your hair, I recommend to read this blog.

Peter Kriens

Community Event

2008-06-05T13:19:00.002Z

You did not forget to register yet, did you?

See you next week in Berlin!

Peter Kriens

P.S. As an extra bonus we will host an SE Radio Interview! Shouldn't miss it if I were you!

Is 9903520300447984150353281023 Too Small?

2008-05-30T07:41:00.007Z

JSR 277's Stanley Ho published a rationale why (in the so far unpublished EDR2) JAva Modules (JAM) invent a brand new version scheme. A rationale that needs a lot of text. I could go in painstaking detail, but I think the rationale derails in one of the first paragraphs where the requirements are (implicitly) described. I highlighted the part that describes in what kind of situations you use the major, minor, micro, update, and qualifier version parts:

Major version number should be incremented for making changes that are not backward-compatible. The minor and the micro numbers should then be reset to zero and the update number omitted.
Minor version number should be incremented for making medium or minor changes where the software remains largely backward-compatible (although minor incompatibilities might be possible); the micro number should then be reset to zero and the update number omitted.
Micro version number should be incremented for changing implementation details where the software remains largely backward compatible (although minor incompatibilities might be possible); the update number should then be omitted.
Update version number should be incremented for adding bug fixes or performance improvements in a highly compatible fashion.
Qualifier should be changed when the build number or milestone is changed.

Can you spot the difference between minor and micro? I can't, and that is the reason that OSGi proposes the following convention: incompatible (major), backward compatible (minor), no API change/bugfix (micro), and builds or variations (qualifier). That is, our micro is Sun's update because Sun's minor and micro appear to have an identical purpose.

And ever thought about the concept of largely backward compatible? Isn't that something like a little bit pregnant?

There are always reason to improve on existing schemes, but any improvement should be balanced against the interests of the existing and future audiences. I am not stating that the OSGi version is the mother of all version schemes, we are as fallible as all of us. Maybe we were too rationalistic; we looked at a lot of schemes and saw how people were also looking for more room at the low-end of the version scheme and hardly ever incremented the first numbers. If you standardize there is always a tension between allowing as much freedom as possible but also minimizing the complexity of the implementations that are depending on the variations you offer. We chose simplicity.

Is the OSGi scheme usable? Well, we have no outstanding requirements or bugs in this area, nor were any proposed in JSR 291, while at the same time it is heavily used. Jason van Zyl, Maven, told me they were thinking of adopting the scheme as well. SpringSource converted almost 400 open source projects to bundles and they did not complain. Seems there is a lot of practical usage out there.

Wouldn't it be a lot more productive for all of us if Sun would just adopt the OSGi scheme? There is lots of work to do on the module API, why not reuse existing specifications where you can? And if the OSGi scheme has burning issues, why not report that as a bug or change request, after all Sun is a distinguished OSGi member.

With an infinite number of builds or variations, 9903520300447984150353281023 possible bug fixes (80.000 fixes per microsecond until the Sun implodes), 4611686014132420609 backward compatible changes, and 2147483647 incompatible releases, the OSGi spec seems to have enough room to breathe for mere mortals.

Peter Kriens

P.S. You did register for the OSGi Community Event in Berlin, June 10-11? Please do so, we have a very strong program prepared with lots of cool demos. Hope to see you there!

BUGs I Like!

2008-05-27T15:07:00.003Z

Last week I blackmailed Ken Gilmer of Buglabs into finally sending me an evaluation set. He had promised me this set for some time and I wanted him to show this kit on the OSGi Community Event on June 10-11 (You did register I hope?). Ken could not come due to other obligations so I promised to make a demo if he would send me an evaluation kit.

So yesterday Fedex delivered a nice red box at my doorstep! It is always nice when a company spends a bit of effort on designing the box; this package reminded me of my first TomTom, a company that also took care in making the unwrapping experience fun.

The content was a BUG with 4 modules. The BUG itself is the core computer that drives (hopefully) cheap modules. The core contains a small LCD, a minimal joystick, a push button and 4 LEDs that can also be pressed. It is powered through a separate (universal) power supply. Unfortunately, the also present small USB connector is not used to power the device; it is only there for the communications with the development computer. A slide-switch acts as on-off button.

There are 4 modules included:

LCD Display
GPS
Motion Detector
Camera

The modules insert into one of the 4 slots of the BUG, there are no specific positions nor are you limited to a single type per BUG. That is, if you want 4 LCD displays, go ahead and do it. I think the modules should be hot swappable but that did not work for me. Hope that will work in the future.

The development environment is based on (surprise!) Eclipse, called DragonFly. They told me to use Eclipse 3.3 (Europa) but I decided to live risky and use my current 3.4 Ganymede copy of Eclipse. At least this way I can only blame myself for some of the things that did not work. DragonFly replaces PDE but there are a number of similarities. Also, the manifest file is the driver for the generation. Overall it was very simple to write a new bundle when you are familiar with Eclipse. DragonFly nicely creates a complete project for you with the classpath setup correctly.

For debugging there is a pretty good emulator that reuses the same Java code and implements the services that abstract the devices. You just ask a project to "Debug As" and the select the Virtual Bug. Very smooth. Once the Virtual BUG is running it appears as a device in the list of Bugs. You can drag and drop your project to these devices and that will install them. This drag and drop interface works very well for device management.

The runtime is of course the most interesting part of this device. It runs the (now) open source PhoneME project. This is a CDC VM that supports most of the core Java SE libraries. On top of this VM, BUGLabs uses a Concierge framework. Concierge took the decision to not move to Release 4 but stick on Release 3 because of size reasons. This makes my feelings toward concierge slightly ambivalent. Release 4 added some very important changes that I hate to miss. I know that size matters, but the difference is not that dramatic with today's memory prices. Ok, I prefer OSGi over no OSGi, but staying behind on release level is very tricky. Lots of bundles now use the Bundle.getBundleContext() method, like Declarative Services, which is not part of R3. Also, the modularity layer got an important boost in R4. However, the biggest problem is compliance. Concierge was not tested for compliance and with the OSGi's current rules we cannot certify it anymore. During the development of R4 we seriously discussed the problem of the size the new features would take. Siemens, who had their own OSGi implementation for the vehicle market, even did a study to the impact. We very consciously decided that these new features were worth the added memory. I guess there is not much we can do about this, but it is a pity that we now have a split in OSGi, a split which we tried to avoid so hard. Anyway, better an OSGi R3 than no OSGi I guess ...

I decided to build a little application to get a feel for the whole thing. I decided to build an automatic camera that would take a picture whenever the GPS indicated that you had moved more than 50 meters. The track thus created would then be available over a web interface. Formatted as a KML file, it can be displayed in Google Earth. Simple.

Well, writing the application was quite smooth and the debugger was great. Getting it to run on the device was a bit harder because it just did not take pictures. It took some time to figure out how to use the Concierge shell. Even that did not help, there was something with the GPS service. Anyway, I guess with some help this will work before the community event. See if I can get my trip to Berlin recorded!

I wish I had had this device while I was working for Ericsson Research in 1998. It is absolutely perfect for trying out automation scenarios. I remember soldering an IO board, a movement sensor and a mobile phone to demonstrate a burglar alarm system based on sending a picture to the police. This scenario would have been a snap with the BUG.

So what are the application areas for the BUG? If you are one of those lucky people that can now and then wander from the virtual world into reality, then this is abslutely your device. You get a quite powerful computer that can be extended with a snap, ehh, 4 snaps. Obviously, if you still can get excited of what you can do with computers (like me), then it is just too much fun to miss. And if you give OSGi tutorials, well, this kind of device makes OSGi shine.

This is not to say they have arrived. BUG labs still has a lot of work carved out for them. The success will depend on amount of modules they can turn out in the coming years. I wish they had developed a breadboard module first so other companies and individuals could make modules. In earlier times I played a lot with the PIC processors from Microchip and it would be quite easy to interface these into the system. Also, a 1-Wire interface would be really fantastic. 1-wire devices are relatively cheap, and very easy to interface. For a few dollars you could extend a BUG with a temperature sensor, humidity sensor, unique ids, switches, cryptography tokens, etc. Remember the famous Java ring? Well, that was a 1-wire device.

The development environment and the documentation also require some work. As a true open source project they use a Wiki for their documentation so do not hesitate to help to make the BUG more popular. There is also a need for more BUG modules as well: Wifi, audio in/out, power switches, video, bar code readers, proximity detectors, RFID readers, fingerprint readers, Zigbee I/O, Bluetooth, infra-red, geiger-tellers, etc., etc.. There is a mighty amount of work awaiting them.

I really do hope other companies will pick up this concept and start to leverage it. Again, I wish I had had this device in 1998 when we were struggling to create demos!

Peter Kriens

P.S. But what on earth is the Teleporter module going to do???

Java Modularity

2008-05-13T09:32:00.011Z

Before JavaOne, there were clearly changes in the air with respect to JSR 277 Modules for Java and JSR 294 Superpackages. The announcement of Glassfish to support OSGi and the project Fuji (an ESB/JBI implementation based on OSGi) represent a significant mind-shift at Sun. We also saw Stanley Ho's interoperability paper on the JSR 277 mailing list. I therefore was happily surprised when the JSR 277 Java Modularity spec leads, Alex Buckley and Stanley Ho, when they asked me if I still wanted to join JSR 277.

This question posed a dilemma for me. Was I still interested? Two years ago when JSR 277 started I was rejected and I made a big fuzz about this, which obviously puts a moral burden on me now. Also, the changes around JSR 294 Superpackages definitely inclined me to join. JSR 294 was now folded back in JSR 277 and its course had changed 180 degrees. Instead of an overly redundant poorly thought out model Alex had kicked the bucket under the superpackages draft and proposed the much simpler concept of an access modifier called module. Though this idea had not been worked out in detail, it clearly was much, much, more feasible. Modules setup this way could be very usable in the OSGi world. My technical me would love to work on this aspect.

However, JSR 277 now consists of two good (Repository and Language modules) and one bad part (JAM). Just before JavaOne 2008 Stanley had published a brief OSGi interoperability document. This document was not only very thin on details, it also referred in many places to an invisible Early Draft #2 (EDR2). I asked the EG members BJ Hargrave and Richard Hall if they had seen this EDR2, but it was unknown to them as well. I had recently complained about the lack of discussions on the JSR 277 mailing list, so it seems that work had been going on stealthily.

During the JSR 277 presentations Alex and Stanley it became clear to me that a lot of work had happened behind the scenes; Without any communications on the mailing lists. For somebody in the audience it was absolutely not clear that almost everything that was said was never discussed in the EG. How could I participate in an EG if the only option would be to approve work done at Sun behind the curtains? How much change can there be if the unpublished EDR2 is presented at JavaOne as the stable result for JSR 277? Should I join an EG where most of the work was already done? How many fundamental changes can you achieve in this situation?

I wish it was differently. Bryan Atsatt did a lot of work to convince Sun to open up, and it worked better than one could have expected. So, there is a lot of moral pressure on me to compromise and accept. As several people said: "Sun will not give up their JAMs, so get over it." Now, it is in my nature to compromise; I prefer working together on technical details instead of fighting. However, I just turned 50 years old and I can recall too many painful events where I compromised against my technical instinct. Years later the forces behind the compromise were forgotten and the inferior technical design remained with my name attached to it. I really do not like that and this seems one of those cases.

So why do I not like JAMs? Well, who needs them? Do they provide something that OSGi not already provides? Alex and Stanley stressed the "fact" that JAMs are "deterministic" and "simple", the Siren's song of simplistic solutions. Only when you do not fully understand the problem is JAM an attractive solution. The truth is, JAM creates unnecessary complexity, the worst of all.

There is only one argument that makes sense for a special module layer for the VM: The boot classes. Obviously during the boot phase a full-blown OSGi framework is not feasible because OSGi is written in Java. However, JAM will have exactly the same problems loading java.lang.Object because the environment to run the framework is not loaded yet. How do you process annotations without having the Class class loaded? Write an annotation layer in C? I will not even go into initialization and custom module binding support. Clearly, a special solution is required to modularize the core language packages. This problem was also faced by Apache Harmony. They also did not want to support a full blown OSGi framework in their VM, but they did understand the importance of existing standards. Apache Harmony modularized the bootpath classes and provides them as OSGi bundles using the standardized metadata. Because headers are in the manifest and easy to parse, no complex redundancy is required as with JAM's annotation support. Using the same metadata makes a seamless transition possible from booting all the way to running one of the many application servers defined on top of OSGi. For management systems it is ideal if they only have to worry about a single format.

We are now at a crucial point in time. Though I'd love to participate in the discussions about modules in the Java language and the repositories, I cannot become responsible for a module system in Java that is creating this complexity for no rational reason. I do hope Sun will take this last step as well, and drop JAM in favor of the much simpler approach of using the OSGi metadata throughout the system. In the last months we have gotten so far! Please?

Peter Kriens

P.S. Did you register at the Community Event yet? There are only 190 places and I think they will be filled up. After Sun's embrace of OSGi, a lot more people got interested. You surely do not want to miss it. Last week we finalized the program it looks surprisingly strong.

P.P.S. Do you regularly read this blog? Do you know you can become an OSGi supporter? This free of charge and it helps us make it more clear to people that you support OSGi, sometimes numbers help, becoming a supporter does help. Of course if you can afford it, we'd love even better to have you as a full member or adopter. Take a look at the conditions. Appreciated!

SpringSource Application Platform + Bundle Repository

2008-05-06T12:44:00.004Z

When SpringSource announced their Application Platform and Bundle Repository I was very excited. I had been informed something was coming and picked up rumors from the grapevine.The bundle repository was in one word: stunning. I had been looking into providing such a service but had decided that I could not afford it, despite the fact that I think that many companies would really benefit from such a service. This bundle repository must have been a terrible job to do. Kudos!

The Spring Source Application Platform was, however, a bit of a shock. Looking through the documentation I found lots of headers that felt like OSGi but that I did not recognize: Import-Library, Import-Bundle, Application, etc. It looked like SpringSource had “improved” OSGi extensively. This was a bit of a shock, especially because they looked so much like standard OSGi headers.

I learned over the past 10 years that just adding a number of headers that solve a particular problem usually backfires. Something that sounds really good, does not have to be that good at all when you inspect it from a different perspective. The problem with standards is, is that it is very easy to add something but impossible to remove. You really have to get right as much as possible the first time; it is therefore better to err on the side of leaving things out. Due to the embedded origin of OSGi we have always been extremely careful to minimize the number of feature. Each additional feature require a lot of eyes and discussions and a very thorough analysis before it makes into the specification. Especially today, with a mature specification the feature interaction can be quite painful.

Do not get me wrong, as a practitioner I do know the importance of getting things to work and standards, by definition, are not always sufficient. I also know that a standardization process feels sometimes excruciatingly slow when you live in the trenches, and I love experiments.

However, Eclipse has the same problem, but they have always been a good citizen in this respect. They always added proprietary headers in their own name space, learned from their experiences, and then proposed a new standard. In all cases so far, the specification process was able to make the concept more general and fit better in the overall OSGi model. Having a separate name space was also a convenience for their customers. A specification process will undoubtedly change the semantics, requiring awkward constructs to distinguish between the SpringSource defined meanings and the OSGi defined meanings. I do realize that OSGi does not own the name space in the manifest, and I do regret heavily that we did no better scope our header names, however, this did not seem overly important 10 years ago.

Why would the standardized version be better? Because more eyes looked at it, from different perspectives (which sometimes can be quite bad for one's ego). One man's convenience is another man's complexity. This is why so far we have always focused on providing the primitives and not convenience functions because convenience is often in the eye of the beholder and can be provided during the development process in the IDE or run-time libraries.

The biggest danger of a standard like OSGi today is feature creep. The leanness of OSGi is one of its main attractions for many newcomers. It really took real hard work and untold man hours to make it this lean, believe me. Maybe the SpringSource headers are a significant improvement over the existing specification, maybe they turn out to be superfluous. Time will tell. I guess it is a sign of becoming more popular that companies try to stretch the specification in their world view. However, I do hope that most developers realize that restricting oneself to the standard may sometimes be a little bit less convenient, but it does provide one with a more options and no vendor lock-in down the line.

So sadly, my feelings are mixed. I love what they have done with the bundle repository, and also many parts of the S2AP look very impressive, Neil Bartlett's blog provides an overview that I can very much sympathize with. I know several Spring Sourcerers and they are very clever and capable people. SpringSource undeniably is meaning a lot for the OSGi organization and the adoption of the specifications. However, multiplying bundle headers feels more like coming from an apprentice than from the experts I know them to be.

Peter Kriens

P.S. On a more positive note. We are reaching the first deadline for registration of the OSGi Community event in Berlin, June 10-11. I am part of the program committee and I can ensure you that it is worth coming. We actually had to split the second day into two tracks because we had so many good sessions submitted. Looking at the number of submissions it will look like we will sell out the places we have this year. This is the premier OSGi event in Europe, we are really expecting all the key OSGi people to be there. Take advantage of the early registration fee and register today!

KISS Principle

2008-04-28T08:42:00.002Z

Wikipedia defines this principle as:

The KISS principle (acronym for "Keep It Simple, Stupid") states that design simplicity should be a key goal and unnecessary complexity avoided. It serves as a useful principle in a wide array of disciplines, such as software development, animation, photography, engineering, and strategic planning.

Today Google Alerts dumped me into two discussions where OSGi seems to be on the wrong side of the divide regarding the KISS principle. The first is an extensive debate in the dev@tomcat.apache.org mailing list, the second is the JSR 277 mailing list.

The Tomcat developers have been asked to osgify their popular web server. This resulted in a discussion that was very interesting to read. Several people seem to have a disdain for the OSGi zealots (which I guess is me) but there seems to be a general uneasiness with this request. However, my key trigger for this blog was that in this discussion I noticed that several people regard OSGi as overly complex and heavy. The JSR 277 discussion raised the same problem from another direction. From the JSR 291 interoperation discussion: "I wanted something in Java that was simpler and cleaner that OSGi-users could leverage if they wanted.". Is OSGi already ossified?

The key problem to decide this question is, as so often the case, requirements. Is Java complex? Hard to answer, if you want to write a "Hello World" it is humongous overkill. If you are writing a large scale web application it is nice to have a solid foundation. The key phrase in the definition of KISS is unnecessary complexity. Complexity is in the eye of the beholder.

First, the OSGi API is very small by any measure. There is only one mandatory package that only has 26 class files, including the security and exception classes. And you can already leverage OSGi without writing Java code. Can you call it intrusive when you can turn a JAR into a bundle by adding one or two manifest lines? All the complexity people complain about is optional. That said, I do agree that even optional things create some kind of conceptual complexity. So maybe there is a lot of cruft in the options?

Let us take a feature like import packages which is often considered hard. Most dependency mechanisms like Maven, Ivy, and also the original JSR 277 are based on the Require-Bundle concept. I.e. you declare a dependency on the wrapper.
Wrapper based dependency models are simple to understand for us humans. It would work actually very well in practice if JARs would be highly cohesive and not change their constituent packages over time. Which raises the question: Do you want the woman, or the dress?

We developers have discovered the wonders of refactoring and things do change. OSGi is a module system, it is about being able to change software while not breaking the deployments. Though import packages might be harder to understand initially than Require-Bundle, Import-Package does create more resilient systems because it minimizes coupling. That is why people that have used both systems usually decide to standardize on Import-Package (and bnd makes this quite easy).

Similar to the uses: directive. In the Tomcat discussion it was stated that the uses directives made the manifest unreadable. It was much nicer to create a manifest by hand, it looked better. I do agree from an aesthetic point of view but who reads a manifest? The manifest is intended to be read by the OSGi Framework. The uses directive is an incredible important concept to maintain classpace consistency when the same class can be loaded multiple times through different class loaders. If you have lived in a world where this model is supported, you understand that uses have a very high value because they minimize runtime problems.

I do not say that OSGi has no unnecessary complexity at all. Obviously, the specification has been evolving over almost ten years. We have to tried to keep it is simple as possible, but we are as fallible as all of us. However, requirements have always come first in OSGi and fulfilling requirements just forbid you to make things too simple because they force you to understand the problem.

Obviously the key question is, are many of these requirements highly specialized cases not useful for the majority. Or are they just solving problems that awaits anybody that ventures into the wonderful land of modularity?

Peter Kriens

First book on OSGi at JAX 2008!

2008-04-25T09:31:00.003Z

Last evening I got back from a very pleasant trip to JAX 2008 in Wiesbaden. I was asked to give the best practice and the OSGi modularity presentations. Rod Johnson had already warmed the audience for OSGi with the result that we had to close the room when people where sitting, crawling, and standing everywhere even though there were still 40-50 people standing outside. Good news for OSGi, but I had to give the presentation for a second time that evening at 21.15, where I was surprised that there were still 50-60 people present that late in the evening. The next day was even better because I got a much bigger room that was also filled. Feels good after gotten used to preaching to audiences of 10 people for so many years.

I also got handed a copy of the first official OSGi book in the world. The book, written by Gerd Wütherich, Matthias Lübken, Nils Hartmann, and Bernd Kolb.

This is a thorough book on OSGi (though Equinox biased) that I recommend to anyone who is using OSGi or wants to know more about it. You can buy it at d.punkt. I hope they translate it in English soon. I already read most of the text during my Christmas vacation to help them with a review. Gerd had already shown me on the first day the thank you section. They had been really nice, and made the thank you in Dutch! Great work! It is great to finally being able to point to a good OSGi book.

JAX is a great conference, very well organized. Well, there is so much to do that I'll keep it short. Though there is one observation, unrelated to OSGi, I can't help myself making. On the way back with Ryan Air I observed that people can be incredibly rude at jumping the queue, and not just the one that look like it, even elegantly looking older people showed an incredible rudeness. Though I admire Ryan Air for many things, their queue management is non existent, seeming to call out the worst behavior in people. I figured people behaved so badly so that to get the best seats in the plane. However, when I boarded at the end of the queue the emergency seats in the middle were still mostly free. So why are people so much in a hurry that they are jumping the queue? One day I have to debug this, but now back to the build!

Peter Kriens

P.S. Talking about books, you can now also order the specification (core + compendium) in printed form for a special price.

Is XML Part of the Problem?

2008-04-19T14:52:00.008Z

Last week I had a great time in Stockholm working two days at ComActivity AB. They are reworking their applications so they can take advantage of OSGi in their server applications. In this process they are investigating lots of technologies. And as is so often the case, one runs in modularity problems while using legacy libraries. Ok, they are not all legacy libraries, they are often brand new and provide valuable functionality, and so often for free as well. What a world!

However, integrating libraries into an OSGi based system is often fraud with class loader problems. In this case we took an in-depth look at the Cinnamon library from www.spicefactory.org. This library is a Remote Procedure Call (RPC) library for Flex based applications using the AMF protocol. AMF is an Adobe (publicized) standard for serializing ActionScript objects from Flash.

At the start of the visit I was shown a very nice demo of the application. Fast, powerful, and looking good despite the austere theme they were using. As can be expected, the application took advantage of the OSGi architecture. A top level application used the service registry to discover modules that implemented a specific application functionality. This module provided the Flex flash SWF file and provided the receiving end of the RPC interface. It was kind of cool to see how they made new bundles with bnd, deployed them with File Install and during the whole morning never restarted their Tomcat that was hosting their OSGi WAR file.

During the demo we went through the design by looking at the source code, modifying things, and then testing it. One thing I noticed was the large amount of configuration and specialized classes to make this application work with Cinnamon. I once write a tiny webrpc application and did not think it had to be that hard. (I am an optimist).

One of the key issues was configuration. The Cinnamon library had two types of configuration: Spring based and a private XML configuration format. The key problem is that these text based configuration files are forced to use dynamic class loading. If these classes must come from other bundles, then OSGi (rightfully) barks. The whole purpose of the OSGi modularity is to have controlled class loading, taking care of versions and other constraints. Doing Class.forName blissfully ignores such mundane details. A large part of the custom code was necessary to find the right classes at the right time.

Obviously, this was working. However, cruft frustrates me and I convinced them we should try to get rid of it. So at the last two hours of the second day we created a new project and started from scratch. No Spring XML and no Cinnamon XML. It was already at the end of the second day so we had little time but it turned out to be surprisingly simple to use the Cinnamon library from Java only, taking advantage of the facilities of the service registry (and that without even reading the manual or javadoc). Still, by getting rid of the XML configuration things became significantly easier. All of the class loading problems disappeared as by magic because from the Java code we had direct access to the right types without requiring dynamic loading.

By single-stepping through the Cinnamon library I also noticed how much code was spent on making the library flexible and adaptable. As so many libraries today, it was doing impressive class loader gymnastics to implement yet another plugin system. I think the library could be significantly simplified if it had been using OSGi to provide this plugin mechanism with bundles and services. It is sad to see how many time the same wheels are invented over and over again.

Anyway, I also remembered how I had a similar experience with Hibernate last year. In an effort to use Hibernate from an OSGi based system, I switched from the XML configuration to do the configuration in Java code. Also in that case, a lot of things that caused class loading issues just disappeared because classes could be manipulated as classes (with the proper versioning) and not as strings.

Different, but still related, the day before we had another experience where we significantly simplified life. They had used Spring to define global configuration for database connections. This is obviously highly specific information for each of their customers. With Spring they could provide this configuration through a bundle. However, if they wanted to change the configuration, they had to update this bundle. Looking closer at the configuration file it was clear that they were specifying several connections to databases. This looked like a clear case for the Configuration Admin's Managed Service Factory. We modified their pool manager to use the Configuration Admin by registering a Managed Service Factory service. This was only a few lines of code. They were already using File Install, and File Install makes it easy to set new configurations by placing a property file in the watched directory. It was kind of sad to see how small and readable the property file was in comparison with the XML configuration it replaced.

Now do not get me wrong. I am very impressed with the Spring, Cinnamon, and Hibernate libraries and I do see the value of centralizing configuration files. Maybe OSGi changes the rules a bit and this XML based model works better in monolithic applications. Lets face it, what is the difference of changing XML or Java code in Eclipse? However, last weeks experience makes me wonder if we should not be a little bit more critical before using XML to solve problems that are much cleaner and better solved in plain old Java?

Peter Kriens

JSR 294 Superpackages No More

2008-04-08T06:22:00.003Z

Alex Buckley posted a mail to the JSR 294 list with was more or less superpackages meeting kryptonite. Though in this case, Alex is in my opinion the good guy! He basically abandons all the unnecessary complexity of the earlier proposal, see my blog about this subject, and comes up with a guiding principle: ''simpler is better''.

The proposal definitely follows that principle. A source file is marked with the word ''module'', just like a package. And that is basically it.


package org.osgi.framework;
module org.osgi.core;

public class AdminPermission {
module void checkCertificate();
...
}

The compiler encodes the module membership in the class file and the VM can enforce that any member attached with the module keyword is not accessible from another member unless it is in the same module. I like it! This can actually be quite useful, also for some of the work we are currently doing in the OSGi Alliance concerning modularization. More detail is needed, but this significant simplification is absolutely in the right direction. There are some rough edges obviously, and I like to weigh my input on them.

Module membership should not be on the source but on the package because it is ''inherently'' safer. Alex argues in his blog that from a technical point of view the best place for module membership declaration is the package (package-info.java), but that from a ''moral'' point of view the class file is better because it allows one to introspect a class file and have all information together. Moral in this case should be seen as the Java philosophy, which he defines as focus on understandability by not letting the compiler infer information, but requiring that the programmer states his intentions clearly, even if this is redundant. I.e.

  
  List<List<List<CharSequece>>> list = new ArrayList<List<List<CharSequence>>>()

Is for me harder to read than:

  
  List<List<List<CharSequece>>> list = new ArrayList<>()

Because I can trust the compiler to do the right thing and I trust it more with these things than my reading capabilities.

I guess this was also attempted in the JLS, which is also highly redundant. Personally I think this decreases readability because you tend to glaze over sections you already have read 5 times in the previous page. However, this makes it too easy to miss small, highly important differences.

Anyway, I guess consistency is important (though I strongly believe Java would be more readable with ''less'' verbosity, less is more!) but I still very much prefer inherent safe models instead of tool enforced safe models. Why would you introduce a data structure that can be wrong if you can also do it in a way that does not allow those errors? Let me explain this a bit more.

Both Alex and I seem to agree wholeheartedly that packages are first class citizens and that you can not have modules criss-cross through packages. I.e. each package member must have the same module membership. That is, you can not have two classes in the same package that would not belong to the same module. So module membership is therefore not on the type but on its package. Placing the module membership in the source of the type just allows you to create invalid configurations when two sources in the same package would have a different module membership, which will just require more tools to fix them.

Putting the membership on the package would also be a good reason to overhaul the reflection methods of Package. Packages have always been second class citizens in the Java VM, for example you can not find its members. By placing the module membership on the package we have a perfectly good reason to fix these deficiencies.

Additionally, I do have some more wishes for the EDR2. I'll go through them shortly.

Module private interfaces - Modules should have module private interfaces. Currently, when you implement an interface, you are forced to make your method public. It must be possible to implement interfaces without forcing you to declare the implementation methods public.
Class Loaders - Modules should be able to cross class loaders (with the restriction that packages do not cross class loaders). This will multiple bundles to form one module.
Dynamic Membership - I do hope the API will allow dynamic module membership. That is, it should be possible to add and remove packages to/from a module in runtime.
Module-Export - I think there is a need to export a member or type to another module only. I.e. if module M1 imports M2, it should be possible for M2 to selectively exports members and types to an importer. This could be done with a keyword or an export on the module. for example:

        M1: Import-Module: M2
      public class Foo {
        void bar() {
          Ok ok = new Ok();           // no problem
          new Fail(); }       // not visible
      }

      M2:
      export module class Ok { }
      module class Fail { }

Scoping - Currently type references are only encoded with their name. It will be necessary to add the module name to a type reference, and while we are at it, it would be terrific if this then also encoded the version it was compiled against. This will allow the module system to calculate dependencies instead of relying on the old model of the serendiptity of the classpath. Obviously the module file should be able to change the hard references from versions (1.3.2) to a range (1.3,1.4).
Versions First class citizens - As show in the previous example, versions should become first class citizens. I do not think annotations are the right way to go for this.

The previous points are requirements, and not solutions. I do not envy Alex's task to come up with JLS changes. This is the same as open heart surgery.

From an OSGi perspective I think modules as proposed by Alex will be a welcome addition to
the language. I think it addresses some of our uses cases we are working on. Nice work!

Peter Kriens

Ghost Town JCP

2008-03-27T08:53:00.006Z

Is JSR 277 a ghost JSR or did the work move elsewhere? The last message on the JSR 277 mailing list was more than a month ago and before that traffic was virtually absent. Since June there is about a message per month on average, of which several are house keeping messages. Even Stanley Ho's blog has not seen a new entry since June. Did the discussions move to the Open JDK process? Looking at the modules-discuss archives, this does not seem to be the case. There is very little activity and the mails that I can read are about builds, and requests for information. The modules-dev list contains nightly build reports, the last one is from Sunday March 2. It therefore seems that work is going on, however, it seems hard to believe that there are no issues to discuss in this phase of the project. I assume that the people working in OpenJDK have another (closed?) mailing list where they discuss issues.

The problem with free organizations (as in free beer) the cost are carried by one party that in exchange demands special rights. For example, the OpenJDK projects is mostly paid for by Sun, just like the JCP. However, the consequence is that if you submit a source code, you must grant Sun a perpetual and full license, with the rights to sub license, for all your code.Though the OpenJDK is available under the GPL license, this choice of license is a Sun decision. You can not submit your code as GPL to the OpenJDK project, the GPL is a license granted by Sun to its users. You therefore have to sign the Sun Contribution Agreement. In this contract you oblige yourself to

Share your copyrights with Sun

License any patents bearing on your contributions to Sun

They key is now that Sun provides the OpenJDK to anyone under GPL and parts of it with the Classpath exception that allows you to run code on it. However, for many companies this is too restrictive because they have special patents they are not willing to share or can not share their source code for other reasons. Obviously, Sun is willing to provide the OpenJDK under an an alternate license for an appropriate fee, or not. It is their call.

If you do serious work in the JCP or OpenJDK, it is highly likely that the value of your donation to Sun significantly exceeds any membership fees of any open organizations.

When I started using Java in 1996 I had a lot of confidence in Sun because I always thought that the way they handled the standardization of NFS in an exemplary way. I still do believe that most people working for Sun have the best intentions. However, for the good of the community the shepherd should not be one of the sheep.

Zucht (Sigh in Dutch). Another blog side tracked about non-technical issues. I wish we could focus on the technical issues so we can show why and how the OSGi service platform is more ambitious and provides more and superior solutions for the modularity problem than JSR 277+294. It feels so sad that instead on working together on a suitable standard, Sun, against a lot of industry pressure, bifurcates the market for no technical reason. I do not claim the OSGi specs are perfect, there is work to do. However, they are mature, proven, have a large audience, and seem to provide more functionality than JSR 277 attempts to implement now (with a large learning curve ahead of them). Would such a situation have arisen when the Java community was shepherded in an more independent way?

Peter Kriens

P.S. I just learned that Neil Bartlett had the same idea and also posted a blog about the same subject ...

OSGi DevCon, Here We Still Are!

2008-03-19T22:50:00.006Z

Sorry for not updating yesterday, but it is kind of hectic here. It seems almost impossible to walk more than 100ft without being asked a question about OSGi. But hey! That is a lot better than desperately getting people's attention. But it makes it kind of hard to update a blog.

Anyway, yesterday we had a full program with lots of OSGi talks. Though I must admit the keynote was very entertaining, even without OSGi. The keynote was given by Dan Lyons (Fake Steve Jobs). Well, I let the pictures do the talking again.

BJ Hargrave (IBM) and Neil Bartlett in their Android for OSGi presentation

Christer Larsson and Eric Wistrand from Makewave working hard to get it to work.

Marcel Offermans and Karl Pauls from Luminis making sure BJ and Neil got it right.

Yesterday evening we had an OSGi BOF, with drinks. There were quite a lot of people and everybody seems to have a good time.

Well, today we had the Bug Labs presentation (cool hardware!) and the Virtual OSGi presentation from Jan Rellermeyer (which has recently become our second Invited Researcher next to Richard Hall).

Sprint, ProSyst, and IBM presented the Sprint Titan program, which uses the OSGi framework in the incarnation of JSR 232, as a cornerstone. These phones support an amazing amount of standards. The Titan platform will become a very interesting platform, the first that will allow you to write code that runs in binary form on a phone, desktop, server, and mainframe.

I did not get a chance to to visit all session, unfortunately. There is so much going on at the same time. One thing that was very nice today, btw, was that I had a meeting with some of the Jason van Zijl, Eugene Kuleshov, Carlos Sanchez, and Jeff McAffer and we discussed aligning the mapping of the Bundle Symbolic Name and version issues. Watch this space.

Tomorrow we will have the services versus extensions panel. This morning we met and it looks like it will be an interesting panel. Lots of people are interested in this subject.

Wish you were here!

Peter Kriens

OSGi DevCon, Here We Are

2008-03-18T13:53:00.002Z

We are getting ready for the second day of the conference. Yesterday, was (free!) tutorial day and the program was packed. I ran between tutorials to take pictures and see how things are going. The OSGi tutorials were very well attended and in several rooms there were not enough places. Anyway, let the pictures do the talking.

All pictures can be found at: Flickr.

So the real conference starts today with the keynote of Fake Steve Jobs. Then the short talks, Android, Services & DI & Extension points, Android, Building large applications and the BOF. And so much more where OSGi is involved. Last year, OSGi was a buzz, this year it feels like a roar!

So if you are at OSGi DevCon/EclipseCon do not forget to come to the BOF at 7.30 pm. It would be great to meet you all. Ok, off to breakfast and a great day ...

Peter Kriens

OSGi DevCon, Here We Come!

2008-03-17T13:21:00.006Z

Today, March 17, we are starting the OSGi DevCon. This year has a tremendous amount of events devoted to OSGi. The program has a special icon for OSGi events (thank you!) and it is littered over the schedule.

Today is tutorial day and for the first time the tutorials are free. Select your tutorials carefully, though it is likely many will be full. I wanted to give you an overview but the OSGi talks are marked so well on the program that I just give you the link to the Monday tutorials overview.

The first sessions have started and we have some photos fresh from the event.

Peter Kriens

OSGi DevCon/EclipseCon

2008-02-27T14:05:00.004Z

I hope you did register at OSGi DevCon/EclipseCon? It is only three weeks to this event and we would really like to see you there. I think this years program is exceptionally strong. For sure the OSGi DevCon program. It was really hard to choose this year from the large number of excellent proposals.

Do you know that this year the registration includes tutorials? This is the first time I see this in a conference and I think it is an excellent idea. It allows you to choose without having to worry about the financial consequences. The only problem is that there are so many interesting subjects that it is hard to choose.

The supposed dichotomy between services and extensions keeps people occupied. I hope we can finally decide this year when and where to use what. Not only do we have a tutorial, a presentation, but also a panel about this subject. To muddy the waters even further, a Jazz member will speak about a proposal that is supposed to merge the concepts. Whatever comes out of it, I am pretty sure it will be interesting.

On Tuesday we have a BOF at 19:30. The OSGi Alliance board has a meeting in the valley and many of them will be present to talk to you. This will be an excellent moment to tell them your ideas, issues, and maybe kudos. Oh yeah, and there will be drinks!

After the OSGi BOF, there will be a Distributed OSGi BOF that I really want to attend. Easy, it will be in the same room. My famous friend Jeff McAffer will have an Equinox/OSGi BOF on Wednesday so you will be able to find me there as well.

There is an overview page with the whole OSGi DevCon program.

I am really looking forward to see you this year at OSGi DevCon/EclipseCon. If there is only one conference you can go to this year, I urge you to pick this one. Don't wait too long with registering. There is a maximum and it is very likely they will reach this maximum this year.

See you all in Santa Clara!

Peter Kriens