The Object Teams Blog

IDE Innovation

Every now and then some folks report that the IDE they’re developing now supports this or that cool new feature. Sometimes I envy them for such progress - but more often than not I end up realizing that the OTDT already has that feature or something very similar. Is that just my personal bias (which certainly I have) or are we cheating in some way, or what?

There’s a little detail in the design of the OTDT that turns out to make such a difference as normally can only achieved by cheating: by the way how the OTDT extends and adapts the JDT it is like saying we’re starting the race not at the line saying “START” but at the other one saying “FINISH” and run on from there. While many projects define “JDT-like user experience” as their long-term goal, the OTDT basically has this since the first release. How come? The OTDT basically is the JDT, with adaptations.

There’s a fine point in the word basically. To tell the truth, every feature that the JDT supports for Java development is not automatically fully available in the OTDT for OT/J development. In fact most JDT features need adaptation to provide equal convenience for OT/J development. It’s just that all these adaptations can be brought into the system very very easily - thanks to the self-application of OT/Equinox. And now, here is actually an example of a JDT feature that lacked OT/J support - until yesterday:

Change Method Signature Refactoring

If you refactor mercilessly the “Change Method Signature” refactoring is certainly one of your friends. Add/rename/remove/reshuffle parameters of a method without (too easily) breaking existing code, cool. It knows about the connections from method invocation to method declaration and about overriding. That’s good enough for Java, but not good enough for OT/J since OT/J introduces method bindings (”callout” and “callin”) that create a wiring between methods of different objects. Obviously, if one of the methods being wired changes its signature so must the method binding.

Technically, the JDT implementation of that refactoring bailed out when it asked a parameter/argument for its parent in the AST and found neither a method declaration nor a method invocation. The JDT refactoring does not know about OT/J method bindings, so it just failed to update those.

After a little of coding this is what happens now when you apply “Change Method Signature” on a piece of OT/J code. Assume you have a plain Java class:

public class BaseClass {
	public void bm(int i, boolean b) {
 
	}
	void other() {
		bm(3, false);
	}
}

and a TeamClass whose contained RoleClass is bound to BaseClass:

1
2
3
4
5
6
7
8
9
10

public team class TeamClass {
	protected class RoleClass playedBy BaseClass {
 
		void rm(int i2, boolean b2) <- after void bm(int i, boolean b);
 
		private void rm(int i2, boolean b2) {
			System.out.println((b2?i2:-i2));
		}
	}
}

Here the right-hand side of the callin binding in line 4 refers to the normal method bm(int,boolean) defined in BaseClass.

What happens if you start messing around with the signature of bm?
Like:

I.e., we are adding a parameter str and also change the order of parameters (from i,b to b,str,i). I don’t have to tell you what this refactoring does to BaseClass, but here’s the preview of those changes affecting TeamClass:


(sorry the screenshot is a bit wide, you may have to click to really see).

The preview shows that the refactoring will do three things:

1. Update the right-hand side of the method binding.
2. Add a parameter mapping (the part starting with “with“) to ensure that the role method rm receives the arguments it needs, the way it needs them.
3. Do not update any part of the role implementation, because that’s what the parameter mapping is for: shield the role implementation from any outside changes.

Some words on these parameter mappings: each element like “b2 <- b” feeds a value from the right-hand side (representing things at the baseclass side) into the parameter at the left-hand side (representing the role). The list of parameter mappings is not ordered, which means further swapping of base side parameters requires no further action. And indeed the current implementation of the refactoring does not attempt to adjust an existing parameter mapping (which might be a quite complex task). If adjustments are required which the refactoring cannot perform automatically, it will inform the user that perhaps a parameter mapping may need manual adjustment.

The refactoring applies no AI to guess what the intended solution should look like, but it performs a number of obvious adaptations and gives note when these adaptations may not suffice and manual cleanup may be needed.

Implementation

Those who have read previous posts may (almost) know the kind of statistics that follows:

• 299 LOC implementation
• 315 LOC testcode
• 170 LOC testdata

I should really show one of these OT/J based implementation, one of these days. For this post I will just give you an Outline, literally:

Role class Processor is bound to the JDT’s ChangeSignatureProcessor, and the nested roles OccurrenceUpdate and MethodSpecUpdate are bound to two inner classes of ChangeSignatureProcessor, which shows how even class nesting at the base level can be mapped to the team & role level. Instances of the innermost roles will only ever come into being, if a Processor role has detected that it needs to work in order to handle OT/J specific code. Inside each role - apart from regular fields and methods - you see those green arrow-things, denoting method bindings between a role and its base. The highlighted binding to createOccurrenceUpdate is actually the initial entry into the logic of this module. Further down you see how the base behavior reshuffleElements is intercepted to additionally add parameter mappings if needed (and yes, I’m hiding the details of role MethodSpecUpdate, but there are no secrets inside, I just more methods and more method bindings).

Voilà, we indeed have a new refactoring for OT/J. I’ve been planning this one for a while but in the end it took me little more than a day

During the last week or so I modernized a part of the Object Teams Development Tooling (OTDT) that had been developed some 5 years ago: the type hierarchy for OT/J. I’ll mention the basic requirements for this engine in a minute. While most of the OTDT succeeds in reusing functionality from the JDT, the type hierarchy was implemented as a full replacement of the original. This is a pretty involved little machine, which took weeks and months to get right. It provides its logic to components like Refactoring and the Type Hierarchy View.

On the one hand this engine worked well for most uses, but over so many years we did not succeed to solve two remaining issues:

Give a faithful implementation for getSuperclass()

This is tricky because a role class in OT/J can have more than one superclass. Failing to implement this method we could not support the “traditional” mode of the hierarchy view that shows both the tree of subclasses of a focus type plus the path of superclasses up to Object (this upwards path relies on getSuperclass).

Support region based hierarchies

Here the type hierarchy is not only computed for supertypes and subtypes of one given focus type, but full inheritance structure is computed for a set of types (a “region”). This strategy is used by many JDT Refactorings, and thus we could not precisely adapt some of these for OT/J.

In analyzing this situation I had to weigh these issues:

• In its current state the implementation strategy was a show stopper for one mode of the type hierarchy view and for precise analysis in several refactorings.
• Adding a region based variant of our hierarchy implementation would mean to re-invent lots of stuff, both from the JDT and from our own development.
• All this seemed to suggest to discard our own implementation and start over from scratch.

Object Teams to the rescue: Let’s re-build Rome in ten days.

As mentioned in my previous post, the strength of Object Teams lies in building layers: each module sits in one layer, and integration between layers is given by declarative bindings:

Applying this to the issue at hand we now actually have three layers with quite different structures:

Java Model

The bottom layer is the Java model that implements the containment tree of Jave elements: A project contains source folders, containing packages, containing compilation units, containing types containing members. In this model each Java type is represented by an instance of IType

Java Type Hierarchy

This engine from the JDT maintains the graph of inheritance information as a second way for navigating between ITypes. Interestingly, this module pretty closely simulates what Object Teams does natively, I may come back to that in a later post.

Object Teams Type Hierarchy

As an extension of Java, OT/J naturally supports the normal inheritance using extends, but there is a second way how an inheritance link can be established: based on inheritance of the enclosing team:

team class EcoSystem {
   protected class Project { }
   protected class IDEProject extends Project { }
}
team class Eclipse extends EcoSystem {
   @Override
   protected class Project { }
   @Override
   protected class IDEProject extends Project { }
}

Here, Eclipse.Project is an implicit subclass of EcoSystem.Project simply because Eclipse is a subclass of EcoSystem and both classes have the same simple name Project. I will not go into motivation and consequences of this language design (that’ll be a separate post — which I actually promised many weeks ago).

Looking at the technical challenge we see that the implicit inheritance in OT/J adds a third layer, in which classes are connected in yet another graph.

Three Layers — Three Graphs

Looking at the IType representation of Eclipse.IDEProject we can ask three questions:

Each question is implemented in a different layer of the system. Things get a little complicated when asking a type for all its super types, which requires to collect the answers from both the JDT hierarchy layer and the OT hierarchy. Yet, the most tricky part was giving an implementation for getSuperclass().

An "Impossible" Requirement

There is a hidden assumption behind method getSuperclass() which is pervasive in large parts of the implementation, especially most refactorings:

When searching all methods that a type inherits from other types, looping over getSuperclass() until you reach Object will bring you to all the classes you need to consider, like so:

IType currentType = /* some init */;
while (currentType != null) {
   findMethods(currentType, /*some more arguments*/);
   currentType = hierarchy.getSuperclass(currentType);
}

There are lots and lots of places implemented using this pattern. But, how do you do that if a class has multiple superclasses?? I cannot change all the existing code to use recursive functions rather than this single loop!

Looking at Eclipse.IDEProject we have two direct superclasses: Eclipse.Project (normal inheritance, “extends”) and EcoSystem.IDEProject (OT/J implicit inheritance), which cannot both be answered by a single call to getSuperclass(). The programming language theory behind OT/J, however, has a simple answer: linearization. Thus, the superclasses of Eclipse.IDEProject are:

• Eclipse.IDEProject → EcoSystem.IDEProject → Eclipse.Project → EcoSystem.Project

… in this order. And this is how this shall be rendered in the hierarchy view:

The final callenge: what should this query answer:

        getSuperclass(ecoSystemIDEProject);

According to the above linearization we should answer: Eclipse.Project, but only if we are in the context of the superclass chain of Eclipse.IDEProject. Talking directly to EcoSystem.IDEProject we should get EcoSystem.Project! In other words: the function needs to be smarter than what it can derive from its arguments.

Layer Instances for each Situation

Let’s go back to the layer thing:

At the bottom you see the Java model (as rendered by the package explorer). In the top layer you see the OT/J type hierarchy (lets forget about the middle layer for now). Two essential concepts can be illustrated by this picture:

• Each layer is populated with objects and while each layer owns its objects, those objects connected with a red line between layers are almost the same, they represent the same concept.
• The top layer can be instantiated multiple times: for each focus type you create a new OT/J hierarchy instance, populated with a fresh set of objects.

It is the second bullet that resolves the “impossible” requirement: the objects within each layer instance are wired differently, implementing different traversals. Depending on the focus type, each layer may answer the getSuperclass(type) question differently, even for the same argument.

The first bullet answers how these layers are integrated into a system: Conceptually we are speaking about the same Java model elements (IType), but we superimpose different graph structure depending on our current context.

Inside the hierarchy layer, we actually do not handle IType instances directly, but we have roles that represent one given IType each. Those roles contain all the inheritance links needed for answering the various questions about inheritance relations (direct/indirect, explicit/implicit, super/sub).

A cool thing about Object Teams is, that having different sets of objects in different layers ( teams) doesn’t make the program more complex, because I can pass an object from one layer into methods of another layer and the language will quite automagically translate into the object that sits at the other end of that red line in the picture above. Although each layer has its own view, they “know” that they are basically talking about the same stuff (sounds like real life, doesn’t it?).

Summing up

OK, I haven’t shown any code of the new hierarchy implementation (yet), but here’s a sketch of before-vs.-after:

Code Size

The new implementation of the hierarchy engine has about half the size of the previous implementation (because it need not repeat anything that’s already implemented in the Java hierarchy).

Integration

The previous implementation had to be individually integrated into each client module that normally uses Java hierarchies and then should use an OT hierarchy instead. After the re-implementation, the OT hierarchy is transparently integrated such that no clients need to be adapted (accounting for even more code that could be discarded).

Linearization

Using the new implementation, getSuperclass() answers the correct, context sensitive linearization, as shown in the screenshot above, which the old implementation failed to solve.

Region based hierarchies

The old implementation was incompatible with building a hierarchy for a region. For the new implementation it doesn’t matter whether it’s built for a single focus type or for a region, so, many clients now work better without any additional efforts.

The previous implementation only scratched at the surface – literally worked around the actual issue (which is: the Java type hierarchy is not aware of OT/J implicit inheritance). The new solution solves the issue right at its core: the new team OTTypeHierarchies assists the original type hierarchy (such that its answers indeed respect OT/J’s implicit inheritance). By performing this adaptation at the issue’s core, the solution automatically radiates to all clients. So I expect that investing a few days in re-writing the implementation will pay off in no time. Especially, improving the (already strong) refactoring support for OT/J is now much, much easier.

Need I say, how much fun this re-write was?