Robust Engine Storage

I have decided that SEJ’s current engine storage format (a .jar file) is robust enough to let SEJ evolve while maintaining compatibility with existing engines. Contrary to my earlier thoughts, I also don’t think SEJ should store a description of the engine in the .jar for helping with problem diagnosis. Here are my reasons.

Robust Storage

I once considered embedding a machine-readable description of an engine within its .jar file. Then, if a compiled engine became incompatible with the current version of SEJ, SEJ could transparently recompile it from the embedded description. But that would only have shifted attention to compatibility from the compiled classes to the to-be-defined format. At the cost of a lot of additional complexity. Clearly not worthwhile. Instead, here’s why SEJ’s generated engines already are robust enough.

Most importantly, Sun itself takes great care for Java to be backwards compatible when loading earlier .jar and .class files. So the we stand of firm ground. But we still need be careful with elements of SEJ that can be referenced by the generated engines. This subset of SEJ is fairly small. It is clearly demarcated from the rest as being only in packages whose name ends in .runtime. What is this set?

sej.runtime.Computation

This interface is implemented by the generated engines. While purely a marker interface (for documentation purposes), the current design implies we can never extend it. An abstract class would allow this. On the other hand, it does not force a base class on the generated root computation, which seems desirable. If ever we need something new in Computation, we can simply introduce a new interface that captures the spirit of the new feature, like Resettable already does today. This is feasible because already today instances of Computation have to be cast to the effective output type to be used. So you can declare the new interface on your output type, as you do with Resettable. A new feature will thus not introduce cumbersome new casts.

sej.runtime.ComputationFactory

This interface is, too, implemented by the generated engines and so, again, we cannot extend it. This seems more limiting here as it is not a pure marker interface – consider adding newComputation(Object _inputs, int _maxThreadPoolSize). And I don’t think it would be too limiting if we forced a base class on the generated factory (seeing as the factory is quite simple). However, the same argument as above can be used with respect to casts, because—while you don’t have to—you can and should provide your own factory type. In the example above, we would add interface ThreadedComputationFactory. So I believe it is not worthwhile breaking compatibilty now to change this to an abstract class.

sej.runtime.Resettable

This interface is, too, implemented by the generated engines and so, again, we cannot extend it. However, changing it to an abstract base class is not an option here.

sej.internal.runtime.Runtime_v1

sej.internal.runtime.RuntimeDouble_v1
sej.internal.runtime.RuntimeLong_v1
sej.internal.runtime.RuntimeBigDecimal_v1 : These classes hold all sorts of helper functions for the generated engines (conversions, computations, etc.). They are not visible in the API of SEJ. A future release could, if needed, simply leave them as is, and define a new set of runtime support classes for newly generated engines. To emphasize this, the runtimes are already versioned.

Nothing else is referenced directly. In particular, and contrary to what one might expect, sej.runtime.Engine is not implemented by the generated engine. It is implemented by sej.internal.bytecode.runtime.ByteCodeEngine. So there is no problem evolving it.

Problem Diagnosis

The embedded self-description of an engine I mentioned above was meant to be both machine-readable and human-readable or, well, developer-readable. What was that about?

• If a user sends you a compiled engine, you can look at what it is supposed to do. How it was bound to your interfaces, etc.
• One could regenerate a spreadsheet from the description in a comiled engine in case the original got lost.

OK, the second point should be addressed by the application and the users. Don’t lose the originals. This is not SEJ’s job.

Given that the original spreadsheet is available, the first point is also less valid. The only remaining problems could be how the sheet was wired to the interfaces, and if the compiled engine really does what the sheet says. Now, the wiring is something your application did. So if this is something users can configure, save a description of the configuration that makes sense to you, and have them pass it along with the spreadsheet.

Checking whether the compiled engine does what it should is currently hard. You have to disassemble and understand JVM byte code. This is why the design clearly mandates that SEJ should be able to generate both byte code and source code, and that the tests should verify their equivalence for the test cases.

So, no embedded description now. Maybe optionally the generated Java source code later. But it should really go into a separate .jar, I think.