Wanting some light reading for this evening, I decided to dig into the Closures for Java draft spec (homepage).
Oh. My. God.
I'm a huge fan of closures. They're elegant, simple, and easy to use. Their semantics are subtle, but not confusing. They allow for very concise implementation of otherwise complicated algorithms, trimming a vast number of "normal" control structures, particularly various looping constructs.
Java already has (anonymous) inner classes which provide similar semantics, though with a marginally clunkier interface. This mechanism is basically how closures in Groovy are implemented, with some compiler magic to do the heavy lifting.
If you go look through the Closures for Java spec, however, what you see is appalling. Closures are about simplicity and elegance, and the spec outlines exactly the opposite. Closures themselves are exactly what you'd expect, using the Groovy (or Ruby) syntax, though types are required for the arguments. But what you don't expect is the function type syntax, the closure conversion rules, generics support, etc. It's a mess. Check this example:
String name;
{int => long} transform;
The first line declares a variable of type "java.lang.String". The second declares a variable of type "function that takes one int argument and returns a long". Note that that is different from a variable declared as "{int => int}", of course. Even better, any closure that can be assigned to the 'transform' variable implicitly implements every single-method interface whose method accepts an int and returns a long. It's unclear to me whether the converse is true – I couldn't take any more dry spec-speak.
The spec is pretty short (less than nine pages), and if you do any Java, it's definitely worth perusing. After reading, you'll realize that anonymous inner classes are just fine, thank you very much, even with the requirement of having to declare the interfaces explicitly. With the amount of baggage they seem to be lugging into Java (mostly because of it's typing), all I can say is "yuck." Closures require a bit more dynamic an environment to be effective, I think, and Java is static. Totally static.
ActionScript 3 provides an interesting middle ground that provides the benefits but culls a lot of the garbage. In AS3 you declare functions with typed arguments and an explicit return type. However, every function is of type function, which means if I have an algorithm that requires an {T, T => int} closure (that's the Comparator interface, by the way), I can happily pass in any old function I want and the compiler won't care. I'll get an error at runtime, of course.
Much the dancing the spec details is to protect developers from this potential situation, and the rest is to deal with backwards compatibility an interoperability with older libraries. I understand that Java is a strongly typed language, and that the compiler is omnipotent, but c'mon. Backwards compatibility is also very useful, but again, to what extent. You can already thrash a generics-based library with reflection, because it's absolved from compiler checks. It seems like there should be a point where you recognize the fact that the language has very deliberately poked holes in it's own security structures, and realize that that means trying to create airtight compiler checking is pointless.
I guess that's why Groovy got created, though. Backwards compatibility is important for Groovy as well, but the compiler (javac, mind you, not groovyc) and all it's type checking wonder has basically been given the finger since everything is just an Object and duck-typed as needed. Quack! I'm a developer that likes transparency in my compiler. Compiler writers are supposed to be smart, they can figure out what I'm doing and tell me if it'll work or not. I shouldn't have to give them explicit instructions, at least not that explicit.
NB: Function pointers (a la CFML) are NOT closures. Closures bind to their definition scope, function pointers just point at a function with no bound scope. I've heard the argument that function pointers are equivalent/sufficient. They're not.
Actually, {int => long} is a specific interface type. I probably got the names wrong, but {int => long} transform gets desugared into java.lang.function.IL, which is an interface with a single method called invoke that takes an int as a parameter and returns a long. That interface in the prototype compiler is generated at compile time; ideally it will be generated by the JVM as array types are.
You cannot do {int => long} transform=new SomeOtherInterfaceThatLooksCompatible(){…}; I think this is bad, as it's not as general as it could be, but you seemed to think the opposite was bad.
The mess that you described is just the result of you never seeing closures in a useful form before. Groovy's Closure type is not generic, so you cannot specify that your method accepts a closure that takes an int and returns a long (and if you could, it'd be checked dynamically, wastefully). This spec's version is more useful than that.
But note that you don't have to use function types to use closures. You can just as easily use a closure to implement ActionListener as you can an equivalent function type, {ActionEvent => void}.
"Closures require a bit more dynamic an environment to be effective, I think, and Java is static. Totally static."
I suggest you look at Scala, Haskell, C#, F#, Clean, Ocaml, and Boost, just to see how well closures work in static typing. They work better than in dynamic typing. To be more technical, how Lisp does closures comes from the untyped lambda calculus. The guy who invented/discovered that, some years later, wrote the typed lambda calculus, which is what most if not all static versions of closures are based on. It's much more powerful.
I think that the closures proposal is overly complicated. It would be much simpler, and more powerful, if a short syntax for instances of anonymous inner classes was used. Closure aren't objects and this leads to all the complications, just make them objects and the problems go away. Whilst you are at you might as well add some type inference, like Scala, to reduce the general boiler plate. See http://www.artima.com/weblogs/viewpost.jsp?thread=182412
I'll get an error at runtime, of course.
This is exactly the thing we dont want in Java. I want static analysis (FindBugs), I want a tool that can tell whether my program will do the right thing before it actually runs (except for runtimeexceptions, etc).
Howard,
Give one example of where a short syntax for instances of anonymous classes is better than BGGA-style closures.
Closures are objects in the BGGA proposal – the JVM doesn't support anything else for passing non-primitive things around so it couldn't be implemented any other way.
I agree about adding type inference. C# lambdas use it well.
Hi Ricky,
The difference is that the closure has one this pointer, it points to the enclosing class. An inner class has two this pointers, one that points to the inherited class and one that points to the enclosing class. Therefore a closure is, in a strict mathematical sense, not in a my opinion sense, a subset of an inner class. There are a whole host of things you can do with an inner class that you can't do with a closure: inherit implementations, fields, more than one method, recursion, etc. Some examples are here: http://www.artima.com/weblogs/viewpost.jsp?thread=202004.
Turn the question round, what can a closure do that an inner class can't. You will soon see that it is a subset of an inner class.
Howard.
The original post talks about 'reading' but not 'coding' or 'prototyping'.
A prototype is available: try it out! Like generics, the syntax is harsh at first but it becomes clear. The BGGA spec is actually quite elegant in many ways, and very powerful. You may or may not agree on elegance but you'll have more credibility after giving it an honest test-drive.
Howard,
A closure has two this pointers in the implementation, but you only see one from code. I'm not sure why you'd need a pointer to an object that has nothing more than a single method, from within that method, other than for recursion, which I'll soon address.
Saying that closures are a subset of inner classes is akin to saying that for loops are a subset of gotos. While it's technically true, it's not a very meaningful thing to say.
"There are a whole host of things you can do with an inner class that you can't do with a closure: inherit implementations"
It's a single method/function – how would it benefit from inheritance?
.."fields" – writing a closure that has a field is the same as writing a closure that uses a local variable from the enclosing scope, in terms of what you can do with it.
.."more than one method" – sure, you're talking about writing a class here, not about a block of code. Cars aren't appropriate for crossing bodies of water.
.."recursion". I once showed this Java code on a Java IRC channel, only to be told it wasn't Java:
System.out.println(new Object(){public int fac(int x) { return x<2 ? x : x*fac(x-1); } }.fac(5));
In other words, the closest you can do to a recursive lambda in Java is code that many people won't understand anyway. I think you can write a combinator to write a recursive lambda using the BGGA prototype, but I haven't quite understood how yet. The usual combinators you can use in lisp don't translate to typed languages particularly well.
lol and perhaps you rediscover soon functors , lambda calculus and strong typing can reduce many of your complexity problems mdr with strong rules for coding ,that xml can be easily map top functunal languages … you trying to add an extender pattern with functional languages facilities in j7 ;)
jfeven aka java fonctors 7even
ps : Now i know why i hate java!proof of program !!!
Ricky, Suppose you want to implement an AbstractAction for an event, easy with an inner class hard with a closure. Putting the field in the outer scope isn't ideal, you increase the visibility and allow errors. Plenty of things that have one public entry point have numerous private methods, easy with an inner class hard with a closure. The list goes on. Anyway, turn the question round – what is better about a closure? Howard.
Howard,
That is a fault with the AbstractAction type. It should be a one-method interface, or should be renamed to NotAbstractEnough :). You can easily create a one-method interface and a wrapper around AbstractAction without losing anything.
I agree about putting the field in the outer scope, but I'd rather not have the field if I can manage that.
You're not getting the point though. Closures are useful where anonymous classes are ridiculous. They are not a complete replacement.
"What is better about a closure?" Readability and 'wrapability'. Wrapability is what Gafter was talking about with his mentions of Tennent's Correspondence Principle. code should have the same meaning as { => code}.invoke(), in all cases. Wrapping code into an anonymous class makes it have a different meaning than it had before, making you have to do awkward things. Consider changing:
for (String s: new String[]{"hello", "world"})
if (s.startsWith("h"))
return s;
into a call to a forEach method that takes a closure/anonymous class. Using an anonymous class, and a simple method signature for forEach (void forEach(Action action);), the caller would have to throw an exception from inside the anonymous class and catch it from outside to achieve the early return. It's a similar story with break, continue, this and writing to local variables from the enclosing scope.
If you have a problem with forEach as an example, because there's already foreach in the language, think of another example.
Ricky,
Re. AbstractAction: I disagree, there are plenty of useful partial implementations that have more than one method that is potentially overridable.
Re. usefulness of closures: I can't think of an example where you couldn't use an inner class with short syntax where you can use a closure. The keys are short syntax, write to locals, and possibly non-local return, break, and continue – see below. I have nothing against improving inner classes with short syntax etc.
Re. Tennant: The closures proposal breaks Tennant, consider:
int f() { return 1; }
now lets wrap it:
int f() {
{ () => return 1; }.invoke();
// Syntax error here – doesn't return an int
}
Oops syntax error before the final }, since it doesn't return an int. Tennant uses this argument to argue against the return, break, and continue statements in his book. To sum up, Tennant himself rejects the return statement as being compatible with his principle. Hence closures don't give you Tennant's principle.
Re. non-local return: If you want this feature just add it to the language in general including inner classes, this way the whole language remains consistent – not different behaviour inside a closure to the rest. Using the BGGA syntax for an inner class with a named return:
String f() {
forEach(
new String[] { "hello", "world" },
{ (String s) => if ( s.startsWith( "h" ) ) { f.return s; } }
);
}
As you can see, exactly the same with an inner class as with a closure using named return. As I said before I think Neal is considering named returns for BGGA since the current syntax for the multiple types of return is confusing and hence I used named return above. I should spell this out; if you decide that non-local return is a good idea then add it to the whole language, not just the latest bolt on goody. Be consistent.
"I can't think of an example where you couldn't use an inner class with short syntax where you can use a closure."
Anywhere where you return, use this, write to local variables from the enclosing scope, continue or break is awkward to use an inner class with or without short syntax.
Anywhere where you don't care to see the types involved because they aren't interesting is annoying to use an inner class. list.map({ int x => x*2 }) is clearer than list.map(new IntToIntFunction(){ public int invoke(int x) { return x*2; } } ). Note that clear doesn't mean 'more explicit'.
The problem you mentioned is about tracing whether code returns or not, and isn't fundamentally about the syntax. If the closure { => return 5; } were tagged by the compiler as always returning, then the method that invokes it were detected to always invoke it, it would be possible to guarantee that the return happens, and the compile error you mentioned would disappear. In practice a big use case of closures is to pass them to other methods, so you'd lose this information quite quickly.
I look forward to seeing a real case where this is a problem.
"Re. non-local return: If you want this feature just add it to the language in general including inner classes, this way the whole language remains consistent – not different behaviour inside a closure to the rest."
It's not different behaviour inside a closure to the rest, it's identical. Return returns from the method it's used in. This does not change when it's inside a closure as a closure is not a method.
Barney, sorry if this conversation is considered noise.
Ricky,
I, like you, am not convinced that you can do a general escape analysis to track that { [closure] }.invoke() never returns. You can probably do simple cases and Neal has added some support with type Nothing.
Your map example could be an inner class, suppose that map was:
public interface List {
public List map( Method mapper );
public List forEach( ( Method mapper ); // see below for forEach comments
…
}
Then why can't:
list.map( { (int x) => return x * 2; } );
be an inner class?
Also the current inner classes interpretation of return can be used like a yield statement in Scala, which is nice. EG:
List strings = asList( "Hello", "World" );
List hStrings = strings.forEach( { (String s) =>
if( s.beginsWith( "H" ) ) { return s; }
return null;
} );
My forEach, in List above, only puts non-null items into its returned list, i.e. return yields the result. The current control structures in Java are a bit odd in that apart from :? they don't return a value (straight from C). This isn't usual in a functional language or even a more modern imperative language. Therefore I don't see much point in adding further support for a feature, control structures with no returned value, that is dated already.
I don't see that return as consistent in BGGA; the closure is much more like an inner class than anything else in the language, I think there will be enormous confusion when a closure is refactored into an inner class or vice versa. EG
String f() {
return new Object() {
public toString() {
String s = super.toString();
if ( s.endsWith( "A" ) ) { return "A"; }
return "Not an A";
} } + " Location";
}
and now refactor into a closure:
String f() {
return { () =>
String s = super.toString(); // not the same toString
if ( s.beginsWith( "A" ) ) { return "A"; } // Doesn't return to the same place
return "Not an A"; // Doesn't return to the same place
} + " Location"; // Never actually invoked, toString will be called on the wrapped closure!
}
This will be confusing. There are numerous places were the two examples above look like they do the same thing but don't!!! Sure I picked on toString, something that all objects have hence double the confusion. But you get the idea, this will not be easy to debug with a complicated example and the compiler won't help. I think people will tend to use the two, closures and inner classes, rather interchangeably. You could write 101 Puzzlers on closures alone!
The best you can say for the closure interpretation of return is that it muddies the water. I think that named returns will be a lot clearer. But we also need to ask the question, do we really need non-local return, break, and continue? Scala drops beak and continue completely, because of their poor interaction with closures. Also Scala is consistent about the behaviour of return throughout the language (it uses yield for example when two types or return are required).
Oh, right, so if Gafter renamed 'closures' to 'inner classes' but kept everything else the same, you'd be happy? I'm really not sure what you mean otherwise. By the way, the return there is unnecessary as are the parens around int x.
Agreed, but it's generally a bit odd to have a method that has side effects also return a value.
About refactoring a closure into an inner class or vice-versa, I expect that to be automated by IDEs. People using closures need to know that closures are blocks of code, not anonymous methods or inner classes. If they worked like anonymous methods or inner classes they would be less useful.
Perhaps, but they'd all be the same puzzler.
Languages should not force naming.
As long as the programmers likely to use closures are likely to think in terms of loops and side effects, return, break and continue are going to be useful. You could argue that closures should try to force them to think functionally instead, but that's not at all what Gafter is aiming for.
Scala drops break and continue because they are useless. Scala encourages composition, not messy for loops filled with ad-hoc renamed gotos like Java encourages. Scala has non-local returns. Returning within a closure in Scala is the same as returning from its enclosing method. I have obtained a NonLocalReturnException in Scala once, on purpose. More often, you don't even use the return keyword. I have 2268 lines of Scala, containing 16 mentions of 'return'.