Web 3d.0 Oooh! Teh shiney!

I am really impressed with Objective-J, and then I'm really impressed again with Cappuccino. They have made it onto my list of things that I know, which I assiduously trim down to the absolute minimum. I don't have a lot of room in my mind for struggling with how to do something, and I really want to get on with what I can do before my limited time and/or attention moves me onto the next thing.

My list is:

* Objective C
* OpenGL
* Cocoa
* GNUstep
* Nu
* Flash

and now

* Objective J
* Cappuccino

That's a list where the items have a great deal in common with each other. They share the same language, concepts, libraries and constructs. They can communicate with each other in the same dialect or in JSON.

This combination of tools allows the following platforms:

* Mac
* Windows
* Linux
* iPhone
* Browser

without needing to port code to other languages, libraries, runtimes, or the need for bridging. On each platform, the code has native access to libraries and the ability to drop down to other levels. It is not cross platform, but it is maximally cross platform in preserving code as much as possible. The differences needed attention anyway.

So if the above tools don't solve the problem, there are other escapes. You can shift into directions like:

* Cilk - parallel programming
* Objective C++ - peeking into Hell
* OpenCL - herds of turtles
* Javascript

Again, all of these are based on C (+ Objective). For knowing C and Smalltalk, that's a lot of results and options for only two simple languages! OK, Nu throws in a little bit of Lisp, and that's coming to the point of this post.

I started on C, then I moved to Ruby, then I saw Objective-C had the same programming model, so I bought a Mac. I then found Ruby's bridging to be awkward (and realise in hindsight that I was using Tim Burks' libraries!), moved onto Objective-C, and then moved to Nu a year ago.

I started chasing down this rabbit hole when I saw "Joy" by AAA+ in action. Not that one, the one that had a runtime Objective-C console. You could cut/paste from source code, fix it up, and paste it back. Objective-J is almost back there, and JSCocoa brings it to just a bridge away.

But it is a bridge. And Tim Burks, who created the Nu in my list, had already been down that path. Read http://programming.nu/rubycocoa-and-rubyobjc to see some of the problems with that approach. Then throw in the things that Lisp can do that Javascript can't.

That's actually not many things, as Javascript is almost Lisp when ignoring syntax. However, I want the additional escape of being able to define a DSL where needed. For example, if I need a bit of logic programming, then Nu lets me define Prolog in a few lines and run my problem through it.

(thankfully that need hasn't happened yet)

Nu also doesn't have much conceptual overhead as it closely models Objective-C. The additional rule is "use lots of ()'s instead of []'s", which was pretty much a given anyway! The visual difference is large, as one screen is full of []'s and the other one is full of ()'s. And that one is full of ;'s...

So while I wonder about automatic conversion between Nu and Objective-C for that Joy-like cut/pasteability, I do actually want to write my code in Ruby flavoured Lisp with Smalltalk infix! It sounds like a crazy combination but it does work!

I am wondering about a C parser in Nu, eg. something like:

(C
int align = (*ptr)->alignment;
if (align > 4) align = 4;
if ((align - 1) & bytes)
bytes = ALIGN(bytes, align);
bytes += STACK_ARG_SIZE((*ptr)->size);
)

or, to also nicely sugar syntax over the math side of things:

(C 45 * (5 + i) / 3)

I'd like to have a go at hooking it up to the ObjC runtime and Nu, so that I can achieve that Joy(!) of working with my Objective-C code in a runtime (with no bridge!). I figure that Nu is the minimum amount of code that makes the ObjC runtime workable as an interpreted environment.

I have no idea what I'm doing, but I'll find out. :)

The developers at 280North have come up with a desktop application in a browser:

http://280slides.com/

More interesting is the framework they developed to implement the Keynote clone:

http://objective-j.org/ (including Cappuccino when released)

While some have taken the view that Objective-C is more powerful than Javacript, the fact that the former has been implemented in the latter shows both languages are Turing complete and the question of "power" is misleading. It is more to do with how software is constructed, and the encouragement that specific languages implicitly lend towards particular approaches.

C++ encourages overloading type hierarchies with communication, thereby creating brittle structures that change when messages change. Smalltalk encourages a focus on messaging with a separate and independent mechanism, so that inheritance is not of such importance. Javascript is based on prototyping objects and comes even closer to Lisp-like freedom.

The trouble with too much freedom is similar to the trouble with C++. Given many ways of doing something, another problem is invoked when people use that freedom do things in many different ways! The situation to be solved then becomes less about how to do something, and more about how to put the different somethings together in a uniform way that can be debugged and maintained.

Features like "cut/paste", "undo", "drag-drop" are uniform features that go across a wide range of interfaces and objects, so they are difficult to implement in languages that have many ways to do things. C++ and Javascript emphasise functions, so a framework like Objective-J is needed to emphasise the more uniform approach of method calls instead.

There is another interesting library:

http://projects.gandreas.com/jskit/

The combination of Objective-J and JSKit creates the compelling possibility of using Objective-C flavoured Javascript as an interpretive layer over Cocoa, with that fast Squirrelfish engine powering the works. One language layer that can control both the browser and the desktop! For a moment there, I considered that and was tempted away from Nu.

Both approaches have a number of similarities, but Objective-J is still based on C (via Javascript) and Smalltalk. This means it is lacking Nu's ability to define domain specific languages and convenient syntax, as well as the ability to extend into logic programming and other conceptual domains. In short, it does not have "code is data, data is code".

As these kinds of language merges become more common (with the current focus on DSLs), this question will arise more often. I feel that Nu has leapt to the sweet spot of DSLs... the original Lisp.

Nu isn't something that I would have predicted to be so good, but then again I would never have come up with Objective-C either. I'm very much a homogeniser and I would have generalised C into something like a simpler version of C++... perhaps like Qt with its 'moc' extension to C++, except to C instead.

This is how Nu achieves operating on multicore and distributed architectures... by being close to the metal:

http://blog.neontology.com/posts/2008/03/02/objective-c-actors

http://blog.neontology.com/posts/2008/03/08/fun-with-distributed-objects

I didn't want to learn Objective-C, but I had to use the GUI API, which is Cocoa, which is Objective-C. Once I was in there, I could see the benefit of having object calls looking very different from C calls. It was something that I had to observe and use to understand, and explaining it to me wouldn't have done it.

When I'm playing around with a SQL select, or a Prolog match, or a Lisp program, my mental model of the computer is doing so much crap work that it makes me sick to think about it. That cute little bit of code in Lisp that maps and selects? So many thousands of lists died in the making of that output. So much memory used.

Great for one-off jobs or higher level control, but if you're going to be (((((list cdr) cdr) cdr) cdr) cdr)'ing down recursively building sub-lists and nonmutable copies per change to stay functional, then that's why today's computers aren't all that much faster than the Colecovision or Atari console game.

So having Objective-C code as the "still a bit uncomfortably inefficient with all that object stuff" compromise with C as the "insane but still human" hammer for all nails, slotting that up into a high level scripting layer language takes care of the rest. The problem is which high level language?

Ruby and Python and Haskell and the others all have the problem of having different concepts and semantics to their host language. Look at the bridging layers to write Ruby code in C, or the ways of translating Python variables to their less capable C equivalence.

In contrast, see how Nu has high level control with C level speed:

http://programming.nu/benwanu

Nu isn't just another language... it has a new idea. It is imitating Objective-C as closely as possible in Lisp, to the point of using keywords to imitate the Objective-C syntax. It is also Lisp, and can be programmed exactly like Lisp. Including having a command line.

The idea is to be able to drop code down to Objective-C for extra speed, or pull it up into Lisp for more control. Code that is as similar as possible makes this easier, and they have the same run-time

Nu is Lisp, but it is intended for people who are not Lisp programmers. Nu is still a full Lisp, but it is for programming in Objective-C.

The power of using Nu is being able to drop slower interpreted Nu code down to Objective-C or C or assembler as needed. If you program in Nu as though it is Lisp, it will be difficult to convert your Lisp code to Objective-C code. Much as you would write in Objective-C as much as possible and drop to C where needed... you write in Nu much like Objective-C, and push up where needed.

So if you need Lisp macros, you have them. If you need lambda and closures, you have them. If you need an interpreter, a console, a debugger, you have all of that too. The purpose of Nu is "C over lambda", and the style of Nu is to program in Objective-C and use the features and power of Lisp where needed. Out of the range of Lisp features, what are the ones that matter the most to the Nu programmer?

One of the reasons why I was interested in Nu was the knowledge that Prolog can be implemented in Scheme in just a few lines. I have a need for logic programming in a very tiny (as tiny as possible please) part of the problems I'm solving.

Using Nu meant that not only did I get other things I wanted, I also wouldn't have to load in a Prolog library and learn the bridge functions and how to translate values and so on. Along those lines, here is something interesting:

http://www.lambdassociates.org/prolog.htm

That is Prolog written in Qi. Qi runs on CLisp , CMU Lisp, SBCL (May 2007) and Allegro Common Lisp (June 2007). Those are presumably written in C...

It's C over lambda over C over lambda all the way down...

A very long time ago, we were swinging around in trees. We got killed occasionally, and starved every now and then. Sometimes the starving wiped out small tribes, but some of us got through.

But we don't really like to starve, so the solution to that is to plant food. Then defend it. Then harvest it. Then a whole range of other problems that are really just based around having food when we want it. Civilisation seemed like a good idea at the time, but it does have an awful lot of problems that come with it. How about solving the original problem?

What is the most direct and straightforward way of dealing with the need for food? The answer is to limit the growth of that need for food, so that any solutions remain valid in the longer term. This is a problem that has been identified in the past with attempted solutions, so what further information do we have now that could change the solution to one that does work?

Rich countries have problems with low birth rates, which is a local problem that would be a solution if it were globally distributed. The primary factor appears to be the long term care that a stable society can bring to the infirm or aged. Raising the standards of living and health care for everyone around the world could stabilise the population to a sustainable level.

The difficulty is that it takes an enormous amount of resources to provide the level of comfort that is needed to achieve lower birth rates. This has been achieved at the expense of those in disadvantaged countries, and various social constructs throughout history. Now the comfortable people have consumed their way to the limits of the planet.

Things are going to have to change. Driving to work and back every day is going to become too expensive. Telecommunication work is going to become the standard way of working. Short term online contracts will mostly replace permanent 9 to 5 work. Growing your own food becomes normal. Local people will become more important.

There are many ways we will be affected, but merely coping with the changes and reducing the standards of living could start another baby boom as people realise that they need children to look after them. The same reasons they have children when their society isn't rich enough to support them. Then there's nothing, and people die.

As we reduce our resource usage through better engineering, we need to improve the quality of the products and achieve more breakthroughs and design ways of addressing human needs efficiently so that we can all be comfortable from cradle to grave. This is what we're already trying to do, and there's nothing better than doing that.

The change from resource limited economies to unlimited resource economies is happening in software as the Internet allowed people around the world to collaborate as effectively as localised companies. Reducing the costs of everything so that anyone can have (almost) anything is the end result of turning everything into information.

Bring on the colour TVs and tiny laptops and other great stuff, using less than half a watt and manufactured from plant byproducts. And when the poorest in the poor countries all have laptops and broadband (and food), we're coming up with solutions for the problems we face now. And the more minds we have on the problems, the better solutions.

It's been a long and winding road that lead me to Nu, a very different destination and yet an inevitable one. Rather than the exuberance of darting from one language to another with the energy of youth, I'm an old fart who tries to do more with less.

I learned to write programs in BASIC. This damaged me, or saved me; for it created a deep appreciation of having an immediate response when I tell the computer to do something. I moved onto C after that, but always felt the lack of direct control compared to the edit-compile-run loop.

C++ did not win me over, and neither did Java. Perl was interpreted, and therefore lacking the speed I needed in places. Yet the pain of writing all application code in a lower level language was also an issue. Apple solved that problem by making Objective C a valid development environment.

Smalltalk is one of the great intellectual achievements in the field of programming languages. It was also rather slow. LISP is another great achievement, and also slow. And they both have the problem of all nice languages becoming ugly when optimised.

The best language is the one that provides a close mapping to the problem domain, yet brings things up to a level that is most comfortable to the human mind (respective to what is being done). The best language for a von Neumann hardware architecture is C.

The next layer of language engineering for the comfort of the human mind is to break problems up into objects. This metaphor is chosen by the efficiency of implementation, rather than other abstractions that can be expensive to achieve (eg, Prolog). The best language for objects is Smalltalk.

The problem with going between different layers of programming is the subtle variations caused by different things looking the same. So the solution is the initially unpleasant, yet actually correct and powerfully so, approach of combining languages that look as different as possible.

The merge of C and Smalltalk is Objective C, and the proof is in the results. Apple have used it to achieve what Microsoft could not. And yet even these amazing results doesn't stop my early BASIC brain damaged self from missing that immediate response to my direct control.

Domain Specific Languages (DSL) are becoming very popular right now. So why not add a scripting layer that gives me an extra level of control? F-Script was promising, but it was C + Smalltalk + Smalltalk, and I wanted more so that it would be worth that extra layer.

So the first time I saw code in Nu, I was amazed at how well the merge between the two worked. In the same way that Objective C bridges between two very different environments, Nu achieved the merge of Smalltalk into LISP, with C as the common ground.

Nu tries to be as much like Objective C as possible, and yet is still obviously LISP. That provides the context for mentally switching to a third language, and the very low cognitive overhead of using what you already know makes Nu an easy and very powerful addition.

Nu is a clever mathematical pun, with the best URL, http://programming.nu ... it means "C over lambda", and that is exactly what it means. Nu is a full LISP and has no problems implementing macros and lambda or anything else. However it is very different from pure LISP, as objects are the fundamental construct.

LISP is based around lists and values. Nu is based around lists and objects. Everything in Nu is an object. Even numbers are objects. And these objects aren't expensive abstractions that only work fast enough with small amounts of test data. They have the solidity of real metal behind them, written in C.

By selecting from a base of stable components, Nu is also a stable mature programming environment despite being so young. It is already possible to write enterprise applications with these tools because the significant portions are not new, and the language itself is not original.

See for yourself:

http://blog.neontology.com/posts/2008/02/24/scripting-nsoperations-with-nu

and LISP abilities:

http://blog.neontology.com/posts/2008/01/13/y-combinator

Yet this very thin layer allows me to drop code down to Objective C, or down to C, or down to assembler. It lets me call sideways to OpenGL or Apple's various Core libraries. I can whip out my copy of SICP and implement Prolog on top of my existing code. I can use macros to write real DSLs for any part of my code.

The results are stunning. Strawberries, chocolate and cream go together like C, Smalltalk and LISP. Add some slices of Apple and Cocoa on a Quartz and OpenGL platter and it's irresistible! The disadvantages of LISP are balanced out by the advantages of Smalltalk. The slowness of both is removed by the ability to drop to C.

And add the blue vein cheese of Linux... ok, that's taking the analogy a bit too far. :)

I 0 therefore I 1!