25 February, 2007

Smalltalk: An Application Language

When I started this project, the largest concern I had was that forcing anyone to use object-oriented programming would be extremely limiting or even detrimental to the final application. I've had some very bad experiences with OOP in the past: lots of hidden execution time and gross obfuscation of - and sometimes impossible to follow - code. OOP is a tool, and sometimes an extremely useful one. But, is it one that should be used exclusively?

My theory was that perhaps OOP wasn't the problem, but rather the implementations that I had used (namely C++ and its derivatives). Perhaps Smalltalk, the grand-daddy of OO languages had it right. Fast forward 7 months....

A typical program might begin with a few modules of "super" code - code that can do anything and everything. Then, as the program develops and molds, functionality is extracted and moved into new modules. As this continues, the code becomes easier to read, maintain, and extend. This is a good thing.

One of the many lessons learned (the hard way) by good programmers is knowing when to stop. Eventually a point will be reached when the returns are minimal, and if continued, the code may become so obfuscated that maintaining and extended (and performance) can be severely impacted.

The OO paradigm can have a way of clouding the judgment of otherwise excellent programmers with the allure of "perfect" code. The kind of code where everything is in it's place, only knows about what it needs to know about, all arrows point in one direction, and hides 90% of what all other code shouldn't be concerned with.

Sadly, that kind of code doesn't exist in the real world. It only exists on paper. I have yet to see any significantly large MFC program where the following macros (or similar functions) weren't declared globally:
#define MY_FRAME ((CMyMainFrame*)AfxGetMainWnd())
#define MY_VIEW ((CMyMainView*)MY_FRAME->GetActiveView())
#define MY_DOC ((CMyDoc*)MY_VIEW->GetDocument())
And then, little by little, the document class starts making assumptions about the view, and now someone adding functionality down the road is in a lot of trouble.

Let's just clear the air right now: arrows only pointing one direction are a great starting point, and should be maintained as long as possible, but someday your application needs to ship, bugs need to get fixed, and the end user is never going to care that your rendering engine just happens to know about this one, special level. The end user will care, however, if it can't render the level properly or efficiently.

Surely, hiding data and code is good, though, right? Well, do you want to just trust that the interface library you just plugged into your code base isn't using 10x as much memory as needed, leaking half of it every frame, and slowing down the rest of your application? I didn't think so. And neither do I.

Performance is always a concern - especially in games. And I get sick to my stomach when I purchase a new 2D, turn-based strategy game like Civilization IV at the store and find that it recommends:
  • Pentium 4 CPU with at least 1.8 GHz
  • 512 MB RAM
  • 128 MB graphics card
  • 1.7 GB free hard disk space
Seriously, we're talking about a 2D, turn-based game! Oh, and my laptop beats those specs, and it still runs slow! Maybe Firaxis shouldn't have just trusted some of the 3rd party code they used.

OOP is a tool, and can be very useful. But, it, like Mr. Worldly Wiseman, has a bad habit of "promising" the removal of a great burden from the programmer: having to write real code at the end of the day. For some reason, many programmers still think with that with the proper framework and a double-click, their application will just appear.

Now, the curtain lifts, and in walks Smalltalk...

- "What? What's gonna happen?"
- "Something wonderful!"

Let me state this unequivocally: Smalltalk is an application language! It's about creating applications; it's not about creating code. And there is a world of difference between the two. It's about removing the mundane barriers from the programmer so that he or she can focus on the real-deal, get it done, and move onto the next task.

Smalltalk dispenses with the "arrow" notion. (This is the term I use for those pretty code design diagrams that show object hierarchies and what systems know about, and how all the arrows should only point in one direction). That's left up to the programmer. It's definitely wise to design up front, and write the code as cleanly as possible for as long as possible. But, when it's a time to swing the hammer and make it work, Smalltalk doesn't make a programmer jump through hoops to get the job done.

Nothing is hidden. Nothing. Not even the compiler. Programmers like Ian Bartholomew can create profilers, that allow profiling every line of my application - including the core Smalltalk image - without having to write one extra line of code. Anyone can modify the core libraries to add, improve, or even completely remove functionality if and when it's needed.

Most languages that pride themselves on enabling the programmer to develop applications quickly are founded on the premise that programmers can't be trusted. Visual Basic and C# both jump to mind immediately.

These languages fall terribly short after the first 30 minutes of use. This is usually due to one, simple development truth: time constraints mean that prototypes have a nasty habit of turning into the final application. How often have you had to fix a bug or add a feature in an existing C# application at work, only to open it up and see this?
public partial class Form1 : public Form {
private void button1_Click(...) {
// ...
}
}
The default names given to forms, buttons, and other common controls haven't been updated. The original programmer was just trying out an idea that the boss then liked, but there wasn't time to go back and do it right. So, now you are wasting time trying to work around default code that is already difficult to follow, because you didn't code it to begin with.

Perhaps you even want to take [what should only amount to] 30 seconds and fix the problem by updating the name from button1 to SendButton, but soon give up when you realize that changing the name doesn't update the auto-generated function names, but did move it from a public place to a private place when it was renamed, and there are several places in code where button1 is directly referenced. Another 30 minutes wasted, and now enough code has changed that QA should probably have another round with it. More money wasted.

And that isn't even the worst case scenario. The worst case is when the button name was set originally, but it's text and implementation is now completely different from the original name given (the SendButton is actually the CheckSpelling button). Now it's harder to maintain, and a good programmer will feel morally compelled to fix it.

These are problems that seriously impede the development of applications. Applications that reduce overhead and iteration time, generate revenue, and give the programmer weekends with the family. An "application language" should free the programmer of such tedious, downright ridiculous, responsibilities, without treating the programmer as a miscreant who doesn't know how to properly manage code.

This is what Smalltalk does for me. I only write the code that matters. If I need to rename a class, all references everywhere else in my sources are automatically updated for me. If I change a method name, a browser window opens allowing me to see every caller so that I promptly fix them. These are just two of many, many, many features, all possible thanks to the reflective nature of Smalltalk, due in no small part, to it's object-oriented approach to problem solving.

I have learned to embrace object-oriented programming [in Smalltalk]. When it is implemented well (dare I say "properly?"), this paradigm doesn't just constitute another tool in the toolbox; it is the toolbox, from which everything else stems.

Since I started this endeavor to create a 2D game engine in Smalltalk, I've not once - wait, let me reiterate this - not once have I stopped, and started over again. I'm still working in the same image I started with in late August, 2006. All my designs began with prototypes (*cough* hacks *cough*) inside the main code base, and yet the code is clean. When they worked, it took minutes to reorganize and re-factor the code properly. And when they didn't, it took minutes to strip out bad code, and restore the old.

I've honestly never had more fun programming! Smalltalk has actually made programming more fun than it already was. It allows me to iterate the code so fast! The speed at which I can try an idea, see what's wrong, switch gears, or continue molding until it's right is so ridiculously fast, that it's fun. It's been stated that programming isn't fun. Problem solving and code design are fun. Programming in Smalltalk is just that - all the time. It's fun.

10 comments:

Sean said...

I'm with you on that.

I stumbled across Smalltalk a couple of years ago now. (2003 i think). It tainted me to the point where my day job (C#/C++) is painful.

Building my own business where I can Smalltalk for a living.

SM said...

What a great post. I am just starting my smalltalk journey, and am just begining to suspect this stuff. Confirmation is a wonderful thing.

Isaac Gouy said...

It's good that you're having fun programming!

Some of the things you wrote are a bit puzzling:

"anyone can modify the core libraries"
- that's true of any language implementation that includes source code
- what mechanism does Smalltalk provide to resolve multiple different versions of the core libraries?

"The original programmer was just trying out an idea that the boss then liked, but there wasn't time to go back and do it right."
I've seen similar problems in Smalltalk code - it's the programmer not the language. Have you ever worked on someone else's Smalltalk codebase?

"If I change a method name, a browser window opens allowing me to see every caller so that I promptly fix them."
You're talking about IDE functionality - have you used Eclipse or IntelliJIDEA ?
iirc Even VisualStudio Express provides a rename refactoring...


(The reason a Smalltalk browser window opens is that without type information we risk renaming all add method call sites, instead of just the blob>>add method call sites we intended to rename, so there's a manual step instead of a fully automated rename.)

Jeffrey Massung said...

Isaac,

It's definitely true that if you have the source to any language, you can modify core libraries. But, I have yet to run across one that makes it as easy as Smalltalk. Forth jumps to mind quickly, but it doesn't provide an image or packages (vocabularies aren't quite as flexible) to save modifications in. I don't think I understand your second question, though.

"I've seen similar problems in Smalltalk code - it's the programmer not the language. Have you ever worked on someone else's Smalltalk codebase?"

I agree. The programmer is always a huge (if not the primary) factor. My comment wasn't intended to imply a good programmer couldn't write great code in C# or VB. But, many times, code is quickly pieced together to try ideas. When those ideas work, it's good to know that molding the result into clean code is fast and easy. This is an area I feel Smalltalk excels in.

But, to answer the question: no, I haven't. In fact, that is one area I definitely want to explore given enough time. Large projects and source control in an image-based language is something I do worry about for the future.

"You're talking about IDE functionality..."

This is something I struggled with in the post. I originally had a paragraph to discuss this, but cut it. Certainly Visual Studio, Eclipse, and other IDEs are progressing and providing some excellent functionality. But the reflective capabilities of Smalltalk make the features I discussed feel far more... inherent.

There are many more features that I didn't cover - most dealing with runtime manipulation of code and updating of objects. These are features that no IDE can provide for just any language. But those features are about a different area of development than what this post was about.

Thanks for the comments! :-)

blake said...

"anyone can modify the core libraries"
- that's true of any language implementation that includes source code


Not to the same degree. When STers talk about core libraries, we're talking about, like, the Integer class. Or the Class and Object classes.

- what mechanism does Smalltalk provide to resolve multiple different versions of the core libraries?

That's an issue for sure.

You're talking about IDE functionality - have you used Eclipse or IntelliJIDEA ?
iirc Even VisualStudio Express provides a rename refactoring...


This is true. But Smalltalk isn't just the language; it's also the environment, which is something that most people don't get until they've used it for quiet a while.

There's a reason that (long after "commercial viability") many good programming ideas emerge from Smalltalk. "Refactoring" as we know it was pioneered in Smalltalk.

You've heard of "first class functions" and "first class objects". In Smalltalk, all users are "first class users". If you decide you need a tool or change to a tool, you can make it and your tool has the same status as an "official" tool.

Isaac Gouy said...

blake wrote: Not to the same degree. When STers talk about core libraries, we're talking about, like, the Integer class. Or the Class and Object classes.
It is true to the same degree when the source code for Integer and Class and Object is included!

blake wrote: "Refactoring" as we know it was pioneered in Smalltalk.
iirc William Opdyke's Ph.D. thesis on "Refactoring Object-Oriented Framework" was about refactoring C++ ...

blake wrote: all users are "first class users"
As long as they have access to the source code of the virtual machine... (Note: Smalltalk implementations differ one from another)

Isaac Gouy said...

Jeffrey Massung wrote ... if you have the source to any language, you can modify core libraries. But, I have yet to run across one that makes it as easy as Smalltalk.
I guess it depends on exactly what you mean by "as easy as Smalltalk" - but following on from blake's comment, simply downloading sun-jdk-1.6.0 gives the source code for Integer and Class and Object. (That didn't use to be true - things change.)


Jeffrey Massung wrote Certainly Visual Studio, Eclipse, and other IDEs are progressing and providing some excellent functionality. But the reflective capabilities of Smalltalk make the features I discussed feel far more... inherent.
That left me unsure whether or not you had actually used Eclipse or IntelliJIDEA.

Once you start talking about the benefits of Smalltalk as an IDE it becomes important to know what IDE you're comparing it to.

Once you say "all possible thanks to the reflective nature of Smalltalk" it becomes awkward not to praise the the reflective nature of Java (!!) when Java IDEs obviously provide class renaming that updates all references, and fully automated method renaming.

... many more features that I didn't cover
As I think you understood, that might be read as moving the goalposts.


Keep having fun, and keep thinking about what it is about Dolphin Smalltalk that makes it comfortable as a single-person exploratory programming environment.

tejón said...

"Large projects and source control in an image-based language is something I do worry about for the future."

Source control is something I do wonder a little about. Packages help a little, but it's not the same as having every class in its own file for convenient CVS (unless, of course, you put every class in its own package). I haven't had the time lately to really explore the community and see what solutions have been found, but a couple of possibilities have occurred to me...

First, maybe CVS is a moot point. The Swiki server uses the Smalltalk image to hold web pages, simultaneously accessable and editable by multiple users. What if instead of serving to web browsers, it were serving to class browsers?

Second, maybe the reason we're both still wondering how Smalltalk handles these issues is that they're much smaller than in most other languages, so solutions aren't shoved in your face as a mandatory part of the coding process (cough Java cough). You've nearly completed a complex Asteroids clone entirely from scratch, starting from zero knowledge of the language, in about half a year -- including the time it took to wrap DirectX.

Now: shift to the perspective of an experienced Smalltalk programmer with a sizable collection of classes (highly re-usable because refactoring is an idle passtime), and knowledge of other good packages the Smalltalk community can already offer. It seems to me that compared to other languages, 'large projects' might be vastly, even exponentially, smaller. I'd say "no pun intended," but I'm getting the idea that this is what the name means in the first place.

The point: could even "huge" projects perhaps have a small enough development team that they can just keep track of things at the package level?

Isaac Gouy said...

tejón wrote Source control is something I do wonder a little about.
As usual it depends on the particular Smalltalk implementation.

Back in the day, there was a very successful source code and configuration management system which worked with several different Smalltalk implementations - OTI Envy/Developer. It continued to be supported as part of IBM's VisualAge Smalltalk, and is still part of instantions VA Smalltalk - see EMSRV

Envy/Developer provides fine-grained method-level versioning - every time you accept a method edit, a new edition of that method is created in the shared code repository, which could be named and published and shared.


tejón wrote The point: could even "huge" projects perhaps have a small enough development team that they can just keep track of things at the package level?
Absolutely not!

tejón said...

On the revision control/concurrent coding topic, have you looked at Croquet?