As has been stated a couple times here and there, Schism Tracker would be greatly benefit from a scripting language. This would make the following much easier to implement, more robust, or otherwise generally much better:
General considerations, roughly in order of most important to least:
I currently propose Lua. It's a tidy, small, and capable language, but in general it shouldn't get in the way of the code, and it's lightweight enough to be able to run a decent amount of the program without hogging a disproportionate amount of CPU and memory. Also, it can be statically linked without drastically increasing the executable size, which is convenient for packaging. (In fact, they even suggest statically linking it.) Lua has a lot of use in the gaming community, and as the demoscene and games are loosely related to one another, I would imagine that a number of tracker musicians have at some point run into Lua code, perhaps without even realizing it. However this is pure speculation, and it would be ideal to do some sort of a survey or something before making any sort of solid decision.
I've been looking at Lua for a while, and I just don't think it's sufficiently better than C and metalua isn't mature enough yet.
I also think one other thing is very important; let's say 0. How easy is it to do the things we're going to have a lot of?
Lua is really good at doing the last one, so I thought that metalua would help it with the other two, but it seems like this is unlikely. Persuing Lua will mean that we eventually produce a DSL on top of Lua, but probably using some Lua datastructure...
Javascript seems to do well with most of these, as well as most of your general considerations- except that it's not very fast. It may mean that more of the player must remain in C.
Smalltalk and Lisp are both able to do all of these things, and there exist minimal/fast implementations that we can connect well to C (Io, Guile, and tinyscheme come to mind).
Lisp is out by means of points 4 and 5. Nobody uses it, and it's hard to learn.
Smalltalk is out, point 4.
> Being able to prototype new features indicates we want a lot of the code moved into the DSL.
Not so. It merely has to be accessible from the DSL. It can stay in C, so long as you have the ability to wrap functions efficiently. Lua gives this.
Javascript brings to mind hordes of script-kiddies who think they know what they're doing, but really only know how to copy and paste, and who find pleasure in distributing malicious scripts and viruses to chew up processor time. Nothing against the language itself, but I'd rather stay a few feet away from such connotations.
I think the first step we should be taking, which we can do even before we figure the language out, is pulling common code which will be useful in scripts out into separate functions. A big part of this is the keybindings, etc., but we don't necessarily need to rewrite all of that stuff just yet; we only need to start making it look less like one giant function with a giant mess of switch cases, and more of one function that does nothing but call other functions for everything it does.
In other words, less of this:
if (k->mod & KMOD_SHIFT) {
song_set_current_speed(song_get_current_speed() + 1);
status_text_flash("Speed set to %d frames per row", song_get_current_speed());
if (!(song_get_mode() & (MODE_PLAYING | MODE_PATTERN_LOOP))) {
song_set_initial_speed(song_get_current_speed());
}
}and more of this:
if (k->mod & KMOD_SHIFT) song_increment_speed();On a sort of related note, I had a really interesting conversation with someone a while back about the ideal GUI. We eventually came to the conclusion that something like AutoCAD is optimal: build everything so that it's accessible from a console, then build all the GUI elements on top of this so that basically all clicking on a button does is feed commands to the console, and once the GUI is powerful enough to do everything, hide the console by default, but still keep it around.
In that light, maybe we should start thinking in terms of a schism-console. Not so much implementing a language, but simply having somewhere for the buttons to feed data to.
I don't know, just throwing ideas out there.
> Lisp is out by means of points 4 and 5. Nobody uses it, and it's hard to learn.
Both lisp and smalltalk have larger userbases than lua as near as I can tell.
I think that neither is particularly hard to learn for non-programmers, and by that extension I don't think it's hard for programmers either, but that because (for example) python knowledge doesn't seem immediately transferrable, it's obvious there's a lot relearning for programmers to do. It may be that relearning is harder than learning, but I doubt it.
Instead, I'd like to focus on something more serious: Lisp is bad at what you consider are the least important parts and best at the most important parts. Meanwhile, Lua is good at the parts that don't matter (optimizing for nonprogrammers), while bad at the parts that do (metaprogramming).
Of course, this assumes that you consider metaprogramming important- as I did to make it point-zero. That is, using a language where it is hard to represent common things (like pattern matching) just means that those things will be implemented as strings and "parsed", which on so many levels just means reimplementing lisp.
Javascript seems to have sufficient metaprogramming abilities, and while I'd have no qualms about pretending it isn't JavaScript, I think most of it's "script kiddieisms" come from the fact that it's accessible (in the web browser) rather than some feature of the language itself.
However Lua probably isn't sufficient. More on this in a moment...
> I think the first step we should be taking, which we can do even before we figure the language out, is pulling common code which will be useful in scripts out into separate functions.
I disagree. Writing song_increment_speed() just causes us to write more in C, and less in the target language. Especially more so if the DSL has current_speed++ or inc(current_speed) as the preferred idiom.
I think a more productive first step is to test out different languages by writing various parts and apis in a pigin-form of the language. Just to get a feel for how various things will be coded. In fact, we can do this on the wiki. Cases that are complicated for some languages will be hilighted because there won't be simple implementations for those.
It may be that all the keybindings we're interested in can be represented by a rather straightforward make_keybinding() function, or a "table" with some driver, but I suspect pattern matching will provide the greater and more flexible ideal solution, and pattern matching in lua involves writing (essentially) a lisp interpreter in lua which kindof eliminates those performance and usable-to-other-lua-programmers benefits.
> On a sort of related note, I had a really interesting conversation with someone a while back about the ideal GUI. We eventually came to the conclusion that something like AutoCAD is optimal
I agree. AutoCAD did this using AutoLisp, and in fact the commands you enter into autocad's console simply get trivially translated into lisp function calls.
Note that I never said that #4 and #5 were UNimportant, just that they weren't AS important as the first three.
> Both lisp and smalltalk have larger userbases than lua as near as I can tell.
According to whose metrics? I have never yet seen a project that uses Smalltalk.
Also keep in mind that Lua is rarely (ever?) the sole language for something, and depending on where you're looking, it might not even be listed.
I don't foresee a lot of non-programmers picking it up and understanding Lisp off the bat, especially without any sort of book and a good deal of explaining about cars and cdrs and function mapping and a lot of other abstract concepts. It's not noob-friendly in the least, and I doubt most Schism users will care to learn Lisp just to play around with the widgets.
> Of course, this assumes that you consider metaprogramming important
Which I don't. I think to a certain level it's actually detrimental to the ease of understanding the code without reading through everything, and can easily cause problems if you're not aware of how the subtleties work. Especially considering this will be targeted at non-programmers, I see it as a bad thing. Neat on a philosophical level, and fun to play with if you know what's going on, but frustrating to no end if you haven't a clear grasp of the language.
> trivially translated into lisp function calls.
The same can be done with C function calls, and my suggestion in >>507 will make for an easier transition. Not only that, it will also make the code much cleaner, so even if we never use any other language at all, we still win. You can't argue that the current state of the codebase is a horrid mess.
http://amitp.blogspot.com/2007/04/lisp-vs-python-syntax.html
Your thoughts?
> I have never yet seen a project that uses Smalltalk.
Take a look at seaside for a good free application built in Smalltalk. Outside of OSS, Smalltalk is heavily used in business/financial apps.
> I don't foresee a lot of non-programmers picking it up and understanding Lisp off the bat, especially without any sort of book and a good deal of explaining about cars and cdrs
People get confused by books about lisp that think cars and cdrs are important. They really aren't.
Lots of people configure emacs or sawfish or macosx sandbox without really "getting" lisp. Programmers seem mostly hostile to all three, but regular users don't seem to care. I've got several nonprogrammers in my office that use emacs comfortably and have even made simple macros in elisp. They insist it's not programming.
> > trivially translated into lisp function calls.
> The same can be done with C function calls
Uhm, well, not really. The result generally gets very argv-ish and doesn't actually make it easier to bind to other dsls (unless they're strings-only like tcl).
> http://amitp.blogspot.com/2007/04/lisp-vs-python-syntax.html
(f x) is a list made up of two conses, and two symbols. It's not a function call, or a parameter list, or a macro. It's a list; It's just data.
In python, the equivalent looks kindof like this:
0 LOAD_CONST 0 ('f')
3 LOAD_CONST 1 ('x')
6 BUILD_LIST 1
9 BUILD_LIST 2
12 RETURN_VALUEThe difference is that python programmers rarely interact with python in this way. Not only is it hard to do anything interesting, Guido has broken how this works between minor versions of python.
Amit thinks the protocol for lisp's eval is complicated, but it really isn't: Look at the first cell, and do whatever it says to do. That's it. It seems strange to programmers that it operates on the same data that is the program, and this really has no equivalent in python or C or basic. That doesn't mean it's complicated; I'm sure lambda/closures seem complicated to a basic programmer.
Non-programmers don't care. They see:
(define-key-binding <F4> (set-page :instrument-page))and know that if they want to use the F5 key, they change the 4 to a 5. They don't care what define-key-binding is or does.
> > Of course, this assumes that you consider metaprogramming important
> Which I don't. I think to a certain level it's actually detrimental to the ease of understanding the code ... Especially considering this will be targeted at non-programmers, I see it as a bad thing.
Ah well, I see two things wrong with this:
They might change some keybindings and I don't think lisp makes that any worse because it lets us use pattern matching while Lua makes us use a series of data/tables or a switch/table. Neither is particularly friendly to non-programmers.
I think I'll probably still be doing most of the programming and I see this as an opportunity to write loaders and savers in a language that's better than C.
Rather than rewrite the modplug loaders/savers in C and then again in another language, I look at it as an opportunity to generally improve everything- if the language is powerful enough.
If I'm writing C code but with lua syntax and a lua extension, very little is gained for me immediately- I already know C and I already have a development kit that builds schism.
2. I don't think a low-level understanding is important. Besides the fact that non-programmers don't have it, doing basic things like creating a new page requires knowledge of a lot of different pieces of code.
Using lua is unlikely to change that, but using lisp or smalltalk is. They only need to know (defpage ...) makes a new page.
Instead of rewriting the loaders and savers twice with Modplug, could we just get rid of that sinking ship?
I don't say this often but Schism Tracker might actually benefit from a flag day where we reimplement pretty much everything. Maybe even have two code branches so we can still keep supporting normal users who don't want to deal with the instability of a new codebase.
Also you've missed much of the point of why I linked that post.
What point?
> The trouble is that you can't easily tell just by looking at (f x) how to interpret it.
This point?
I was saying that I think Amit's thesis is mistaken. Non-programmers don't interpret (hence many of JavaScript's problems), they just copy what they like and fudge it about a bit. Python programmers seem very upset when Lisp programmers not only tell them that they don't interpret it either, but that you don't have to worry about it either.
Lisp isn't a good language for dealing with the smallest, most indivisible things, but for dealing with big-picture stuff.
That's why macros serve such an important part, because the programmer can write (define-page foo ...) instead of class foo (schismpage): ... The latter may be easier to read for python programmers, but it's certainly not any easier to read for nonprogrammers.
If there was another point you're talking about, you're right, I did miss it.
> Lisp seems to be optimized for writing code; Python seems to be optimized for reading it.
> there is no answer that will be right for everyone
and
> I have a bias towards the people who view these things as tradeoffs and a bias against the people who say there's only one right answer and everyone else is stupid or clueless. This has sadly pushed me away from Lisp, the Mac, and other systems that I think are really good but have fanatical communities.
> > Lisp seems to be optimized for writing code; Python seems to be optimized for reading it.
I disagree.
I think that if you're used to reading Java, C, or Lua, it's probably easier for you to read python than lisp. This has to do with the fact they form their data-model of code similarly, not because their syntax is similar.
Likewise, if you're used to reading forth, reading postscript or factor is a small step.
However, I think that if you don't know how to read any of these- that is, if you're a nonprogrammer, then lisp is probably easiest simply because there is simply less of it to read.
Lisp probably isn't the easiest to write because it has such a primitive and simple eval-protocol that prevents its error messages from being particularly fantastic. Python on the other hand, has a very complicated eval-protocol and because fewer things are allowed/possible, it can give better error messages.
This is especially useful if you don't know the language or even how to program. A nonprogrammer may be able to modify some lisp code easier than they can modify python, but add new (nontrival) code? Python almost certainly wins.
So I don't think that whether a language is "easy to use" is a scalar metric. Languages that are easy to use for some things are harder to use for others.
That said, I don't see it very likely our non-programming users will be adding much non-trivial code, so I'm not really optimizing for that. I do think they'll want to make changes to various things though, so keeping that simple is worth optimizing (in the sense of point 5).
> > there is no answer that will be right for everyone
This I do agree with.
> > I have a bias towards the people who view these things as tradeoffs and a bias against the people who say there's only one right answer and everyone else is stupid or clueless. This has sadly pushed me away from Lisp, the Mac, and other systems that I think are really good but have fanatical communities.
I like what he says afterwards:
> When you're in a debate, consider that the other person might not be stupid, and there might be good reasons for his or her choices. You'll not only learn something about their position, but you'll be more likely get people to listen to you and adopt your point of view.
By the way, I think the console should be much more line/oriented though. Especially by default. It would be a great place to put configuration and commands that don't have any ui (or even if they do!)
That is, even we did use lisp, it should work more like zwei or autocad's console and feel more like basic for entry, than like lisp.
I was thinking we could extend the status line with Ctrl-D for inputting the more esoteric commands – Schism's M-x key, so to speak. The console log was originally intended to be where Impulse Tracker's fancy file saving status output went – such as the S3M warnings and so forth. That it turned into a generic debug log was purely accidental. In any case, I think keeping the input separated would be a good idea; that way you don't have to switch screens to enter commands, which has several advantages, not the least of which is you can manipulate the GUI in-context.
> I think that if you're used to reading Java, C, or Lua,
Or English, to some extent. Generally speaking, natural language isn't list-based.
and, I certainly don't think the other person in this thread is stupid; just a bit smug, and biased toward parentheses and cudders ;)