24 replies to this topic

[Nyx]

1. My opinion (rant) on existing scripting languages...

I've been playing around with Ruby recently and found it had some rather neat aspects. But then again, there were things I didn't like about the language. The most obvious being a total disregard for convention. The language uses strange keywords, allows you to do the most elementary things in 5 different ways, and has some rather loose syntactic rules. More power to the programmer? Not really, just more confusion!

Anyways, the point is, I've tried numerous scripting languages, even made a simple one of my own (a LISP dialect and interpreter implementation)... But I can't say I've ever felt "satisfied" with one thus far. It seems to me that Python has an unclean syntax and encourages bad programming practices (everything public by default?). I'm also getting sick of the mislead idealism some of those languages put forward "If you don't program in the way we think you should, you're a bad programmer". Ruby has a ton of "learn by example" tutorials, but I haven't been able to find a proper reference manual (like that of PHP). It seems like every of these language says.

And lastly, one thing I found very lame is that, it seems people who make scripting languages present them as "opposed to" compiled languages. Python and Ruby are both interpreted, and they both basically tell you that you have to choose between flexibility OR speed, but you can't have both. Hence, programming a simple portable GUI app in python is doable, but programming a raytracer or a 3D FPS game... Not really, unless you code all of the CPU intensive code in an external C/C++/Java library, which, in my opinion, defeats the entire purpose of a scripting language!

Ah, did I say you could program a simple portable GUI app in Python? Well, no, that's not really true. It's not simple at all. Last time I tried, I had to deal with this Tcl/Tk external dependency that really made everything a pain... Because it doesn't ship with windows, and the python library apparently doesn't ship with python either.

So that's my opinion on current scripting languages... They're not very present. Python is currently the "dominant" one... And Python is cool for several things, but it's not really all that a scripting language can be. It didn't revolution the world of simple GUI apps... It's not very fast, and in the end, it's rather impractical to program with when your project becomes complex. Ruby is neat, but the only reason it's remotely popular is "Ruby on Rails"... RoR is a great product, but nobody would really want to program in Ruby. It's complicated, doesn't have such a good language reference, and well, if you show your code to anyone, they won't really know what's going on because of the obscure syntax, and probably won't be able to help you solve your problems.

My scripting language idea...

I'd like to combine the best qualities of existing languages. Why not have a scripting language with GC, that isn't strongly typed, and that has built-in support for unicode or utf-8 strings, vectors and hash maps... But that is also fast. I would like such a language to have a clean syntax, while offering the speed of bytecode compilation, and the power of a compiled language.

For example, because it would be compiled, such a language could have the ability to properly deal with circular references. Python simply tells you that you shouldn't have them, because it means you're a bad programmer, and then forces you to bastardize your code to deal with the issue. This is only an excuse for the fact that their interpreted linearly interpretes code. But a proper multipass compiler should be able to deal with this issue flawlessly, without forcing lame compromises on you!

There's also the issue of multithreading. AFAIK, most scripting languages don't support it, or support it in a limited way. Most programmers don't really use multithreading in scripting languages because the languages are limited to applications that don't require speed anyways. With a proper, fast scripting language, some quality MT support (such as that found in Java), could be implemented, and could actually be useful.

And finally... I also brought up the topic of GUI apps, and how cumbersome they can be in python. Well, it's not just a matter of being impractical... There's also the fact that using Tcl/Tk is going to limit you in how you make your GUI app look, visually speaking. My idea to solve this problem involves essentially DROPPING support for existing GUI frameworks. Instead, the interpreter could be made to support OpenGL, which is typically supported on every platform that has a graphical interface nowadays... And a custom, default GUI framework would be implemented in the scripting language on top of OpenGL (simple libs to get the mouse and keyboard input within a GUI environment would still be provided to make this possible). This sort of system could make it possible to have an interpreter that supports the same GUI facilities on every platform out of the box... In addition to supporting 3D graphics out of the box.

Now you might think I'm just some idealist myself. You might think that making a bytecode compiled scripting language that is as powerful as python is impossible, but I believe not. I have actually thought of ways to implement this in a fair level of detail. I know it's doable and I know it's not that difficult to do either. Of course, building such a language to have all the libraries anyone could possibly want is the tricky part... But if such a language had just file IO, networking and graphics, it would already have a huge potential. It could also be possible to make a lightweight interpreter that can be integrated in specialized applications (such as games).

Just to give you an idea of how I view the syntax of this language...

import foo
import bas

# Some class...
class Bar extends Foo

    attribute x
    attribute y

    method Bar()
    end

    method Bif()

        # Set some values in a hash map
        y = { "foo":5, "bar":7.5 }

        # Create a new object of type Bas
        bas = new Bas()

        # If x is not 0, do something
        if x != 0
            bas.doSomething()
        end

    end

end

I'm thinking of giving the language a simple name, such as "Go". To promote the language, once ready for release, I had an idea that it would be neat to have a short trailer movie (possibly all CG), with a touch of humor.

Anyways... Just wanted to share an idea. Feedback and language feature suggestions are welcome, I'm also willing to discuss ideas regarding the syntax.

And if anyone is interested in helping me in this project, or participating in the design, don't hesitate to propose your help and to send me a private message.

[dave_]

Nyx said:

Python and Ruby are both interpreted, and they both basically tell you that you have to choose between flexibility OR speed, but you can't have both. Hence, programming a simple portable GUI app in python is doable, but programming a raytracer or a 3D FPS game... Not really, unless you code all of the CPU intensive code in an external C/C++/Java library, which, in my opinion, defeats the entire purpose of a scripting language!

Ah, did I say you could program a simple portable GUI app in Python? Well, no, that's not really true. It's not simple at all. Last time I tried, I had to deal with this Tcl/Tk external dependency that really made everything a pain... Because it doesn't ship with windows, and the python library apparently doesn't ship with python either.

Have a look at somthing like IronPython
This is the CLR implementation of Python. It has vast improvements in performance over normal Python, and you can use all the .NET libraries by default.

If you do decide to make a language, it may be worth prototyping it at least using the CLR. This will allow you to use all the libraries for free and will mean that you dont have to worry about x86 optimisations.

[Nyx]

I have no intention of tying this up with the .net framework. The main reason being that developers would be very likely to use .net libraries, and thus, tie their application to the windows platform. Plus, I intend for my bytecode to be more oriented towards what my scripting language actually does (eg: higher level bytecode).

Not to mention that, I kind of doubt the portability of .net bytecode itself. Surely someone will point out that there is mono and that it runs X, Y and Z applications, but the fact remains that mono is a 3rd party implementation, and there will necessarily be platform differences which can make portability a pain. Even if there aren't none you know of at this time, I'm sure microsoft will find ways to introduce such issues. There's a reason it's called MSIL, and not ISOIL.

[Kenneth Gorking]

It's quite fun, I have been having almost the same thoughts about scripting languages, with the exception that I would like it to be strongly typed but with a generic type. I have been working on my own scripting language on the side for some time now, and I will soon begin implementing some of the features you mentioned. It is currently pretty basic right now, but it works and I use it in my engine for simple tasks. I am also able to map script-classes to native classes inside my engine, I even have a script that creates the material for rendering grass, and it also sets up the shaders and textures to use during rendering. It uses unicode for the strings, the compiler is multpass so you can safely inherit from classes not declared yet, and it is event based.

I also had an idea for an 'iterator operator', if you could call it that. It's basically just a simplified version of the for_each operator:

array<Bar> bars;

// ... add elements to array

bars:doSomething();	// call 'doSomething()' on every Bar in the array

bars:x += 5;		// Add 5 to x of every Bar in array

A thing I also miss in c++ is a more flexible switch/case statement that works with other types and comparisons:

float f = randf();

switch(f)

{

    equal 0.0:		// if(f == 0.0)

	DoThis();

	break;

    upto 0.2:		// if(f <= 0.2)

	DoThat();

	break;

    above 0.2:		// if(f > 0.2)

	GoHome();

	break;

}

I am curious to know how you plan to handle networking. Will you tag variables for transfer, or is left to the user?
What will the graphics interface look like?

[Nyx]

Your "iterator operator" sounds like vector operations to me. You could use something like that to apply an operation between multiple arrays or vectors (provided they are the same size) as well. It's a smart idea, although I personally find it would fit better in a language that natively supports lists.

As far as networking goes, I will make it similar to sockets (but simplified). It's imperative that the programmer be able to decide exactly what gets sent on a per-byte basis for compatibility with existing software to be possible. I plan on having stream classes with operators (like in C++). Although it could be possible to implement some "higher layer" functionality destined for interactive software where variables can be remotely updated and functions remotely called.

I can't tell you what the GUI would look like at this time for the simple reason that I haven't really given it that much thought. The answer is that the default GUI would probably be very plain, but if it were designed on top of OpenGL, it would be possible to extend it in any kind of way you might want to.

And well, for the switch statement... I think one thing that would be nice is a "handle" statement that can perform actions based on the type of the object being passed.

Perhaps we should unite our effort at some point to work on a common language, if you're interested.

[Kenneth Gorking]

Nyx said:

I can't tell you what the GUI would look like at this time for the simple reason that I haven't really given it that much thought. The answer is that the default GUI would probably be very plain, but if it were designed on top of OpenGL, it would be possible to extend it in any kind of way you might want to.

Well I was refering more to how the user would communicate with OpenGL. Just wondering if you plan to implement all the Gl functions (e.g. glVertex()) or create a render class for passing data to OpenGL. If the latter approach is chosen, one could also create wrapper classes for e.g. vertex and frame buffer objects, using the optimal method supported by the system or emulating it if not supported by the hardware. What method was used would be transparent to user, it would just work.

Nyx said:

And well, for the switch statement... I think one thing that would be nice is a "handle" statement that can perform actions based on the type of the object being passed..

Do you mean something like this? Because that would be cool...

method Bleeh(attribute a)

{

	switch( a.type )

	{

	case int:	// ...

	case vector:	// ...

	}

}

Nyx said:

Perhaps we should unite our effort at some point to work on a common language, if you're interested.

I am interested. :)
I need to do some research on untyped languages though, I haven't really used them that much.

[Nyx]

Kenneth Gorking said:

Well I was refering more to how the user would communicate with OpenGL. Just wondering if you plan to implement all the Gl functions (e.g. glVertex()) or create a render class for passing data to OpenGL. If the latter approach is chosen, one could also create wrapper classes for e.g. vertex and frame buffer objects, using the optimal method supported by the system or emulating it if not supported by the hardware. What method was used would be transparent to user, it would just work.

I would tend to think that people in general would prefer to have a full OpenGL 1.4 and/or OpenGL 2.0 interface, which would essentially consist of a library with functions wrapping all the GL functions (and using the same name). This is easy to code for us, and people would like it because they would feel "at home", if they have any experience with OpenGL. That being said, having done a fair share of 3D programming of my own, I do tend to think that vector and matrix math classes tend to be lacking from most languages. So it might be nice to provide the users with something like that, and perhaps a higher level, simpler OpenGL wrapper, while giving them the choice to use that or not.

Quote

Do you mean something like this? Because that would be cool...

method Bleeh(attribute a)
{
	switch( a.type )
	{
	case int:	// ...
	case vector:	// ...
	}
}

Yes, I mean something like that, but with an even more straighforward syntax, like

handle a

    type vector
        // Handle vector type
    end

    type int
        // Handle int type
    end

end

Quote

I am interested. :)
I need to do some research on untyped languages though, I haven't really used them that much.

If you can try Python, its a pretty good example of an "untyped" language. I think the real term is weakly typed, because all programming languages necessarily have types to some extent... It's just that scripting languages often tend to do alot of implicit type casting so as to make them nearly invisible to the user (http://en.wikipedia....ki/Weak_typing).

I won't lie to you though, I haven't really devised this language in its entirety, and there are some issues which still need to be resolved in terms of implementation that could affect the way the language works. But it would be cool to have someone else work with me on the project, because I have a hard time trying to think of everything ahead of time just on my own!

By the way, I sent you a PM with my IM contact info if you want to chat.

[dave_]

Nyx said:

I have no intention of tying this up with the .net framework. The main reason being that developers would be very likely to use .net libraries, and thus, tie their application to the windows platform. Plus, I intend for my bytecode to be more oriented towards what my scripting language actually does (eg: higher level bytecode).

Not to mention that, I kind of doubt the portability of .net bytecode itself. Surely someone will point out that there is mono and that it runs X, Y and Z applications, but the fact remains that mono is a 3rd party implementation, and there will necessarily be platform differences which can make portability a pain. Even if there aren't none you know of at this time, I'm sure microsoft will find ways to introduce such issues. There's a reason it's called MSIL, and not ISOIL.

Although you've clearly got a high level of technical competence you would do well not to be so arrogant, you clearly don't know everything!
Its actually called CIL, here's the standards: ISO & ECMA

And note I suggested you use it for prototyping, so you can see if its feasible.


fscking students they think they know everything....

[Nyx]

dave_ said:

Although you've clearly got a high level of technical competence you would do well not to be so arrogant, you clearly don't know everything!
Its actually called CIL, here's the standards: ISO & ECMA

And note I suggested you use it for prototyping, so you can see if its feasible.


fscking students they think they know everything....

My point stands.

[dave_]

It sort of stands, but with more holes than a block of swiss cheese...

[Nyx]

dave_ said:

It sort of stands, but with more holes than a block of swiss cheese...

Do you have anything interesting to say about *scripting language design*?

[dave_]

Nyx said:

Do you have anything interesting to say about *scripting language design*?

I made very sensible suggestions.

• Designing a decent language is not an easy task. There are loads of languages out there, none of them 'perfect'.
  
• Creating a decent JIT VM is not an easy task. (Or if you're going native, an optimising compiler)

Why do you believe you can create both with limited resources better than anyone else out there?
A good approach when creating software is to solve one problem at a time. Divide and conquer. You can't kill these two birds with one stone.

Perhaps you could suggest why my idea of using the CLR (or another alternative) VM as a base for a prototype is a bad idea, now that I've blown all your arguments out the water?

[Nyx]

dave_ said:

I made very sensible suggestions.

• Designing a decent language is not an easy task. There are loads of languages out there, none of them 'perfect'.
  
• Creating a decent JIT VM is not an easy task. (Or if you're going native, an optimising compiler)

Why do you believe you can create both with limited resources better than anyone else out there?
A good approach when creating software is to solve one problem at a time. Divide and conquer. You can't kill these two birds with one stone.

Perhaps you could suggest why my idea of using the CLR (or another alternative) VM as a base for a prototype is a bad idea, now that I've blown all your arguments out the water?

Have you ever written a compiler, an interpreter or a VM? I actually have experience, having already created compilers, VMs and interpreters before, so I have a clue what I'm talking about.

Using the .net framework would actually complicate things alot, so its definitely not a quick way to prototype anything. I'm already working on the project with a friend and I'm quite sure we can get our own VM running fairly quickly (that means one or two weeks).

I don't know if I want it to be JIT compiled at all, because there are some things I want to allow in my VM that could be made difficult by this.

[juhnu]

Nyx, it wouldn't hurt to be a bit more polite and NOT to take every response as a personal attack.

[Razor]

juhnu said:

Nyx, it wouldn't hurt to be a bit more polite and NOT to take every response as a personal attack.

I think he's handled himself adequetely, dave's posts have been verging on personal attacks.

dave_ said:

Although you've clearly got a high level of technical competence you would do well not to be so arrogant, you clearly don't know everything!
...
fscking students they think they know everything....

dave_ said:

It sort of stands, but with more holes than a block of swiss cheese...

dave_ said:

...now that I've blown all your arguments out the water?

[juhnu]

Razor said:

I think he's handled himself adequetely, dave's posts have been verging on personal attacks.

It's not only this thread I was talking about, but I agree that not all comments haven't been so nice to him either. I've just this feeling that if he was a little bit less defensive it would help him to deliver the message and get better responses in this forum.


Edit:

Oh and Nyx indeed has some good points in his rant. Was interesting to read.

As dave_ suggested I would definitely implement/prototype this over CLI as well as I just don't see the point reinventing the wheel in every possible place. Desigining a good language alone is a big task alone without any burden of building and maintaining actual support libraries, which .NET would offer you free. It takes people years to build just these libraries so I think it's fair to ask why do you think implementing all this by yourself would be a better idea and also could you elaborate why would using the CLI complicate things?

[Nyx]

juhnu said:

could you elaborate why would using the CLI complicate things?

Because all it provides is low level bytecode. It's not adapted to our needs. What we would need to do is essentially implement another layer on top of it. We would need to implement a support environment for our language in that language, and compile our script in it at the same time, while its in fact a bit too low level for our needs.

That, and using the .net framework is bound to impose us some restrictions which we may not want to deal with.

Designing our own VM, on the other hand, allows us to do exactly what we want. We can tailor it for our needs rather easily, instead of having to adapt our code to someone else's.

[juhnu]

Have you checked out Nemerle(http://nemerle.org)? It's a strongly typed language, but because of the "type inference" it's simpler to write code compared to traditional strongly typed languages such as C++ or C# and it has bunch of other features worth checking as well, such as it's macro system or lambda function support. Might give you some ideas how to develop your own language further.

[Christian Weis]

Hello all,
I'm currently also developping my own scripting language because none of those scripting languages out there is REALLY suited for game develpoment. Except UnrealScript has inspired me a lot and let me explain why.
The way UnrealScript works is somewhat cool. You can define a (base-)class in c++ and derive that class in script code for adding new functionality. If you need speed implement that method in c++ and if you need flexibility and all the other cool features that UnrealScript has, implement it in script. Doing so you have the best things out of both languages.
You also don't need a JIT compiler for such a hubrid language, because you have c++ for efficiency reasons.
If you are interrested in UnrealScript you may have look at (http://unreal.epicga...nrealScript.htm) where Tim Sweeney also explains why an interpreted scripting language is "good enough"!

cu,
Chris

[dave_]

Nyx said:

Have you ever written a compiler, an interpreter or a VM? I actually have experience, having already created compilers, VMs and interpreters before, so I have a clue what I'm talking about.

Yes, Yes and Yes. Real experience.

So far, I've determinded that effeciency and not reinventing the wheel are low on the list of your priorities.

You want an interpretted language.

You haven't really described the language.
What is your starting point? The syntax looks sort of similar to Python.
What do you intend to add, and for what reason? So far all I've seen is that you want weak typing and type switches.
Are you going to take anything away?
Why not just create a dialect of Python? You dont have to use the VM from Python, but it would save you from having to write the entire grammar yourself.
I dont care about the syntax. But it must be consistent and readable. I dont care if you use 'end' or '{...}' to define scope. Its a non issue.

I'd also suggest that you need to ensure that the grammar needs to be written so you can use either LALR or LL. I'd go for LL if you want your parser to be simpler or LALR for ease of specification.

Oh and BTW when I was telling Nyx not to be arrogant, I was refering to all the other threads Nyx posted to.