So, would you believe it... I found some free time. Probably because everyone at home is ill, or sleeping but time none the less. So I decided to read through the current GM suggestions listand there's some interesting stuff on there. I suspect much of it wil have to wait for the rewrite, but we can still do a fair bit during the many updates to 8.x we hope to do in the future.
Types... Not for 8.x (or not fully that's for sure)
Values
we'd like to get ints/bools etc. in there, but not sure when. Enums are also a must at some point - I hate having to define constants in the menu system. Unicode is a long term goal for Game Maker, but I think it may well be too painful for 8.x.
Collections.
Actually, I don't really like any of the collections. I think it's one of the few areas where you really NEED "class" based operations. So instead of giving you a handle to something, you actually get the object back. This will be a hotly debated topic at YoYo, that's for sure. We don't want to break everyone's source, but at some point, you have to make a change. This may mean keeping the old ones and giving some new ones, then removing the older stuff in a later version. But at some point, I think the old "handle" system should go. Again.... will be a huge debate internally on this, and we'll then need to open that debate up to external devs as well.
Scope.
Yes. Need Scope. I'm not a huge fan of C++ style namespaces, but they work well on C#. That said... they aren't vital. So not sure, another one to discuss. Multiple inheritance...Mmmm...prob not, but who knows. def not for 8.x. As to uninitialised variables not being 0, well.... that's needed for beginners, there's no doubt. We are coming to the conclusion that there may be a new level though - Pro (or something), where things get really interesting! Time will tell on that one.
Scripts.
Default args would be better. I suspect we may well start allowing proper script definitions, then all this would be possible. But it would also allow us to keep the current method for beginners. But I prefer proper definitions as it means you get proper error checking when calling functions as it knows the number of args it needs. We can work it out I guess simply by checking how many args it uses IN the script, but it's cleaner and less error prone to define it. I don't think you need pass multiples args back. Again, if you had proper defined functions you could specify "outs", but other than that, you can currently just use globals and set them. It does the same thing.
Flow control - not for 8.x
Multi level break? Don't be silly.
Editor.
non-modal editors. Sooooooooooooooooooooo YES! I hate the horrible modal dialogs! Hate them!! Hate them!! Hate them!! Hate them!! Depending on time, This might get in to 8.1, however we need to discuss with the community to see if there's any valid reasons to keep them. Mark thinks there is... but couldn't remember, so we'll open it up for discussion soon.
Better window management. Yes. def. Tabs would be nice.
Multiple selection/editing of tree resources. Yes.
Dragging resources between projects. This would be nice... but no idea when.
Group duplication. Another nice but "when we have time..." one.
Portable Installation.. We'd have to look into that due to piracy issues.
Escape closes... yeah.
Debugging
Whole debugger needs redoing, but that'll probably be 9.x or so. You need the new scripting engine for this. Once thats in, then pretty much all this is possible, and will be done. Not sure about dynamic recompiling though. It's nice, but might be tricky... Probably between game "ticks" is the simplest way.
Code
I've said before. The code editor is reasonably good, but just needs a little update to make it very good. Most of these should be done at some point, with luck during 8.x life cycle. I'd remove the external editor, but more on this later....
Objects
Bypassing D&D - yes.
Inheritance graph. I guess. I never use them, but on others games it might have been handy to see, and I know others use them.
Show GML equivalent of D&D. We hope to go one better than that...
undo.. yeah... need a better undo system over all. But that's probably a 9.x thing.
Rooms
Instance stuff. Yes. The idea at some point in 8.x is to move to a more select and change system, rather than the current kinda mushed one. We will start making everything brushes, and that will let us do all manner of things. Group selection/editing, grabbing chunks from the map, painting with them and all the rest. You should also be able to set some instance data directly from the editor. rotation, scale, colour etc...
Zooming. Oh HELL yes. Will be there in 8.1, in fact... it's almost complete. We might post a video soon to show it working. It's a massive MASSIVE improvement.
Layers/tiles. Yes. In fact, I want to completely change the tiling/instance system to be more "playfield" based. I find this is much less error prone, and will let the graphics engine draw everything much, MUCH faster. Currently tiles are really just very cheap instances, and everything is drawn pretty slowly. But if you do a proper tile/layer system, then you can blast whole layers very quicly indeed. Rendering would speed up massively. Not sure if this will make it into 8.x as again, we will need to take care not to break everything.
Grid offset/colour - sure.
Sprite.
onion skinning. Yes. Might be 9.x
Selection/manipulation... not sure.
Curves. Probably.
default precise checking. If this is the pixel checking for game collision, then no. This is really REALLY slow... so you should be forced to think about it. If its something to do with the editor, then not sure. Preload. I think all preload stuff is going.
Triggers.
Not used them yet, so won't comment.
Files.
split gmks... More on this later....
Remove redundant compression. The file format will be changing. More on this later.
conditional compilation. Yes. Would be nice. Not sure when.
Editors
Sound editor. Perhaps...
Particle/effect. Yes, at some point. Preferably right inside the room editor. Prob a 9.x thing, but you never know...
Extensions
DLL access to in game resources. Yes, would be nice. But tricky. At the very least getting texture handles and a pointer to variable "values" would be good, but I don't know how possible this is yet.
IDE mods. Not for 8.x
extensions in general will be changed in 9.x I think. We want to do a lot more with them. 8.x ones are pretty limited.
Drawing.
Animated tiles - yes.
isometric tilesets. Not high on the list. (I presume you all mean free assets here)
alpha in colour - oh hell yes! Full 32bit colour values rather than alpha, and colour would save LOADS of effort in the engine.
draw_*_tiled_region... not sure what this means. Draw a sprite./background tiled? yes, sure.
More vector support. more prims. bigger better drawing in general... yep
Layers - I'm a BIG fan of layers...
Correct normals for 3D solids??? They aren't correct? (spheres, cubes etc?)
More 3D formats - 9.x (at some point)
bloody hell this list goes on some!
Collision
Not used the more complex collision very much.. so won't comment. It's pretty slow though, so needs an overhaul.
Instances - sure. Though don't hold your breath for Lambda stuff.
Sound
Sound instance. Yes
formats - yes.
seek - probably
length - probably.
modification - probably
mute - yes. Master volume should mute media player music too really.
Rooms - yes and yes.
I'd also like to add scissor regions. So you can hardware clip to a rectangle.
Networking.
Total rewrite required.
Reflection.
Yes. But not sure when.... 9.x prob
Maths.
Well, better maths in general.
Files
ini files. INI files aren't too bad. But need some kind of instance or handle so you can have multiples open at once. You also need to be able to flush an ini file, to get rid of old stuff. Basically, a refresh would be nice. xml etc would also be good.
read whole file - yes. soon I hope.
game save/load. This whole thing needs redone. But it could throw an event.
Input
yes. Allow multiple mouse clicks at once as well (allows for multi-touch devices)
Joystick - sure.
Timing.
Mmm...
Naming alarms would be nice. I don't like just having alarm[3] either.
Hires timer would be great. It's available, so we should provide it.
Interface.
Message boxes. This is an on going debate. I think we should make all message boxes the OS native ones. This allows for Macs to look like Macs, and windows to look like Windows. But there is a backward comparability issue, in that the code allows your to change the look/feel. I think if you want to do that... do your own dialogues. SO not sure if it'll be in 8.x.
Not sure about the rest.....
Runner.
We will be changing the runner format, and making it harder to decompile. Will be done in 8.x time-frame.
Bytecode generation at build time. Yes. Not sure when. Might need the c++ runner for that.
remove debugging in release builds - would also be nice.
Unused resources. Yes, however... since you can refer to resources as strings, and then later make them into actual object references, then it becomes hard to actually know if someone has used an object. However, we hope to make this an option so if you don't do this, then we can strip out all the crap. We've seen sketches and photos in a games .exe as the developer as used it as a project workspace, but the final program has gotten it all as well. This should be cut in some way.
Wow. Okay, that was much longer than I though it was going to be..... Now just because I've said yes doesn't mean it'll happen tomorrow. And just because I've said no, doesn't mean it'll never happen. ALL these things are still up in the air, and under discussion. We're very aware that anything we change will have a massive impact on the community and developers work flow, so we want to be very careful as to how we proceed. I think I'll probably start some GMC topics to open discussion with the community about some of the changes, and we will speak directly to the devs we've been working with before anything major is affected. In general, additions will probably go right in, while changes may take some time to validate the need, and the effect on the community.
I've mentioned before just how excited I am to start working on Game Maker itself. It's a fabulous product, with a great community, and huge scope for change and improvement. I think we can drive Game Maker to be the tool to use for indi devs, but it'll take time. The real improvements will only come with the rewrite, but we should be able to start the process in this iteration.
In the next few weeks, you'll hear whats going to be in our first update, although you all know about the room editor zoom. We hope to do small iterations, more often. So with luck, for 8.x there won't be any more massive waits, you'll get updates much more often, and new features will slowly start to appear.
EDIT: Oh yeah... The documentation needs an overhaul as well. It should be more like DirectX with actual examples of function use, not just listing them - that's no use to anyone! However, this is a massive task. But one day, we will.
Showing posts with label runner. Show all posts
Showing posts with label runner. Show all posts
Saturday, January 22, 2011
Monday, October 11, 2010
Writing cross platform code...
You know, it occurred to me that some folk may find it interesting to see how we manage to port the Game Maker runner to other platforms, so I thought... Let's write about it. I then thought, well... it's not specific to the runner, Russell and I have been doing this for almost 20 years. I'll use the runner as the example, but the method holds true no matter what you do.
So the first thing you have to get into the habit of doing is abstraction. This doesn't mean layer upon layer of code and calls, but a very thin level that can remove any real platform dependence. Now, since I'm a graphics guy at heart, I'll talk about abstracting graphics code, but the theory is the same be it networking, audio or a basic file system. So here goes...
When we got the C++ runner it was all tangled up with windows specific code, from DirectX to MFC (Windows Foundation Classes), so the first thing we had to do was remove all the specifics, well... as much as we could. We spent months removing MFC and started using a simple WIN32 interface as this is much simpler and gave us control over the main loop which was vital to actually running on other platforms. We also started to add a thin layer between Game Maker, and DirectX. This meant that instead of calling D3D directly, it now called one of our functions, and we called D3D. This was done throughout the code, from creating textures and surfaces, to rendering lines and triangles.
The first thing we did was takeover the screen/canvas creation. This means we're now creating the device and have access to all the normal D3D functions inside our little world, and as far as Game Maker knows, it's simply asked to OpenWindow().
Next, we want to get something drawn, so we change all the rendering calls to a call to our triangle rendering instead so we now have access to all the vertices and can draw flat/coloured triangles where all the sprites would be. Most of these changes are pretty straight forward, and are a simple search/replace. Change DrawPrimitive() into a DrawArray(). Once that's done, we can start to change ALL the primitive types from being D3D specific, into our own custom values. So rather than using D3DPT_TRIANGLELIST, we now use our own ePrimType_TRILIST. So far so good.
Now there's lots of drawing code inside Game Maker, fonts, sprites, tiles and backgrounds and so on, and we really don't want to have to rewrite these every time. However, now that we have something that will render triangles in an abstracted way, we can rewrite them once, into OUR format. So the font rendering will now make vertices into our buffers, and use an ePrimType_TRILIST to draw them. Hay-Presto! The font code no longer needs D3D to render things. But what about the textures it uses?
Well, we created a new CreateTexture() call as well, which returns a simple void* which then allowed the internals of texture management to do whatever it liked. It also has the advantage that the texture system can now resize/resample textures if it liked, as Game Maker will never know; very handy when memory is tight! After all, Game Maker doesn't really care what the call is, as long as it can set the texture, so this works well.
So now we have the ability to render primitives, and create/change textures, so what else? Well there are lots of render states to do, so we'll need to abstract them as well. This means things like the D3D culling renderstate (i.e. D3DCULL_CW) now changes to eCull_CounterClockwise and so on, until every state, every D3D call has been abstracted away into a new interface. But why bother? Well, for a start it means that we don't have to include the D3D headers on other platforms! D3D obviously has windows specific includes internally and that would end up being a nightmare, but my abstracting things a little, the other platforms have become much simpler to port to.
And, although it seems odd... D3D is in effect our first port. Game Maker now runs using YoYo's interface and API, which then has a set of classes which translate things into D3D calls (which has been upgraded to DX9 BTW). Theres lots of other functions which we abstracted, matrix operations, viewports, grabbing screens, surfaces and render targets and the like, but at the end of it, we have a clean, non-D3D interface. And we can now port it easily.
Now, this method is true of any platform specific API you care to mention. We also changed the Audio system on windows to XAudio2 (the latest DirectX audio system), and we did this by simply having calls to Load a sample and return a void* which Game Maker can then use as a handle to the sound,midi or MP3.
Now... the real trick here is to get most of the code to be platform independent. Although things like fonts draw to the screen, they don't have to know anything about the underlying API. All it wants to do is set a texture, some blend modes, and then give you some vertex data. The interface then handles the rest.
Once you have a set of files that ARE platform dependent, you can then port these to any system you like. The real win for this, is that any new feature you add to Game Maker, will usually appear on all the platforms at once. Only the most obscure thing like grabbing screen rects without using the CPU (the PSP needs this) will have to be hand crafted. But other things like optimising font rendering will be the same on all platforms, and only require to be written once.
So, for the record.... we now have a Win32 DirectX 9 render, and PSP render, and an iOS OpenGL ES render. Making a Mac OpenGL render would now be trivial as the OpenGL ES is actually a subset of it, so we can actually ADD features!
As you can see, making a true platform independent bit of code is fairly simple, as long as you think ahead and don't try to be too clever. It also means that all systems benefit from any core changes, and porting to another system can sometimes be achieved rapidly if required.
Now, porting to something like iOS is interesting because your supposed to use Objective C, but in reality... anything that can call C++, can use the whole runner. We use Objective C to create the display, flip the screen and get the touch input, but everything else is done in the normal way using the C++ runner. Any system that allows C++ code, can be done like that making it a trivial port (i.e. simple to do, but takes a little time). If a platform doesn't allow calls to C++ (like XNA), then this means you have to write directly in the provided language, and that then becomes a monster task to rewrite the whole engine. It also means any improvements to the C++ code must then be replicated to the new system, which isn't very nice.
The Game Maker C++ Runner is now very portable thanks to the abstraction it's received, and we also know it's very hardware independent as it runs on Intel, MIPS and ARM cpus. Different CPUs can also give some headaches as byte order can flip, and many systems don't let you access INTs on non-INT boundaries and so on. We've now been through all of that, and now have a very portable engine which is looking good for the future.
So there you go... This is how we ported the runner over, and how other platforms suddenly seem to spring up. All that's really missing from it are some of the features we removed initially to get it all working, and they will come back in time, and one day, the C++ runner will be THE way to run Game Maker games.
So the first thing you have to get into the habit of doing is abstraction. This doesn't mean layer upon layer of code and calls, but a very thin level that can remove any real platform dependence. Now, since I'm a graphics guy at heart, I'll talk about abstracting graphics code, but the theory is the same be it networking, audio or a basic file system. So here goes...
When we got the C++ runner it was all tangled up with windows specific code, from DirectX to MFC (Windows Foundation Classes), so the first thing we had to do was remove all the specifics, well... as much as we could. We spent months removing MFC and started using a simple WIN32 interface as this is much simpler and gave us control over the main loop which was vital to actually running on other platforms. We also started to add a thin layer between Game Maker, and DirectX. This meant that instead of calling D3D directly, it now called one of our functions, and we called D3D. This was done throughout the code, from creating textures and surfaces, to rendering lines and triangles.
The first thing we did was takeover the screen/canvas creation. This means we're now creating the device and have access to all the normal D3D functions inside our little world, and as far as Game Maker knows, it's simply asked to OpenWindow().
Next, we want to get something drawn, so we change all the rendering calls to a call to our triangle rendering instead so we now have access to all the vertices and can draw flat/coloured triangles where all the sprites would be. Most of these changes are pretty straight forward, and are a simple search/replace. Change DrawPrimitive() into a DrawArray(). Once that's done, we can start to change ALL the primitive types from being D3D specific, into our own custom values. So rather than using D3DPT_TRIANGLELIST, we now use our own ePrimType_TRILIST. So far so good.
Now there's lots of drawing code inside Game Maker, fonts, sprites, tiles and backgrounds and so on, and we really don't want to have to rewrite these every time. However, now that we have something that will render triangles in an abstracted way, we can rewrite them once, into OUR format. So the font rendering will now make vertices into our buffers, and use an ePrimType_TRILIST to draw them. Hay-Presto! The font code no longer needs D3D to render things. But what about the textures it uses?
Well, we created a new CreateTexture() call as well, which returns a simple void* which then allowed the internals of texture management to do whatever it liked. It also has the advantage that the texture system can now resize/resample textures if it liked, as Game Maker will never know; very handy when memory is tight! After all, Game Maker doesn't really care what the call is, as long as it can set the texture, so this works well.
So now we have the ability to render primitives, and create/change textures, so what else? Well there are lots of render states to do, so we'll need to abstract them as well. This means things like the D3D culling renderstate (i.e. D3DCULL_CW) now changes to eCull_CounterClockwise and so on, until every state, every D3D call has been abstracted away into a new interface. But why bother? Well, for a start it means that we don't have to include the D3D headers on other platforms! D3D obviously has windows specific includes internally and that would end up being a nightmare, but my abstracting things a little, the other platforms have become much simpler to port to.
And, although it seems odd... D3D is in effect our first port. Game Maker now runs using YoYo's interface and API, which then has a set of classes which translate things into D3D calls (which has been upgraded to DX9 BTW). Theres lots of other functions which we abstracted, matrix operations, viewports, grabbing screens, surfaces and render targets and the like, but at the end of it, we have a clean, non-D3D interface. And we can now port it easily.
Now, this method is true of any platform specific API you care to mention. We also changed the Audio system on windows to XAudio2 (the latest DirectX audio system), and we did this by simply having calls to Load a sample and return a void* which Game Maker can then use as a handle to the sound,midi or MP3.
Now... the real trick here is to get most of the code to be platform independent. Although things like fonts draw to the screen, they don't have to know anything about the underlying API. All it wants to do is set a texture, some blend modes, and then give you some vertex data. The interface then handles the rest.
Once you have a set of files that ARE platform dependent, you can then port these to any system you like. The real win for this, is that any new feature you add to Game Maker, will usually appear on all the platforms at once. Only the most obscure thing like grabbing screen rects without using the CPU (the PSP needs this) will have to be hand crafted. But other things like optimising font rendering will be the same on all platforms, and only require to be written once.
So, for the record.... we now have a Win32 DirectX 9 render, and PSP render, and an iOS OpenGL ES render. Making a Mac OpenGL render would now be trivial as the OpenGL ES is actually a subset of it, so we can actually ADD features!
As you can see, making a true platform independent bit of code is fairly simple, as long as you think ahead and don't try to be too clever. It also means that all systems benefit from any core changes, and porting to another system can sometimes be achieved rapidly if required.
Now, porting to something like iOS is interesting because your supposed to use Objective C, but in reality... anything that can call C++, can use the whole runner. We use Objective C to create the display, flip the screen and get the touch input, but everything else is done in the normal way using the C++ runner. Any system that allows C++ code, can be done like that making it a trivial port (i.e. simple to do, but takes a little time). If a platform doesn't allow calls to C++ (like XNA), then this means you have to write directly in the provided language, and that then becomes a monster task to rewrite the whole engine. It also means any improvements to the C++ code must then be replicated to the new system, which isn't very nice.
The Game Maker C++ Runner is now very portable thanks to the abstraction it's received, and we also know it's very hardware independent as it runs on Intel, MIPS and ARM cpus. Different CPUs can also give some headaches as byte order can flip, and many systems don't let you access INTs on non-INT boundaries and so on. We've now been through all of that, and now have a very portable engine which is looking good for the future.
So there you go... This is how we ported the runner over, and how other platforms suddenly seem to spring up. All that's really missing from it are some of the features we removed initially to get it all working, and they will come back in time, and one day, the C++ runner will be THE way to run Game Maker games.
Friday, October 1, 2010
Runner/Progress follow up...
So... a quick reply to a what's been said so far, both here and on the GMC.
(Sorry... another long one)
First and foremost, please remember that while we plan to do great things with GM, it WILL take time. Lots of time. It's not something that will happen overnight, or even over a few months. Some of the "big" improvements may take a year or so to appear. Now this might seem drastic, but let me explain a little... First, YoYo is a reasonably small team, we don't have the 50 developers some games companies have, and that means we have other things to do as well as improve Game Maker itself. If we can get the runner onto lots of platforms we can make more money (for us and the developer), and that lets us hire more folk to do more with Game Maker. So first off we've take a long time to get the runner into a very portable format, and while that meant cutting it back, it also means we can start getting games out on many other platforms.
While doing this, we've obviously noticed many performance issues, some we've fixed, some we've addressed in a small way, and some require extensive rewrites and reorganisation on the code. Let's take the GML interpreter. The original by Mark was a pretty mean beast, functional, but unwieldy. Russell has taken some of the work Mark did and converted it into a virtual machine. This was a massive step, but at the same time a really small one. So instead of having lots of nested C++ code calling code blocks, calling loops, calling functions, calling code blocks and so on, we now have a very simple virtual machine loop. This opens up massive speed ups for the future, but also opens up full script level debugging. Allowing you, for the 1st time to stick a breakpoint on a script line, and then step through the code when the breakpoint is hit. Now... we don't have all the fancy GUI stuff. We have some VERY simple stepping code. Adding the rest into Game Maker will take huge amounts of work. And this is only the beginning of what's needed.
The problem with the current Virtual Machine is that it doesn't really give us the massive speed up we needed. This us because of the way variables and constants are currently handled inside Game Maker. The next monster task for Russell and the VM is to identify the correct "types" and link them with the correct functions. This would stop every Game Maker function having to check to see that the argument it's just been passed is of the right type. Very simple Virtual Machine opcodes like ADD should be a few lines long, but due to the way its arranged just now, it's pretty huge, and every opcode is like that. Over time, this will all be addressed, and GML will sped up massively, way beyond what is currently achieved.
Now, I only explain all this to show just how much work we have to do in the runner to get it up to the speed WE want, never mind the improvements that everyone else wants. Make no mistake, the VM will be huge, but it will take a long time to implement and make the changes to the rest of the runner so that it works the way it should. Now while Russell's doing that, I'll probably be doing other performance stuff, or even just adding functionality back in; as I said, the runner has been stripped bare.
All this means Game Maker won't change overnight. But it will change. Now, none of these things have been prioritised, meaning we may well do some Game Maker stuff well before the huge VM changes, so don't worry too much, but don't expect everything to come at once.
Now as to some of the other issues... Screen Tearing. This is usually done because it allows the game to "flip" screens quicker, it doesn't have to wait for a vertical blank and will present a more even game experience. When you're in a window, you don't really have a lot of say in how flipping is handled, but when you're in full-screen mode you can tell it not to tear. However... if your game drops from 60fps to 59, then you don't get 59, you get 30. It's pretty harsh. Now all this said... We can fix the tearing in future versions; preferably as an option. The ports don't "rip", because we modify each game we tune it to a specific (manageable) frame rate, and then it's tied to that. so 60fps or 30fps. In fact, it's a TRC failure on the PSP to flip outside a Vertical Blank.
Full screen BLUR. This is tricky. Many drivers override whatever you select. But we can see the need so will try.
Win98... sorry, it's dead. I loved Win98 se (MUST be the SE edition!), but then I also loved the Amiga, but both are dead, time to move on. :)
File size... It's not a huge concern in today's broadband world, but we'll always try and reduce when we can. But when most games are hundreds of Mbs in size, GM games are tiny by comparison, so it's not a worry.
OGG. Yes, when we have time to do it, OGG will be built in.
The rendering inside the C++ runner has been updated to DirectX9, not DirectX8. Audio has been totally rewritten, although it needs completed. Many functions no longer work and have to be re-implemented.
Networking has been removed from the C++ runner. It was an old dated monster that was in there (Direct Play). We will get round to doing a totally new networking system, but it's pretty low on the list.
No one asked about this, but it's actually important. point/texel sampling has been unified. The original code added little hacks to every sprite drawn to get correct texel selection. This slowed everything down and was a nightmare across platform. This has all been removed and done properly.
There's two reasons we won't add backwards file saving. First, as a company we want people to buy the new version so we add lots of new features to encourage you to swap. And second, because we've added new features, you can't save as an old version because those features don't exist. Things fall apart pretty quickly from there...
Graphically linking objects...yes could do. Not high on the list... but could do.
Vector collision code. Again, yes... could do. But there's lots of other stuff to "fix", so again, not high on the list.
External DLL's. While I'd love it so you didn't NEED external DLL's, we'd be stupid to assume we knew everything and thought we'd added all you'd ever needed. There's too many "what-ifs". We WILL incorporate some of the features of the most popular ones (one day.... over the rainbow), as that just makes sense... but what if you wanted to make a 3D glasses version for that new toaster that has a screen and windows Windows? Well... we won't be supporting that unless 100million people buy that toaster, until then use an external DLL.
Please stop calling me "Dailly", you're not my old school teacher and I've not been "bad". It's rude. :)
Lastly... remember that Russell and I have only really been part of YoYo for a short time, and as Sandy said at the time we're still a limited resource. We've had other priorities, some of which you know about and some you don't (and we're not about to tell you yet!), We can't do everything overnight. It's getting better, but it'll take time to get everything the way we want it.
Okay...REALLY lastly. We do read the GMC when we can, but there's simply too much there for us to keep track of everything, but when something "big" comes to light we will try and respond if we have time.
Now... as it's almost 10pm. I'm off to have my tea if you don't mind! Don't say we don't work hard here! :)
(Sorry... another long one)
First and foremost, please remember that while we plan to do great things with GM, it WILL take time. Lots of time. It's not something that will happen overnight, or even over a few months. Some of the "big" improvements may take a year or so to appear. Now this might seem drastic, but let me explain a little... First, YoYo is a reasonably small team, we don't have the 50 developers some games companies have, and that means we have other things to do as well as improve Game Maker itself. If we can get the runner onto lots of platforms we can make more money (for us and the developer), and that lets us hire more folk to do more with Game Maker. So first off we've take a long time to get the runner into a very portable format, and while that meant cutting it back, it also means we can start getting games out on many other platforms.
While doing this, we've obviously noticed many performance issues, some we've fixed, some we've addressed in a small way, and some require extensive rewrites and reorganisation on the code. Let's take the GML interpreter. The original by Mark was a pretty mean beast, functional, but unwieldy. Russell has taken some of the work Mark did and converted it into a virtual machine. This was a massive step, but at the same time a really small one. So instead of having lots of nested C++ code calling code blocks, calling loops, calling functions, calling code blocks and so on, we now have a very simple virtual machine loop. This opens up massive speed ups for the future, but also opens up full script level debugging. Allowing you, for the 1st time to stick a breakpoint on a script line, and then step through the code when the breakpoint is hit. Now... we don't have all the fancy GUI stuff. We have some VERY simple stepping code. Adding the rest into Game Maker will take huge amounts of work. And this is only the beginning of what's needed.
The problem with the current Virtual Machine is that it doesn't really give us the massive speed up we needed. This us because of the way variables and constants are currently handled inside Game Maker. The next monster task for Russell and the VM is to identify the correct "types" and link them with the correct functions. This would stop every Game Maker function having to check to see that the argument it's just been passed is of the right type. Very simple Virtual Machine opcodes like ADD should be a few lines long, but due to the way its arranged just now, it's pretty huge, and every opcode is like that. Over time, this will all be addressed, and GML will sped up massively, way beyond what is currently achieved.
Now, I only explain all this to show just how much work we have to do in the runner to get it up to the speed WE want, never mind the improvements that everyone else wants. Make no mistake, the VM will be huge, but it will take a long time to implement and make the changes to the rest of the runner so that it works the way it should. Now while Russell's doing that, I'll probably be doing other performance stuff, or even just adding functionality back in; as I said, the runner has been stripped bare.
All this means Game Maker won't change overnight. But it will change. Now, none of these things have been prioritised, meaning we may well do some Game Maker stuff well before the huge VM changes, so don't worry too much, but don't expect everything to come at once.
Now as to some of the other issues... Screen Tearing. This is usually done because it allows the game to "flip" screens quicker, it doesn't have to wait for a vertical blank and will present a more even game experience. When you're in a window, you don't really have a lot of say in how flipping is handled, but when you're in full-screen mode you can tell it not to tear. However... if your game drops from 60fps to 59, then you don't get 59, you get 30. It's pretty harsh. Now all this said... We can fix the tearing in future versions; preferably as an option. The ports don't "rip", because we modify each game we tune it to a specific (manageable) frame rate, and then it's tied to that. so 60fps or 30fps. In fact, it's a TRC failure on the PSP to flip outside a Vertical Blank.
Full screen BLUR. This is tricky. Many drivers override whatever you select. But we can see the need so will try.
Win98... sorry, it's dead. I loved Win98 se (MUST be the SE edition!), but then I also loved the Amiga, but both are dead, time to move on. :)
File size... It's not a huge concern in today's broadband world, but we'll always try and reduce when we can. But when most games are hundreds of Mbs in size, GM games are tiny by comparison, so it's not a worry.
OGG. Yes, when we have time to do it, OGG will be built in.
The rendering inside the C++ runner has been updated to DirectX9, not DirectX8. Audio has been totally rewritten, although it needs completed. Many functions no longer work and have to be re-implemented.
Networking has been removed from the C++ runner. It was an old dated monster that was in there (Direct Play). We will get round to doing a totally new networking system, but it's pretty low on the list.
No one asked about this, but it's actually important. point/texel sampling has been unified. The original code added little hacks to every sprite drawn to get correct texel selection. This slowed everything down and was a nightmare across platform. This has all been removed and done properly.
There's two reasons we won't add backwards file saving. First, as a company we want people to buy the new version so we add lots of new features to encourage you to swap. And second, because we've added new features, you can't save as an old version because those features don't exist. Things fall apart pretty quickly from there...
Graphically linking objects...yes could do. Not high on the list... but could do.
Vector collision code. Again, yes... could do. But there's lots of other stuff to "fix", so again, not high on the list.
External DLL's. While I'd love it so you didn't NEED external DLL's, we'd be stupid to assume we knew everything and thought we'd added all you'd ever needed. There's too many "what-ifs". We WILL incorporate some of the features of the most popular ones (one day.... over the rainbow), as that just makes sense... but what if you wanted to make a 3D glasses version for that new toaster that has a screen and windows Windows? Well... we won't be supporting that unless 100million people buy that toaster, until then use an external DLL.
Please stop calling me "Dailly", you're not my old school teacher and I've not been "bad". It's rude. :)
Lastly... remember that Russell and I have only really been part of YoYo for a short time, and as Sandy said at the time we're still a limited resource. We've had other priorities, some of which you know about and some you don't (and we're not about to tell you yet!), We can't do everything overnight. It's getting better, but it'll take time to get everything the way we want it.
Okay...REALLY lastly. We do read the GMC when we can, but there's simply too much there for us to keep track of everything, but when something "big" comes to light we will try and respond if we have time.
Now... as it's almost 10pm. I'm off to have my tea if you don't mind! Don't say we don't work hard here! :)
Wednesday, September 29, 2010
Game Maker runner performance
I’d like to briefly address some of the concerns as to performance and what exactly we’re doing for the future of Game Maker.
First there have been several posts about the current Delphi runner and how bad it is. While this is a little exaggerated there are indeed several problems with it. The first thing any developer has to realise is that a platform will never be fully to his liking, ever. DirectX has lots of things I really hate about it, stuff that slows me down and makes my code ugly, but I don’t just sit back and complain about it, or simply demand Microsoft do something about their “core gaming tech” or I’ll swap to OpenGL, I work with and around the problem to get what I need. All good coders do this because every platform is flawed.
That said… there are some short comings inside Game Maker’s API, some of them I consider fairly nasty. Aragon1029 posted some tests on the GMC so I’m going to start out by looking at these. I’m not completely sure what “some” of them are about, but I’ll address them as best I can.
The fact that .EXE’s are made read-only (I think this is what he means) is something we can look at, but you really shouldn’t be changing the .exe so I’m not sure why this is an issue to anyone, and it’s certainly something you can get round easily for now.
The decompiler. We will look at it in the next version, but anything we do is fundamentally crackable; eventually. We’ll see what we can come up with.
Crashes. We need reproducible tests and bug reports. We have a bug reporting system, if something’s causing problems, use it. Mac users will tell you that there are regular updates for them, and once we get the same system in place for windows, you’ll get that too!
File access. While I’ve no idea why you would want to read/write very large files in GM, I can see that it’s slow. The main reason for this is reading/writing to any file system a byte at a time is ALWAYS slow, no matter what language you’re using. It needs a “block” read/write badly. This will be added in later versions. What we’ll probably do is some sort of very simple memory allocation with peek/poke access, and then let you save “blocks” from this. This would allow fast access to data and allow nice reading/writing of “types” from files, particularly if we allow something like Peek_Float(memory_handle,offset), Peek_String(memory_handle, offset) etc… Oh, and no... the program hasn't crashed when it's "not responding", it's simply not responding because it's taking a long time. Leave it for a while and it will eventually come back. This is what happens if you sit in a large loop without letting windows in.
The Format. Not sure what he means about this one…??? We do want to make some changes to allow it to be source controlled however.
Dragging the window = frozen game “while” dragging the window.. This is a windows issue. It doesn’t give you messages while it’s being dragged. So the timing stops. All windows apps experience this… *shrug*
The “varied” parsing of GML ( a=0 or a:=0 etc..). Okay, this is a feature of GML. It’s primarily a learning language so is very forgiving of these, kinds of things. In some respects I’d love to make it much stricter, in another, we don’t want to hinder beginners by making it hard to get up and running. However it IS possible to have a “PRO” parsing mode, so that when you want strict parsing, you can have it. But this is something we will think long and hard about so that we don’t affect beginners. At the end of the day, a lot of our customers are schools and universities that teach to non-technical students, so we have to be very careful about what we do with this.
C/Delphi string handling. [0] or [1]…. Being a programmer, everything starts at 0 for me. I suspect we’ll standardise this in the future, one way or another. Outside of that should throw an error. It’s hardly a show stopper though….
Font rendering. First the example given isn’t terribly realistic, just printing row upon row of outline text isn’t exactly common in games, and if it was slow, you’d simply do it a different way. That said, font rendering is definitely slow. We have already seriously sped this up inside the C++ runner, and I expect this to get faster in the future when formatting is all done on the fly. However… a full screen of “outlined” text will always be slow, but we might be able to speed it up more by giving dedicated GML commands to actually draw a full outlined font making it faster yet. So there’s several things we can do here, although the main culprit has been changed in the C++ one already. Speaking of fonts, there was actually a very good talk at SigGraph a few years back that allows smooth, crisp font scaling. This means we can reduce the number of fonts you actually have to include in your game, having a single font you simply scale, rather than having 5 different levels. We’ve used this before and the results were superb.
Runtime memory. We have addressed a lot of the memory issues, with some games using a fraction of the previous amount. This was caused by the runner creating every room and every instance at start up. So any game with many rooms had a huge overhead. We simply now create them from the “reduced” source asset when needed. This gave a huge memory boost. We’ll continue to address this as time goes on.
Now, there are plenty of other areas we really need to improve. The Code editor really needs to stay open, even if this means losing the external text editor I think this is a price worth paying. As long as the code editor is good enough, losing the external “text” editor option isn’t really an issue, but it’s one we’ll evaluate closer the time.
We also really need to revamp the room editor and there’s some very cool ideas for this, in fact I’ll probably do a whole post describing what we’re planning for this so everyone can read it in detail; some of the new features will be very cool. To give you a rough idea…. We’re planning on adding zooming, and allowing you to put down sprites/text as well as objects. This means you can decorate your level without creating loads of objects. The new tile system is still under discussion but will likely include multiple playfields with each “whole” playfield being at a single user defined depth, this will speed up the room drawing hugely. More on this later.
On top of all this, there are some internal operations that could just be done better. They work fine, but we can speed many of them up quite a bit. Collision for example, can be sped up a lot by simply being sneakier about how we handle it. Also precise collision is pretty slow, but as that only happens when things actually hit, it’s not the end of the world.
I’ve said this before, you write for the platform as best you can, and you work around issues you have to. All development is about doing that. If not, then I could draw 20,000,000 particles for a candle sitting on a table and not care. It’s about doing as little as you can to achieve the desired effect. Yes, we can and will speed things up in future versions. Yes, the C++ runner is much faster than the Delphi one, and is getting faster and more efficient all the time, and yes, windows/mac versions will get it at some point in the future instead of the Pascal runner. These things take time, a long time.
So there you go. I plan on doing another post to actually discuss the runner. Where it’s at, changes made, and discuss the benefits over the Delphi runner, and why the C++ one will be much quicker overall. So stay tuned.
First there have been several posts about the current Delphi runner and how bad it is. While this is a little exaggerated there are indeed several problems with it. The first thing any developer has to realise is that a platform will never be fully to his liking, ever. DirectX has lots of things I really hate about it, stuff that slows me down and makes my code ugly, but I don’t just sit back and complain about it, or simply demand Microsoft do something about their “core gaming tech” or I’ll swap to OpenGL, I work with and around the problem to get what I need. All good coders do this because every platform is flawed.
That said… there are some short comings inside Game Maker’s API, some of them I consider fairly nasty. Aragon1029 posted some tests on the GMC so I’m going to start out by looking at these. I’m not completely sure what “some” of them are about, but I’ll address them as best I can.
The fact that .EXE’s are made read-only (I think this is what he means) is something we can look at, but you really shouldn’t be changing the .exe so I’m not sure why this is an issue to anyone, and it’s certainly something you can get round easily for now.
The decompiler. We will look at it in the next version, but anything we do is fundamentally crackable; eventually. We’ll see what we can come up with.
Crashes. We need reproducible tests and bug reports. We have a bug reporting system, if something’s causing problems, use it. Mac users will tell you that there are regular updates for them, and once we get the same system in place for windows, you’ll get that too!
File access. While I’ve no idea why you would want to read/write very large files in GM, I can see that it’s slow. The main reason for this is reading/writing to any file system a byte at a time is ALWAYS slow, no matter what language you’re using. It needs a “block” read/write badly. This will be added in later versions. What we’ll probably do is some sort of very simple memory allocation with peek/poke access, and then let you save “blocks” from this. This would allow fast access to data and allow nice reading/writing of “types” from files, particularly if we allow something like Peek_Float(memory_handle,offset), Peek_String(memory_handle, offset) etc… Oh, and no... the program hasn't crashed when it's "not responding", it's simply not responding because it's taking a long time. Leave it for a while and it will eventually come back. This is what happens if you sit in a large loop without letting windows in.
The Format. Not sure what he means about this one…??? We do want to make some changes to allow it to be source controlled however.
Dragging the window = frozen game “while” dragging the window.. This is a windows issue. It doesn’t give you messages while it’s being dragged. So the timing stops. All windows apps experience this… *shrug*
The “varied” parsing of GML ( a=0 or a:=0 etc..). Okay, this is a feature of GML. It’s primarily a learning language so is very forgiving of these, kinds of things. In some respects I’d love to make it much stricter, in another, we don’t want to hinder beginners by making it hard to get up and running. However it IS possible to have a “PRO” parsing mode, so that when you want strict parsing, you can have it. But this is something we will think long and hard about so that we don’t affect beginners. At the end of the day, a lot of our customers are schools and universities that teach to non-technical students, so we have to be very careful about what we do with this.
C/Delphi string handling. [0] or [1]…. Being a programmer, everything starts at 0 for me. I suspect we’ll standardise this in the future, one way or another. Outside of that should throw an error. It’s hardly a show stopper though….
Font rendering. First the example given isn’t terribly realistic, just printing row upon row of outline text isn’t exactly common in games, and if it was slow, you’d simply do it a different way. That said, font rendering is definitely slow. We have already seriously sped this up inside the C++ runner, and I expect this to get faster in the future when formatting is all done on the fly. However… a full screen of “outlined” text will always be slow, but we might be able to speed it up more by giving dedicated GML commands to actually draw a full outlined font making it faster yet. So there’s several things we can do here, although the main culprit has been changed in the C++ one already. Speaking of fonts, there was actually a very good talk at SigGraph a few years back that allows smooth, crisp font scaling. This means we can reduce the number of fonts you actually have to include in your game, having a single font you simply scale, rather than having 5 different levels. We’ve used this before and the results were superb.
Runtime memory. We have addressed a lot of the memory issues, with some games using a fraction of the previous amount. This was caused by the runner creating every room and every instance at start up. So any game with many rooms had a huge overhead. We simply now create them from the “reduced” source asset when needed. This gave a huge memory boost. We’ll continue to address this as time goes on.
Now, there are plenty of other areas we really need to improve. The Code editor really needs to stay open, even if this means losing the external text editor I think this is a price worth paying. As long as the code editor is good enough, losing the external “text” editor option isn’t really an issue, but it’s one we’ll evaluate closer the time.
We also really need to revamp the room editor and there’s some very cool ideas for this, in fact I’ll probably do a whole post describing what we’re planning for this so everyone can read it in detail; some of the new features will be very cool. To give you a rough idea…. We’re planning on adding zooming, and allowing you to put down sprites/text as well as objects. This means you can decorate your level without creating loads of objects. The new tile system is still under discussion but will likely include multiple playfields with each “whole” playfield being at a single user defined depth, this will speed up the room drawing hugely. More on this later.
On top of all this, there are some internal operations that could just be done better. They work fine, but we can speed many of them up quite a bit. Collision for example, can be sped up a lot by simply being sneakier about how we handle it. Also precise collision is pretty slow, but as that only happens when things actually hit, it’s not the end of the world.
I’ve said this before, you write for the platform as best you can, and you work around issues you have to. All development is about doing that. If not, then I could draw 20,000,000 particles for a candle sitting on a table and not care. It’s about doing as little as you can to achieve the desired effect. Yes, we can and will speed things up in future versions. Yes, the C++ runner is much faster than the Delphi one, and is getting faster and more efficient all the time, and yes, windows/mac versions will get it at some point in the future instead of the Pascal runner. These things take time, a long time.
So there you go. I plan on doing another post to actually discuss the runner. Where it’s at, changes made, and discuss the benefits over the Delphi runner, and why the C++ one will be much quicker overall. So stay tuned.
Subscribe to:
Posts (Atom)