next »

[Multi core]

So here we have another dillema: what about multi core support?

Multi core
- All processors will be multi core in the future. Not using them would be a plain waste. We can get a great performance boost.
- It is much harder to program for multi core processors. Especially in a game, where handling events in the right order is key.

Single core
- Much easier to program for.
- On multi core CPU's, you're going to waste a lot of power.

And it's a big dillema... because if we make a multi core engine, it won't be able to run on a monocore. And even if the multicore is the future, there will be always a large part of people who will be always on mococore in 2/3 years...
And there any easy way to transform a multicore engine to a monocore engine, the engine need to be recoded.

In another side like 2PG said, the multicore offers a great performance!

NB: for those you want to know if the multicore is really the future, take a look at this:

because if we make a multi core engine, it won't be able to run on a monocore.

well, that's not entirely true. see, a multi core program runs on multiple threads (or processes, if you like, though that's not really true). one processor can run multiple threads, but multiple processors can't run the same thread (yet).

My main point of doubt is the difficulty. From what I understand, writing for multiple cores is a whole lot more difficult than for a single core.

The benefit is obvious, and increasing in time, so that is my reason for doubt. What is the difficulty/gain ratio? I do not know the answer, and I think noone really does, until we are at a point that we can test our engine. And the first test likely won't be the most positive compared to the end result, so even then we'll still be guessing.

Ok, let's consider it from this point:
- Multi core is the future. 2play and Darvin will both undoubtedly get involved in some project that is written for multiple cores. This project is not just for fun, we also want to learn a bit, like Darvin said in the introduction topic.
- We do want to automate a lot, which means it will be heavy on the processor. By the time the project is done, even people with 2-3 year old PC's likely have a dual core ( It's more worth for the money to buy a dual core than a single core these days)

If you think it is possible, then I say give it a try.

Solinx

My biggest concern is that there is a HUGE untapped fanbase in the developing world for cheap games that run on old machines.  With openwar's focus on modularity, we could easily have a variety of graphics options to cater to a wealth of different groups with access to different levels of machine performance.  On the other hand, development and media attention requires us to cater to the North American and European markets which want top of the line performance based.

We're essentially left with a conundrum.

You are right, a conundrum indeed.

- With the programming becoming harder, how many programmers won't join because it's too difficult?
- Or how many will join, just because it's a challenge?
- Or how many will join because they need experience with multiple core programming?
- How many amateur programmer's won't make their own game because we chose multi core programming?
- In 5 years, will using an engine written for single core processors be considered an option?
- ...

So many more questions that are left to guessing

Solinx

It's all a question of markets and economics, which unfortunately I can't evaluate with any greater depth than what my intuition allows.  What we see there is a prediction based on a company which has a profound interest in increasing investor confidence.  In other words, it's a very VERY biased source of information.  It's a great indication of where Intel wants to go, but not necessarily where the markets will take them.

This is actually the usual "technology transfer" hurdle.  Essentially, it's a matter of markets.  If the gaming population is primarily single-core, developers have no choice but to develop single-core, because even if they can achieve better results on multi-core, it doesn't matter if they are incapable of selling to people that lack multi-core, which make up the lion's share of the market.  If developers aren't developing multi-core games, then single-core gamers don't feel the need to upgrade.  It's a vicious little cycle.  Unless the developers feel comfortable enough to lose their single-core market, it can't take off.  If developers DO produce a wealth of multi-core games, it will take off.  It's as simple as that.  I think that unlike Blue-Ray, which suffers from this same conundrum, multi-core processors have a better chance of success.  However, nothing is certain.

With that said, we must remain aware of what our focus is, what our target is, and what we aim to achieve.  I don't think we stand a chance of competing with a professionally designed graphics engine, so chances are we're aiming at systems with lower specs to begin with.  Can we expect enough of those systems to be multi-core?  Also, can we expect multi-core gamers to forgive us for performance if we cost nothing and offer great gameplay?

As far as learning goes, I'd expect a project like this will be rewarding whether it's single-core or multi-core, both in terms of experience and when it lands on your resume.

With the programming becoming harder, how many programmers won't join because it's too difficult?

I don't think that would be too much of a problem. Every programmer will handle a part of the engine, and not all parts will become (considerably) more difficult if we use multithreading.

Or how many will join, just because it's a challenge?

Let's hope a lot

How many amateur programmer's won't make their own game because we chose multi core programming?

Huh? What has that got to do with it? The way people are going to make a game will be the same, regardless.

In 5 years, will using an engine written for single core processors be considered an option?

Seeing as somebody predicted we'll have 80 cores in 10 years, the answer is probably "no".

Well, I think that last question/answer kinda makes it clear what we need to choose

EDIT: now in response to Darvin

I don't think we stand a chance of competing with a professionally designed graphics engine

I don't quite agree there. It's not hard to create an engine for stunning graphics, it's making the actual content for it that's hard. But seeing as that is not our responsibility...

Also, can we expect multi-core gamers to forgive us for performance if we cost nothing and offer great gameplay?

I don't think so. The average player doesn't think like we do, they need something that's performing perfectly, or they will just whine and not play it. Also, what about the computers with 16 cores (not so impossible in the future), each one working at about 300MHz? The engine will definitely have trouble with those, if it is single-core.

Interesting. Just today I've been skimming through a nice book about open source projects in all it's aspects... it costed a friggin 55 euro's, so it's definetly not moving to my bookshelf, but the little I read was interesting.

I'll first answer 2play.

With the programming becoming harder, how many programmers won't join because it's too difficult?

I don't think that would be too much of a problem. Every programmer will handle a part of the engine, and not all parts will become (considerably) more difficult if we use multithreading.

Or how many will join, just because it's a challenge?

Let's hope a lot

How many amateur programmer's won't make their own game because we chose multi core programming?

Huh? What has that got to do with it? The way people are going to make a game will be the same, regardless.

The answer to the first question also renders the last question as rubish, which I suspected but not being a programmer, didn't know for sure. The second question I'll get back to later.

In 5 years, will using an engine written for single core processors be considered an option?

Seeing as somebody predicted we'll have 80 cores in 10 years, the answer is probably "no".

Well, I think that last question/answer kinda makes it clear what we need to choose

Exactly. My questions are quite biased, it's pretty clear now I reread them (this was not on purpose). And now I find myself agreeing with myself. (scary )

The theory that Darvin has written about it true, only in this case, I find it incomplete. Yes, the roadmap is unlikely to become reality, no doubt about that, however, both Intel and AMD are, and already have been, focussing on multicore processors. They no longer continue their struggle with raw power, but with the number of cores and for the more knowledgeable also with memory, etc. From what I've heard, neither of them really continues development of their single core processors. (If I'm wrong in this, then you are very likely to be right Darvin, let that be clear.)
Last time I checked, the basic AMD dualcore, the Athlon X2 3800+, could be bought for about 100 euro's. Not the cheapest of processors, but certainly not expensive either. The marketing of both AMD and Intel is directed to multicore, together they can force this on the consuments if they would want or need to, which is not even the case. The marketing has been succesful, lot's of fanboys will tell you any day that multi core is the only way to go, and the average user with too little knowlegde to assemble his own PC will get one from Dell with a nice Intel Duo Core, since that was really neccessary for IE to run according to the sales man. (ok, exaggerated quite a bit, but you do get the point.) The case of multi core processors is a technology push by the producers, both producers, all (significant) producers, resulting in a sort of cooperative attempt to get customer acceptance for multi cores. Well, synergy is more appropiate than cooperation I guess.

btw. Blue Ray isn't in the same position, far from it. In the portable disk market, there is no cooperation to get customer acceptance, there is competition to become the sole survivor.

I don't think we stand a chance of competing with a professionally designed graphics engine

I don't quite agree there. It's not hard to create an engine for stunning graphics, it's making the actual content for it that's hard. But seeing as that is not our responsibility...

When we are succesful enough with the engine and , we will be able attract artists that produce the same quality, perhaps even better. Open source projects have the of possibility to attain higher quality results than commercially produced projects. Reasons for this are that there is no monetary goal that limits the resources (time), ideas come from bottom up, artist have more abilities to work on their own ideas and make them part of the project, this promotes the project originality. Open source projects are often used to train skills, to show skills and capabilities, to teach others, to get recommendations by others and a few other reasons I forgot.

That is what can become of a good project. We are not there yet, but we have only just begun

Now, as for the second question:

Or how many will join, just because it's a challenge?

Let's hope a lot

The book I read named this as one of the main motivators for contributers to join a open source project, together with expanding personal knowledge and experience, either for future professional purpose, or just for increase their knowlegde and having fun in the community.

The book also named a 'scratching-itch' effect. Sometimes a visiting programmer sees something he really got to try, a challenge perhaps, or to give a helping hand to solve a problem, that he too adds his bit to the project and sometimes decides to stay in the community.

Before you start with: "that is only one book!" I know, but I believe it was based on emperical research. In any case, it had over 100 of well documented sources, making this a bit more dependable than your average internet blog.

Also, can we expect multi-core gamers to forgive us for performance if we cost nothing and offer great gameplay?

I don't think so. The average player doesn't think like we do, they need something that's performing perfectly, or they will just whine and not play it. Also, what about the computers with 16 cores (not so impossible in the future), each one working at about 300MHz? The engine will definitely have trouble with those, if it is single-core.

sadly, I think 2play is right. Many users won't be very forgiving or understanding.

Now, I probably forgot a few things, but I really don't know anything more to write for now

Solinx

hmm, what's the name of that book?

2play already knows for some time, but here it is for other interested in it:

Open Source, A multidisciplinary approach, written by Moreno Muffato

Solinx

In the last months I have been doing some multithreaded programming (in C and Java) and it's easier than it sounds. We could very well use it in the engine. A good example would be to let the AI run in a seperate thread, or the graphics (though that would be a little more difficult). In fact, if you do network programming, you can hardly get around it.

I would like to mention the famous quote originating from Tony Hoare and restated by Donald Knuth: "Premature optimization is the root of all evil".

Don't solve the problems you do not yet have
-- yuvi.

Erm...I don't really understand the relevance (sp?) of that quote here

You have the pre-projects as a proof-of-concept for the main project, In these projects you can estimate what your bottle-necks will be. That is why i think it is to early to talk about multi-core support - As this might not be our main concern for performance...

We might find out that our problem is memory usage, and then we will have to concentrate our resources  (and design) to low memory usage, that is why I said, we should not solve the problem we do not yet have.
 
-- yuvi

In fact, if you do network programming, you can hardly get around it.

And using a single processor may not be it's weakest point now, but perhaps it will be later. From what I understand, it's not something you change half way through.

Another reason is that from the looks of things, multicore is the future. Having it as a feature in this project allows the programmers to learn what they're likely going to use at their jobs. Furthermore it'll be another selling point of the engine.

It's a good point UVGroovy, but in this case there are enough reasons to go ahead anyway.

Solinx

Many times multithreading is easier than using a single thread.

For instance, if you are going to do some heavy work and want to display a loading bar, it's better to seperate that into 2 threads. If not, you will have to design your own scheduling system whichs alternates the heavy work with updating the progress bar.

If you're going to want an adaptable, forward-looking game engine, you're simply going to have to have multi-core support.

I'm not a real programmer so I can't tell you how much more difficult it is to support multi-cores rather than a single-core, but I can tell you from a hardware stand-point, if you want an engine to last, you're going to need to support multiple processing cores.

We're really moving away from seeing increases in IPC and single-threaded performance, and moving torwards just seeing more processing cores. That's not to say we won't see improvements in processor efficiency; Core 2 Duo proved that IPC could still be improved and Penryn (45nm C2D) is proving that even more, with higher efficiency and clockspeeds. But if you look at future roadmaps, multi-core is coming and it's coming fast.

Quad-core is here now, eight-core will be here soon, probably with Nehalem in 2008. Intel has said they will have 80-core by 2011; these cores are going to be a lot more simple than current processing cores, but with so many of them performance will still be much greater. The future processor is really a Cell-like design, with several "big" cores like you see today, and many more small, simple cores.

Overall, you're going to have to support multi-core if you want an engine that will last and will be able to support high-end games with complex physics and eyecandy. All the big engines/games are moving to multi-core; SupCom uses multi-core, so does Unreal Engine 3, CryEngine 2, Source (with an update), Alan Wake, and alot of the new RTS games like Company of Heroes. RTS games have always been CPU-dependant, so I think you're going to want to get all the performance you can get. You first game might not really need multi-core to perform well, but later on when you (or someone else) wants to make a more complex title, you're going to want that performance.

With a quad-core processor, you can have seperate threads for rendering (preparing data for GPU), physics, AI, and (maybe, I know Alan Wake has it) for audio. There's just no