Tim Sweeny on Unreal Engine 3

[P4Man]

First, if you havent already, download the video <A HREF="http://beyondunreal.jetstreamgames.co.nz/demos/unreal3engine_e3.zip" target="_new"> here </A> (warning, 100+ Mb).

Then read Tim's comments <A HREF="http://www.beyondunreal.com/content/articles/95_1.php" target="_new"> here </A>

A few interesting quotes:

Well, we are aiming at the kind of PC that we think will be mainstream in 2006. We will also be able to scale it down. Basically DirectX 9 cards will be minimum spec, so any DirectX 9 shipping today will be capable of running our game, but probably at reduced detail. If you only have a 256 meg video card you will be running the game one step down, whereas if you have a video card with a gig of memory then you'll be able to see the game at full detail.

A gig of memory on the videocard... Well, *cough*, seems like you want be upgrade safe for 2 years, even with a 9800U or X800 XT .

BTW, 1 gig of videomemory basically requires a 64 bit version of windows. I just don't see how you can map 1 GB twice (thats the way windows does it) into 2 GB of address space, and have anything left for the OS and apps. I figure 512 MB of videomemory is pretty much the practical limit of 32 bit windows. Are you still so sure there is no need for 64 bit computers any time soon ?

= The views stated herein are my personal views, and not necessarily the views of my wife. =<P ID="edit"><FONT SIZE=-1><EM>Edited by P4Man on 05/18/04 06:27 PM.</EM></FONT></P>

 

[Mephistopheles]

"Anytime soon?"

2006 is <b>not</b> anytime soon, P4Man.

But good point anyway.

<i><font color=red>You never change the existing reality by fighting it. Instead, create a new model that makes the old one obsolete</font color=red> - Buckminster Fuller </i>

 

[TechMan]

Yes, indeed, 18 months not too far away. And yet today we've only got 64-bit procs, and beta 64-bit OSes, very minimal 64-bit driver support.

At any rate, these are just pronouncements.

Maybe developers should look into multi-core cpu's instead of memory-resource extensive paradigm. Taking adavantage of the former is, IMHO, the way to go.

 

[Mephistopheles]

18 months is a long time from now in my perspective... If moore's law still holds (which I doubt) this means yet another processor performance doubling.

I agree on that stuff about multi-core; it truly is very promising!

<i><font color=red>You never change the existing reality by fighting it. Instead, create a new model that makes the old one obsolete</font color=red> - Buckminster Fuller </i>

 

[P4Man]

>2006 is not anytime soon,

that is a matter of perspective. Its definately not that far either; its closer to us than summer 2002, at which time you could buy a 3 GHz P4 with a Radeon 9700 PRo, hardly an obsolete machine today.

Don't tell if you bought such a combo in august 2002, that you'd expect to have to upgrade/replace it again today.. ?

Similary if I'd buy a truly high end gaming rig today, I would expect it to last two years without major upgrades and without sacrificing much if anything on eyecandy and framerates.

= The views stated herein are my personal views, and not necessarily the views of my wife. =

 

[juin]

Personnaly i have never like this guy.1GB of ram that 8 more that actual mainstream card.In 2010 ok i would have buy that but not in 2006.

A favor to one of is good friend AMD.

i need to change useur name.

 

[P4Man]

>Maybe developers should look into multi-core cpu's instead
>of memory-resource extensive paradigm

I'm sure they are looking at it, but no matter how many cpu cores you have, if you want such high res, normal mapping on your poly's like showcased in that video, you'll need the memory. Multicore really isnt a substitute to this.

What might be a partial substitute is using several PCI-E video cards in parallel, each drawing only part of the screen, which would double rendering performance but even there afaik, each card would have to hold all the textures, therefore have that much memory. So in fact, in such a scenario you might even need *twice* as much memory and address space, limiting you to around 256 MB/per card (512 MB total, doubled by Windows) which is common even today and only a quarter of what Unreal Engine 3 would require optimally.

1 GB of video memory may sound excessive now, but consider even the cheapest low end DX9 cards today are offered with 128 or even 256 MB of RAM (FX5200). Two years ago that would have been 32 MB or less... I fully expect 512 MB to be mainstream and 1 GB to be common for high end cards in 2006, and like I said, I can't see how this would work with 32 bit Windows. Let alone two of those cards...

= The views stated herein are my personal views, and not necessarily the views of my wife. =

 

[Darkmatterx]

So since I have a 128mb video card, XP holds 256mb of my RAM as well?

In the BIOS isn't there a setting for AGP memory which is generally suppose to be double whatever your card has? Is this the setting that tells XP to hold 256 ram?

 

[P4Man]

>So since I have a 128mb video card, XP holds 256mb of my
>RAM as well?

No, not your RAM. But 256 MB (2x128 MB) is mapped into kernel memory, which means, your windows kernel has only 2 GB minus 256 Mb of address space for itselve. Virtual memory address space, nothing to do with how much ram you have, all of your ram can be (and will be) used.

As such this isnt a problem yet, XP under most circumstances can do with less than a gigabyte of kernel address space (using the /3GB switch, it only allocates a single GB, leaving 3 GB for the apps).

Its only with 256 MB cards and up that it becomes problematic. 512 MB cards would simply be unusable with with the /3GB switch (which would already be a realworld problem today for workstation with highend OpenGL cards), and 1 GB cards would be downright impossible with a 32 bit OS.

>In the BIOS isn't there a setting for AGP memory which is
>generally suppose to be double whatever your card has? Is
>this the setting that tells XP to hold 256 ram?

AGP aperture. I'm honestly not too sure what it does, I think it reserves "AGP memory", enabling your GPU to use system RAM when it runs out of local memory -something that would have horrible performance issues, so you don't want that anyway. In the old days, the idea was to use main memory as extra videocard memory through a "fast" AGP bus. At 66 MHz and 32 bit using 66 or 100 MHz RAM that was more or less feasable as videocard memory wasnt all that much faster those days.

But today you'd be *nuts* to let your GPU address textures over the slow AGP bus to main memory instead of using the local 1 GHz 256 bit memory busses. The difference is far to big.

So, I would not set AGP apertyre to 2x your videomemory if you have a 128+ MB card. Few games if any require 256 MB today, and if you would let a game use 256 MB textures on a 128 MB card, performance would be horrible.

Also, this would further decrease the ammount of address space available to the OS. If I understand AGP aperture correctly, it should also be mapped into kernel space, and twice so, so it would mean taking the ammount of videomemory, adding the AGP aperture, doubling it (windows, but apparently Linux does the same), and then subtract that number from the initial 2GB reserved for the OS and pray you have enough left.

With a 128 MB card that shouldnt be a problem, even using 2x the size for aperture, you'd still have:
2 GB - (128+256)x2 = ~1.2 GB of kernel addres space. Plenty.
Now try with a 256 MB card:
2 GB - (256+512)x2) = 512 MB. Oops.. doubtfull that will work

With a 512 Mb card or more, its simply impossible, even if you do not use AGP aperture... I think that pretty much explains why you havent seen 512+ MB cards, even if 256 is offered on the lowest end FX5200's and Radeons.

= The views stated herein are my personal views, and not necessarily the views of my wife. =

 

[Terracide]

I a a bit bewildered here?
Can anyone post any technical data/link to why the Video-RAM should be affected by the System RAM limitations in a 32 bit OS?
As Video-RAM and System memory are 2 totally different things?!

Terracide...

Don't pretend - BE!

 

[P4Man]

Cant find a good explication right now, but will google some more. Here is already a small article on MS knowledge base that "proves" my point:
<A HREF="http://support.microsoft.com/default.aspx?scid=kb;en-us;319043&Product=winxp" target="_new"> link </A>

SYMPTOMS
If you use the /3GB switch in the Boot.ini file to support a program that can use more than 2 GB of virtual address space, a driver may not be loaded when Windows starts. This is more likely to occur with video adapter drivers. <b>This is especially likely to occur if the hardware device contains a lot of onboard random access memory (RAM) that is used as a buffer. </b>

CAUSE
This problem occurs because the driver is trying to reserve a large block of contiguous virtual memory addresses in the kernel address space to map the device's buffer. When you are use the /3GB switch, the memory that is available to the kernel is halved. The kernel might not be able to allocate a large enough block of memory to satisfy the driver's request.

I'll look for a better explication or maybe write something myself if I can't find anything. Very short, if you remember the old DOS days, you may remember the "memory hole" between 640 Kb and 1 MB (HMA, UMB, all that ?). 16 bit cpu's and the 16 bit DOS could address up to 1 Mb, yet applications where limited to 640 Kb because the rest of that space was reserved for devices (audio, video, network, ..) and later parts of it where freed up by Dos 6.0 to load drivers and part of DOS.

Pretty much the same is done in 32 bit OS's, only that now its not 64 kb needed for CGA or even VGA graphics but hundreds of megabytes and the division is not 640Kb/1 MB but 2 GB/4 GB or 3GB/4GB. Interestingly enough, the ratio's are roughly the same Think of /3GB as the modern variant of Dos=High, UMB. A temporarely hack.

= The views stated herein are my personal views, and not necessarily the views of my wife. =

 

[P4Man]

Here is another link they may help explain. It focuses on AGP Aperture, but should help nevertheless:
<A HREF="http://www.ocfaq.com/article.php/overclocking/vidcard/43" target="_new">http://www.ocfaq.com/article.php/overclocking/vidcard/43</A>

Look at the diagram and see how video memory is mapped into the 0-4 GB system address space (actually this is gross simplification, since it doesnt take virtual memory into account). As I suspected and speculated above, AGP aperture gets added to the address space already in use by the GART to map video RAM. Its apparently done dynamically though, so no address space is wasted when your game/app isnt actually running out of videoram, so it should not be a big concern.

Also note the last paragraph:

The actual usable amount of this 'virtual' AGP memory is less than half the AGP Aperture size set in the BIOS. This is because the Aperture is divided into two areas. One uncached half and another write-combined area.

IOW, you loose 2x the address space for each useable AGP aperture page. AFAIK, this is not only true for AGP aperture, but also for all mapped video memory, so including those 128 or 256 MB on your videocard. But I haven't been able to dig up any hard evidence of this, so if someone here can, please share.

I hope this clears it up somewhat, it if not, just ask.

= The views stated herein are my personal views, and not necessarily the views of my wife. =

 

[Terracide]

So your saying that if you have a system with 2 GB system RAM and a GFX with 256/512 RAM, this is not possible?

Terracide...

Don't pretend - BE!

 

[P4Man]

>So your saying that if you have a system with 2 GB system
>RAM and a GFX with 256/512 RAM, this is not possible?

No. You can have 16 GB of system RAM in a Pentium Pro (provided you'd find a motherboard with enough memory slots) running NT 4.0 and you could plug in a 512 Mb videocard allright. It just wouldnt be usefull.

There is an important difference between virtual memory address space and physical RAM that you seem to not know about. I have written so many posts on the subject I'm not sure I have the energy to explain it once again. do some googling, and learn about virtual memory and 32 bit address space limitations. They are very real, but not related to the ammount of RAM you can install in your system.

The extreme short version is that under a 32 bit OS a process is limited to 2 (or 3 if you use /3GB) GB of virtual memory space, regardless if you have 128 MB or 16 GB or RAM. The OS requirs the other 2 (or 1 GB) for itselve, and for device drivers, even if you only have 128 Mb of system RAM.

And related to this post, its impossible to have a videocard with more than 512 MB of RAM under a 32 bit (windows) OS, period. Its probably not even possible to have a 256 Mb card and run the /3GB switch succesfully on most configurations.

= The views stated herein are my personal views, and not necessarily the views of my wife. =

 

[Darkmatterx]

Hmm I've been having a problem with 3dmax running out of memeory on a large file. I have 2 gig of ram but it still happends from time to time. I also haven't been using this 3gb trick.

So if I lower my AGP appature size to 128 and use this trick I may fix the problem?

If yes, how do I use this trick?

 

[P4Man]

>So if I lower my AGP appature size to 128 and use this
>trick I may fix the problem?

No. AGP aperture isnt allocated until you run out of videomemory, and further more it is allocated in the kernel memory space, not in user memory, so it doesnt affect the ammount of address space available to your app (which is ~2 GB regardless).

You could try running windows with the /3GB switch though.Just edit boot.ini to get something like this:

multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Microsoft Windows XP" /fastdetect /3GB

I'm not sure if this works with XP Home, but it does on XP PRo. This also only helps if 3DS is designed to take advantage of the /3GB hack, and it could give other problems like with device drivers (things like SCSI drivers), but its worth trying. Oh, and if you do run into trouble with /3GB, switching to a video card with less memory would help. Long live 32 bits ! 64 bit is not needed until 2007, intel says so, so it must be true

= The views stated herein are my personal views, and not necessarily the views of my wife. =

 

[Darkmatterx]

wow that didn't go well.

After adding the 3gb trick I couldn't boot my computer and windows wouldn't recognize the HD for me to undo the change in the boot.ini file. BIOS saw the HD, but not a boot disk, and I couldn't go into safe mode either. Ended up having to go into recovery console and renaming the file so windows defaulted to a standard boot.ini file.

My HD is IDE and I have nothing out of the ordinary on my system so I dunno what that happened. Only thought is that all my parts required more then 1gb of ram to use which caused XP to reboot during loading.... Fun times.... 😛

Oh well, was worth a shot

 

[P4Man]

Hmmm.. that's very odd. I don't see how changing boot.ini would render your harddisk unbootable .. are you sure you didnt alter anything in the "multi(x)disk(x)rdisk(x)partition(x)" part ?

Try this; edit boot.ini again and copy paste the line that defines your standard windows boot installation, change the name (between the " " quotes) and add /3GB to the second line. Also add a bootup delay of a few seconds. You should end up with something like this:
[boot loader]
timeout=5
default=multi(0)disk(0)rdisk(0)partition(1)\WINDOWS
[operating systems]
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Windows XP Pro" /fastdetect
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Windows XP experimental" /fastdetect /3GB

(Obviously, dont copy paste this, since I don't boot from C:!).

That should give you a bootmenu with the two obvious options. If it doesnt, you're screwing up something somehow. When it does, try the "experimental". IF anything fails (like because of a lack of kernel space) you should at the very least get a blue screen, and more likely, it will boot but just not load all required drivers or modules. Check system log after boot.

/3GB is tricky, but really it can't render your bootpartition invisble to the bios which seems to have happened to you. You are using XP Professional are you, and not XP Home ?

= The views stated herein are my personal views, and not necessarily the views of my wife. =

 

[Darkmatterx]

all I did when I first tried was add /3GB at the end of the line with /fastdetect

The BIOS COULD see the drive, but a boot disk couldn't. And the computer would reboot as soon as it would have gone to the windows loading screen

 

[P4Man]

Ah, so you got to the windows boot screen.. that makes somewhat more sense then, I thought you got a BIOS error like 'no bootable disk found'.

If you want to persue this any further, try going into bootmenu, and start windows (/3GB) in VGA and enable bootlogging. Check the logs (if booting fails).

I just checked MSDN and came up with this link that may apply to your case:
<A HREF="http://support.microsoft.com/default.aspx?scid=kb;en-us;328269" target="_new">http://support.microsoft.com/default.aspx?scid=kb;en-us;328269</A>

In short: if that is your problem, better wait for SP2
Told you /3GB was a hack. It works...
...sometimes.

Further more I have not found out yet if 3DS supports /3GB. If it doesnt, you'd be wasting your time as it wouldnt make a difference at all.

Your best bet would be to get a 64 bit capable machine next time you upgrade. Once 64 bit windows is there, even 32 bit apps will get the full 4GB of address space instead of just 2 now (3 if you get /3GB to work and the app to use it), which should help significantly. Of course, a 64 bit port of 3DS will be done sooner or later, completely curing you from "out or memory" errors for the next few decades.

But hey, people will keep claiming 64 bit is only for marketing and not needed....

= The views stated herein are my personal views, and not necessarily the views of my wife. =

 

[P4Man]

Oh, and one more thing.. how much memory does your videocard have ? If you have 256 Mb card, its not unlikely you can simply forget about using /3GB. Too much kernel address space would be taken up by mapping the videomemory. You might want to consider plugging in a <=128 MB card for that one big project...

= The views stated herein are my personal views, and not necessarily the views of my wife. =

 

[Darkmatterx]

Its a 128 card. sadly I don't have my 64mb card anymore. :/

 

[P4Man]

then try booting /3GB in VGA mode. If the videodrivers arent loaded, I don't see how the videomemory could be mapped (it should be invisible to the OS). VGA is like 64kb or so, so if that doesnt work, its definately something else holding you back.

= The views stated herein are my personal views, and not necessarily the views of my wife. =

 

[Darkmatterx]

Ok I tried what you suggested. VGA mode still did not load. It just rebooted as soon as I hit enter to continue with the bootup

I then went and told it to log the bootup but when I went back into windows after I saw that the boot log was from 1 year ago... Windows doesn't even get started enough to START the damn log file....

I don't think this is going to work for me. At least I'm back in my system with the normal boot.ini file.

Thanks

 

[P4Man]

Well doh.. it was worth a shot I guess.
Maybe you can try again when Service Pack 2 is released, in case you are hit by the issue I linked above.

And if not, better start saving for a 64 bit machine by the time windows is released for it

= The views stated herein are my personal views, and not necessarily the views of my wife. =