What got this started:
A few years ago I downloaded and started playing the DKZ version of NS and was impressed but really agravated at the crashes and poor performance. I then noticed some things that were obvious problems:
First one I hit was the 64K+1 array index crashes that occured right from the start.
The next thing I noticed was that one of the dialog files was huge, this is a problem for the LUA script engine as it forces a paged array to place it in ram. I have broken this one huge file into several that stay within the efficient limits of the LUA table structure.
At this point I shared my fixes to a few friends and next thing I knew, my wonderful (NOT!) server company shut me off as I had too much traffic... from all over... all thanks to a STALKER that goes by the handle: PsuedoStomp aka Snork. Those of you who like this mod and the direction it's going should be shure to voice a thanks to him sometime.
OK...
Let me try and explain what I have done to solve the limited memory problem without violating the Steam user agreement but still taking advantage of large addresses.
First you need to understand the real problem that causes those cstack errors. This game was written before huge addressing was available from the compiler and although recompiling can improve this to some extent, it's not an option on Steam.
So...
The game actually has these limitations to work with:
A stack that is 64k, a windows managed heap that is 1meg in Vista and 4meg in Win 7.
When you run with the modified xrGame.dll you are getting more paging memory for textures, sounds and shaders up to approx. 3gig but not for the engine.
Solution...
As is obvious by the performance gains so far that the following concept though simple will help get this mod growing (simplistic version):
There are several lists that are searched constantly and then there are some that only get searched once or twice per level, all of these are local tables at run time. This fills up the heap quickly and is also dependant on the garbage collection system to keep old stuff cleaned up. This whole process causes a situation where a table is called and unfortunately there is not enough heap left and CTD.
I took three of the worst offenders and converted the tables to .LTX files formatted in a certain way and then opened them in a file stream to search them and get the data. WHY? Windows 7 (Vista too but not as well) catches the LUA file calls and automatically uses 'streams' and they allocate OUTSIDE the 1 or 4 meg heap region as far out as 16gig which is Windows limit at present. Thus I have prevented these from overflowing by getting them completely out of the engines memory space.
Those using SSD drives will see this speed increase even more dramatically as these LTX files get cached by the drive and that's even more memory and performance gained.
Status...
Since this has proven to work so well I am going to convert all large tables to this method and streamline the data base system so a real dynamic system can emerge that will aid us in the Zones expansion. The Collector is already expanding the Zone using this new method and soon I hope others will embrace it.
As you can see, when you start as The Collector, that the door dialog is different. We are hooking into the game manager and choosing between the STALKER game mode and The Collector game mode. The Collector takes place after the final credits of NS or as a start option, this is our Free Play hook. The game play actually is initiated as a Multi-Player game and we save important play settings in those portions of the USER.LTX file instead of the ALL.SPAWN or game save files.
We are also preparing tools that will make creating game play easier, like:
Walk and Spawn - Imagine, wander to a spot, indoors or out, and open the spawn menu, select Snork, set how many, and press OK. Out pop snorks!!!, waiting to be told what to do. After you place your alife and items, you then use the command 'JUST DO IT STALKER' and a mod template is created for you to finish with dialog and actions.
Level Layering - We are in the process of segmenting resources and game play so that we can start a new mod like Secret Trails in the RECON level after there are no more tasks for NS there.
More - We will then use this as our Multi-Player environment instead of Death-Match style, it will be free roaming accross the WHOLE ZONE. Use the PDA to send messages to others playing, even though you are in the Cordon and they are all the way to the North. Cooperate in the quests and missions, become faction members and start a conlict together or apart. Modding tools that work within the game instead of having to use editing tools and much, much, more...
