Logo

shenlei flasheart · 01-22-2010, 07:23 AM

Last night those of us on the grid had an opportunity to experience another first with OpenSim. In reading the following cross-post from the Science Sim Users Group, please keep in mind that three years ago, OpenSim was a concept, nothing more. It's made doubly poignant for those of us users who don't write a scrap of code (like me) because I remember all too well those first ventures into OpenSim in September 2007: Ruth on Dot, and nothing was persistent, and nothing really worked, except there was this alternate frontier.

I learned about the scale tests while I was standing in one of our park areas that I was busily planting sculpted, rotating swans. OpenSim has just come so far from those early days.

Scale test post from Mic Bowman follows:

this is just fyi... and a very positive comment about how far opensim has come in recent months!

as part of sizing the hw requirements for a mirror world project we're exploring... we wanted to do some scalability tests on the capacity of individual simulators in terms of the total number of regions. we wanted baseline numbers that focus on just the most simple region configuration: a completely empty region with default terrain. that is... this is JUST simulator overhead.

all tests are done on one of our blades... dual proc, quad core with 16G ram running 64bit ubuntu. mono 2.6. and the tests are hosting 1024 regions in a 32x32 grid. the simulator configuration was our standard sciencesim config (XEngine, ODE, groups, wind, sun, etc).

the first configuration ran all 1024 regions in *one* simulator. i honestly figured this would crash quickly. it didn't. we managed to get all 1024 regions created and running well enough to walk around. it did take almost 8 hours to start. the first couple regions were created in 1-2 seconds each. by the end, it was taking 4-5 minutes per region. clearly there is something quadratic in there (stop using linear lists, they are evil! :-). but it could have been the mono garbage collector. who knows... not sure its worth too much investigation because i can't imagine ever running a config like this for real. the simulator did crash when we opened the map in the viewer. the crash was caused because we ran out of sockets. while it was running, the simulator used just over 10G of ram and was running at about 700% CPU utilization (kind of scary to see load averages in the 900 range!).

the second configuration ran 16 simulators each with 64 regions. startup took about 30 minutes (each of the 16 simulators avoided the quadratic "knee" we hit with the one big simulator). consumed about 11G of memory and was again consuming essentially all cycles on the machine (completely idle regions aren't very idle). all 16 simulators died just after startup with a "too many open files" error. not sure what caused it, but all of them died loading the same terrain dll as part of some wildcard function looking for dlls. that one is an interesting bug.

the final configuration and the one we're shooting for in the short term runs 16 simulators, each with an 8x8 megaregion. again startup was around 40 minutes. we might be able to cut that time down by starting all 16 simulators simultaneously rather than 4 at a time. i really just wanted to make sure that some of the spikes we see in startup didn't cause failures (some race condition causes all threads to consume maximum processor cycles randomly on startup right now). and, well... it just worked. i figured the viewer would die horribly (it can't handle 250K "real" prims very well) but it survived just fine. turn off far clip with 8x8 megaregions providing neighbors and you are "capable" of contacting simulators in a 24x24 region range!. the view is pretty cool though it seems to not go beyond 12x12. :-) there are a bunch of problems (like you can't see the terrain in the region you're standing in)... but there are a LOT of little humps of terrain in view. oh... and it takes a lot of patience to get everything loaded.

so... the conclusion... this opensim thing is pretty amazing! good work everyone.

--mic
*******

Great work, folks. I'm looking forward to Year Four.

Shenlei E. Winkler
Author, Designing Dreams: Best Practices for the Art & Business of Avatar Apparel Design & Development
Author, Shengri La Spirit: A Designer's Perspective of the Making of OpenSim
CEO
Fashion Research Institute, Inc.

Fashion Research Institute's Shengri La

Register for FTT495 VIRTUAL FASHING
Buffalo State College - Course Listings for Summer 2010 - Session A

WarKirby Magojiro · 01-22-2010, 07:38 AM

I'd be interested to know how transitions between such sims work. is it any smoother than SL ?

megasims as described are pretty much a wet dream of every vehicle/motorsport/transit oriented community in SL.

01-22-2010, 07:23 AM	#1 (permalink)
shenlei flasheart Utopian Rocket Queen Join Date: Jul 2007 Location: New York Posts: 14	Some Scalability Tests from ScienceSim Last night those of us on the grid had an opportunity to experience another first with OpenSim. In reading the following cross-post from the Science Sim Users Group, please keep in mind that three years ago, OpenSim was a concept, nothing more. It's made doubly poignant for those of us users who don't write a scrap of code (like me) because I remember all too well those first ventures into OpenSim in September 2007: Ruth on Dot, and nothing was persistent, and nothing really worked, except there was this alternate frontier. I learned about the scale tests while I was standing in one of our park areas that I was busily planting sculpted, rotating swans. OpenSim has just come so far from those early days. Scale test post from Mic Bowman follows: this is just fyi... and a very positive comment about how far opensim has come in recent months! as part of sizing the hw requirements for a mirror world project we're exploring... we wanted to do some scalability tests on the capacity of individual simulators in terms of the total number of regions. we wanted baseline numbers that focus on just the most simple region configuration: a completely empty region with default terrain. that is... this is JUST simulator overhead. all tests are done on one of our blades... dual proc, quad core with 16G ram running 64bit ubuntu. mono 2.6. and the tests are hosting 1024 regions in a 32x32 grid. the simulator configuration was our standard sciencesim config (XEngine, ODE, groups, wind, sun, etc). the first configuration ran all 1024 regions in one simulator. i honestly figured this would crash quickly. it didn't. we managed to get all 1024 regions created and running well enough to walk around. it did take almost 8 hours to start. the first couple regions were created in 1-2 seconds each. by the end, it was taking 4-5 minutes per region. clearly there is something quadratic in there (stop using linear lists, they are evil! :-). but it could have been the mono garbage collector. who knows... not sure its worth too much investigation because i can't imagine ever running a config like this for real. the simulator did crash when we opened the map in the viewer. the crash was caused because we ran out of sockets. while it was running, the simulator used just over 10G of ram and was running at about 700% CPU utilization (kind of scary to see load averages in the 900 range!). the second configuration ran 16 simulators each with 64 regions. startup took about 30 minutes (each of the 16 simulators avoided the quadratic "knee" we hit with the one big simulator). consumed about 11G of memory and was again consuming essentially all cycles on the machine (completely idle regions aren't very idle). all 16 simulators died just after startup with a "too many open files" error. not sure what caused it, but all of them died loading the same terrain dll as part of some wildcard function looking for dlls. that one is an interesting bug. the final configuration and the one we're shooting for in the short term runs 16 simulators, each with an 8x8 megaregion. again startup was around 40 minutes. we might be able to cut that time down by starting all 16 simulators simultaneously rather than 4 at a time. i really just wanted to make sure that some of the spikes we see in startup didn't cause failures (some race condition causes all threads to consume maximum processor cycles randomly on startup right now). and, well... it just worked. i figured the viewer would die horribly (it can't handle 250K "real" prims very well) but it survived just fine. turn off far clip with 8x8 megaregions providing neighbors and you are "capable" of contacting simulators in a 24x24 region range!. the view is pretty cool though it seems to not go beyond 12x12. :-) there are a bunch of problems (like you can't see the terrain in the region you're standing in)... but there are a LOT of little humps of terrain in view. oh... and it takes a lot of patience to get everything loaded. so... the conclusion... this opensim thing is pretty amazing! good work everyone. --mic ******* Great work, folks. I'm looking forward to Year Four. Shenlei E. Winkler Author, Designing Dreams: Best Practices for the Art & Business of Avatar Apparel Design & Development Author, Shengri La Spirit: A Designer's Perspective of the Making of OpenSim CEO Fashion Research Institute, Inc. Fashion Research Institute's Shengri La Register for FTT495 VIRTUAL FASHING Buffalo State College - Course Listings for Summer 2010 - Session A __________________ Shenlei E. Winkler Founder & CEO, Fashion Research Institute, Inc. Author: Shengri La Spirit: A Designer's Perspective of the Making of OpenSim Author: Designing Dreams: The Art and Business of Avatar Apparel Design and Development www.fashionresearchinstitute.com www.shenlei.com

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

Logo

Site Navigation