I’ll keep this one brief for once!

I spent the past several days working on a solution to get high quality trees both looking and performing well.  Aside from setting up the art assets themselves a big effort went into creating a high performance LOD system that could handle enormous amounts of objects without slowing things down.  LOD means “Level of Detail” and is a common concept in game development.  The idea is that objects that are closer to the camera are more detailed while those that are further away are more simplistic and less detailed.  The goal is to improve rendering performance while still producing as seamless an experience for the player as possible.  LOD in general should not be very noticeable by the player.

I started off working on the code and implementing a solution that was extremely fast.  Here’s a video of the first iteration of development:

Blocks that appear in green represent the highest detail level, those in blue a level below green, those in yellow a level below blue and finally red representing the lowest detail level.  Realistically not every object in the game will have the same number of detail levels.  Some may have 2, others may have 4 or 5.  The system had to be created in such a way that it would transition appropriately regardless of how many actual detail levels existed for any particular object.  I also wanted it to be able to scale to hundreds of thousands of objects and have minimal impact on the CPU, garbage collection, memory use and other system resources that would slow the game down.

After a lot of work I was able to go from the block demo in the first video to a test with 250,000 high detail trees loaded at once and each randomly rotated.  So far I’m pleased with the result.  Note that there’s no editing in this video.  Everything was recorded in real time to demonstrate the speed and performance of the new LOD system (which I’m calling RealmLOD internally — creative name right?).

We don’t plan on having any areas with hundreds of thousands of trees (or anything else for that matter) packed together at the same time but this stress test helped validate the performance of the system.  It should easily handle what we have planned for realistic numbers of environment objects.

Hope you enjoyed this peak into the ongoing internal development of Realm Explorer.  Share your thoughts in the comments!

EDIT:

After re-watching the video on youtube I noticed that unfortunately the video quality came out really bad.  It may give at least a little idea of the performance (there’s no lag at all) but it definitely doesn’t give a good impression of the visual quality of the environment.

Here are a few screenshots from the same test:

April 24, 2017 Zeruel Zeruel No comments

For the past few weeks I’ve been working on integrating Realm Engine into Unity. I started off by working through the steps of launching Realm Engine from within Unity. Originally Realm Engine would start from within a console application called BasicServer. This program would handle most of the basic tasks for start; reading in startup parameters, creating a new realm if one does not exist, loading an existing realm if requested to do so, starting the main game simulation / server process and proceeding from there. When a player would play a single player game we would have the game client launch an instance of BasicServer invisibly in the background with special single player startup parameters and have the client make a local connection to the server. For dedicated server hosting a server admin could either directly launch the BasicServer program or they could utilize a server creation / management / launching tool we had created (generally called RealmEngine.Server.Configuration.UI). Either way the game server would always start up inside of the BasicServer console app and generally be invisible in the background.

The first step in integrating Realm Engine into Unity was to replicate some of the things that BasicServer handled for us; primarily reading in startup parameters, creating/loading a realm and instantiating the main GameServer class on a separate thread and starting the GameServer. Since we’re now starting up our server process from within Unity itself (with the goal of having communication between Unity and Realm Engine) we needed to make it clear within our Unity code whether we’re acting as a client or server. Additionally if we’re acting as a server we need to transition between a few different scenes in Unity (for example, skipping the main menu scene where the player can log in, change game options, connect to a server, create a character and so on).

Getting through these initial stages of integration were fairly easy. The entire integration was not quite as seamless, however. Several immediate issues arose that had varying degrees of difficulty in resolving. For example, we had to streamline how we are dealing with paths to the file system. Previously our code acted under the assumption that assemblies were in the root directory. After integrating into Unity all assemblies moved to the \RealmExplorer_Data\Managed\ directory. If our code expected that we were in the root directory we really needed to look 2 levels up from where we were actually starting to find directories and files we were expecting.

Another issue that came about was a problem with how we get information specific to the game we are dealing with. Since Realm Engine is a generic technology not tied to any specific game we have an interface called IGameInfo. Each game is responsible for creating a class that implements this interface and provides certain common data that various areas of the game use; for example, the name of the game, copyright information and other data to identify each game uniquely. Realm Engine has a concrete class that we can program against to get this information. The class within Realm Engine dynamically (via reflection) looks through the the available assemblies to locate which one contains a class that implements IGameInfo and then loads that data once. While this worked perfectly for our stand alone builds we ran into an issue within Unity where the entire editor would freeze when compiling the game.

After working with Unity’s support staff (not knowing what the exact issue was at the time) they were able to help us determine that some kind of deadlock was occurring during compilation. This ended up being because Unity was in some way inspecting the static properties of the engine-level class which caused it to attempt to dynamically examine other assemblies. With the general information in mind that we were running into a deadlock during compilation from the Unity editor we were able to track down the cause and modify how it worked. Previously we would sort through a specific child directory for .dll files and then do an Assembly.ReflectionOnlyLoadFrom(file) to see if we could find the assembly that had the class we were looking for. If we found the class we would then fully load that assembly, instantiate the class via Activator.CreateInstance(…) and perform our one-time read. We worked around this issue by simply using AppDomain.CurrentDomain.GetAssemblies() to examine already loaded assemblies (which Unity takes care of for us) and finding the type that implements the expected interface. This was a very puzzling issue for several days (in part because there was absolutely no information of any kind in the Unity logs nor any kind of exception, error or anything that we could act on) but it was ultimately resolved.

Finally there were several challenges with handling Realm Engine’s dynamic script compilation from within Unity. Realm Engine (and thus Realm Explorer) has an incredibly powerful scripting system; our scripting system is one of the pillars of our engine technology. When Realm Engine starts up is locates script folders (which can be configured on a per-realm basis), reads in all of the script files, compiles them and then integrates them into the Entity-Component system. Unfortunately the CSharpCodeProvider and VBCodeProvider classes have some implementation issues in Unity. After a lot of testing and research none of the proposed work arounds that others had discovered were desirable. Fortunately an idea I came up with worked without a hitch. Ultimately my solution involved creating a separate compiler executable project and having the Realm Engine process (now hosted within Unity) launch the compiler executable and provide some startup parameters to it. The compiler would then perform the compilation and generate a compiled DLL with a file name and in a location based on instructions from the process requesting the compilation. From there the compiled assemblies could be easily loaded into Unity and the script contents loaded with reflection as per usual.

(By the way, we’re including the vast majority of gameplay-related scripts as raw C# files that users can view, modify and learn from. This will enable motivated players to do things like creating new items, craft recipes, NPCs and more. Players with a little C# or VB .NET programming capability will be able to do much more!)

The process of integrating Realm Engine within Unity ended up having many more obstacles than expected (some of which were quite difficult to resolve) but after working through all of these issues this milestone has been achieved.

I know I’ve been teasing some juicy in-game features (and they’re still coming!) but next time I’m going to talk a little bit about synchronizing calls from Realm Engine into Unity. Getting to this point was one of the major reasons for integrating Realm Engine into Unity in the first place and the end result is extremely satisfying!

March 3, 2017 Zeruel Zeruel 2 comments

As I’ve mentioned a few times recently we have a code base for our own proprietary multiplayer RPG / sandbox engine called Realm Engine.  We’ve created this as a generic server-side game engine but we’re using it exclusively for Realm Explorer right now.  Our server technology has evolved numerous times throughout the alpha period.  The server code used for the very first release was extremely primitive in comparison to our current code base.  The pre-cursor to Realm Engine was released several versions ago while the code was still tightly integrated for Realm Explorer.  Further refactoring helped us to fully realize the goal of creating Realm Engine as a unique standalone server technology that we could build our games against.  As of right now the current internal build of Realm Explorer is built entirely on Realm Engine.

Early on we had a goal (both philosophically and due to technical considerations) to make our server technology independent of Unity.  We use Unity3D for the client side (rendering, handling user input [keyboard and mouse input], playing sounds, playing music, etc.) but our server technology was developed to exist outside of Unity.  One of the reasons for this was that when we started working on Realm Explorer Unity’s 64bit support was hit or miss (and the Unity editor itself was 32 bit).  This meant that we were limited in how much total memory we could use.  The Marching Cubes algorithm we were using for the terrain would use a lot of memory and was difficult to try and handle situations where multiple players could be in completely different parts of the world.  Having 1 player with terrain loaded around him would already use of a lot of memory – having a dozen players each in different parts of the world all needing their surrounding terrain loaded was essentially impossible!

Don’t get me wrong – Unity is an incredible piece of technology but we knew that we would need a high performance and light weight solution for implementing the server code and all of the RPG/Sandbox rules for a truly huge world with support for a large number of simultaneous players.

The benefit of rolling our own server solution was that we could have complete control over every aspect of it.  We also make extensive use of dynamic data loading/unloading, multithreading, dynamic runtime script compilation, implementation of API endpoints for further third party integrations and support and several other big features. The downsides (aside from having to create such a program from scratch) included not being able to leverage some of Unity’s powerful features (in particular anything related to 3D geometry, physics, collisions and so on).  Over the past 2 weeks we’ve had a fairly major paradigm shift.  Let me elaborate.

Previously we had our game client done entirely in Unity and our server technology as a series of separate C# .NET projects making up the server application.  We had some common code libraries that both the client and server would share but otherwise these two programs were independent.  Since Realm Engine is pretty mature at this point the major goals we set out for it have been mostly accomplished.    The big change that’s recently come about has been that we now host Realm Engine inside of Unity on a separate thread from Unity.

You might be saying to yourself, “Didn’t you just tell us why you created Realm Explorer as a separate program to live outside of Unity???”.  Why yes, that’s correct.  We’re now at a point where Realm Engine is able to do its heavy lifting very efficiently and having worked with it so long we can clearly see where and how we can leverage a direct interface with the Unity engine to further improve several things.  Over the past few weeks I worked through all of the integration work so that we could spawn a dedicated server process inside of Unity.  The next step (which is almost done) will be to create a communication layer between the Unity engine and Realm Engine.  This way they can both run separately but marshal commands and information about objects between each engine.

Aside from being able to now get some useful information that we haven’t been able to have on the server before (such as collision between objects, performing accurate raycasts on the server side and so on) this also gives us a huge leg up on how we’re handling our dynamic infinite terrain and building construction.

Oh no it looks like we’re out of time for today.  We’ll get into the details of terrain generation, biomes, lakes(??), rivers(?!), caves and dungeons(?!?!), AI (!!!) and more in future posts as we work towards another alpha release (speaking of which, the current plan is to only provide new builds [when they are ready] to people who picked up the game during the brief alpha period – Realm Explorer won’t be posted for sale again until it’s ready!).  For now I’ll be continuing with finalizing the Realm Engine -> Unity integration and the engine-to-engine communication layer.

In my next post I’ll likely be taking a look at new building and house construction.  Maybe a new video for this?  That’s all for now!

December 29, 2016 Zeruel Zeruel 1 comment

A question that I get fairly often is, “How many people are working on Realm Explorer?”.  Gabriel pointed out to me recently that when I respond to emails like this I usually come across sounding really formal and corporate.  The truth is our team is small.  How small?  Well, Gabriel and I are the only two programmers.  I created most of the game functionality, the majority of Realm Engine (our proprietary moddable multiplayer server engine), most of the game client features, the scripting / mod system and the majority of the networking code, etc.  Gabriel has done an incredible job creating virtually 100% of our UI (including making it skinnable / moddable), tons of user-interaction stuff (all of the special behaviors for drag and drop inventory, smooth transitions for chat messages, beautiful styling and so on).  Gabriel also does all of our web stuff (including RealmSource .NET account creation / account management, game key management, player sessions, password recovery email functionality and all of that good stuff).

We also work with a long-time associate of ours (who I’ve had the pleasure of meeting on several occasions now despite us living on completely different continents) ArmedBee who helps get us the artists we need to do specific projects — everything from 3D modelling, animation, concept art and more.  Without his help the in-game graphics would probably be a couple of 3D blocks moving around the screen (backed by our super powerful multiplayer RPG engine, of course).  ArmedBee has also helped with a lot of technical aspects of setting up art content and working through the workflow of setting up new visual features.

As of right now… that’s everyone!  We’ve had a few others come and go over the years contributing pieces here and there but the majority of artwork and code that we’re using right now has been the result of the continued efforts of the three of us.

Having said that there are a lot of people I would love to work with in the future (on the music side especially!  I’m a huge fan of great game music) but I firmly believe in making sure people get paid for their work.

That’s all for now.  Check back next week for some news on recent work on Realm Engine integration and other technology improvements that are letting us do more great stuff!

December 9, 2016 Zeruel Uncategorized No comments

Lots of stuff is in the works lately including a ton of work on terrain (generation, digging, etc.), new and improved structure building, more art assets (and some work related to animation improvement) and more.  Gabriel has been putting a lot of work into improving the UI framework as well.  Have I even shown the new server browser / server connection UI?  Well, here it is:

The other things I mentioned aren’t ready to show yet (and they’ll best be represented by video anyway) but here’s a little look at some of the further improvements to materials and environment (these look much better in live action instead of just still photos too but these will have to do for now!)

As always we’re still building on our core Realm Engine technology (officially a thing for a few months now) and Realm Explorer is still multiplayer, moddable (C# and VB .NET on the server side), skinnable UI (HTML5-based with support for tons of web technologies like CSS, TypeScript and more [Gabriel will have to fill in the details here]) and with nearly infinite dynamically generated worlds to explore.

There’s more to discuss so keep an eye out for the next post!

November 18, 2016 Zeruel Zeruel 1 comment

It’s been awhile since my last post but progress is still continuing with Realm Explorer.  The team has been ramping up with tons of new work across all areas of the game — new artwork, new game content, new features, improved performance and more.

One of the areas that hasn’t seen attention for awhile (until now) has been character customization.  We’re up to 29 male hair styles, 20 female hair styles (all with hair physics) as well as the addition of 6 total facial hair styles for male characters.  We also recently added 4 new head styles for the male characters (with 2 more on the way).  This will bring us to a total of 9 different base faces in addition to all of the hair and beard style and color options.  You can also now change your character’s underwear color (yay?).

There’s a lot of new stuff that hasn’t yet been seen (and a lot still in the works) but I’ll leave you with a few screenshots demonstrating the new and improved visuals we’ve been working on (don’t miss the UI details either!):

October 26, 2016 Zeruel Zeruel 5 comments

First a word from our leader Zeruel:

Realm Explorer definitely is still in development.
Our team has been working on it every day. You may not be aware of this but our main distribution platform was Desura. Unfortunately Desura had serious financial problems and hadn’t paid us in over a year — hundreds of other developers weren’t paid either. Desura finally went bankrupt and closed down a few months ago so we have no chance of being paid by them. As you can imagine this has slowed things down for us but we’re still working on the game regardless. Desura also made it very difficult to get new builds of the game released. In fact, right now it’s impossible to release any new updates to Desura at all.

We’ve committed tens of thousands of dollars from our own pockets (as well as thousands of hours programming) this past year alone to help keep development going.

Our goal is to release Realm Explorer on Steam in 2016. If you have a RealmSource .NET account we will send out an email when things are ready in the next few months. Even if you didn’t purchase Realm Explorer during the alpha period you can still sign up for a free RealmSource .NET account at https://www.realmsource.net/Account/Register. Our target is to have the game as close to “complete” as possible for the Steam release. We’ll continue to support it with patches and new content after the Steam release.

In addition to this we have an incredibly powerful new mod system — nearly every aspect of the game can be modified by C# and VB .NET scripts (your choice of programming language) from the server-side. The client UI can also be skinned and customized using common web-development technologies like HTML and CSS. We support drop-in UI skin packs so that people can create their own User Interface changes to change the look, feel and layout of UI elements (or use packs made by other people should they choose). Users with some programming knowledge will be able to develop their own content – everything from craft recipes and item qualities, NPCs and AI behavior to terrain generation and completely new game systems.

In conclusion Realm Explorer is far from dead — our internal code base and art assets are the best they’ve ever been. Desura is dead and we can no longer support releases on the platform (in addition to not having been paid for any games sold there). In order to do a proper Steam release we really want the game to be the best it can be so that we can ensure a strong release.

Thanks for your continued interest in Realm Explorer!

If you’d like to recover your Realm Explorer key from Desura you can find instructions we made for this here. If you don’t have a RealmSource.NET account, you can register with us here, or if you just need to add it to your pre-existing account. you can do so here,

Since I made the last blog post in May, we’ve completed over 80 features for Realm Explorer as well as continuing to fine-tune existing features, fix bugs, and more.

If you’ve got any questions about our continued development of Realm Explorer, leave a comment!

January 11, 2016 Gabriel Gabriel 8 comments

May has flown by and we’ve been doing a lot of work that is all coming together nicely.

Here I am going to go through our tracker and summarize the work we’ve been doing.

Over at https://www.realmsource.net I added a bit of “two-factor authentication”. When you login to your account there we’ll send you an e-mail if we don’t recognize the device you used. We take your account security seriously, and this is just one of several new safeguards we’ve added to our systems recently.

In the game UI, status bars went on a diet, they’re now about half the size they were. And to give a “hud-less HUD” effect, if any of your status bars are full, they’ll fade out after 5 seconds. With the action bar, it fade outs after you’ve changed the selection of your active item as well, but we still show your items and slots as minimally as possible. I’ll be adding a game option soon to disable this new feature if you’re so inclined.

We bring everything into full view when your inventory is open. You can also see that the equipment slot graphics have been updated with outlines to enhance use-ability.

Completely new is the in-game radar, which shows you nearby objects like npcs, plants, players, containers, corpses, items. You might be able to see a sheep off to my left in the darkness, but you can definitely see the blue dot on the radar for it…

You can customize the radar in the options menu. There’s a lot to configure here, so we now have a “Restore Defaults” button on the general options page.

The container UI has been expanded to support action slots and equipment slots. Here I am inspecting a sheep corpse:

The options UI now offers new video settings for Clouds, Antialiasing, and SSAO which makes the game really shine if your PC can handle it! We also vastly increased the range of the mouse sensitivity slider (by ten times!) for folks with all sorts of input needs.

The multiplayer browser UI has also been changed a bit from what I previewed in an earlier post. It also now remembers your preferences for passworded/full/empty/server sort order, and will let you know when those options are preventing any servers from showing up in the list. You’ll also be able to tab to your single player realms all from this one window.

The server Configuration UI has been revamped and has a whole new look. We also added a new “HTML Log” option which makes the log a lot more readable with pretty colors. Server admins can also disable radar or just specific blips on the radar, coordinates, and other settings:

The server Admin UI now has a “time slider” which allows you to quickly change the time by moving the range knob around. This is great for making videos or for us to see how light and darkness affects materials in game.

We also fixed a handful of bugs that cropped up along the way too. Most importantly we discovered a pretty serious and frequent crash that was occurring and after working with Coherent Labs we have an updated version of the software that displays the UI.

And according to Matt, Hostile NPCs are just a few days away… Dyox says we’ll see a completely overhauled terrain system within a month.

May 28, 2015 Gabriel Gabriel No comments

Well it’s been a busy 10 days since my last post, tons of work is getting done. It’s all good 🙂

I updated the login process for the game, you’ll now be presented with the login screen when the main menu appears.

Everything is now streamlined, the register, license activation, and forgot password functionality is all in the same view. The cleans up a lot of state transfer that we were doing before.

If you don’t have an internet connection we of course give you the option to play offline up to 30 days after your last login.

In regards to the login process itself, we were previously using a strong encryption to store and transmit your password, however, now I’ve beefed this up even more. The next version of the game will automatically forget your login information and all you need to do is re-enter your username and login and you’ll be all set.

Once you’ve logged in, you’ll be able to logout (or login, if you’re playing offline) with a button available at the sign post in the main menu.

With this change, along with the new multiplayer browser, there’s no more login page at the billboard, no more multiplayer mode selection. This means as soon as you click the “Multiplayer” board, you’ll be taken straight to the multiplayer browser.

Another initiative that was part of this whole login re-work was to remove the reliance on a .NET networking technology we were using. This removal will benefit us in the future when it comes to porting to other platforms. We’ve moved to a more standard system that enables us to be a lot more flexible when it comes to how we handle requests coming from the client as well.

In game, we now have a new progress indicator for interacting with objects. Here you can see I have picked some cotton from this plant already and am now gathering more. In the lower right, we now have icons indicating the icon for the item you acquired. You can also see the compass I mentioned in my last post in action.

Also, with no more sites to display in the community panel on the in-game esc menu, I’ve simplified the menu-ing there as well. We could display this blog but who actually reads this stuff?

That’s all for now, see you around.

April 30, 2015 Gabriel Gabriel No comments

Well, what the heck have I been up to…? Oh I know! Let’s check the commit log…

We were using WCF for calls to our login and other services, however these would only work on Windows game clients. WCF in Unity also led to some undesirable delays which might briefly freeze up the client which we seemed to be experiencing more of in recent builds. We’ve moved to a web API using HTTPS, I too the opportunity to also beef up and require security across our services.

The loading screen timeout for displaying the “Cancel” button has been changed to 20 seconds since the previous 90 seconds was very long given how fast we are able to load into the game now.

Added a compass to just above the action bar which now tells you what direction you are walking in. The compass can be disabled in the game options.

Improved the look of our buttons in the game, there is now a “mouse down/clicked” state for buttons which makes it obvious the button is being/will be clicked, instead of just the two normal/hover states we had previously. I also found a bunch of buttons weren’t making sounds when clicked so I updated them to do so.
If you find a button that isn’t making noise let me know!

April 20, 2015 Gabriel Gabriel No comments