Skip to Content

author

Author: Zeruel

Integrating Realm Engine into Unity

For the past few weeks I’ve been working on integrating Realm Engine into Unity. I started off by working through the steps of launching Realm Engine from within Unity. Originally Realm Engine would start from within a console application called BasicServer. This program would handle most of the basic tasks for start; reading in startup parameters, creating a new realm if one does not exist, loading an existing realm if requested to do so, starting the main game simulation / server process and proceeding from there. When a player would play a single player game we would have the game client launch an instance of BasicServer invisibly in the background with special single player startup parameters and have the client make a local connection to the server. For dedicated server hosting a server admin could either directly launch the BasicServer program or they could utilize a server creation / management / launching tool we had created (generally called RealmEngine.Server.Configuration.UI). Either way the game server would always start up inside of the BasicServer console app and generally be invisible in the background.

The first step in integrating Realm Engine into Unity was to replicate some of the things that BasicServer handled for us; primarily reading in startup parameters, creating/loading a realm and instantiating the main GameServer class on a separate thread and starting the GameServer. Since we’re now starting up our server process from within Unity itself (with the goal of having communication between Unity and Realm Engine) we needed to make it clear within our Unity code whether we’re acting as a client or server. Additionally if we’re acting as a server we need to transition between a few different scenes in Unity (for example, skipping the main menu scene where the player can log in, change game options, connect to a server, create a character and so on).

Getting through these initial stages of integration were fairly easy. The entire integration was not quite as seamless, however. Several immediate issues arose that had varying degrees of difficulty in resolving. For example, we had to streamline how we are dealing with paths to the file system. Previously our code acted under the assumption that assemblies were in the root directory. After integrating into Unity all assemblies moved to the \RealmExplorer_Data\Managed\ directory. If our code expected that we were in the root directory we really needed to look 2 levels up from where we were actually starting to find directories and files we were expecting.

Another issue that came about was a problem with how we get information specific to the game we are dealing with. Since Realm Engine is a generic technology not tied to any specific game we have an interface called IGameInfo. Each game is responsible for creating a class that implements this interface and provides certain common data that various areas of the game use; for example, the name of the game, copyright information and other data to identify each game uniquely. Realm Engine has a concrete class that we can program against to get this information. The class within Realm Engine dynamically (via reflection) looks through the the available assemblies to locate which one contains a class that implements IGameInfo and then loads that data once. While this worked perfectly for our stand alone builds we ran into an issue within Unity where the entire editor would freeze when compiling the game.

After working with Unity’s support staff (not knowing what the exact issue was at the time) they were able to help us determine that some kind of deadlock was occurring during compilation. This ended up being because Unity was in some way inspecting the static properties of the engine-level class which caused it to attempt to dynamically examine other assemblies. With the general information in mind that we were running into a deadlock during compilation from the Unity editor we were able to track down the cause and modify how it worked. Previously we would sort through a specific child directory for .dll files and then do an Assembly.ReflectionOnlyLoadFrom(file) to see if we could find the assembly that had the class we were looking for. If we found the class we would then fully load that assembly, instantiate the class via Activator.CreateInstance(…) and perform our one-time read. We worked around this issue by simply using AppDomain.CurrentDomain.GetAssemblies() to examine already loaded assemblies (which Unity takes care of for us) and finding the type that implements the expected interface. This was a very puzzling issue for several days (in part because there was absolutely no information of any kind in the Unity logs nor any kind of exception, error or anything that we could act on) but it was ultimately resolved.

Finally there were several challenges with handling Realm Engine’s dynamic script compilation from within Unity. Realm Engine (and thus Realm Explorer) has an incredibly powerful scripting system; our scripting system is one of the pillars of our engine technology. When Realm Engine starts up is locates script folders (which can be configured on a per-realm basis), reads in all of the script files, compiles them and then integrates them into the Entity-Component system. Unfortunately the CSharpCodeProvider and VBCodeProvider classes have some implementation issues in Unity. After a lot of testing and research none of the proposed work arounds that others had discovered were desirable. Fortunately an idea I came up with worked without a hitch. Ultimately my solution involved creating a separate compiler executable project and having the Realm Engine process (now hosted within Unity) launch the compiler executable and provide some startup parameters to it. The compiler would then perform the compilation and generate a compiled DLL with a file name and in a location based on instructions from the process requesting the compilation. From there the compiled assemblies could be easily loaded into Unity and the script contents loaded with reflection as per usual.

(By the way, we’re including the vast majority of gameplay-related scripts as raw C# files that users can view, modify and learn from. This will enable motivated players to do things like creating new items, craft recipes, NPCs and more. Players with a little C# or VB .NET programming capability will be able to do much more!)

The process of integrating Realm Engine within Unity ended up having many more obstacles than expected (some of which were quite difficult to resolve) but after working through all of these issues this milestone has been achieved.

I know I’ve been teasing some juicy in-game features (and they’re still coming!) but next time I’m going to talk a little bit about synchronizing calls from Realm Engine into Unity. Getting to this point was one of the major reasons for integrating Realm Engine into Unity in the first place and the end result is extremely satisfying!

Realm Explorer vs Realm Engine

realmenginevs

As I’ve mentioned a few times recently we have a code base for our own proprietary multiplayer RPG / sandbox engine called Realm Engine.  We’ve created this as a generic server-side game engine but we’re using it exclusively for Realm Explorer right now.  Our server technology has evolved numerous times throughout the alpha period.  The server code used for the very first release was extremely primitive in comparison to our current code base.  The pre-cursor to Realm Engine was released several versions ago while the code was still tightly integrated for Realm Explorer.  Further refactoring helped us to fully realize the goal of creating Realm Engine as a unique standalone server technology that we could build our games against.  As of right now the current internal build of Realm Explorer is built entirely on Realm Engine.

Early on we had a goal (both philosophically and due to technical considerations) to make our server technology independent of Unity.  We use Unity3D for the client side (rendering, handling user input [keyboard and mouse input], playing sounds, playing music, etc.) but our server technology was developed to exist outside of Unity.  One of the reasons for this was that when we started working on Realm Explorer Unity’s 64bit support was hit or miss (and the Unity editor itself was 32 bit).  This meant that we were limited in how much total memory we could use.  The Marching Cubes algorithm we were using for the terrain would use a lot of memory and was difficult to try and handle situations where multiple players could be in completely different parts of the world.  Having 1 player with terrain loaded around him would already use of a lot of memory – having a dozen players each in different parts of the world all needing their surrounding terrain loaded was essentially impossible!

Don’t get me wrong – Unity is an incredible piece of technology but we knew that we would need a high performance and light weight solution for implementing the server code and all of the RPG/Sandbox rules for a truly huge world with support for a large number of simultaneous players.

The benefit of rolling our own server solution was that we could have complete control over every aspect of it.  We also make extensive use of dynamic data loading/unloading, multithreading, dynamic runtime script compilation, implementation of API endpoints for further third party integrations and support and several other big features. The downsides (aside from having to create such a program from scratch) included not being able to leverage some of Unity’s powerful features (in particular anything related to 3D geometry, physics, collisions and so on).  Over the past 2 weeks we’ve had a fairly major paradigm shift.  Let me elaborate.

Previously we had our game client done entirely in Unity and our server technology as a series of separate C# .NET projects making up the server application.  We had some common code libraries that both the client and server would share but otherwise these two programs were independent.  Since Realm Engine is pretty mature at this point the major goals we set out for it have been mostly accomplished.    The big change that’s recently come about has been that we now host Realm Engine inside of Unity on a separate thread from Unity.

You might be saying to yourself, “Didn’t you just tell us why you created Realm Explorer as a separate program to live outside of Unity???”.  Why yes, that’s correct.  We’re now at a point where Realm Engine is able to do its heavy lifting very efficiently and having worked with it so long we can clearly see where and how we can leverage a direct interface with the Unity engine to further improve several things.  Over the past few weeks I worked through all of the integration work so that we could spawn a dedicated server process inside of Unity.  The next step (which is almost done) will be to create a communication layer between the Unity engine and Realm Engine.  This way they can both run separately but marshal commands and information about objects between each engine.

Aside from being able to now get some useful information that we haven’t been able to have on the server before (such as collision between objects, performing accurate raycasts on the server side and so on) this also gives us a huge leg up on how we’re handling our dynamic infinite terrain and building construction.

Oh no it looks like we’re out of time for today.  We’ll get into the details of terrain generation, biomes, lakes(??), rivers(?!), caves and dungeons(?!?!), AI (!!!) and more in future posts as we work towards another alpha release (speaking of which, the current plan is to only provide new builds [when they are ready] to people who picked up the game during the brief alpha period – Realm Explorer won’t be posted for sale again until it’s ready!).  For now I’ll be continuing with finalizing the Realm Engine -> Unity integration and the engine-to-engine communication layer.

In my next post I’ll likely be taking a look at new building and house construction.  Maybe a new video for this?  That’s all for now!

100 Programmers or more?

rs

A question that I get fairly often is, “How many people are working on Realm Explorer?”.  Gabriel pointed out to me recently that when I respond to emails like this I usually come across sounding really formal and corporate.  The truth is our team is small.  How small?  Well, Gabriel and I are the only two programmers.  I created most of the game functionality, the majority of Realm Engine (our proprietary moddable multiplayer server engine), most of the game client features, the scripting / mod system and the majority of the networking code, etc.  Gabriel has done an incredible job creating virtually 100% of our UI (including making it skinnable / moddable), tons of user-interaction stuff (all of the special behaviors for drag and drop inventory, smooth transitions for chat messages, beautiful styling and so on).  Gabriel also does all of our web stuff (including RealmSource .NET account creation / account management, game key management, player sessions, password recovery email functionality and all of that good stuff).

We also work with a long-time associate of ours (who I’ve had the pleasure of meeting on several occasions now despite us living on completely different continents) ArmedBee who helps get us the artists we need to do specific projects — everything from 3D modelling, animation, concept art and more.  Without his help the in-game graphics would probably be a couple of 3D blocks moving around the screen (backed by our super powerful multiplayer RPG engine, of course).  ArmedBee has also helped with a lot of technical aspects of setting up art content and working through the workflow of setting up new visual features.

As of right now… that’s everyone!  We’ve had a few others come and go over the years contributing pieces here and there but the majority of artwork and code that we’re using right now has been the result of the continued efforts of the three of us.

Having said that there are a lot of people I would love to work with in the future (on the music side especially!  I’m a huge fan of great game music) but I firmly believe in making sure people get paid for their work.

That’s all for now.  Check back next week for some news on recent work on Realm Engine integration and other technology improvements that are letting us do more great stuff!

More Environmental Improvements

Lots of stuff is in the works lately including a ton of work on terrain (generation, digging, etc.), new and improved structure building, more art assets (and some work related to animation improvement) and more.  Gabriel has been putting a lot of work into improving the UI framework as well.  Have I even shown the new server browser / server connection UI?  Well, here it is:

serverbrowser

The other things I mentioned aren’t ready to show yet (and they’ll best be represented by video anyway) but here’s a little look at some of the further improvements to materials and environment (these look much better in live action instead of just still photos too but these will have to do for now!)

charactercustomization01 charactercustomization02 charactercustomization03 charactercustomization04 environment01 environment02 environment03 environment04

As always we’re still building on our core Realm Engine technology (officially a thing for a few months now) and Realm Explorer is still multiplayer, moddable (C# and VB .NET on the server side), skinnable UI (HTML5-based with support for tons of web technologies like CSS, TypeScript and more [Gabriel will have to fill in the details here]) and with nearly infinite dynamically generated worlds to explore.

There’s more to discuss so keep an eye out for the next post!

Ongoing Progress

It’s been awhile since my last post but progress is still continuing with Realm Explorer.  The team has been ramping up with tons of new work across all areas of the game — new artwork, new game content, new features, improved performance and more.

One of the areas that hasn’t seen attention for awhile (until now) has been character customization.  We’re up to 29 male hair styles, 20 female hair styles (all with hair physics) as well as the addition of 6 total facial hair styles for male characters.  We also recently added 4 new head styles for the male characters (with 2 more on the way).  This will bring us to a total of 9 different base faces in addition to all of the hair and beard style and color options.  You can also now change your character’s underwear color (yay?).

Character Customization

There’s a lot of new stuff that hasn’t yet been seen (and a lot still in the works) but I’ll leave you with a few screenshots demonstrating the new and improved visuals we’ve been working on (don’t miss the UI details either!):

Character Selection In the woods Running