FreeRails: How to untangle code?

There are three basically different approaches

• Study the code, find the crucial parts that are connected but should not, find a solution to untangling them, then do the programming
• Study the code and untangle code wherever you find it, until all but the crucial part remains, then tackle it
• Study the code and rewrite the code (in a simpler form) in another set of sources which then could replace the original code at some point

The last method is very risky and may easily fail. The first one is less risky and overall probably taking the shortest amount of time. But somehow my head is not big enough to grasp the full complexity of FreeRails yet. So I go for the slow but relative safe way of taking the second option currently, slowly reading through the code base, frequently running the tests and running the game, optimizing locally, untangling the code one piece at a time. It's quite an effort, but it's also the only good way I know. It will take some time.

Civil: Next steps

There seems to be a major issue with the game. That is that the server does not seem to advance the game world except for the time. Units do not move, probably do not fire at each other too. There was an overlay layer for the action phase in the interface but it is commented out. Also the description of version 0.83 (the last official release) reads as if the game was real time then, so one would expect a continuous update. However, maybe that was not fully implemented in 0.83?

The question is: How can the game be fixed in this regard?

The second issue is that although the editor kind of works now with PyQt5 there is still a lot of work to do with it. I would like to use PyQt5 also for the main game in order to reduce dependencies, use less code (PyQt5 brings many features already).

Finally I would like use as much as possible from the old code, at least the whole game model and the graphics and the layers.

The crucial next steps should therefore be:

• Understand the inner workings of the code base
• Identify and separate into model, view, controller, server
• Find out why the game model is not updating as it should
• Replace the view and controller with PyQt5
• Add a more or less working editor
• Release
• Make a plan for the future development

Android coloring app for kids

Two years ago I started a coloring app for kids on Android. It's completely open source and loosely based on another open source coloring app. It's basically not a very challenging project and serves to also get to know the Android environment better.

The code is on Github.

However, after some work I stopped in an intermediate state. Recently, I rediscovered it after it got even some interest of others on Github and continue working on it now.

Statistics of open source games

The list is by no means complete but I thought I would share some statistics about the ~200 investigated open source games projects.

State

Approximately 50% are mature, 25% in development and not yet mature and about 20% abandoned.

This may be skewed though because games in development are definitely harder to spot than mature projects.

Most used Programming languages

• C++ (40%)
• C (24%)
• Java (8%)
• Python (8%)
• Lua (4%)
• Javascript (2%)

A solid knowledge in C/C++ or Java or Python will be very useful for aspiring open source contributors.

Copyright

Most often the code is licensed by

• GPL-2.0 (37%)
• GPL-3.0 (19%)
• MIT (13%)
• Custom (7%)

 which means that open source projects in general prefer to keep modifications open source too.

Civil: Refactoring, refactoring, refactoring

While upgrading the code base from Python 2, PyGame 1.5 and PyQt3/4 (state of the art in 2004) to Python 3, PyGame 1.9 and PyQt5 you need to approximately change only 20-30% of the code base and often enough it can be done by some refactoring.

However, with Python being a dynamically typed language, refactoring is not so simple. Also the Civil project misses unit tests. What worked in the end was:

• Heavy usage of regular expressions to modify code that follows a certain naming structure to, for example, exchange arguments. I learned quite a lot there too.
• Starting the program again and again, doing the same stuff over and over and fixing runtime errors one by one, so next time it runs a bit further.
• Using a source versioning system and saving intermediate steps often (with Git I often amend commits to be able to revert uncommitted changes if needed) so one can go back if nothing else helps.

By far the most typical runtime error I encountered was

AttributeError: 'module' x has no attribute 'y'

which mostly indicates that the variable has a different type of what people expected, which can have many different causes. Still, fixing things can be very satisfying. :)
 

FreeRails: How to reduce complexity of the code?

I looked at the code base of FreeRails now for quite some time. It's not simple to understand. There is quite a lot of complexity involved.

On the one hand that is expected. Managing railways isn't simple. Incorporating the logic of trains running on tracks with a delivery schedule, cargo producers, financial transactions, extending the track network in a changing world, ... that is even a difficult problem in the real world. :)

Mapping this to a client/server framework where the server even has to defend itself against illegal moves (no underwater tracks so far) of the client or offer some kind of undo capability adds its share to the problem.

Add some common practices like missing documentation or using overly complex (cyclic) dependencies and the code soon mutates into a huge spaghetti monster. (That is by no means an accuse of anyone, it's just what happens naturally. Also reading code is often harder than writing it.)

How to untangle the spaghetti, reduce the complexity and facilitate further development? That is the big question!

I'm not aware of an easy way to do that. It rather takes much time and the first steps are usually the hardest while later most things more or less fall in place. But the project is open source and on Github, so everyone can take part.

Railway Empire looks so nice

Most of the times commercial games look much better than open source games. A large group of skilled professionals working full time for many months with highly refined tools and frameworks - that cannot help but must outshine all attempts of a few partly skilled volunteers in their free time.

And so it is here too. There are indeed some Railway management games out there and since at least 10 years in quite good 3D graphics.

But the soon to be released Railway Empire (to be released on 26th January 2018) by Kalypso Media/Gaming Minds Studio just stuns me by the beautiful graphics.

Need a proof? Look at that gorgeous screenshot of a train in a landscape.

Makes me wonder how meaningful the revival of FreeRails is anymore? On the one hand it's fun to go through the code and improve it and surely the look can be improved dramatically from the look back then even with modest means, but on the other hand professional, commercial products have improved so dramatically from the early 2000s to now. Maybe there was a demand and a reason to clone RailRoad Tycoon in 2000-2005, but I'm not sure there is any reason now anymore.

Civil: Screenshots of 0.84

The main screen

This is the main screen as it looked in 2004 but in 2018.

FreeRails: Refactoring, refactoring, refactoring

The complexity of the current code base of FreeRails is quite high. There are many classes and methods which are highly (sometimes cyclically) dependent on each other and often enough comments are missing. Additionally the game was thought to be a complex multi-player game with a server-client structure, with pathfinding for the trains, and complicated updates of the game world state (especially the trains).

That might be a part of the reason why development on FreeRails more or less stalled after 2005. It's very quite difficult to read this code and possibly extend or modify it.

On the other hand, the original makers used tests to continuously check their code and they wrote down their design ideas in a document. Also Java as a strongly typed language running on a standardized virtual machine, offers many tools to entangle such difficult to understand code bases without loosing the ability to execute the game.

That's why I could refactor the code base now for a week without loosing anything. Using static code analysis (code inspections in IntelliJ) and removing unused code, applying automatic fixes and manual improvements, replacing custom solutions by Java standards (which did not all exist in 2005 like Lambda functions in Java 8) I could reduce the number of code lines from 39k lines of code in version 0.4.1 to 33k lines of code now which means roughly 15% less code to understand. And as I said without loosing anything in functionality, the game looks and runs as before.

Civil: Minor release 0.84

After getting the game to run, I decided to release the current state of the code as version 0.84. Downloads are hosted on PyPI and there is a Windows installer on bintray.

I used the right to increase the version of the GPL-2.0 open source license and published the code under the GPL-3.0 license. However, the content and look is probably very similar to version 0.83 from 2004. The editor is not yet working.

There are two ways to get Civil 0.84:

1. Use the Windows installer

Windows Installer (18 MB) comes bundled with its own Python runtime.

2. Use PyPI

Install with 'pip install civil'.

Civil: Getting the game to run

Getting a game that saw last development many years ago to run is a hard task. It used Python 2, I wanted to use Python 3. It used PyGame 1.5.3, now the actual version is 1.9.3. And because there was no ' binary distribution' I cannot be sure if ever it run. And there are parts written in C which I would like to avoid. Python is still there and even more popular now, but being a weakly typed language, makes it harder to refactor or detect unused code.

In the end I managed (and could restructure the project too) by

• Repeatedly (many, many times) starting the game, letting errors happen, looking at the stack trace, setting breakpoints, inspecting variables, fixing code.
• Replacing old  code with fixed code in multiple location simultaneously using regular expressions and project wide find&replace in PyCharm
• Using PyCharms refactoring capabilities
• Using the 2to3 script to update Python 2 to Python 3 code

A mixture of all these methods finally made Civil run for me on a modern system with the limitation, that units do not move on the battle field. I'm not sure, if the original developers actually fixed that.

Civil seems to be free of obvious bugs.

FreeRails: Minor release 0.4.1

After an initial round of cleanup of the code base, and building the original FreeRails 0.4.0 code with Java 8, I released this slightly improved version as 0.4.1. Downloads will be hosted on bintray.

Download 0.4.1

I used the right to increase the version of the GPL-2.0 open source license and published the code under the GPL-3.0 license. However, the content and look is almost exactly the same as version 0.4.0 from 2008.