How to rewrite someone else's project (in another language)
How to rewrite someone else's project (in another language)
A couple of months ago I embarked on a journey. There was an app which I adored and which was a victim of inevitable technological obsolescence. It was a cross-platform app, a feature that demands a certain level of effort to get right. So, reasonably, it was based on a cross-platform runtime which would abstract away anything platform-specific and let you focus on the meat of your project instead.
It used Flash (or Adobe AIR to be precise), and Flash is dead.
So I had an idea that it is within my skills to recreate this app in something more modern and with a bit more longevity to it. An open source game engine with excellent app-making capabilities, perhaps. This was a couple of months ago, and today I am a happy author of a library and a front-end application for said library, both of which are well beyond a concept and well within reach from the state of completion.
In a span of 7 weeks I ported about 85% of the functionality of the library the original app was based on, and then the last couple of weeks I was making a stable progress with the app itself (a reveal this week, perhaps?). The original project was done entirely in ActionScript 3, targeting Flash native API, with everything created through code. My project, on the other hand, is a lot of C++ and some GDScript, leveraging Godot types, abstractions, and UI tech.
I want to share how this has come to be, and maybe give you some ideas how to approach similar problems.
First thing first, can I even do this?
Legal implications of porting a project made by someone else may not be a huge concern if you're doing it for fun, for personal use, and with no intent to share the results with the world. But in my case the goal was specifically to share a tool with other people. This meant that I had to know ahead of time that I was allowed to do such a thing.
There is always the hard way. The clean room approach — a reverse engineering strategy where you are only allowed to investigate the behavior as a user and can never look at the implementation. The following article will conveniently avoid this subject, as it doesn't apply to my circumstances. But do consider this approach very carefully, especially if this is the only way it would be possible for you to complete your project. Preferably, consult a lawyer.
In my case, prospects were favorable. The original project was open sourced under a permissive license (BSD-2-Clause-Views), and so was the underlying library (MIT). This meant that derivative work was permitted.
Now, even permissive OSS licenses aren't explicitly clear about source code rewrites in another language. My suggestion here is to do everything in your power to reinforce the idea that your work is done in good faith. Always credit the original project, and if you can, reach out to the original creator for a blessing. Chances are, they are going to be thrilled to learn that someone is willing to carry the torch.
Technical evaluation, or how to fit a square peg in a round hole
A layman and a neophyte may consider coding software to be some sort of magic (kudos if you're a grizzled vet and still consider it so, by the way!). But the reality of it is that a thought expressed in a programming language is a straightforward and self-consistent statement which is going to be true in any other programming language, adjusted for syntax and other peculiarities.
This means that if you follow an algorithm line by line, function by function, there is no reason why any specific process cannot be expressed and continue to work in a different framework, engine, runtime, or operating system. Trust me, at one point in my life I ported Java code to Minecraft command functions. Still, that's speaking generally and being very generous with how far you actually can follow said algorithm.
As you know, the environment for which you're writing your code provides you with tools, and built-in, standard APIs — to simplify and abstract away certain common tasks. Unless you are migrating the codebase to a plug'n'play replacement of the original environment, there are going to be differences between native capabilities of the old and the new platform.
So, your first consideration must be native dependencies. Investigate all declared dependencies and all imports/includes to get a complete picture. In ActionScript, for example, there are both implicit (shared namespaces, scope injections) and explicit dependencies (import statements). The latter can also be declared with a wildcard, meaning an entire namespace of objects and functions can be referenced with one line.
It may take a bit of time to chase down and understand how native APIs are used, but there is a good revelation at the end of this tunnel. If you strip away platform calls, you know for certain that what remains can be replicated almost verbatim in your own code. Of course, doing it this way may not be the best idea, but it's at least a possibility and a decent starting point.
Besides native dependencies the project can also depend on third-party code. The good news is that third-party code and logic can be replicated all the same, just like your main project. The more lukewarm news is that each third-party dependency may add to the scope of your work. It is possible that the dependency is pre-built/pre-compiled and you cannot get source access to it (or it is not permissively licensed). It is possible that the dependency is too complex and requires domain-specific knowledge.
You have to decide on two things: do you need it, and do you have to port it yourself. For instance, my work started with the goal to port a certain music making app. The app in question had a few dependencies which helped it with various mundane things, like input handling. Given that I use a game engine as my means to remake the app, input handling, rendering, OS interactions, and even the main loop have already been solved for me. So I simply don't need to worry about those.
However, at the core of the app was a software synthesizer library. While there were, without a doubt, other synthesizers which I could use as a replacement, I knew that I had to port this one. The synthesizer was responsible for the exact sounds that the app produced, it was critical to get it right. So I had to extend the scope of my work to include the library, SiON. In fact, it became my primary task, as without this library there was no project.
SiON, in turn, was mostly self-contained. It did, however, rely on integration with the Flash standard library to output sound — the Flash.Sound object to be specific. There was nothing that could be done about that. The new implementation had to be built around a new solution, leveraging Godot's audio streams instead.
So, to conclude, each dependency presents you with one of three options: drop it, find a replacement for it, or reimplement it yourself.
Establish a base line, people will thank you later
At this point, you can slowly start implementing your new project, decide on the code style and patterns which are going to be used throughout the codebase. Just one final thing before you go there.
You need to be able to compare your results with some base line. It helps if the original project has a decent coverage with unit tests which you can borrow, or just has any tests in general. But those can be imperfect, and bugs can creep up on you anyway. Make sure you can build and debug the original, that you can modify it and experiment with it live. You can even consider writing your own tests for it to make sure that some specific part of your port behaves exactly the same as before.
In my case, I needed (no-longer-Adobe) AIR SDK, which is still accessible and supported by Harman, plus some VS Code extensions. The exact build commands were given with the project, and I was able to get it up and running quickly. I even fixed a few compilation bugs in my local copy of SiON 👀
Code with patience, go nuts with spreadsheets
You're all set, and the rest is in your hands as a programmer and a software engineer. Your approach to coding is uniquely yours, and you should lean into what makes you the most productive. But I do have a few pointers that helped me:
1. Focus on the most direct path to getting results from your work.
This was a particularly hard task for me. A lot goes into the synthesizer, and I had to erect a lot of scaffolding for even the simplest case. It took me weeks before I could produce any sound (and boy, how happy I was to hear it!).
As a general rule I recommend going from the outside, from the API, from basic user interactions. Find a thread that you can pull to get some output, and then follow it deeper into the codebase to address the functionality that is not yet implemented or doesn't work correctly. At the same time...
2. Don't commit to entire classes, namespaces, and other sorts of scopes from the get go.
Instead of going all-in, implement them in chunks and spend time only on those components that you need right now, and stub the rest. Be careful to keep track of what's done, what's in progress, and what's not touched at all. For the reference, I used 4 states to classify readiness of each class:
- Pending — this class doesn't exist in the new codebase;
- Not implemented — this class is present in the new codebase, but it's an empty shell with maybe a couple of properties or methods which I immediately need;
- Stubbed — this class is present, and its every property, method, struct, enum, etc (public or protected, at least) is declared, but not implemented;
- and, Done — this class is fully implemented in the new codebase; it may be lacking polish or tweaks (and testing).
Perhaps this is my highly-organizational nature speaking, but spreadsheets are your best friend here. Before I started writing any code, I had made an Excel table with every file that I needed to consider for the port, and then kept track of the state of each of them. This was on top of adding code comments of a similar nature. As an additional perk, you can calculate a percentage of completion using such a table, for that sweet morale boost in the darkest times.
3. Keep the original logic, but leave a TODO/FIXME to improve it later.
Some changes you will inevitably have to do on the fly. For instance, ActionScript 3, being an interpretation of the ECMAScript specification, considers functions their own objects and makes it easy to pass them around. Things are considerably different in the C++ land, where class methods are not as mobile. On top of that, Godot and the GDExtension API provide their own means to solve this problem, called Callables, which in turn creates implications for your entire codebase (for one, every class that needs to rely on this also needs to be registered with the ClassDB system of the engine and thus needs to be a Godot-compatible class).
So there are options, and this is something that needs an immediate resolution, and you will have to power through such case. But not every problem is going to be like that. I noticed a handful of bugs or questionable implementations in the SiON codebase, and in a lot of cases I left it as is, until I had an opportunity to revisit it later.
If it works and compiles, it can be just good enough for you to keep moving. Not worrying about redesigning the codebase removes that one extra level of complexity which can turn a feasible task into an extended chore, and making yourself miserable shouldn't be your goal.
4. Try to pay attention to language-specific differences in behavior.
Throughout this article I tried not to make any assumptions about one's proficiency with either the source or the target programming language. That said, a wider understanding of programming languages, their design and choices that go into making them, is a critical tool under your belt. Some languages, even excluding purely esoteric ones, can differ quite a lot from the conventions of their more mainstream counterparts. Not every language is C-like, and not every C-like language behaves the same way.
Not without a note of pride I can claim that my port of SiON has worked almost first try, despite the majority of the process being blind. Now, I haven't tested every bit of functionality, but what I could compare worked pretty much the moment I managed to produce a sound. Except...
Except for those pesky integer division errors. You see, ActionScript is very nice with its implicit type coercion. You just divide a number-ish value by another number-ish value, and if the result is not a whole number, it becomes a floating point value. C++, however, is very particular with its types. So a division of two integers can never produce a floating point value. A truncated int is all that you get.
I know this, and I even don't have a problem with such behavior, but I haven't been paying attention in the moment. And it ended up tripping me. This was the reason why the first sound GDSiON ever produced was completely wrong. And then it happened a couple of times more.
Learn from my mistakes, if you can.
And good luck!
Rewriting someone else's work in an entirely different language for an entirely different technology stack is a challenging task. But every challenging task is gratifying upon completion. You always learn something new, you get the best form of practice. And maybe you even help to preserve software, make it accessible again.
Hope you've enjoyed the reading!
If you have questions, don't hesitate to ask them. Make sure to check out GDSiON on GitHub, and keep an eye open for the upcoming reveal of the music making app that I'm creating with it.
Please consider becoming a patron, even if you can only help a little <3 I want to create, or recreate in this case, amazing and free tools and technology for developers like yourself, and I cannot do it without your help!
Cheers
Loading comments and feedback...