Warning: There is some really cursed code down there!
Last week I ported Bosca Ceoil Blue for the web platform. For the most part everything just worked, as both Godot (the engine that I use) and GDSiON (the synth library that I use) can run natively on web via WebAssembly. However, an application running in a browser environment needs to consider a couple of things to maintain reasonable user experience and compromise on no feature. In this two-parter I will cover how I went on adapting Bosca for Web, so you can do this too for your project!
In the second part I will talk about making a better landing page, loading experience, and compatibility checks. But let's start with how saving and loading works a.k.a. file I/O.
Bosca Ceoil, being a good app, offers its users a way to store and load back up their compositions. On top of that it provides an export and an import tool, that integrates Bosca with different external formats. The saving and exporting process consists of the user clicking a button, a dialog being prompted to select the destination for the created file, and then the file is generated and written to disk. For loading and importing the user clicks on a button and is similarly prompted to select the file to read, after which the file is loaded into the application memory and parsed.
Straightforward stuff! So, how is it different in a browser? Web browsers have a long history of restricting their features so the simple act of surfing wouldn't compromise your device in any way. In the olden days embedded technology, like Flash, could circumvent some of these restrictions, but ever since HTML5 and ECMAScript 2015 the approach to web standards and security has drastically changed. Long story short, at the time of writing vendors and committees are very concerned about providing means for web pages to access your local file system. There is a promise of FileSystem API exposing methods like showOpenFilePicker and showSaveFilePicker, but we aren't there yet.
It's not all hopeless in today's world though!
Naturally, you've likely saved files a bunch of times while browsing. In fact, you can go ahead and save Bosca Ceoil right now! Except we call it downloading. And that's what is going to do the trick for us.
How does downloading work on web? You click a link and your browser requests a new web resource based on its URI. Depending on its metadata (response headers) the resource can be either displayed or downloaded as a file. And you can also force it to always be downloaded with some HTML markup. The neat part is that you can replicate all of this from code: create an anchor element (<a>), set it up with correct attributes, add it discreetly to the page, simulate a click, and remove it.
Doing so in the browser, with JavaScript, would be trivial:
const a = document.createElement('a');
a.href = url; // Set the resource URI.
a.download = name; // Force this to be a download and give the file a name.
a.style.display = 'none'; // Hide the element with CSS.
document.body.appendChild(a);
a.click();
a.remove();
There are a few things to consider, but the key limitation here is that this must happen from a user interaction. Browsers are capable of detecting if the executed code has been triggered by such interaction, and they use it as a security measure, so websites don't just download arbitrary junk as you visit them. So, in layman's terms, the user must click on the page right before this happens. Luckily, user clicking on a button in our application still counts!
But, Yuri, you ask, this is JavaScript! How does this interact with Godot? Well, in this particular case it's actually painfully trivial. Godot exposes a special wrapper exclusively for the web platform, JavaScriptBridge. This is a class that allows opaque interaction with the web runtime. It's really powerful and we'll utilize it to a larger extent later on. For file saving, though, we just need to call one method, JavaScriptBridge.download_buffer().
It takes two required arguments, a byte array and a name, and triggers the exact same code on the browser side that I've shown above. In fact, I just took it from the engine sources 🙃 So, now it's time to integrate it into the file I/O pipeline that we have in the application.
In Bosca Ceoil Blue specifically, I have a dedicated global manager, called IOManager which exposes methods like save_ceol_song(). Similar methods are available for exporters. The flow is thus:
func save_ceol_song() -> void:
if not Controller.current_song:
return
var file_name := Controller.current_song.get_safe_filename()
var save_dialog := Controller.get_file_dialog()
save_dialog.file_mode = FileDialog.FILE_MODE_SAVE_FILE
save_dialog.title = "Save .ceol Song"
save_dialog.add_filter("*.ceol", "Bosca Ceoil Song")
save_dialog.current_file = file_name
save_dialog.file_selected.connect(_save_ceol_song_confirmed, CONNECT_ONE_SHOT)
Controller.show_file_dialog(save_dialog)
func _save_ceol_song_confirmed(path: String) -> void:
if not Controller.current_song:
return
var success := SongSaver.save(Controller.current_song, path)
if not success:
Controller.update_status("FAILED TO SAVE SONG", Controller.StatusLevel.ERROR)
return
Controller.update_status("SONG SAVED", Controller.StatusLevel.SUCCESS)
static func save(song: Song, path: String) -> bool:
var file := FileAccess.open(path, FileAccess.WRITE)
var error := FileAccess.get_open_error()
if error != OK:
printerr("SongSaver: Failed to open the file at '%s' for writing (code %d)." % [ path, error ])
return false
# Generate the contents as a string.
var file_contents := ""
# ...
file.store_string(file_contents)
error = file.get_error()
if error != OK:
printerr("SongSaver: Failed to write to the file at '%s' (code %d)." % [ path, error ])
return false
return true
This code can run on web too, but with two limitations: you cannot use a native file dialog, and you can only have access to a virtual file system that is stored in the browser. Which is not very useful when we want to save a file from the app to the user's device. Some edits are due. For the JavaScriptBridge.download_buffer() method we need a byte array representing file contents, so to make things a bit easier for us we will still generate the file using existing logic. The file will be written to the virtual file system, and then read from it into a buffer for the download.
We don't need to prompt the user for the save location, and in fact we cannot. But we can use this step to set up a path for the temporary file in the virtual file system. So the first call becomes this:
func save_ceol_song() -> void:
if not Controller.current_song:
return
var file_name := Controller.current_song.get_safe_filename()
if OS.has_feature("web"): # On web confirm immediately with a temporary file path.
_save_ceol_song_confirmed("/tmp/" + file_name)
return
# The rest remains as is ...
We don't need to change the second step, but we do need to augment the last one. This is where we integrate the JavaScript bridge. At the end of the saving routine, before we return true, we need to trigger the download.
static func save(song: Song, path: String) -> bool:
# Same as before ...
if OS.has_feature("web"):
file.close() # ???
var download_name := file.get_path().get_file()
var file_buffer := FileAccess.get_file_as_bytes(file.get_path())
var error := FileAccess.get_open_error()
if error != OK:
printerr("SongSaver: Failed to open the file at '%s' for downloading (code %d)." % [ path, error ])
return false
JavaScriptBridge.download_buffer(file_buffer, download_name)
return true
And now you have it, a file is downloaded by the browser when the user presses "Save"! Depending on the browser, it may just get stored to a default download location, or the user may be prompted for the destination path. Either way, it's out of our hands now.
Implementation note: there are different ways to access the contents of a file as a byte array/buffer, but there seem to be engine bugs. Since we have the file open for writing already, we should be able to just get its contents with this:
file.seek(0)
var file_buffer := file.get_buffer(file.get_length())
This, however, doesn't work. The length is read correctly, but the buffer ends up being empty. The seek(0) call is required here to make sure we read from the top, but that still doesn't solve the issue. You may have also spotted a suspiciously marked line above, with a call to file.close(). And indeed, unless you do that or call file.seek(0), even FileAccess.get_file_as_bytes(file.get_path()) returns an empty buffer. Seems like some internal file handlers leak into each other's contexts. This may be fixed in time, so feel free to experiment with a simpler approach.
Just like saving is not that different from downloading, opening files for editing is pretty much uploading them. And you've likely uploaded files before, like a profile picture on a social network or a text document to Google Docs. So the general idea for a workaround is familiar as well.
When you upload a file on a webpage, there is an input element with the file type. Sometimes websites prettify it and create nice drag'n'drop areas, but in the most basic form it's just a button with a label next to it. Clicking on it prompts the user with a file dialog to select the file for the upload. At this point the JavaScript runtime can access and read the file from your local file system. This is used in many web applications to ensure they don't need a server to run. And so we can read bytes, and we it's safe to assume we can pass these bytes back to our Godot application.
Unfortunately, there is no built-in alternative to JavaScriptBridge.download_buffer() specialized in loading files. We'll have to do everything ourselves. And that's where the true power of JavaScriptBridge comes into play: you can access objects that exist on the webpage and write browser code directly from GDScript! There are three handy methods that we will use:
JavaScriptBridge.get_interface() allows you to access any existing global object from the browser, like window or document.JavaScriptBridge.create_object() is used to create new instances of types available in the browser.JavaScriptBridge.create_callback() is necessary to pass a function reference where JavaScript API expects one connecting it directly to a Callable in Godot.With these tools at our disposal, anything that can be done from JavaScript can be done from our app's scripting too. So let's create an input element:
var document := JavaScriptBridge.get_interface("document")
var element := document.createElement("input")
element.type = "file"
element.accept = ".ceol"
document.body.appendChild(element)
And then let's connect a callable to a web event. One thing here needs to be explained beforehand, though. As per documentation, the object returned by JavaScriptBridge.create_callback() will not be kept in memory unless you explicitly do that in your code. I'm not sure why this is the case, but what it boils down it is you need to have a persistent variable that holds this object. Like a class property.
The callable that we bind in this manner receives an array with all the arguments passed to it. It's your responsibility to know how many arguments there can be and what they mean. In case of events and event listeners there is always one argument and it's always a reference to the event object.
So, for simplicity, I made a small wrapper to handle both these nuances.
var _event_handlers: Array[JavaScriptObject] = []
func _add_event_handler(object: JavaScriptObject, event: String, callback: Callable) -> void:
var callback_ref := JavaScriptBridge.create_callback(func(args: Array) -> void:
callback.call(args[0]) # The event object.
)
_event_handlers.push_back(callback_ref)
object.addEventListener(event, callback_ref)
And now we can finally connect our callables:
var _element: JavaScriptObject = null
func show() -> void:
var document := JavaScriptBridge.get_interface("document")
_element = document.createElement("input")
_element.type = "file"
_element.accept = ".ceol"
_add_event_handler(_element, "change", _file_selected)
_add_event_handler(_element, "cancel", _dialog_cancelled)
document.body.appendChild(_element)
func _file_selected(event: JavaScriptObject) -> void:
pass
func _dialog_cancelled(event: JavaScriptObject) -> void:
pass
This is truly the best way to upset both web developers and game developers!
Back to the task at hand, we still need to load the file. Once again, I think the best approach is to utilize as much of the existing infrastructure as possible. So we want to do the reverse of what we did with downloading: take the bytes from the user's file system, store them in the virtual file system of the application as a temporary file, use that file's path in the loading/importing routine as if the file was selected locally. After all, the path is what the built-in FileDialog returns, so that's what we must return as well.
Let's start with reading bytes. The input file element provides a blob for each selected file, which can be a good starting point. Alas, decoding the blob into a byte array is an asynchronous operation, and during this I was not ready to look into how async might work between GDScript and JavaScript. One way to resolve this is to just do the bulk of the work on the browser side instead of calling JavaScript API from GDScript. I wanted to avoid that and keep things in the application codebase, if I could help it. Promises might also look... promising. But doing the whole JavaScriptBridge.create_callback() dance with them will probably lead to an absolute mess of code.
Luckily, there is the FileReader API. Instead of callbacks or async it uses events, and that's something we can easily handle now. The irony here is that if this was a JavaScript codebase, using FileReader would be a more verbose option. But here it's actually the simplest and cleanest one.
func _file_selected(_event: JavaScriptObject) -> void:
if _element.files.length > 0:
var file_name: String = _element.files[0].name.get_file()
var file_reader: JavaScriptObject = JavaScriptBridge.create_object("FileReader")
_add_event_handler(file_reader, "load", _file_loaded.bind(file_name))
file_reader.readAsArrayBuffer(_element.files[0])
And here's the final piece of the puzzle. The result of this operation is an ArrayBuffer, a type which we cannot use directly. We must construct Uint8Array out of it so that the data is interpreted as bytes. And then that we can convert into Godot's PackedByteArray.
func _file_loaded(event: JavaScriptObject, filename: String) -> void:
var buffer := PackedByteArray()
var byte_array: Variant = JavaScriptBridge.create_object("Uint8Array", event.target.result)
for i: int in byte_array.byteLength:
buffer.push_back(byte_array[i]) # This only works if the byte_array is untyped. Some indexing operators missing?
We're ready to write it into a virtual file, which involves no fancy tricks.
func _file_loaded(event: JavaScriptObject, filename: String) -> void:
# Same as before ...
var path := "/tmp/" + filename
var file := FileAccess.open(path, FileAccess.WRITE)
var error := FileAccess.get_open_error()
if error != OK:
printerr("FileDialogNativeWeb: Failed to open the file at '%s' for writing (code %d)." % [ path, error ])
return
file.store_buffer(buffer)
error = file.get_error()
if error != OK:
printerr("FileDialogNativeWeb: Failed to write to the file at '%s' (code %d)." % [ path, error ])
return
# Success!
Now path can be fed to the rest of the system. Something that I haven't mentioned yet is that I wrapped all this code into a FileDialogNativeWeb class, so it can be used as an almost seamless replacement for the standard FileDialog. I didn't implement all possible functionality, but it covers the cases that I need. Feel free to adopt it for your projects and modify it to fit your needs!
And here it is integrated:
func load_ceol_song() -> void:
if OS.has_feature("web"):
var load_dialog_web := Controller.get_file_dialog_web()
load_dialog_web.add_filter(".ceol")
load_dialog_web.file_selected.connect(_load_ceol_song_confirmed, CONNECT_ONE_SHOT)
load_dialog_web.popup()
return
var load_dialog := Controller.get_file_dialog()
load_dialog.file_mode = FileDialog.FILE_MODE_OPEN_FILE
load_dialog.title = "Load .ceol Song"
load_dialog.add_filter("*.ceol", "Bosca Ceoil Song")
load_dialog.current_file = ""
load_dialog.file_selected.connect(_load_ceol_song_confirmed, CONNECT_ONE_SHOT)
Controller.show_file_dialog(load_dialog)
Hope this helps you work around file I/O limitations in browsers for your Godot project! You can see all this work in action in the most recent beta version of Bosca Ceoil Blue exported for web:
And all the code is, naturally, open and freely available on GitHub.
If you liked the article, or enjoy the fact that Bosca will soon be available on Web once again, please consider supporting my work via itch.io or Patreon. Making tools for and sharing knowledge with other developers is my full-time occupation, and I'd love to continue doing this for the foreseeable future!
Cheers and happy holidays <3
Loading comments and feedback...