What are your thoughts on a VFS module?

Hello, crystal community friends!

I’m glad to find a community where we can discuss. I’d like to share an issue I encountered.

I wrote a license generator with some YAML template files. It runs fine locally, but the single executable generated by CI doesn’t work properly. This is because it doesn’t embed these template files.

I also looked at some community libraries. I prefer using my own library, but when I import it into the project, I found it was completely unusable. Its API was completely inconsistent with the standard library.

While writing this, I realized there’s ECR. I looked at the source code and saw it has a lexer process, which isn’t the simple implementation I expected.

I noticed that Go has truly integrated the FS feature into its standard library.

Could Crystal provide a VFS abstraction layer?
Implementing it myself revealed inheritance chain issues, and I remain uncertain about which APIs are truly essential to provide.

File handles offer atomic operations—an area I’m unfamiliar with.

With a VFS abstraction layer, we should be able to easily implement our own RoFS and MemFS.

To embed files I have been using ralsina/baked_file_system (which is a small fork of schovi/baked_file_system)

I have a suspicion that it makes compile times longer if you embed a lot (because things are embedded as strings, which then are compiled) but will have to measure if that’s real.

If it is, then a objcopy-based solution is probably doable.

I figure for the specific use case, baked file system (or a similar solution) should be entirely sufficient. You can simply embed your template files into the executable.
That’s simple and straightforward without unnecessary complexity.
It’s also quite easy to use a layered approach to allow overriding the baked files with local paths (see crinja/src/loader.cr at master · straight-shoota/crinja · GitHub, crinja/src/loader/baked_file_loader.cr at master · straight-shoota/crinja · GitHub for example).

A virtual file system abstraction is a much more complex feature and IMO only makes sense when you need to integrate different libraries using file system operations.

It could really be useful for the ecosystem. But for a simple app it’s over-complicating things.