Prebuilds, again


08:05 - 8 Apr 2016

In the last weeks it could seems there have been too few movement in the project due to several reasons: NodeOS core is stabilized after the RC1, and both me and other members of the project we have been busy with work and classes (I hope to read my thesis in May! :-D ). At the end, we are working on NodeOS on our spare time and we have some real-life responsabilities…

The point is that we have been banging our head against some walls while moving towards the 1.0 version. For example, @tylerl0706 has been getting some non-reproducible issues while trying to fix the pending bits to port nodegit so we can have a working git client on NodeOS (needed to use git repos with npm). In my case, although I got some good progresses reimplementing nsh as a bash-languaje interpreter running on a single process (that would lead in the future to write a bash-2-javascript transpiler) instead of just executing the user commands, the fact is that I’ve found a bug on Node.js where stdin stream is not correctly processed and makes it unusable and the alternative is to not be able to auto-detect when it’s running on an interactive shell, and also trying to fix download module to decompress on real time instead of hold the files in ram (making it impossible to download NodeOS dependencies on memory constrained systems) lead to some cryptic errors or random incomplete downloads without notification. Really frustrating :-( Good things are, building an OS a such a complex task that always there’s something to do, and if something it’s burning yourself you can try with another thing :-)

One recurrent issue people is having is that nodeos-cross-toolchain module dissapears when using latest Node.js 5. The reason is that it includes npm 3, that has changed the organization of the node_modules folder to being maximally flat, deleting its folder on the way. Until now the only solution is to still use Node.js 4 and npm 2, but it will someday gets support and it’s time to move on, so since it was almost autonomous I’ve finally converted it on an independent npm module thanks to a modified version of the use of prebuild to be able to use some prebuild releases of general purpose binaries. This allow to use nodeos-cross-toolchain as a dependency of the NodeOS layers without needing to build them again and again, but also move forward to the npm 3 support and more important to allow to test it independtly and reduce NodeOS build time to the half (yes, the cross-compiler takes so much time as NodeOS itself…), so now instead of pick some microwave pop-corn and go to see a movie it’s just a matter of see one chapter of your favorite soap opera :-P It has helped to reduce build time not only to convert the cross-toolchain on a precompiled npm module, but also I’ve created prebuild modules for genfatfs, genext2fs and qemu. Yes, you’ve read it right: now you can use QEmu as a npm dependency, no kidding :-D

But to be honest, upcoming npm 3 support what not the main reason to move cross-toolchain as an independent module, but NodeOS prebuild images. NodeOS build process has got to be so much complex and heavy weight that the CI server was getting random errors constantly (and in fact I’m thinking to move to another one due to this, but I’ve not found any other free one as fast… maybe is it time to give another opportunity to TravisCI?), so the only possible solution was to split the build process in several independent fragments and test them separately, tha obvious first one was the cross-toolchain since it only would need changes when a new version of GCC or Linux kernel arises. So after adding the dependencies and tuning-up the build scripts and fix some bugs on the publish-release module… finally we have prebuild NodeOS images again after 4 months of silence, just ready so people can test the upcoming 1.0 version :-D

Last but not least, another tiny bit that has been floating around since some time ago was the possibility to add an administrator mode that would allow to admin some system global configs like to allow to logon to create new users, while at the same time don’t allow users to log in the system to prevent problems like the fact in WIndows most people (specially in domestic environments) access to the system with administrator permissions. It was easier to implement that what I though on a first time, just by forcing the user to a REPL if system is booted with the single flag and don’t execute the users init processes (specially the one for root, that start logon). This also help me to find a bug on nodeos-init where it was trying to exec as a command any positional argument that it was being given so it though single was a command (when it’s not :-) ), so now it detects if the first one is an absolute path (since at boot the kernel doesn’t define itself the $PATH environment variable used to locate the binaries) and if not, it gives all the parameters as arguments for the default /sbin/init binary that mount the partitions and boot the system :-)

So as you can see, steady NodeOS is moving towards it’s 1.0 version! :-D Let’s hope the issues that have appear on its dependencies modules can be fixed soon and we can release something we can be proud of :-)


NodeOS is made possible by its many contributors around the world. Source code licensed under the MIT license.

Made with ❤️
and 🍺