Dear node.js/node-webkit people, what's the matter with you?

I wanted to try out some stuff that requires node-webkit. So I try to use npm to download, build and install it, like CPAN would do.

But then I see that the nodewebkit package is just a stub that downloads a 37MB file (using HTTP without TLS) containing pre-compiled binaries. Are you guys out of your minds?

This is enough for me to never again get close to node.js and friends. I had already heard some awful stories, but this is just insane.

Update: This was a rage post, and not really saying anything substantial, but I want to quote here the first comment reply from Christian as I think it is much more interesting:

I see a recurring pattern here: developers create something that they think is going to simplify distributing things by quite a bit, because what other people have been doing is way too complicated[tm].

In the initial iteration it usually turns out to have three significant problems:

  • a nightmare from a security perspective
  • there's no concern for stability, because the people working on it are devs and not people who have certain reliability requirements
  • bundling other software is often considered a good idea, because it makes life so much easier[tm]

Given a couple of years they start to catch up where the rest of the world (e.g. GNU/Linux distributions) is - at least to some extent. And then their solution is actually of a similar complexity compared to what is in use by other people, because they slowly realize that what they're trying to do is actually not that simple and that other people who have done things before were not stupid and the complexity is actually necessary... Just give node.js 4 or 5 more years or so, then you won't be so disappointed. ;-)

I've seen this pattern over and over again:

  • the Ruby ecosystem was a catastrophe for quite some time (just ask e.g. the packages of Ruby software in distributions 4 or 5 years ago), and only now would I actually consider using things written in Ruby for anything productive

  • docker and vagrant were initially not designed with security in mind when it came to downloading images - only in the last one or two years have there actually been solutions available that even do the most basic cryptographic verification

  • the entire node.js ecosystem mess you're currently describing

Then the next new fad comes along in the development world and everything starts over again. The only thing that's unclear is what the next hype following this pattern is going to be.

And I also want to quote myself with what I think are some things you could do to improve this situation:

  1. You want to make sure your build is reproducible (in the most basic sense of being re-buildable from scratch), that you are not building upon scraps of code that nobody knows where they came from, or which version they are. If possible, at the package level don't vendor dependencies, depend on the user having the other dependencies pre-installed. Building should be a mainly automatic task. Automation tools then can take care of that (cpan, pip, npm).
  2. By doing this you are playing well with distributions, your software becomes available to people that can not live on the bleeding edge, and need to trust the traceability of the source code, stability, patching of security issues, etc.
  3. If you must download a binary blob, for example what Debian non-free does for Adobe Flashplayer, then for the love of all that is sacred, use TLS and verify checksums!
Recurring pattern

I see a recurring pattern here: developers create something that they think is going to simplify distributing things by quite a bit, because what other people have been doing is way too complicated[tm].

In the initial iteration it usually turns out to have three significant problems:

  • a nightmare from a security perspective
  • there's no concern for stability, because the people working on it are devs and not people who have certain reliability requirements
  • bundling other software is often considered a good idea, because it makes life so much easier[tm]

Given a couple of years they start to catch up where the rest of the world (e.g. GNU/Linux distributions) is - at least to some extent. And then their solution is actually of a similar complexity compared to what is in use by other people, because they slowly realize that what they're trying to do is actually not that simple and that other people who have done things before were not stupid and the complexity is actually necessary... Just give node.js 4 or 5 more years or so, then you won't be so disappointed. ;-)

I've seen this pattern over and over again:

  • the Ruby ecosystem was a catastrophe for quite some time (just ask e.g. the packages of Ruby software in distributions 4 or 5 years ago), and only now would I actually consider using things written in Ruby for anything productive

  • docker and vagrant were initially not designed with security in mind when it came to downloading images - only in the last one or two years have there actually been solutions available that even do the most basic cryptographic verification

  • the entire node.js ecosystem mess you're currently describing

Then the next new fad comes along in the development world and everything starts over again. The only thing that's unclear is what the next hype following this pattern is going to be.

Comment by christian
Node is not to blame here
While node.js ecosystem has a severe case of NIH, WebKit is for all intents and purposes a different runtime. It happens to share the JavaScript part, but that's all. The situation would really be the same if you wanted to use WebKit from Python or Ruby. Downloading a binary (verifying it would be nice though) and running it as a separate process is the only option.
Comment by a
comment 3
i think this post is incomplete without your ideas on how that problem could be resolved in a different way
Comment by actionless.loveless
Answers

Christian, mine was just a rage post, because I needed to vent the frustration. Your comment was actually informative and well written! May I quote it as an addendum?

a, actually it is not the only option. For starters, they somehow build these binary images that you download, why not download the sources and build them too? It might be tricky to deal with all the dependencies, specially when there is a weird build system involved, but this has been done since forever, so it is possible. In fact, if you take an equivalent tool to npm, cpan -the CLI tool- would download source code, compile, and install (locally or globally) your desired package and all dependencies. And of course, all distributions do this routinely and automatically for thousands of packages.

actionless, you are completely right. As I told Christian before, mine was a rage post. Now, to try to address your criticism here, some things I think could be done better:

  1. You want to make sure your build is reproducible (in the most basic sense of being re-buildable from scratch), that you are not building upon scraps of code that nobody knows where they came from, or which version they are. If possible, at the package level don't vendor dependencies, depend on the user having the other dependencies pre-installed. Building should be a mainly automatic task. Automation tools then can take care of that (cpan, pip, npm).
  2. By doing this you are playing well with distributions, your software becomes available to people that can not live on the bleeding edge, and need to trust the traceability of the source code, stability, patching of security issues, etc.
  3. If you must download a binary blob, for example what Debian non-free does for Adobe Flashplayer, then for the love of all that is sacred, use TLS and verify checksums!
Comment by tincho
Re: Answers

Christian, mine was just a rage post, because I needed to vent the frustration. Your comment was actually informative and well written! May I quote it as an addendum?

Sure. I just want to note that my comment here was a bit more polemic than I intended to, but I could identify with your frustration on this issue so much that I couldn't help myself.

Comment by christian
comment 6
Christian: done, thanks :)
Comment by tincho
you're confusing node.js and npm and...

npm can let you do a lot of horrible things, like the ones you mention. node.js runs perfectly well without npm ! Especially if you're using the debian packages (which are, granted, in a bad state).

Also node-webkit is some sort of ugly experiment. A bit less ugly is https://github.com/atom/electron though you still have to carry and compile chromium from source, which isn't pretty.

Comment by kapouer
And what about the license?

Apart from the security problems that most non-system package installers impose on their users, there is another severe problem: Uncertainty about software freedom.

When I use apt with official repositories from Debian, I can be confident, that all stuff is free software and license incompatibilies are unlikely. When using npm, pip, and friends, I faced many problems in the past: Very often compressed, minified, combined (JS) files with totally insufficient information about their authors and respective licenses. Or convenience copies of other programs or parts of other programs with minor or not-so-minor changes without clear indication, what this software was and by whom, and what the changes were and by whom.

Comment by debacle