Saturday, August 23, 2014

Making a package.json from exact installed npm package versions

Attention conservation notice: Google-food for nodejs users.

It's easy to get a list of just the packages you directly depend on:

npm list --depth=0

However, there's no built-in way to get npm list to output results in package.json format. Here's a little shell recipe:

npm list --depth=0 \
  |tail +2 \
  |sed '/^\s*$/d' \
  |gawk -F ' ' '{print $2}' \
  |gawk -F '@' '\
     NR > 1 { printf ",\n" } \
     { printf "\"%s\": \"%s\"", $1, $2 } \
     ENDFILE { printf "\n" }'

This isn't a complete package.json, but it's a format that can easily be copied into one, via a second shell recipe or whatever else you like. (If your pipe-fu is strong you can probably figure out how to extend this to do the whole package.json in a one-liner.)

Motivation: When you npm install --save or npm install --save-dev, npm inserts packages with "the default semver operator" by default. It's easy to forget to pass --save-exact; or, if you're just doing exploratory hacking, you might not even want --save-exact. But when you're ready to cut a build for deployment, you need to capture exact package versions, because semver is basically bullshit can't be relied upon. Hence the recipe above, which can be used to generate or update the dependencies section of the package.json in a deploy directory.

Saturday, August 09, 2014

A few problems with Firefox

Disclaimer: I worked at Google for 6.5 years, although not on the Chrome team. I am currently independent. I also worked a little bit with Rob at IBM in 2001.

Rob O'Callahan has a good post explaining why you should use Firefox. I am sympathetic to this argument, but I can't bring myself to switch yet. I try periodically, and every time I end up bouncing off again. Alas, Firefox is currently inferior in specific ways that are cripplingly bad for my needs.

First, as a web developer:

  1. The developer tools are really janky.
  2. The profiles functionality is buried and has no in-browser UI. I need quick, simple profile management and switching when testing my apps. (Don't refer me to the add-on market; add-ons are a cesspool.)

Second, as someone who recommends tech to my family:

  1. Multi-process isolation and sandboxing have still not shipped.
  2. Firefox's updates are still not as timely as Chrome's. On my personal machines, I often find that I'm running an older version of Firefox weeks after a new version is released. In fact, just as I was writing this, I found that the machine I'm typing it on was still running 30.0, when 31.0 was released on July 22.

As a result, Firefox is plausibly the least secure major web browser shipping today. I can't recommend Firefox to my family until these things are fixed. I won't expose them to a high risk of exploitation, here and now, solely to protect them from a theoretical risk that they'll be harmed by the Chrome team's product choices in the future.

In addition to all of the above, I think Rob overstates the extent to which Google is (1) winning and (2) likely to use that position to harm users in the foreseeable future.

Obviously, I am biased w.r.t. (2), so I don't think it's productive for me to try to convince you of my point of view in this post. At any rate, Rob is better-informed than me about browser politics, not to mention much smarter than me, so I am willing to believe that he has good reasons for believing what he does.

However, w.r.t. (1):

  • On desktop, Chrome doesn't even have majority market share. It is trending upwards, but it's a long, long climb from its present ~40% position to the 80-90% that Microsoft once had in desktop operating systems. And Google's competitors today are a lot more impressive than Microsoft's were in the 1990s. I don't think it's plausible that a large fraction of the web will build strictly for Chrome, or even Blink-based web browsers, anytime in the foreseeable future, unless all of Google's competitors fumble the ball mightily.
  • On mobile, iOs isn't going away any time soon. And I would bet (a small amount of) money that forks of Android, from China and elsewhere, will reach rough market share parity with Google Android in the long run.[0].

Today's giants always look more invulnerable than they really are. Apple looked unstoppable just a few years ago, Microsoft not that long before that. Facebook looked like it would become the identity layer for all human interaction; now it's just a boring and somewhat declassé social media site for middle aged people (plus a server farm for a few flashy acquisitions). Google may seem like a juggernaut, and to be fair I think it is much more competent on average than any of the three companies I just mentioned, but it's vulnerable in ways that aren't even obvious to us today and I'll be very surprised if we look back in 2024 and find that Google is dictating terms to the rest of the technology industry, in the old Microsoft (or new Apple) mold, rather than being merely one influential player among many.


[0] Incidentally, since Google has let the AOSP web browser languish, and restricted Google Chrome(TM) to its increasingly-tightly-constrained partners, making Firefox run amazingly well on non-Google Android — well enough that non-Google Android users almost universally either get Firefox shipped with their device or install it themselves — might be a more plausible path to getting large mobile market share globally than Firefox OS. (I do think Firefox OS is an important and worthwhile project as well.)

Monday, June 23, 2014

More on taxis

I will fully cop to the fact that my previous bitching about taxis concerned a classic San Francisco yuppie firstworldproblem but here are some more stories about how taxis in many cities serve people of all stripes exceptionally poorly.

Relatedly, I took an UberX with a friend the other night and the experience was awesome.

I might have liked the option of tipping the the driver (via the app, after you get out of the car), but UberX doesn't allow that. On the other hand, maybe tipping would establish a social norm for companies like Uber to underpay their drivers with the expectation of tips? I really like traveling abroad in countries that don't have a tipping culture; it seems more rational and arguably in the long run leaves labor in a better position since their compensation is assured by contractual terms rather than manners. So maybe it's great that Uber doesn't have tipping, only reviews.

Wednesday, June 18, 2014

What happens when you let people pay to remove ads

Disclaimer: Obviously I worked at Google for 6.5 years so take this with the appropriate grain of salt.[0]

"I wish Google would just let me pay them money to remove ads from this service. WTF GOOGLE IS SO DUMB AND EVIL WHY WON'T THEY DO THIS."

How many times, in various corners of the Internet, have you read some variation of the preceding sentiment?

Now, it turns out, Google is trying to do this with a subset of music videos on YouTube. But when you do this, you need a contract with the rights holder. This is complicated. The rights holder might not agree to your contract terms. For example, they might want more money. So you end up in negotiations. While those negotiations are ongoing, or if they break down, you can't include that music in your paid service.

The Internet's collective reaction has been: "WTF GOOGLE IS SO DUMB AND EVIL WHY ARE THEY DOING THIS?"

Now, YouTube will still host music videos that are not opted into its monetization program, i.e., if you are a band and want to put music videos on the Internet for free, YouTube will host those videos for you. Let's be clear what this means: if you want to distribute a video to a hundred million strangers on the Internet, YouTube will pay for the software and servers and a petabyte of network bandwidth and a small army of SREs holding pagers who will wake up at 3am if too many people in Kuala Lumpur click on your video and get an HTTP 50x error, and it will do so without you paying YouTube a dime. None of that's changing. But if you opt into YouTube's monetization program, you will have to opt into its full, updated monetization program: ads for non-subscribers, and no ads for subscribers.

Note that there is no sensible way to let users pay to remove ads from music videos while also still showing ads to those same users for some music videos. If YouTube did this, you can bet the Internet would collectively scream, once again, "WTF GOOGLE IS SO DUMB AND EVIL WHY DON'T THEY REMOVE ADS ON MUSIC VIDEOS WHEN I PAID TO HAVE ADS REMOVED? I'm unsubscribing from this bullshit service!" The subscription service would fail and YouTube would have to revert to its old model of monetizing via ads only.

In short, "blocking" — i.e., excluding unlicensed music from the monetization program — is an inevitable consequence of having a paid subscription service.

The press has done an abysmal job covering this. It seems that every year my contempt for (most) journalists finds reasons to grow greater and greater. I'm pretty disappointed that the Financial Times, reputedly pretty reliable, appears to be Ground Zero for this particular blast of misinformation.

p.s. As for claims that YouTube has anything approaching a monopoly on online video sharing, I'm honestly puzzled by the claim. Just to take one example competitor, Vimeo is pretty reliable and seems to have rough feature parity, including embedding in external sites. Vimeo's content is included in Google's video search corpus (example) and therefore shows up as a thumbnail image in Google web search, just like YouTube videos. If you need to make money, Vimeo has a few built-in monetization options; if those are insufficient, there's a lot of innovation occurring in the world of funding, and in my opinion the Patreon/Subbable model seems much more promising for creators with a small-to-medium-sized audience than Spotify-style monetization, which currently amounts to "we'll give you 0.0001 pennies per stream play so that both the company and the artists can lose money hand over fist!" Is it really true that Adele fans will face substantial (or even non-substantial) barriers to watching her music videos if they're on Vimeo? And that's just one competitor. The upshot is that indie labels are probably wrangling with YouTube over licensing terms not because it has anything like a monopoly, but because they think wrangling with YouTube will make them more money than all the other alternatives. Which is perfectly fine — more power to them, negotiating the best deal is essential in business, etc. — but we should not misread the situation.


[0] On the other hand, one benefit of no longer working there is that I can write stuff like this post, just like I used to before I worked there. Arguably, working at Google made me less predisposed to harshly criticizing misinformed critics of Google.