By Jay LaPorte on May 29, 2014.
Ever since we released the Forecast API, we’ve been asked for translations of our text summaries (such as “Light rain starting in 15 minutes, stopping again 35 minutes later.”) into other languages. It’s also something we’ve wanted, ourselves, in order to facilitate internationalization of Dark Sky and Forecast. I’m pleased to announce that, as of today, this feature is finally public, with initial support for English, French, German, Dutch, and (believe it or not) Tetum. Our plan is to add additional languages on a continual basis.
If you’re a developer wishing to use this feature, please consult our API documentation. If you’re a multilingual developer that would like to contribute another language, please visit our translation project on Github. If you’re a user of Dark Sky or Forecast and want to see them translated into a different language, just hold tight: we’ll be working on it as time permits. Finally, if you’re a curious reader wanting to know why a seemingly simple task took us fifteen months from start to finish, well… we ran into a couple big problems.
The first problem was a technical one. Most computer programs are static when it comes to language: they effectively store a big list of all possible messages used in the program, which one can simply translate and substitute in. (The Forecast app itself is a fine example of this: most of the English on the page doesn’t ever change. For example, the messages “Right Now”, “Next Hour”, and “Current Location” could be translated by any bilingual person, and then if the user wants the messages in Spanish, we could simply select “Ahora Mismo”, “Siguiente Hora”, or “Ubicación Actual” from the list, instead.) The API, in contrast, is much more dynamic: since there are so many different types of weather and different ways that they can occur, we were forced to make our text summary robots extremely versatile. Rather than select a message from a list, what they do is much more akin to building sentences out of their constituent nouns, verbs, and adjectives. Language tends to follow very regular rules, so this doesn’t sound so very difficult, either; but the problem (as any grade-school English student knows) is that these rules have a surprising number of exceptions in order to make the words “flow” well. In other words, making a robot build a sentence that makes sense is pretty easy: making a robot build a sentence that sounds natural is much, much more difficult.
(As a quick example: if you want to say that rain will occur during the afternoon, you’d say “rain in the afternoon.” But at night? “Rain in the night” sounds awfully awkward. One option is to say “rain during the afternoon” and “rain during the night,” but that sounds too formal; we opted to change it to say “rain overnight” instead. But then you have to be careful to not do that substitution all the time, or you might end up with a sentence like “rain until overnight,” which is just silly.)
The process we eventually settled on was this: instead of generating English directly (with it’s many rules and exceptions), we expose those LEGO-like building blocks to a translation system that is responsible for figuring out all of those exceptions on it’s own, separately, for each language. While this made the problem tractable, the downside was that we couldn’t just ask any old bilingual person to translate sentences for us: we need bilingual software developers to write the code to transform the building-blocks into natural-sounding sentences. (We’ve created a number of tools to make this process easier, but even then, some languages have necessitated a lot of revision to these tools. Since we at The Dark Sky Company are monolingual, we don’t have the knowledge to anticipate these kinds of situations. Luckily, we never had to go all the way back to the drawing-board, though I still worry that some exotic language or other may necessitate it!)
The second reason that translations took us so long is the best kind of problem to have: we’ve become the victims of our own success. At the beginning of March 2013, when we started planning the translations, we had 82,000 Dark Sky downloads and were processing about 550,000 API requests per day. As of right now, we have 600,000 Dark Sky downloads (that’s over seven times as much) and are processing 6,000,000 API requests per day (that’s over ten times as much).
In all life cycles, growing pains are simply a part of getting older, and as Dark Sky and our API have grown, much of the resources of our tiny little software studio have had to be invested in the maintenance and system health of our server network. We have devoted entire months to increasing our API’s capacity, and this has naturally come at the expense of adding new features to our software—we can only do so many things at once.
Frankly, we’d much rather work on making the best weather API around than on keeping the machinery running, and having such a slow turn-around time for new development just won’t do. Of course, on the other hand, we want our company to stay small and personable. With the amount of operational overhead that we currently have, this just isn’t possible: however, we may have hinted that we’re working on some new developments that we think will let us have our cake and eat it, too.
Thanks for all of the support you’ve given us so far: we’re humbled by and grateful for it. Please enjoy the new translations, and keep an eye out in the coming months for some exciting news.
“Rosetta Stone” photograph by Hans Hillewaert.