Archive for April 26, 2017

Wednesday, April 26, 2017

Swift With a Hundred Engineers

Tuomas Artman (via Quinn):

Basically both of these problems; the architectural problems in the current application and the full redesign on the UX, lead us to the decision that Ben told us yesterday (in his talk) we shouldn’t do - which is just; change everything. Start from scratch. Not try to look at the architecture and fix it going forward, but start from scratch.


So we call it “Riblets”, that means; Router, Interaction and Builder and possibly a Presenter and a View. Those are the core components of one piece of the application. It’s sort of a take on VIPER. We looked at MVVM, we looked at VIPER, we looked at MVC and we came up with our own innovation on top of VIPER. The big thing that we wanted to do is obviously compartmentalize everything and make everything testable.


All of this created a lot of code. We had protocols between everything. We had components that compromised a Riblit, that compromised five different files. So in our codebase we have over five thousand files and over half a million lines of Swift code.


Our crash-free rate target is 99.99% and we are very close. I’ve never seen anything like this. First launch of an application and it’s almost crash free.


We named everything the same, everything worked in the same manner and Swift basically was the language that enabled us to do that. If we had done it in Objective-C, I don’t think we could have come to such a close relationship with the Android engineers or even have the architecture be the same across both sides.


The first thing that we found out is that, testing is pretty hard. Swift is a static language and as such you can’t really rely on your mocking frameworks that you had used in Objective-C.


On the other bad side, tooling issues.


So we built all these frameworks and then we have a post-build step that takes all the symbols out of these frameworks and links them together in your static binary and that’s how we get away with the startup speed that we had.


But if you add a user-defined custom flag; SWIFT_WHOLE_MODULE_OPTIMIZATION and set it to yes, as well as set the optimization level to none. It will do whole module optimization without optimizing. And it’ll be super fast.

The Internet Archive and Robots.txt

Mark Graham:

Over time we have observed that the robots.txt files that are geared toward search engine crawlers do not necessarily serve our archival purposes. Internet Archive’s goal is to create complete “snapshots” of web pages, including the duplicate content and the large versions of files. We have also seen an upsurge of the use of robots.txt files to remove entire domains from search engines when they transition from a live web site into a parked domain, which has historically also removed the entire domain from view in the Wayback Machine. In other words, a site goes out of business and then the parked domain is “blocked” from search engines and no one can look at the history of that site in the Wayback Machine anymore.


We see the future of web archiving relying less on robots.txt file declarations geared toward search engines, and more on representing the web as it really was, and is, from a user’s perspective.

Nick Heer:

I get where Graham is coming from here. The Internet Archive is supposed to be a snapshot of the web as it was at any given time, and if a robots.txt file prevents them from capturing a page or a section of a website that would normally be visible to a user, that impairs their mission.

But, much as I love the Internet Archive, I think Summers’ criticism is entirely valid: ignoring robots.txt files would violate website publishers’ wishes.

Google Rewrites Search Rankings to Bury Fake News

Mark Bergen:

The Alphabet Inc. company is making a rare, sweeping change to the algorithm behind its powerful search engine to demote misleading, false and offensive articles online. Google is also setting new rules encouraging its “raters” -- the 10,000-plus staff that assess search results -- to flag web pages that host hoaxes, conspiracy theories and what the company calls “low-quality” content.


“It was not a large fraction of queries -- only about a quarter percent of our traffic -- but they were important queries,” said Ben Gomes, vice president of engineering for Google.

Danny Sullivan:

How’s Google learning from the data to figure out what’s authoritative? How’s that actually being put into practice?

Google wouldn’t comment about these specifics. It wouldn’t say what goes into determining how a page is deemed to be authoritative now or how that is changing with the new algorithm. It did say that there isn’t any one particular signal. Instead, authority is determined by a combination of many factors.


My best guess is that for infrequent and unusual queries, Google has been giving more weight to pages that seem a better contextual match, even if they lack strong authority.

Ben Thompson:

This simply isn’t good enough: Google is going to be making decisions about who is authoritative and who is not, which is another way of saying that Google is going to be making decisions about what is true and what is not, and that demands more transparency, not less.

Again, I tend to agree that fake news is actually more of a problem on Google than it is Facebook; moreover, I totally understand that Google can’t make its algorithms public because they will be gamed by spammers and fake news purveyors. But even then, the fact remains that the single most important resource for finding the truth, one that is dominant in its space thanks to the fact that being bigger inherently means being better, is making decisions about what is true without a shred of transparency.