Archive for August 8, 2017

Tuesday, August 8, 2017

QuickCheck for Swift

SwiftCheck (via Andy Bargh):

What makes QuickCheck unique is the notion of shrinking test cases. When fuzz testing with arbitrary data, rather than simply halt on a failing test, SwiftCheck will begin whittling the data that causes the test to fail down to a minimal counterexample.


SwiftCheck implements random generation for most of the types in the Swift Standard Library. Any custom types that wish to take part in testing must conform to the included Arbitrary protocol. For the majority of types, this means providing a custom means of generating random data and shrinking down to an empty array.

Lazy Permutations in Swift

Joshua Emmons:

Why is our generic type X an iterator but we force Y to conform to Collection? If you look at the nested loop in our naive example above, you’ll see that while we iterate over xs only once, we actually loop over ys a number of times (an xs.count number of times, to be precise).

IteratorProtocol allows us to iterate over a set exactly once, so it’s perfect for our X type. But only Collection guarantees us the ability to non-destructively traverse a sequence over and over. So Y must be a little more constrained.


And so, finally, we can lazily iterate over every ordered pair of our collections — all while only storing the collections themselves, not their product.

Kryptonite: Protect Your SSH Private Key

Kryptonite (via Steven Frank):

On iOS, Kryptonite generates a 4096-bit RSA key pair using the Apple iOS Security framework or optionally an Ed25519 key pair using libsodium. Kryptonite stores the private key in the iOS Keychain with accessibility level “kSecAttrAccessibleAfterFirstUnlockThisDeviceOnly”.


The role of a private key in an SSH login is to sign the SSH handshake. When a signature is required, your workstation calls out to Kryptonite running on the paired phone with the data that must be signed. If authorized, Kryptonite performs the signature using the private key and returns only the signature to the workstation.


Upon install, Kryptonite adds a few lines to your SSH configuration (at ~/.ssh/config) that cause SSH to offer your Kryptonite key. Your other keys will still be presented and your Kryptonite key will only be used if it has access to the service you are connecting to.

The potential benefit is that your private key doesn’t have to be stored at a known path on your Mac, where other apps can access it. Unfortunately, due to iOS limitations, there’s no easy way to ensure that Kryptonite isn’t sending your private key somewhere.

Previously: HandBrake Proton Trojan.

Making the Internet Archive’s Full Text Search Faster

Giovanni Damiola (via Hacker News):

Analyzing the hot threads we discovered that our bottleneck was in the parsing of the lucene‘s inverted index.

We have a lot of high-frequency words, some of them present in almost all of the 35M documents.

The combination of large documents (we index a whole book as a single document) and the need to support full phrase queries (and show the user the correct results) leads to a serious problem: when a phrase query includes a specific token present in a large portion of the documents, all of these candidate documents must be examined for scoring. The scoring process is very IO intensive because relative proximity data for the tokens in a document must be determined.


What if we were to encode proximity information at document index time? And then somehow encode that in the inverted index? This is what the Common Grams tokenizer was made for.

For the few thousand most common tokens in our index we push proximity data into the token space. This is done by generating tokens which represent two adjacent words instead of one when one of the words is a very common word. These two word tokens have a much lower document hit count and result in a dramatic reduction in the number of candidate documents which need to be scored when Lucene is looking for a phrase match.