Archive for July 16, 2014

Wednesday, July 16, 2014

Compacting VMware Disks and Avoiding Snapshots

I use VMware Fusion to test my products on different versions of Mac OS X. This has worked very well over the years. Every time I installed a new version of the OS, I would make a snapshot, and I ended up with a tree of them. At first, snapshots were relatively space-efficient. If space got tight, I would delete all but the leaf snapshots, i.e. Mac OS X 10.7.5, 10.8.5, and the current stable and pre-release versions of 10.9 and 10.10.

I thought this was the smart way of doing things, using snapshots to let VMware efficiently manage the storage and share the bits that hadn’t changed among multiple OS versions. Unfortunately, I got to the point where VMware was using 108 GB of disk space to store only 27 GB of data. My MacBook Pro’s SSD was almost full. I thought it would be a simple matter to compact the virtual disk and reclaim the unused space. However, after some research, it looks to me like there is no good way to do this.

The General tab in the VMware settings showed that none of the disk space in my virtual machine was reclaimable. I found a post in the VMware forums that describes how to compact a virtual machine, but the first step is to delete all of your snapshots. If I were going to do that, I might as well start a new virtual machine!

Surprisingly, there does not seem to be any way to export a snapshot as a new virtual machine. I think the best I could do is to boot from each snapshot and clone the disk to a drive or disk image outside of VMware. Unfortunately, there also does not seem to be a way to create a new virtual machine from an existing drive (except that of a Windows PC). There are some command-line tools for creating a new virtual disk from a mounted volume, but I kept getting errors like “Failed to convert disk: You do not have access rights to this file (0xd).” Since the virtual machines are just clean OS installations, I decided it would be easier to just built new virtual machines by reinstalling each OS version.

Because snapshots don’t work with compaction, I decided to create wholly separate virtual machines for each major OS version. If necessary, I can copy the .vmwarevm packages in the Finder to manage my own heavyweight snapshots.

It is then possible to compact the virtual disks:

  1. From the Virtual Machine menu, choose Install VMware Tools.

  2. A disk will mount in the virtual machine. Run the installer that’s on it.

  3. In Terminal (in the virtual machine) enter the command:

    sudo /Library/Application\ Support/VMware\ Tools/vmware-tools-cli disk shrink /
  4. This will take a while, and the OS will complain about running out of disk space, but eventually it will finish and reboot.

After installing Mac OS X 10.7 and using the Mac App Store to update to 10.7.5, my virtual machine was using 18.2 GB. After the above procedure, it shrunk to 9.8 GB.

For another virtual machine, I installed Mac OS X 10.8 and updated to 10.8.5 with the combo updater. This brought it to 15.4 GB, which shrunk to 7.1 GB.

I have not yet built my 10.9 and 10.10 virtual machines, but it seems clear that the total of four independent ones will not be much more than the 27 GB with snapshots for each major version. Duplicating a virtual machine package in the Finder is actually not much slower than saving a snapshot. And I should no longer have problems with virtual machine bloat, since virtual machines without snapshots can be compacted. So I now believe that there is no compelling reason to use snapshots the way I had been using them. Using separate virtual machines also means that I can run multiple instances simultaneously.

Update (2014-08-18): For reasons I don’t understand, VMware started reporting the error “The parent virtual disk has been modified since the child was created,” even though I thought I had made fully independent copies of my virtual machines via the Finder. I am now using the Create Full Clone command, which hopefully will work better.

Overcast 1.0

Marco Arment:

It’s been a long road, but 1.0 is finally done. I’m proud of what I’m shipping today.

Overcast FAQ:

The servers do the heavy lifting of constantly checking for updates so your device doesn’t need to download each show’s entire feed every few minutes to make sure you get new episodes quickly. The servers send only what’s new to the app, and it only takes a few seconds. This saves substantial battery power and data usage over time.

Jason Snell:

Overcast is free, though feature-limited. “I want to offer a better alternative for the mass market, so it must be free,” Arment says in a statement on the Overcast website. For a $5 in-app purchase, however, users can unlock numerous additional features, including: support for downloads over cellular, features that modify or improve sound output (more on those below), and unlimited playlists and episodes in playlists (there’s only one playlist, with 5 episodes, by default).

[…]

Overcast does the best job of speed-alteration I’ve heard, and now I listen to many podcasts on slightly higher than 1x speed.

Separately, Overcast offers a Smart Speed feature that intelligently removes silence from podcasts, shortening episodes even if you don’t choose to listen at a higher speed. (You can also use the two features simultaneously for even more time saving).

It sounds good to me, but I doubt there’s a good way to import my current podcast state: which episodes I’ve listened to, deleted, partially played, imported separate from loose files, etc.

Update (2014-07-17): See also these reviews from Federico Viticci, Rene Ritchie, and Josh Centers.

Update (2014-07-22): Kirk McElhearn:

Below, two screenshots show what you see when playing an episode. To the left, I’ve scrolled up on the podcast’s icon; it shrinks and displays show notes, with clickable links. To the right, you can see the Effects screen, which is the feature that has won me over. This lets you speed up podcasts, without the sort of Alvin and the Chipmunks sound that most podcast apps give you. The Smart Speed setting cuts out bits of silence, helping you save a bit more time when listening to podcasts, and Voice Boost equalizes the podcasts for vocal frequencies, making them clearer. Altogether, I find this the best playback of any podcast app I’ve used.

Introducing Realm

Realm (via Tim Burks):

Today we are very proud to introduce Realm, the first database built from the ground-up to run directly inside phones, tablets and wearables, and the fruit of several years of labor since we came out of YCombinator. Realm uses very little resources, is incredibly easy to use, and lets you interact with your data faster than any current alternative.

[…]

Realm’s designed to give you amazing ease-of-use: data is directly exposed as objects and queryable by code, removing the need for ORM’s riddled with performance & maintenance issues. Plus, we’ve worked hard to keep our API down to just 3 common classes (Object, Arrays and Realms) and 1 utility class (Migrations): most of our users pick it up intuitively, getting simple apps up & running in minutes.

[…]

Realm also offers a full set of checks and guarantees to make it easier for you to build apps without shooting yourselves in the foot, including full ACID transactions by default and an object schema that is enforced directly through your object definitions. Finally, a lot of you will be happy to hear that Realm databases are safe across threads so you can easily asynchronize tasks without any cognitive overhead whatsoever, and finally explore all the concurrency potential of multi-core handsets.

[…]

We’ve seen many libraries try to offer a similar level of features on top of SQLite, at the expense of speed. In contrast, Realm is faster than even raw SQLite on common operations, while maintaining an extremely rich feature set.

[…]

Realm’s efficiency comes from years of work on a custom C++ core designed to fit the needs of much more primitive devices, that leverages bit-packing, caching, vectorization and a zero-copy architecture to realize truly amazing gains in memory usage and speed. Since Realm internals allow for easy concurrency and measurement of deltas and follows an MVCC model, we are also working on adding sync capabilities to Realm.

Here’s the GitHub project.

Achieving Zen With Auto Layout

Justin Williams:

Achieving Zen With Auto Layout is the eBook companion to my talk of the same name, but with the goal of being much more expanded than what I am able to do in a 45-60 minute on-stage presentation.

This is a beta book right now. There are no screenshots (iOS 8 does still have a tiny bit of NDA that prevents those), copy editing hasn’t been done, and I still need to hire an illustrator to do a cover and some other things in the book for me. Content-wise though, I’m proud of what is there so far, and I’m excited to finish the rest of the book in time for the iOS 8 launch later this fall.

How to Flawlessly Predict Anything on the Internet

Andy Baio:

This is a modern update to a classic confidence game—find a risky scenario with limited possibilities, bet on every single combination, and then hide your failures. The result is that you look like you’re either psychic or a goddamned genius.