Friday, April 22, 2016

Swift Proposal: Mutability and Foundation Value Types

As you know from our announcement of Swift Open Source and our work on naming guidelines, one of our goals for Swift 3 is to “drop NS” for Foundation. We want to to make the cross-platform Foundation API that is available as part of swift-corelibs feel like it is not tied to just Darwin targets. We also want to reinforce the idea that new Foundation API must fit in with the language, standard library, and the rapidly evolving design patterns we see in the community.

You challenged us on one part of this plan: some Foundation API just doesn’t “feel Swifty”, and a large part of the reason why is that it often does not have the same value type behavior as other Swift types. We took this feedback seriously, and I would like to share with you the start of an important journey for some of the most commonly used APIs on all of our platforms: adopting value semantics for key Foundation types.

SE-0069:

The pervasive presence of struct types in the standard library, plus the aforementioned automatic bridging of all Cocoa SDK API when imported into Swift, leads to the feeling of an API impedance mismatch for key unbridged Foundation reference types.

[…]

The following value types will be added in the Swift overlay. Immutable/mutable pairs (e.g. Data and MutableData) will become one mutable struct type[…] The overlay is deployed back to the first supported release for Swift, so the implementation of these types will use the existing reference type API.

[…]

Some of the struct types will gain mutating methods. In general, the implementation of the struct type will forward to the underlying reference type, so as to allow a subclass to customize the behavior. If the struct is not initialized with a reference type (using a cast), then it is free to implement as much or as little behavior as it chooses either by delegation to the standard Foundation reference type or via a customized Swift implementation. However, our first version will rely heavily on the existing logic in the Objective-C Foundation framework.

[…]

The most obvious drawback to using a struct is that the type can no longer be subclassed. At first glance, this would seem to prevent the customization of behavior of these types. However, by publicizing the reference type and providing a mechanism to wrap it (mySubclassInstance as ValueType), we enable subclasses to provide customized behavior.

There’s a lot to consider here, but my initial reaction is positive. I don’t fully understand the memory and CPU overhead of the wrapping and bridging yet.

Update (2016-04-25): More about the bridging:

When these types are returned from Objective-C methods, they will be automatically bridged into the equivalent struct type. When these types are passed into Objective-C methods, they will be automatically bridged into the equivalent class type.

[…]

Swift has an existing mechanism to support bridging of Swift struct types to Objective-C reference types. It is used for NSNumber, NSString, NSArray, and more. Although it has some performance limitations (especially around eager copying of collection types), these new struct types will use the same functionality for two reasons:

We do not have block important improvements to our API on the invention of a new bridging system.

If and when the existing bridging system is improved, we will also be able to take advantage of those improvements.

Bridged struct types adopt a compiler-defined protocol called _ObjectiveCBridgeable. This protocol defines methods that convert Swift to Objective-C and vice-versa.

I have written about some bridging performance issues here. The proposal claims:

In almost all API in the SDK, the returned value type is immutable. In these cases, the copy is simply a retain and this operation is cheap. If the returned type is mutable, then we must pay the full cost of the copy.

However, as with the current bridging of string and collection types, it’s not clear what you can do in the event that there is an unwanted copy that affects performance. Likewise, there may be cases where you want to pass references to mutable objects back and forth. In your own Objective-C code, you could probably defeat the bridging by typing the parameters and return values as id, but what about in OS code?

Some interesting posts from the mailing list:

Tony Parker:

In reality, the stored reference object is a Swift class with Swift-native ref counting that implements the methods of the Objective-C NSData “protocol” (the interface described by @interface NSData) by holding a “real” NSMutableData ivar and forwarding messages to it. This technique is actually already in use in the standard library.

[…]

This means that if an Objective-C client calls retain on this object (which is a _SwiftNativeNSData), it actually calls swift_retain, which mutates the Swift ref count. This means that a call to _isUniquelyReferencedNonObjC from Swift actually still works on this class type.

Michael Buckley:

If the answer to this is that the MyMutableData instance is copied in the transfer from Objective-C to Swift so that mutations in Objective-C don’t affect the Swift instance, what about the case where I used var foo instead of let foo? Does that mean that if I modify the data in Objective-C that it won’t get modified in Swift (and vice-versa)?

Brent Royal-Gordon:

Have you looked at the performance and size of your value type designs when they’re made Optional? In the general case, Optional works by adding a tag byte after the wrapped instance, but for reference types it uses the 0x0 address instead. Thus, depending on how they’re designed and where Swift is clever enough to find memory savings, these value types might end up taking more memory than the corresponding reference types when made Optional, particularly if the reference types have tagged pointer representations.

Douglas Gregor:

I went down this rabbit hole a few weeks ago, trying to determine if we could make an isUniquelyReferenced that works for Objective-C-defined classes. One obvious issue is that you can’t always trust retainCount to do the right thing for an arbitrary Objective-C class, because it may have been overridden. We can probably say “don’t do that” and get away with it, but that brings us to Tony’s point: Objective-C weak references are stored in a separate side table, so we can’t atomically determine whether an Objective-C class is uniquely referenced. On platforms that have a non-pointer isa we could make this work through the inline reference count (which requires changes to the Objective-C runtime and therefore wouldn’t support backward deployment), but that still doesn’t give us isUniquelyReferenced for Objective-C classes everywhere.

Interestingly, while Swift’s object layout gives us the ability to consider the weak reference count, the Swift runtime currently does not do so. IIRC, part of our reasoning was that isUniquelyReferencedNonObjC is generally there to implement copy-on-write, where weak references aren’t actually interesting. However, we weren’t comfortable enough in that logic to commit to excluding weak references from isUniquelyReferencedNonObjC “forever", and certainly aren’t comfortable enough in that reasoning to enshrine “excluding weak references” as part of the semantics of isUniquelyReference for Objective-C classes.

Automatic Reference Counting (ARC) Cocoa iOS Language Design Mac Memory Management Objective-C Optimization Programming Swift Programming Language