Monday, June 12, 2023

SwiftData

Apple (Hacker News):

Combining Core Data’s proven persistence technology and Swift’s modern concurrency features, SwiftData enables you to add persistence to your app quickly, with minimal code and no external dependencies. Using modern language features like macros, SwiftData enables you to write code that is fast, efficient, and safe, enabling you to describe the entire model layer (or object graph) for your app. The framework handles storing the underlying model data, and optionally, syncing that data across multiple devices.

We’ve been anticipating this for years, and it finally happened. Apple was waiting until it could utilize Swift Concurrency and macros, and I think that was a good decision. As expected, it’s more a Core Data wrapper than a complete reimagining. This is making some people unhappy, but I think the underlying Core Data design is still pretty solid. Perhaps over time Apple will reimplement the back end, as they are doing with Foundation. I’d certainly like to see less overhead at the managed object level, and database operations should be able to go a lot faster if they can work directly with Swift’s UTF-8 strings.

Since it requires Sonoma, and the initial version is completely inadequate for my needs, SwiftData probably won’t show up in my own code for years. But I’m pleased to see that Apple is moving in the same direction as I’ve been doing with my own Core Data code: schemas defined in code, type-safe predicates, a Collection for batch processing, and migrations decomposed into stages of lightweight table alterations with interleaved fixups.

I was excited to see that there were five WWDC sessions about SwiftData, but it turns out that they were rather short, repetitive, and lacking in depth. Apple seems to have deliberately avoided comparing it to Core Data and giving us a map of what’s different vs. the same and what’s not supported (yet?). So here are some things that I found notable and some questions that were unanswered:

Just about everything was renamed. In some cases this makes sense. For example, I never really understood why we had NSEntityDescription instead of just NSEntity. Now it’s Entity. In other cases, it’s a bit confusing because NSManagedObject has become PersistentModel, whereas “model” used to mean something completely different that is now called Schema. “Transient” was not renamed but its meaning has changed. There’s no more store coordinator and no replacement for some of its functionality.

The XML and binary store types are gone. I’m guessing that the memory store is now based on SQLite, though I don’t think Apple has said.

It sounds like Predicate does not yet support everything that NSPredicate did, but it’s not clear to me exactly what the differences are or to what extent you can use NSPredicate from SwiftData. I guess a lot of questions can probably be answered from the code.

Likewise with various schema features. I didn’t see anything about indexes or validation predicates. Having .unique just do an UPSERT when there’s a conflict is kind of cool, but what if I want to handle constraint violations in a different way? Composite uniqueness constraints still seem to be handled as arrays of strings. If you want to include components of a composite attribute, I guess you’re supposed to use the mangledName?

How do PersistentIdentifiers work? I don’t see a way to convert it back and forth to NSManagedObjectID or URL, which seems like a major problem. There also doesn’t seem to be the concept of a temporary identifier. Temporary IDs were a major source of bugs with Core Data, but I don’t see how they could have been eliminated without causing major performance problems. If they are still there but opaque to us, that seems bad.

PersistentModel seems to be missing a lot of lifecycle hooks and control over faulting and refreshing. I was surprised to see that relationships are handled as arrays instead of as sets, even though presumably they are still sets at the database level. There does not seem to be support for ordered relationships. Built-in support for non-object attribute types is great. For structs and enums, it’s not fully clear to me when they get automatically destructured into composite attributes, when it just uses Codable to transform them into a blob of data, and whether I can choose.

Stuff at the context level is more automatic and gives you less control. Auto-saving that can be turned off is nice. But there doesn’t seem to be a way to set a merge policy between the context and store, and I didn’t see anything about merging changes between contexts. There are also no more parent contexts.

There’s no ModelStore class, just ModelContainer and ModelContext. I guess containers can still have multiple stores because they support a configurations array, but I’m not sure whether you can add to this. There doesn’t seem to be a way to assign the store for a new object, nor to scope a fetch to certain stores.

Persistent history tracking is enabled by default, but you can seemingly only access the history via Core Data.

Contexts now have much more convenient APIs for batch enumeration (with mutation checking!) and batch inserts/deletes/updates. The batch insert API seems less efficient, though, since you have to give it fully realized Swift dictionaries (no shared keys or provider). There does not seem to be a way to do count or dictionary fetches.

Lastly, SwiftData was clearly designed to work with Swift Concurrency, but Apple didn’t say much about this. I assume the idea is that the language will prevent you from passing model objects out of their context’s actor—but maybe not.

See also:

Previously:

Update (2023-06-13): Stuart Breckenridge:

There’s no way to turn off write-ahead logging in SwiftData.

Indeed, there doesn’t seem to be access to any of the coordinator options or SQLite pragmas.

Gwendal Roué:

So… one WWDC video mentions that SwiftData performs upserts when there is a conflict on an unique attribute. Corollary: SwiftData does not perform uniquing, at least not like Core Data, and we may end up with two distinct model instances that refer to the same persisted database row. “Identity” of models promises to be a subtle source of surprises 😅

I’m guessing that it will do the same thing as Core Data and give you a detached object with a different ID. That still seems to be possible to detect because PersistentModel.context is optional. But it would be good for Apple to explain this.

Jack Palevich:

I wish Apple would publish a “theory of operation” for both CoreData and SwiftData, that documents and explains the design choices they have made.

There are a lot of subtle tradeoffs when designing an ORM. When you see something like this upsert/uniquing tradeoff, it would be nice to know if it’s an intentional tradeoff, and if so, what the motivations is.

Update (2023-06-19): Stuart A. Malone:

This was the first error I encountered using Swift Data, too. I don’t have the code in front of me, but I believe I solved it by inserting both objects in the relationship into the context before creating the relationship. It didn’t like having an object in a context related to an object without a context.

With Core Data, new objects are inserted into a context by default.

Helge Heß:

OMG, a 10k batch size did it. It took 4 hours and peeked at over 70GB of RAM usage, but SwiftData finally managed to import my huge 25MB SQLite database 🙂

Keith Harrison:

I attended a WWDC23 SwiftData lab and asked questions in the data-frameworks Slack QA session. This is my summary of what I learned.

Update (2023-06-21): milutz:

Nevertheless, if I have a unique constraint on an (String) attribute and try to insert the same again, I end up in the debugger in the generated getter of the attribute[…]

Update (2023-06-23): Paul Hudson:

I just filed a whole bunch of feedback reports for Apple regarding SwiftData. If you see any of these and want the same, please file your own report asking for it – every feedback counts, particularly now as we’re still in the early betas.

Update (2023-06-26): Helge Heß:

There is no way to do a case-insensitive compare/contains in SwiftData predicates yet, right? It tried a few things, but they don’t seem to work.

Nor normalized comparisons.

Update (2023-06-30): Helge Heß:

The Predicate macro as used in gave me a bit of a head scratch, but I think I have figured it out. The init of the Predicate struct takes a builder function, which gets a Variable as the input. Really just a placeholder. But it has to have a VariableID, that is picked by the Variable.init internally. Currently a UInt sequence.

Helge Heß:

Here are the overloads required to get Codable SwiftData models. [Update (2023-09-08): No longer needed.]

Update (2023-07-05): Mohammad Azam:

This article is structured into several sections, each delving into different aspects of the SwiftData framework. First, we will explore the foundational concepts of SwiftData, followed by an examination of its architectural design, relationship management, migration capabilities, and more.

Paul Hudson:

It’s not long until the window closes for SwiftData changes in iOS 17.0. Please file feedback for things that affect you! Two massive ones for me: an equivalent of NSFetchedResultsController to make MVVM work, and an equivalent of (or at least support for) NSCompoundPredicate.

Ian Dundas:

1: PersistentIdentifier encodes to blank JSON, and 2: when it fails to migrate it doesn’t just throw an error - it also wipes the database and starts fresh!!

Helge Heß:

Looks like using a DateComponents value for like say a birthDate doesn’t work in SwiftData. It fails already in the Schema setup (hence can’t be worked around using an accessor extension for the PersistentModel like Codable).

Update (2023-07-10): Stewart Lynch:

In SwiftData, if I create a model that has a property that is an enum like the following, what it does is create a property for each case of the enum rather than a single case for the property itself.

Helge Heß:

Is it possible to undo changes to a SwiftData model object / view context? I would have thought that modelContext.rollback() would do that (as a peer to save()), but maybe that's for transactions only?

I’m quite surprised to hear that rollback() does not work like with Core Data. There’s no documentation yet about what it’s intended to do.

fatbobman:

Here are some questions and considerations I have compiled regarding SwiftData (originally posted in a tweet, without a more systematic categorization)[…]

[…]

In the current version, data created through other contexts (ModelContext) is not automatically merged into the view context.

[…]

Neither PersistentModel nor ModelContext are Sendable (ModelContainer is Sendable), and they are thread-limited like Core Data.

[…]

Derived options for Attribute have been deprecated.

Is this referring to NSDerivedAttributeDescription?

Via Malcolm Hall:

Query (an alternative to FetchRequest) does not provide a method for dynamically switching predicates and sorting.” crazy it was released with out this!

Malcolm Hall:

Query for a relationship doesn’t auto-update yet!

Update (2023-07-25): Ian Dundas:

Macro magic - expanding it shows that I guess a schema is defined with a default value would only be evaluated once. I guess the way around this is set the random values inside the init(name:).

In other words, because of the @Model macro, initial property values (which become default values in the schema) don’t behave the same way as with a regular class, even though they look the same.

Jessy:

Why does a SwiftData Model allow Array? How do you really store one?

Order is not preserved, meaning the Array that exists in memory is not likely the one that will get persisted and reloaded. That’s not really conceptually an Array—that’s halfway to a Set.

Helge Heß:

Maybe it is just me, but I just don’t find much “Swiftyness” in SwiftData. In fact it doesn’t feel Swifty at all to me. Well, maybe when that #Predicate with 3 expressions gets back to you with the “unable to resolve blub in time” 🙈

E.g. Swiftlang has gone (extraordinarily annoying) lengths at making sure that the initialization contract is ensured. No more “half initialized objects” anymore. And then we get this 🤷‍♀️

And it will crash any time the values aren’t available, not just mid-initialization. It’s strange because Core Data went to the trouble of adding shouldDeleteInaccessibleFaults so that exceptions (e.g. from objects that no longer exist) don’t make Swift code crash. Instead, the Objective-C code will return default values for properties. But then actually accessing an absent value from Swift will crash as it tries to bridge an unexpected nil value. You can, however, check for absent values by casting to an optional (even though the property is not optional).

Keith Harrison:

As with Core Data, SwiftData marks the object as changed if you call the setter on any of the properties of the object. That’s the case even if you don’t change the value of the property.

Previously:

Update (2023-07-27): Ian:

Wow, watch out for this SwiftData bug in b5: Simply add a comment on the same line as a Query (!). The macro tries to pull it in as code and the result won’t build[…]

That’s a macro failure mode that I didn’t expect.

Previously:

Update (2023-08-10): Donny Wals:

The more I explore SwiftData by trying to implement the topics I cover in Practical Core Data the more I realize SwiftData is very much a beta framework.

I fully plan to write Practical SwiftData but right now I’m wondering if iOS 17 is too soon for me to be able to write something that’s more insightful than just the basics along with a bunch of “this isn’t supported right now” notes…

Helge Heß:

Since the release notes have been a little “shallow” for beta6, here is what I found for SwiftData so far:

  • getValue(for:) => getValue(forKey:)
  • Entity => Schema.Entity, etc
  • Property => SchemaProperty
  • deleteRule => not an option anymore, own arg
  • objectID => persistentModelID
  • Entity.mangledName gone
  • Property.isRelationship() func is now a property
  • Attribute.nested gone
  • superEntityName => superentityName 🙈
  • ctx.object(with:) => ctx.model(for:)

Tim Schmitz:

I’m starting to wonder if SwiftData is going to make it into iOS 17.0. The list of known issues is pretty long, and it’s not the kind of thing you’d want to ship half-baked. I hope they’ll take the time to get it to a stable place rather than rush it out the door.

Update (2023-08-17): Guilherme Rambo:

Every new iOS 17 beta build causes SwiftData apps built with the previous Xcode 15 beta to crash on launch due to binary incompatibility. Apple released iOS 17 beta 6 yesterday, which did the same, but didn’t release a new Xcode 15 beta 🥲 I’m glad I haven’t shipped any public TestFlight builds yet.

Helge Heß:

SwiftData is still in heavy flux and changes in major ways every beta.

At release time they’d have to pin down the ABI which makes me think it’ll be dropped for the first iOS 17 release. (cutoff should be within weeks and it seems far from ready)

Update (2023-08-24): Helge Heß:

My summary of SwiftData b7 changes[…]

Helge Heß:

Here is the expansion of the Model macro, it has some really funky stuff. I can’t even compile the thing when manually expanding the macros? 🤔 Why would that be? Special compiler support? This time around the regular properties seem to exist as real instance variables. An _ peer is generated, w/ _SwiftDataNoType 💥

Helge Heß:

In Beta7 SwiftData allows initialization as part of the property declaration again. But since they are moved to a different place by the macro, they can’t use type inference. They do seem to set both the model object ivar value and the default in the storage (which I think makes sense).

Update (2023-09-06): Helge Heß:

Interesting, the SwiftData Model macro doesn’t have an originalName. Isn’t that necessary for table renaming/migration?

Helge Heß:

I think there is a different wrt transient properties between SwiftData and CoreData. The latter still tracks transient attributes, it just doesn’t persist them.

SwiftData doesn’t seem to include them into the schema at all (neither in the KVC metadata). Also: They do not register w/ Observation, I wonder whether that is a bug. I.e. a change to a transient property may not trigger a SwiftUI view refresh.

Update (2023-09-14): Helge Heß:

As far as I can tell there are no major changes to the SwiftData API in Xcode 15 RC. The RC generally seems to be a major improvement, also in swiftc, it successfully compiles a test app again. Most of my tests run, if I disable a lot. I still have the impression that it gets confused w/ types. I suspect it is related to having the same non-qualified type names (e.g. @model class CountTests.Item and also a @model class FetchTests.Item, different in structure).

10 Comments RSS · Twitter · Mastodon

"Composite uniqueness constraints still seem to be handled as arrays of strings."

No link for this, unfortunately. Has anyone found a way to specify a uniqueness constraint that spans multiple columns? I keep looking without luck. I don't like using strings but it would be better than nothing.

In Core Data, I would do this in the model inspector by adding a single line of comma-separated property names to the Constraints box. I tried converting one of my existing entities from Core Data to SwiftData using Xcode's converter. It gave me a vague warning message about unsupported functionality and then created a model with no uniqueness constraints at all.

I would love to see an example of how to do this if anyone has actually done it.

@Justin It’s a property of the Schema.Entity. It’s an array of an array of strings because you can have multiple constraints.

Embarrassing! Hard to believe I missed this. Thanks for the link!

The array of array of strings makes perfect sense, and mirrors Core Data's list of comma-separated strings in the editor.

However, maybe it's not so hard to believe I missed it. Having watched all five WWDC SwiftData videos, it seems they only ever talk about the Schema class in relation to migrations. They want SwiftData to infer the schema as defined by your @Model classes, and it doesn't seem like you can define this type of constraint in your model. Perhaps, though, the importance of the Schema class just went right over my head.

I'll have to re-watch the videos with this new info in mind. Especially the "Dive Deeper" video, as I recall them talking more about Schemas there.

@Justin Yeah, like I was saying above, the sessions really only covered the basics. They wanted to show the happy path where the Schema and Predicates are created automatically using macros, not how you would create them dynamically. It looks like you can create an entire Schema from scratch in code, and then use it to create a ModelContainer. But maybe you can just modify the default macro-generated Schema on the convenience ModelContainer if all you need to do is tweak it…

There is also the undocumented PersistentModel.schemaMetadata() function, which may allow for some customization.

I don't think you are supposed to build a *schema* dynamically, and it is kinda impossible, e.g. you can't set a model type on an Entity (it is somewhat surprising that those types are even exposed). And I think that this is kinda fine and the idea. You declare the schema in Swift using the macros, everything else is none of your business.

The worse thing is that you can't build dynamic predicates w/o getting creative (stay tuned), the structure has to be typed out statically.

The schemaMetadata provides the reflection information which maps a string name to the key path to access the model (essentially KVC), plus a default value and the property information if set explicitly (via @Attribute, @Relationship). It is useful, fancy things can be build around this.

It’s a shame that the macros seem to rely on private API. And the predicates limitation is really unfortunate and I don’t think was even discussed in the pitch. (I guess it bypassed the proposal process?)

What do you mean by the "macros seem to rely on private API". I don't think that this is the case. (the `schemaMetadata` is even declared public, I think it has to be because it is part of the PersistentModel protocol, i.e. public.)
Once it gets actually incorporated into the system (iOS 17.5?), it also has to be ABI stable.

Also note that the Predicate macros (predicates are Foundation btw, not CoreData) are not open source (yet?) and explicitly disabled for Linux and Windows.

The observation macros certainly do. I have not looked at the SwiftData macro expansions, but I assumed they also did, based on your statement that it’s not possible to build a schema dynamically. If it’s all public API, you should be able to implement your own schemaMetadata() that provides what you want. Or are you saying that it’s only impossible in that the Entity created by that metadata is locked to the model type that provided the schemaMetadata() (i.e. no managedObjectClassName)?

If it’s all static at compile time, and there’s no base class that implements the model protocol, I guess that means it’s impossible to build something like Core Data Lab that can work with arbitrary models not known to the app?

The `Schema.Entity` class (the peer to `EOEntity`, whatever that is called in CoreData, Description sth), is a final class that provides no way to access the associated Swift model type anymore (i.e. sth like `managedObjectClassName`). It still has this internally of course, both as a mangled name and as the actual type.

You can implement your own schemaMetadata() function and I think the whole schema buildup is just based on this. And yes, the Entity is tied to this. You can create/init a Schema.Entity on your own, but you can't attach a type to it, so I'm not entirely sure why the inits are even public, they seem to be of no use?
But note that schemaMetadata is build around the statically typed Swift KeyPath'es, not KVC. Not sure how dynamic that can be made, maybe it can (via dynamicMemberLookup).

I don't know how Core Data Lab is implemented, I would assume it doesn't even use CD but goes straight to the storage? Right now, I would assume it would just work w/ SwiftData, because the backing storage _still is_ CoreData.
Not sure this actually works just yet, but let's assume SwiftData allows you to use an enum (or actually an arbitrary Codable type, as it seems to be the goal) as a property. A dynamic app could only ever display this as JSON or sth (maybe w/ some smarts). I would imagine that enums in particular would be very popular in the Swift world.

It depends a little on what someone wants to accomplish. Currently everything is still very dynamic, but that might change. Actually I think I could build an API that is compatible to SwiftData, but in a Lighter.swift way. I.e. predicate macros would build the actual SQL required at Swift build time, and model macros would do the same (create tables etc). (I'm super surprised it isn't that way already, macros seem to be built for that kind of thing)
Or the other - dynamic - way around, you could also build a model macro that directly sits on top of CoreData, giving you the same code based schema construction SwiftData has, but w/ just regular CoreData in a back porting way.

I'm not sure either makes sense yet, I recently asked in a dev forum and essentially 95% of the answers was that they don't want CoreData or SwiftData (mostly how much they dislike CoreData and what they'd want from SwiftData, ignoring that SwiftData exists already 🙈). Which mirrors my broader view of the thing, this could have used a SwiftUI like treatment, something completely new targeted at Swift, not a "let's see how this ages old thing might work in Swift".

You’re right, I think there’s not much you can do with typed Swift KeyPaths. :-(

My understanding is that Core Data Lab loads the actual managed object model and uses real Core Data to view/search stuff, only with the generic NSManagedObject instead of whatever custom classes are named in the model (schema). Yes, it should work for now, while the storage is compatible, except that SwiftData may not necessarily write the MOM to disk. Sometimes you can get a cached version from the store’s metadata…

Yes, for me SwiftData is neither here nor there. It’s so lacking in features compared with Core Data but it’s not a great leap forward, either, just a bit more ergonomic. I already have my own Swift code to make Core Data more ergonomic, and it works with the full feature set and deploys back to El Capitan. I kind of wish Apple had done something like that, but official, if they weren’t going to go full SwiftUI. I basically like Core Data and will continue to use it, though the existence of SwiftData makes me a bit worried that Apple isn’t going to stick with it. If I were going to switch it would probably be to something other than SwiftData. There are a bunch of choices that are faster, open source, have features that Core Data lacks, etc. If SwiftData remains schema-compatible and eventually matches the Core Data feature set, I suppose I’ll eventually migrate to it.

Leave a Comment