Archive for June 12, 2023

Monday, June 12, 2023

Microsoft Edge Removes Button to Delete Cloud Data

Venkat (via Hacker News):

Starting version 114, Microsoft Edge, no longer offers “reset sync” button that deletes data from Microsoft Servers. The button has been replaced with “Re-sync now” option. So if you’re privacy conscious, beware, there is no way to remove your Edge browsing data from Microsoft’s cloud now.

[…]

While Reset deletes sync data from Microsoft servers and re-syncs local browsing data with cloud when you turn on sync. Now, Re-sync just combines Edge’s local browsing data with sync data stored on Microsoft Servers.

However, you can use the special edge://sync-internals URL to access a hidden button to Clear Server Data.

YouTube Tries to Shut Down Invidious

Andy Maxwell (via Mike Rockwell):

The developers of Invidious, a privacy-respecting alternative front-end for YouTube, have received a cease-and-desist notice from YouTube’s legal department. The free and open source software, which provides a YouTube experience minus advertising and user tracking, has been instructed to shut down within seven days. As things stand, cooperation isn’t on the agenda.

[…]

The main problems apparent in Google’s cease and desist are straightforward; Invidious does not use YouTube’s API, and as a result, the project’s developers never agreed to any associated terms of service. As anyone who foolishly left their own instance open to the public will confirm, Invidious is effectively a proxy service, one with a penchant for bandwidth.

Investing 10% to Pay Back Technical Debt

Alex Ewerlöf (via Christian Tietze, Hacker News):

Almost everyone in the team had less experience than me (at least on paper) yet a simple task would take me multiple days longer than I thought. Yet, I felt dumb and helpless.

[…]

You see, the leadership did not care about the code quality as long as the stories were delivered on time. Corners were cut, tests were skipped[…]

[…]

Every other week, we had the “Tech Debt Friday”. These days were not planned for a specific issue or story. […] Engineers looked forward to the Tech Debt Friday. The team would happily remind management that this day cannot (under any circumstances) be planned for regular feature/bugfix work. Although we fixed some bugs along the way, this was primarily an investment to make future feature development cheaper while improving the maintainability and reliability.

Initially it was hard to defend spending 10% of the team bandwidth on tech debt, but over time the payback was huge[…]

kace91:

I’ve seen “x% time for tech debt“ rules in several companies and sadly it didn’t work too well.

Since the problem was the culture of continually pushing half baked features in the first place, the rule was quickly corrupted: people would design a good system, throw anything that’s not required for a POC into the tech debt backlog and deliver a barely functioning version.

“This is a technical debt task” was used to prevent everything that wasnt new Features taking time of the other 90% of the sprint.

Basically, if you assign a block of time to quality, you risk people taking that as an excuse to not focus on quality outside that block.

Previously:

Update (2023-06-23): Collin Donnell (Mastodon):

There are only two times technical debt will ever be addressed: upfront or never.

[…]

The thing you can do upfront is refactor as you go (don’t overdo it) and make it as easy possible to change your mind later. Delay, delay, delay. Just make as few decisions as you can, so when you have to do the next thing.

SwiftData

Apple (Hacker News):

Combining Core Data’s proven persistence technology and Swift’s modern concurrency features, SwiftData enables you to add persistence to your app quickly, with minimal code and no external dependencies. Using modern language features like macros, SwiftData enables you to write code that is fast, efficient, and safe, enabling you to describe the entire model layer (or object graph) for your app. The framework handles storing the underlying model data, and optionally, syncing that data across multiple devices.

We’ve been anticipating this for years, and it finally happened. Apple was waiting until it could utilize Swift Concurrency and macros, and I think that was a good decision. As expected, it’s more a Core Data wrapper than a complete reimagining. This is making some people unhappy, but I think the underlying Core Data design is still pretty solid. Perhaps over time Apple will reimplement the back end, as they are doing with Foundation. I’d certainly like to see less overhead at the managed object level, and database operations should be able to go a lot faster if they can work directly with Swift’s UTF-8 strings.

Since it requires Sonoma, and the initial version is completely inadequate for my needs, SwiftData probably won’t show up in my own code for years. But I’m pleased to see that Apple is moving in the same direction as I’ve been doing with my own Core Data code: schemas defined in code, type-safe predicates, a Collection for batch processing, and migrations decomposed into stages of lightweight table alterations with interleaved fixups.

I was excited to see that there were five WWDC sessions about SwiftData, but it turns out that they were rather short, repetitive, and lacking in depth. Apple seems to have deliberately avoided comparing it to Core Data and giving us a map of what’s different vs. the same and what’s not supported (yet?). So here are some things that I found notable and some questions that were unanswered:

Just about everything was renamed. In some cases this makes sense. For example, I never really understood why we had NSEntityDescription instead of just NSEntity. Now it’s Entity. In other cases, it’s a bit confusing because NSManagedObject has become PersistentModel, whereas “model” used to mean something completely different that is now called Schema. “Transient” was not renamed but its meaning has changed. There’s no more store coordinator and no replacement for some of its functionality.

The XML and binary store types are gone. I’m guessing that the memory store is now based on SQLite, though I don’t think Apple has said.

It sounds like Predicate does not yet support everything that NSPredicate did, but it’s not clear to me exactly what the differences are or to what extent you can use NSPredicate from SwiftData. I guess a lot of questions can probably be answered from the code.

Likewise with various schema features. I didn’t see anything about indexes or validation predicates. Having .unique just do an UPSERT when there’s a conflict is kind of cool, but what if I want to handle constraint violations in a different way? Composite uniqueness constraints still seem to be handled as arrays of strings. If you want to include components of a composite attribute, I guess you’re supposed to use the mangledName?

How do PersistentIdentifiers work? I don’t see a way to convert it back and forth to NSManagedObjectID or URL, which seems like a major problem. There also doesn’t seem to be the concept of a temporary identifier. Temporary IDs were a major source of bugs with Core Data, but I don’t see how they could have been eliminated without causing major performance problems. If they are still there but opaque to us, that seems bad.

PersistentModel seems to be missing a lot of lifecycle hooks and control over faulting and refreshing. I was surprised to see that relationships are handled as arrays instead of as sets, even though presumably they are still sets at the database level. There does not seem to be support for ordered relationships. Built-in support for non-object attribute types is great. For structs and enums, it’s not fully clear to me when they get automatically destructured into composite attributes, when it just uses Codable to transform them into a blob of data, and whether I can choose.

Stuff at the context level is more automatic and gives you less control. Auto-saving that can be turned off is nice. But there doesn’t seem to be a way to set a merge policy between the context and store, and I didn’t see anything about merging changes between contexts. There are also no more parent contexts.

There’s no ModelStore class, just ModelContainer and ModelContext. I guess containers can still have multiple stores because they support a configurations array, but I’m not sure whether you can add to this. There doesn’t seem to be a way to assign the store for a new object, nor to scope a fetch to certain stores.

Persistent history tracking is enabled by default, but you can seemingly only access the history via Core Data.

Contexts now have much more convenient APIs for batch enumeration (with mutation checking!) and batch inserts/deletes/updates. The batch insert API seems less efficient, though, since you have to give it fully realized Swift dictionaries (no shared keys or provider). There does not seem to be a way to do count or dictionary fetches.

Lastly, SwiftData was clearly designed to work with Swift Concurrency, but Apple didn’t say much about this. I assume the idea is that the language will prevent you from passing model objects out of their context’s actor—but maybe not.

See also:

Previously:

Update (2023-06-13): Stuart Breckenridge:

There’s no way to turn off write-ahead logging in SwiftData.

Indeed, there doesn’t seem to be access to any of the coordinator options or SQLite pragmas.

Gwendal Roué:

So… one WWDC video mentions that SwiftData performs upserts when there is a conflict on an unique attribute. Corollary: SwiftData does not perform uniquing, at least not like Core Data, and we may end up with two distinct model instances that refer to the same persisted database row. “Identity” of models promises to be a subtle source of surprises 😅

I’m guessing that it will do the same thing as Core Data and give you a detached object with a different ID. That still seems to be possible to detect because PersistentModel.context is optional. But it would be good for Apple to explain this.

Jack Palevich:

I wish Apple would publish a “theory of operation” for both CoreData and SwiftData, that documents and explains the design choices they have made.

There are a lot of subtle tradeoffs when designing an ORM. When you see something like this upsert/uniquing tradeoff, it would be nice to know if it’s an intentional tradeoff, and if so, what the motivations is.

Update (2023-06-19): Stuart A. Malone:

This was the first error I encountered using Swift Data, too. I don’t have the code in front of me, but I believe I solved it by inserting both objects in the relationship into the context before creating the relationship. It didn’t like having an object in a context related to an object without a context.

With Core Data, new objects are inserted into a context by default.

Helge Heß:

OMG, a 10k batch size did it. It took 4 hours and peeked at over 70GB of RAM usage, but SwiftData finally managed to import my huge 25MB SQLite database 🙂

Keith Harrison:

I attended a WWDC23 SwiftData lab and asked questions in the data-frameworks Slack QA session. This is my summary of what I learned.

Update (2023-06-21): milutz:

Nevertheless, if I have a unique constraint on an (String) attribute and try to insert the same again, I end up in the debugger in the generated getter of the attribute[…]

Update (2023-06-23): Paul Hudson:

I just filed a whole bunch of feedback reports for Apple regarding SwiftData. If you see any of these and want the same, please file your own report asking for it – every feedback counts, particularly now as we’re still in the early betas.

Update (2023-06-26): Helge Heß:

There is no way to do a case-insensitive compare/contains in SwiftData predicates yet, right? It tried a few things, but they don’t seem to work.

Nor normalized comparisons.

Update (2023-06-30): Helge Heß:

The Predicate macro as used in gave me a bit of a head scratch, but I think I have figured it out. The init of the Predicate struct takes a builder function, which gets a Variable as the input. Really just a placeholder. But it has to have a VariableID, that is picked by the Variable.init internally. Currently a UInt sequence.

Helge Heß:

Here are the overloads required to get Codable SwiftData models. [Update (2023-09-08): No longer needed.]

Update (2023-07-05): Mohammad Azam:

This article is structured into several sections, each delving into different aspects of the SwiftData framework. First, we will explore the foundational concepts of SwiftData, followed by an examination of its architectural design, relationship management, migration capabilities, and more.

Paul Hudson:

It’s not long until the window closes for SwiftData changes in iOS 17.0. Please file feedback for things that affect you! Two massive ones for me: an equivalent of NSFetchedResultsController to make MVVM work, and an equivalent of (or at least support for) NSCompoundPredicate.

Ian Dundas:

1: PersistentIdentifier encodes to blank JSON, and 2: when it fails to migrate it doesn’t just throw an error - it also wipes the database and starts fresh!!

Helge Heß:

Looks like using a DateComponents value for like say a birthDate doesn’t work in SwiftData. It fails already in the Schema setup (hence can’t be worked around using an accessor extension for the PersistentModel like Codable).

Update (2023-07-10): Stewart Lynch:

In SwiftData, if I create a model that has a property that is an enum like the following, what it does is create a property for each case of the enum rather than a single case for the property itself.

Helge Heß:

Is it possible to undo changes to a SwiftData model object / view context? I would have thought that modelContext.rollback() would do that (as a peer to save()), but maybe that's for transactions only?

I’m quite surprised to hear that rollback() does not work like with Core Data. There’s no documentation yet about what it’s intended to do.

fatbobman:

Here are some questions and considerations I have compiled regarding SwiftData (originally posted in a tweet, without a more systematic categorization)[…]

[…]

In the current version, data created through other contexts (ModelContext) is not automatically merged into the view context.

[…]

Neither PersistentModel nor ModelContext are Sendable (ModelContainer is Sendable), and they are thread-limited like Core Data.

[…]

Derived options for Attribute have been deprecated.

Is this referring to NSDerivedAttributeDescription?

Via Malcolm Hall:

Query (an alternative to FetchRequest) does not provide a method for dynamically switching predicates and sorting.” crazy it was released with out this!

Malcolm Hall:

Query for a relationship doesn’t auto-update yet!

Update (2023-07-25): Ian Dundas:

Macro magic - expanding it shows that I guess a schema is defined with a default value would only be evaluated once. I guess the way around this is set the random values inside the init(name:).

In other words, because of the @Model macro, initial property values (which become default values in the schema) don’t behave the same way as with a regular class, even though they look the same.

Jessy:

Why does a SwiftData Model allow Array? How do you really store one?

Order is not preserved, meaning the Array that exists in memory is not likely the one that will get persisted and reloaded. That’s not really conceptually an Array—that’s halfway to a Set.

Helge Heß:

Maybe it is just me, but I just don’t find much “Swiftyness” in SwiftData. In fact it doesn’t feel Swifty at all to me. Well, maybe when that #Predicate with 3 expressions gets back to you with the “unable to resolve blub in time” 🙈

E.g. Swiftlang has gone (extraordinarily annoying) lengths at making sure that the initialization contract is ensured. No more “half initialized objects” anymore. And then we get this 🤷‍♀️

And it will crash any time the values aren’t available, not just mid-initialization. It’s strange because Core Data went to the trouble of adding shouldDeleteInaccessibleFaults so that exceptions (e.g. from objects that no longer exist) don’t make Swift code crash. Instead, the Objective-C code will return default values for properties. But then actually accessing an absent value from Swift will crash as it tries to bridge an unexpected nil value. You can, however, check for absent values by casting to an optional (even though the property is not optional).

Keith Harrison:

As with Core Data, SwiftData marks the object as changed if you call the setter on any of the properties of the object. That’s the case even if you don’t change the value of the property.

Previously:

Update (2023-07-27): Ian:

Wow, watch out for this SwiftData bug in b5: Simply add a comment on the same line as a Query (!). The macro tries to pull it in as code and the result won’t build[…]

That’s a macro failure mode that I didn’t expect.

Previously:

Update (2023-08-10): Donny Wals:

The more I explore SwiftData by trying to implement the topics I cover in Practical Core Data the more I realize SwiftData is very much a beta framework.

I fully plan to write Practical SwiftData but right now I’m wondering if iOS 17 is too soon for me to be able to write something that’s more insightful than just the basics along with a bunch of “this isn’t supported right now” notes…

Helge Heß:

Since the release notes have been a little “shallow” for beta6, here is what I found for SwiftData so far:

  • getValue(for:) => getValue(forKey:)
  • Entity => Schema.Entity, etc
  • Property => SchemaProperty
  • deleteRule => not an option anymore, own arg
  • objectID => persistentModelID
  • Entity.mangledName gone
  • Property.isRelationship() func is now a property
  • Attribute.nested gone
  • superEntityName => superentityName 🙈
  • ctx.object(with:) => ctx.model(for:)

Tim Schmitz:

I’m starting to wonder if SwiftData is going to make it into iOS 17.0. The list of known issues is pretty long, and it’s not the kind of thing you’d want to ship half-baked. I hope they’ll take the time to get it to a stable place rather than rush it out the door.

Update (2023-08-17): Guilherme Rambo:

Every new iOS 17 beta build causes SwiftData apps built with the previous Xcode 15 beta to crash on launch due to binary incompatibility. Apple released iOS 17 beta 6 yesterday, which did the same, but didn’t release a new Xcode 15 beta 🥲 I’m glad I haven’t shipped any public TestFlight builds yet.

Helge Heß:

SwiftData is still in heavy flux and changes in major ways every beta.

At release time they’d have to pin down the ABI which makes me think it’ll be dropped for the first iOS 17 release. (cutoff should be within weeks and it seems far from ready)

Update (2023-08-24): Helge Heß:

My summary of SwiftData b7 changes[…]

Helge Heß:

Here is the expansion of the Model macro, it has some really funky stuff. I can’t even compile the thing when manually expanding the macros? 🤔 Why would that be? Special compiler support? This time around the regular properties seem to exist as real instance variables. An _ peer is generated, w/ _SwiftDataNoType 💥

Helge Heß:

In Beta7 SwiftData allows initialization as part of the property declaration again. But since they are moved to a different place by the macro, they can’t use type inference. They do seem to set both the model object ivar value and the default in the storage (which I think makes sense).

Update (2023-09-06): Helge Heß:

Interesting, the SwiftData Model macro doesn’t have an originalName. Isn’t that necessary for table renaming/migration?

Helge Heß:

I think there is a different wrt transient properties between SwiftData and CoreData. The latter still tracks transient attributes, it just doesn’t persist them.

SwiftData doesn’t seem to include them into the schema at all (neither in the KVC metadata). Also: They do not register w/ Observation, I wonder whether that is a bug. I.e. a change to a transient property may not trigger a SwiftUI view refresh.

Update (2023-09-14): Helge Heß:

As far as I can tell there are no major changes to the SwiftData API in Xcode 15 RC. The RC generally seems to be a major improvement, also in swiftc, it successfully compiles a test app again. Most of my tests run, if I disable a lot. I still have the impression that it gets confused w/ types. I suspect it is related to having the same non-qualified type names (e.g. @model class CountTests.Item and also a @model class FetchTests.Item, different in structure).