Archive for April 7, 2020

Tuesday, April 7, 2020

Apple’s Information Systems & Technology Division

Alex Kantrowitz (MacRumors, Hacker News):

IS&T is made up largely of contractors hired by rival consulting companies, and its dysfunction has led to a rolling state of war. “It’s a huge contractor org that handles a crazy amount of infrastructure for the company,” one ex-employee who worked closely with IS&T told me. “That whole organization is a Game of Thrones nightmare.”

Interviews with multiple former IS&T employees and its internal clients paint a picture of a division in turmoil, where infighting regularly prevents the creation of useful software, and whose contract workers are treated as disposable parts.


When IS&T’s projects are finally completed, they can cause even more headaches for Apple employees, who are left with a mess to clean up. Multiple people told me their Apple colleagues were forced to rewrite code after IS&T-built products showed up broken.

From what I’ve heard, this is a longtime problem, and it’s a mystery to me why this group has been immune to the Cook Doctrine. Apple buys forests to manage the paper used in its packaging and designs the desks its employees use and even the pizza boxes for its cafeteria. But when it comes to building the software that runs the company, that’s not considered a core competency.


Update (2020-04-08): chubot:

If you’ve worked in a huge org like Apple or the government, you’ve seen incredible inefficiencies. How can all these smart and often well-paid people be doing something so wrong (i.e. wasting time with political battles, etc.)?

But that’s only true if you look at the “medium view” of the organization. If you look at the large view, how the money actually flows, then it might be beneficial for one part like IS&T to be kind of broken, as long as the rest of the company works.

I have seen this dynamic at other companies. Internal tools can sometimes hold too much leverage over the organization. They can almost “blackmail” people into getting their way because they have a literal monopoly over what they do. It might be better to let multiple teams fight battles amongst themselves, which seems inefficient if you’re working as a regular joe, but could be efficient from the CEO’s perspective.


I was part of a team inside Apple that developed internal-facing tools for the retail teams. Our team was formed because of the extremely high cost in dollars and time that IS&T wanted to charge to develop some fairly straightforward tools. My team was taken from the departments that used the tools, and we developed things that were very custom-fit to what those departments needed.

Eventually politics overcame us, and IS&T finally managed to take over our team. We all became contractors in order to support the migration to their infrastructure, and continue development of the tools. A bit later we all became fired as our project was outsourced to contractors in India, managed by IS&T PMs who had no idea what our tool was used for and had never been inside a retail store. The tools all died shortly after that, some killed by IS&T, some petered out due to lack of use now that they no longer worked correctly or were a good fit for the users.

As far as we could tell, IS&T was run as a unique company inside Apple, which did it’s fair share of price gouging in order to make itself money to keep going. Its purpose never appeared to be helping Apple customers or Apple employees, it was simply to get bigger, absorbing more money and power wherever possible, with no apparent reigning in from the parent org.


I had a similar experience w/Apple. A startup I worked for was brought into Cupertino for a meeting w/their internal business teams. They wanted us to build them an app which seemed relatively simple involving their internal Cafe Mac cafeterias, data centers, and possibly retail locations. It was something that a company like Apple could build in their sleep. But the business folks we met with said that’s how it is at Apple, all the engineering talent goes towards the product side and almost nothing is left for internal IT. They told us how they struggled to get anything done and there were almost no resources available, so the business teams had to go and hire their own IT if they needed things done.


Is there any big consumer product company out there that doesn’t deprioritize management and engineering talent for internal tools and systems ?

Adam Bodnar:

Former IS&T. It’s been that way since 2003 and pretty sure long before then. Cook even made it worse at one point where he forced all non product dev team to be inside IS&T. Not sure if it’s changed, but at the time they were managed by someone that didn’t want to change.

See also: Hacker News, John Gruber, Slashdot.


Personally I find the discussion about IS&T less interesting than how SWE is broken at Apple — e.g., the engineering quality and resources in Eddy-org vs Craig-org, etc

and yes IS&T tools are trash but some of the SWE tools are just as bad

One of my favourite examples about Apple’s broken SWE tooling is the insanely bad crash reporting website that performs like it runs on a Mac Mini on an engineer’s desk.

Meanwhile Uber, with 1/20th of Apple’s resources, has crash analytics and reporting infra that is years ahead

Steve Troughton-Smith:

There’s a lot of talk about Apple’s IS&T group dysfunction without referencing how it’s relevant to any of us: IS&T run Radar. If you’ve wondered why it’s been so hard to improve Radar for external devs, there is a big part of the reason — we’re not IS&T’s customers

See also: John Gruber (tweet).

Marco Luoma:

A “friend” who works at the local Apple Store constantly complains about the bugginess of the POS software on the handheld devices. “Like sending soldiers into battle with rubber swords” is a memorable quote.