{"id":30767,"date":"2020-11-23T17:01:50","date_gmt":"2020-11-23T22:01:50","guid":{"rendered":"https:\/\/mjtsai.com\/blog\/?p=30767"},"modified":"2020-11-25T16:11:50","modified_gmt":"2020-11-25T21:11:50","slug":"m1-memory-and-performance","status":"publish","type":"post","link":"https:\/\/mjtsai.com\/blog\/2020\/11\/23\/m1-memory-and-performance\/","title":{"rendered":"M1 Memory and Performance"},"content":{"rendered":"

Marcel Weiher<\/a> (Hacker News<\/a>):<\/p>\n

The M1 is apparently a multi-die package that contains both the actual processor die and the\nDRAM. As such, it has a very high-speed interface between the DRAM and the processors.\nThis high-speed interface, in addition to the absolutely humongous caches, is key to keeping the various functional\nunits fed. Memory bandwidth and latency are probably the<\/em> determining factors for many\nof today’s workloads, with a single access to main memory taking easily hundreds of clock cycles\nand the CPU capable of doing a good number of operations in each of these clock cycles.\nAs Andrew Black wrote<\/a>: “[..] computation is essentially free, because it happens ‘in the cracks’ between data fetch and data store; ..”.<\/p>

[…]<\/p>

The benefit of sticking to RC is much-reduced memory consumption. It turns out<\/a> that for\na tracing GC to achieve performance comparable with manual allocation, it needs several\ntimes the memory (different studies find different overheads, but at least 4x is a conservative\nlower bound). While I haven’t seen a study comparing RC, my personal experience is that the\noverhead is much lower, much more predictable, and can usually be driven down with little\nadditional effort if needed.<\/p>

So Apple can afford to live with more “limited” total memory because they need much less\nmemory for the system to be fast. And so they can do a system design that imposes this\nlimitation, but allows them to make that memory wicked fast. Nice<\/em>.<\/p><\/blockquote>\n\n

Mike<\/a>:<\/p>\n

The memory bandwidth on the new Macs is impressive. Benchmarks peg it at around 60GB\/sec–about 3x faster than a 16” MBP. Since the M1 CPU only has 16GB of RAM, it can replace the entire contents of RAM 4 times every second.<\/p>\n

[…]<\/p>\n

Some say we’re moving into a phase where we don’t need as much RAM, simply because as SSDs get faster there is less of a bottleneck for swap. […] However, with the huge jump in performance on the M1, the SSD is back to being an order of magnitude slower than main memory.<\/p>\n

So we’re left with the question: will SSD performance increase faster than memory bandwidth? And at what point does the SSD to RAM speed ratio become irrelevant?<\/p><\/blockquote>\n\n

Graham Lee<\/a>:<\/p>\n

And that makes me think that a Mac would either not go full NUMA, or would not have public API for it. Maybe<\/em> Apple would let the kernel and some OS processes have exclusive access to the on-package RAM, but even that seems overly complex (particularly where you have more than one M1 in a computer, so you need to specify core affinity for your memory allocations in addition to memory type). My guess is that an early workstation Mac with 16GB of M1 RAM and 64GB of DDR4 RAM would look like it has 64GB of RAM, with the on-package memory used for the GPU and as cache. NUMA APIs, if they come at all, would come later.<\/p><\/blockquote>\n\n

Previously:<\/p>\n