{"id":18244,"date":"2017-06-27T14:02:15","date_gmt":"2017-06-27T18:02:15","guid":{"rendered":"https:\/\/mjtsai.com\/blog\/?p=18244"},"modified":"2017-12-28T14:05:47","modified_gmt":"2017-12-28T19:05:47","slug":"bug-in-skylake-and-kaby-lake-hyper-threading","status":"publish","type":"post","link":"https:\/\/mjtsai.com\/blog\/2017\/06\/27\/bug-in-skylake-and-kaby-lake-hyper-threading\/","title":{"rendered":"Bug in Skylake and Kaby Lake Hyper-threading"},"content":{"rendered":"<p><a href=\"https:\/\/lists.debian.org\/debian-devel\/2017\/06\/msg00308.html\">Henrique de Moraes Holschuh<\/a>:<\/p>\n<blockquote cite=\"https:\/\/lists.debian.org\/debian-devel\/2017\/06\/msg00308.html\"><p>This advisory is about a processor\/microcode defect recently identified\non Intel Skylake and Intel Kaby Lake processors with hyper-threading\nenabled.  This defect can, when triggered, cause unpredictable system\nbehavior: it could cause spurious errors, such as application and system\nmisbehavior, data corruption, and data loss.<\/p><p>It was brought to the attention of the Debian project that this defect\nis known to directly affect some Debian stable users (refer to the end\nof this advisory for details), thus this advisory.<\/p><p>Please note that the defect can potentially affect any operating system\n(it is not restricted to Debian, and it is not restricted to Linux-based\nsystems).  It can be either avoided (by disabling hyper-threading), or\nfixed (by updating the processor microcode).<\/p><p>Due to the difficult detection of potentially affected software, and the\nunpredictable nature of the defect, all users of the affected Intel\nprocessors are strongly urged to take action as recommended by this\nadvisory.<\/p><\/blockquote>\n\n<p>Via <a href=\"https:\/\/twitter.com\/atomicbird\/status\/879404454780563457\">Tom Harrington<\/a>:<\/p>\n<blockquote cite=\"https:\/\/twitter.com\/atomicbird\/status\/879404454780563457\">\n<p>Check your Mac CPU with &ldquo;sysctl machdep.cpu&rdquo; and compare to this. [<a href=\"http:\/\/ark.intel.com\/products\/codename\/37572\/Skylake\">Skylake list<\/a>, <a href=\"http:\/\/ark.intel.com\/products\/codename\/82879\/Kaby-Lake\">Kaby Lake<\/a> list]<\/p>\n<p>Developers who are concerned can use Instruments to disable hyperthreading until reboot. See Instruments prefs.<\/p>\n<\/blockquote>\n\n<p>Unfortunately, the &ldquo;Hardware Multi-Threading&rdquo; setting in Instruments <a href=\"https:\/\/apple.stackexchange.com\/questions\/196027\/how-to-disable-hyperthreading-on-mac-os-x-yosemite\">does not persist<\/a> after the Mac reboots <em>or sleeps,<\/em> so you have to keep re-applying it. The good news is that Apple should be able to offer a software update that applies Intel&rsquo;s microcode patch.<\/p>\n\n<p>Update (2017-07-05): <a href=\"http:\/\/gallium.inria.fr\/blog\/intel-skylake-bug\/\">Xavier Leroy<\/a> (via <a href=\"https:\/\/news.ycombinator.com\/item?id=14630183\">Hacker News<\/a>):<\/p>\n<blockquote cite=\"http:\/\/gallium.inria.fr\/blog\/intel-skylake-bug\/\"><p>Late April 2016, shortly after OCaml 4.03.0 was released, a Serious Industrial OCaml User (SIOU) contacted me privately with bad news: one of their applications, written in OCaml and compiled with OCaml 4.03.0, was crashing randomly. Not at every run, but once in a while it would segfault, at different places within the code. Moreover, the crashes were only observed on their most recent computers, those running Intel Skylake processors.<\/p>\n<p>[&#8230;]<\/p>\n<p>SIOU didn&rsquo;t take my suggestions well, arguing (correctly) that they were running other CPU- and memory-intensive tests on their Skylake machines and only the ones written in OCaml would crash. Clearly, they thought their hardware was perfect and the bug was in my software. Great. I still managed to cajole them into running a memory test, which came back clean, but my suggestion about turning HT off was ignored. (Too bad, because this would have saved us much time.)<\/p><\/blockquote>\n\n<p id=\"bug-in-skylake-and-kaby-lake-hyper-threading-2017-12-28\">Update (2017-12-28): I contacted Apple in early August and was told that the bug would be addressed in High Sierra. However, as of macOS 10.13.2 there is no update for the processors. Mine still shows:<\/p>\n<pre>machdep.cpu.brand_string: Intel(R) Core(TM) i7-7700K CPU @ 4.20GHz\nmachdep.cpu.family: 6\nmachdep.cpu.model: 158\nmachdep.cpu.extmodel: 9\nmachdep.cpu.extfamily: 0\nmachdep.cpu.stepping: 9\nmachdep.cpu.signature: 591593\nmachdep.cpu.microcode_version: 88<\/pre>\n<p>I spent a couple weeks in December going back and forth with Apple&rsquo;s support people, trying to get an answer. None of the advisors had even heard of the bug. Eventually, a senior advisor escalated the issue to engineering. The engineer confirmed that Apple is aware of the bug but does not yet have a fix. They will not say when or even if Apple will ship a fix. Apple&rsquo;s policy is not to comment on unreleased products.<\/p>\n\n<p>Looking at <a href=\"https:\/\/www.intel.com\/content\/dam\/www\/public\/us\/en\/documents\/specification-updates\/7th-gen-core-family-spec-update.pdf\">Intel&rsquo;s spec sheet<\/a>, it appears that KBL095\/KBW095 were not fixed as of August, so perhaps Apple is waiting on Intel for the Kaby Lake fix. (Intel has had an update for Skylake CPUs since at least June, but it&rsquo;s not clear to me whether Apple has shipped it.)<\/p>\n\n<p>For now, Apple recommends keeping hyper-threading turned off to prevent data corruption. This can be done by unchecking the &ldquo;Preferences &#x2023; CPU &#x2023; Hardware Multi-Threading&rdquo; setting in Instruments. Unfortunately, this setting is not preserved if you sleep or restart your Mac. Apple confirmed that there is no way to make the setting stick. (Years ago, the <code>nvram<\/code> command could be used.)<\/p>\n\n<p>My recommendation is to set the Mac not to sleep the CPU and to add Instruments as a login item so that you remember to disable hyper-threading when your Mac starts up.<\/p>\n\n<p>It&rsquo;s not clear to me whether the Xeon W processors in the <a href=\"https:\/\/mjtsai.com\/blog\/2017\/12\/12\/the-imac-pro\/\">iMac Pro<\/a> are affected.<\/p>\n\n<p>See also: <a href=\"https:\/\/arstechnica.com\/information-technology\/2017\/06\/skylake-kaby-lake-chips-have-a-crash-bug-with-hyperthreading-enabled\/\">ArsTechnica<\/a>, <a href=\"https:\/\/news.ycombinator.com\/item?id=14630183\">Hacker News<\/a>, <a href=\"https:\/\/forums.macrumors.com\/threads\/skylake-kabylake-bug.2053588\/\">MacRumors<\/a>, <a href=\"https:\/\/caml.inria.fr\/mantis\/view.php?id=7452\">OCaml Mantis<\/a>, <a href=\"https:\/\/www.reddit.com\/r\/programming\/comments\/6jfgfp\/warning_intel_skylakekaby_lake_processors_broken\/\">Reddit<\/a>, <a href=\"https:\/\/security.stackexchange.com\/questions\/170895\/understanding-the-scope-of-intel-hyperthreading-errata-kbl095\">StackExchange<\/a>.<\/p>","protected":false},"excerpt":{"rendered":"<p>Henrique de Moraes Holschuh: This advisory is about a processor\/microcode defect recently identified on Intel Skylake and Intel Kaby Lake processors with hyper-threading enabled. This defect can, when triggered, cause unpredictable system behavior: it could cause spurious errors, such as application and system misbehavior, data corruption, and data loss.It was brought to the attention of [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"apple_news_api_created_at":"","apple_news_api_id":"","apple_news_api_modified_at":"","apple_news_api_revision":"","apple_news_api_share_url":"","apple_news_coverimage":0,"apple_news_coverimage_caption":"","apple_news_is_hidden":false,"apple_news_is_paid":false,"apple_news_is_preview":false,"apple_news_is_sponsored":false,"apple_news_maturity_rating":"","apple_news_metadata":"\"\"","apple_news_pullquote":"","apple_news_pullquote_position":"","apple_news_slug":"","apple_news_sections":"\"\"","apple_news_suppress_video_url":false,"apple_news_use_image_component":false,"footnotes":""},"categories":[2],"tags":[131,255,1297,1586,225,261,30,1381,1529,814,260],"class_list":["post-18244","post","type-post","status-publish","format-standard","hentry","category-technology","tag-bug","tag-compiler","tag-imac","tag-imac-pro","tag-instruments","tag-intel","tag-mac","tag-macos-10-12","tag-macos-10-13","tag-ocaml","tag-processors"],"apple_news_notices":[],"_links":{"self":[{"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/posts\/18244","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/comments?post=18244"}],"version-history":[{"count":3,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/posts\/18244\/revisions"}],"predecessor-version":[{"id":19954,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/posts\/18244\/revisions\/19954"}],"wp:attachment":[{"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/media?parent=18244"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/categories?post=18244"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/tags?post=18244"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}