Intel/Linux 20% slowdown

[Keisha]

Another random thought, devices are getting smaller and smaller and have less surface area to disapate heat. This slows down CPUs. If this slows down the CPU enough then maybe it will run as slow as the rest of the components.

The above benchmarks were run with Fedora's default Powersave governor.

I just used CPUFreqUtility to set the CPU's to run with the Performance governor and by raising the minimum frequency to 3.0 GHz the CPU CryptoHash score rose to 3894 with pti=on. Which is 23% faster than with pti=off on the Powersave governor.

However FPU Raytracing on the Performance governor with minimum frequency set as above slowed down to 47 seconds. Meanwhile, gkrellm showed the cpu temps rising to 140+. The temps never passed 130 on FPU Raytracing with the default Powersave governor.

Perhaps, as you suggest, thermal throttling is happening in FPU Raytracing.

The CPU's on this machine are cooled only by air. Next week I'll install decent liquid cooling CPU heatsinks and then benchmark them again, with and without pti=on, trying both governors.

If, in order to compete against AMD's Ryzen and Epyc CPU's, Intel Xeon CPU's must use a performance governor and liquid cooling...then Intel is facing catastrophe.

I wonder if the CEO of Intel acted on insider foreknowledge of this evidently catastrophic bug when he sold off all his Intel stock, retaining only just enough to qualify to keep his CEO job, last November?--see
https://finance.yahoo.com/news/intel-ap ... 00267.html

[Sky Aisling]

[s243a]

[Keisha]

Well, I'll try to answer this; bear in mind, I am not an expert...

My reading of all the news leads me to believe the short answer depends on whether you have an AMD or Intel CPU.

The most succinct overview I have found is here:
https://mybroadband.co.za/news/hardware ... -know.html

But...

AMD CPU's are not vulnerable to any of these attacks, according to AMD's own statement issued yesterday (https://www.cnbc.com/2018/01/03/amd-reb ... chips.html) (***EDITED: AMD has admitted that at least some AMD CPU's are vulnerable to Spectre #1. A kernel patch is needed to fix the vulnerability. Exactly what this patch is, I don't yet know. AMD says performance is not affected by the patch.***). Core Linux developers Linus Torvalds and Thomas Gleixner evidently believe AMD's assurances (***EDITED: insofar as #3, "Meltdown."), see https://www.phoronix.com/scan.php?page= ... le-x86-PTI.

If you have an Intel CPU then I believe the answer is "yes, the bug allowing the two Spectre and one Meltdown types of attacks does affect Puppy run from a Live CD." I believe so since the vulnerability is at the level of the CPU and an attack can be introduced either at the level of the OS (through a java or flash script introduced through the browser, or via other infected software downloaded and run in the current session) or by addressing the CPU directly over the internet connection via the route described at http://murga-linux.com/puppy/viewtopic.php?t=112465 (***EDITED: which involves the remote management capabilities of Intel chipsets. Apparently these can be at least partially turned off by patching the BIOS***).

If you have the misfortune, like most people, to be using an Intel CPU, then starting Puppy with the pti=on parameter will thwart the Spectre and Meltdown attacks from being mounted via an attack at the operating system level (***EDITED: --provided you have a kernel with the pti/kpti patch***).

The attacks can *maybe* be thwarted from being mounted at the CPU level by putting into practice the countermeasures (specifically: disabling the remote management capabilities --AMT, SPS, or ME-- in the BIOS) described at https://www.intel.com/content/www/us/en ... tware.html
(specifically, by doing what the individual motherboard manufacturers recommend, as linked at the bottom of that article).

On my own machine, at the moment I'm out of luck on this latter defense, because the remote management in my wonderfully fast vintage-2016-BIOS dual-E5v3-Xeon motherboard is only version 3 SPS, and Intel no longer supports SPS versions earlier than 4. Therefore Intel does not at present offer a way to turn off SPS on this motherboard.

According to the article you cite, the Spectre attack can succeed against AMD CPU's. However, I can find only one actual research report indicating so --namely,
https://spectreattack.com/spectre.pdf
and this report does not give specifics. It merely says, on page 3, "We have also verified the attack’s applicability to AMD Ryzen CPUs" but nothing more is stated. (***EDITED: AMD now, Jan. 4, admits its CPU's are vulnerable to Spectre #1, but maintains they are not vulnerable to #2 or #3).

Meanwhile, yesterday AMD issued the statement, cited above, that "To be clear, the security research team identified three variants targeting speculative execution. The threat and the response to the three variants differ by microprocessor company, and AMD is not susceptible to all three variants. Due to differences in AMD's architecture, we believe there is a near zero risk to AMD processors at this time." (***EDITED: what they meant was, apparently, "we are vulnerable only to #1 and there is near zero risk to AMD processors from #2 and #3."***)

AMD's rebuttal is unequivocal (***EDITED: only as far as #2 and #3***)

Therefore, I strongly suspect that the single report of AMD vulnerability to Spectre, repeated endlessly through the echo chamber of the financial and technical press, is intentional misdirection, done to stave off a catastrophic meltdown of Intel's stock price, and dramatic rise in AMD's stock price, long enough to give big Wall Street portfolio managers time to reposition themselves accordingly ahead of the breaking wave of catastrophe, i.e. sell Intel and buy AMD stock before the general investing public becomes aware of how bad the situation really is for Intel. (***EDITED: probably more deception on the part of Intel than AMD. Intel's first public response was through their general counsel.***

Intel stock is down this morning despite the overall market reaching new record highs, and AMD's stock is up, which tends to support my suspicion. The Wall Street mavens on CNBC are saying they still like Intel and are viewing this price dip as a buying opportunity in Intel, they are saying that the vulnerability will not materially affect Intel's financial well-being. (***EDITED next day: market at record highs again, Intel stock has recovered slightly, AMD is down slightly.***)

But if Intel can no longer match AMD's performance without liquid cooling and using more electricity (i.e. using the Performance governor), as my own rudimentary local experiments positively indicate...then companies who operate large-scale server farms, cloud providers and suchlike, are going to turn en masse to AMD, and there is a distinct possibility of class-action lawsuits being brought by such large customers against Intel. (***EDITED: as of Jan. 5 I still believe Intel may be harmed. Don't know how much it will benefit AMD though.***)

I hope that such pressure will persuade Intel to extend its mitigation efforts to include older motherboards such as mine, in the realm of defending against page table isolation attacks such as Spectre and Meltdown mounted at the CPU level. Starting Linux with pti=on in the kernel line of the bootloader, should be sufficient to stop attacks at the OS level. (***EDITED: provided you have a patched kernel***)

Short version: start Linux with pti=on in the kernel line of the bootloader (***EDITED: provided you have a patched kernel***), and turn off remote management in the BIOS as per the procedures given in individual motherboard manufacturers' links at the bottom of https://www.intel.com/content/www/us/en ... tware.html. (***EDITED: and you'll need a microcode update to combat #2. As of Jan. 5 Intel has not yet AFAIK provided such an update.***)

Then Puppy started from a live CD on an Intel-based computer will not be vulnerable, according to the best information presently available.

[fabrice_035]

The attack relies on running the attackers code on your computer. If you don't run the attackers code then the exploit won't work. Because puppy minimizes the amount of software that one is istalled by default the chance of a fresh puppy having the exploit in installed software is very low.

Some think very soon a script (run in your web browser) will be able to perform an exploit.

I hope a update kernel for Tahrpup and other Puppy distro. Waiting ...
Regard.

[rufwoof]

Intel stock is down this morning despite the overall market reaching new record highs, and AMD's stock is up, which tends to support my suspicion. The Wall Street mavens on CNBC are saying they still like Intel and are viewing this price dip as a buying opportunity in Intel, they are saying that the vulnerability will not materially affect Intel's financial well-being.

Intel's CEO has maximised liquidation, minimised his stock holdings. https://www.fool.com/investing/2017/12/ ... stock.aspx

choosen to keep only the shares that he's required to by Intel's rules

[8Geee]

I've had a read of many things today, nor'easter at hand, and have figured out the following...

This is all about "out-of-order execution (OOOE)" a trait encoded into most CPU's since the 90's.

Intel's version has a leak, and also executes the delayed instruction without regard to security or privacy. (In other words it does not know that the delayed instruction information is a password.) The leak allows the end-user to be tricked into allowing the OOOE stack to be read.

AMD claims that their processors are not subject to Intel's problems. They remain silent about leakage.

Very few Intel products are exempt from the Intel OOOE flaw,
in particular;

1.) HP/Intel Itanium Servers
2.) Intel Atom "Bonnell technology" (Original)
Some of these exempt Atoms are 64-bit.

Atom CPU's using Silvermont (2013+) are NOT EXEMPT. These include Bay-Trail and Cherry-Trail.

So yes, there is a huge problem. Apple, and Microsoft are patching their engines (kernels). And as of Jan.2 kernel dot org has updated all supported kernels. Google is sending mixed signals, and FUD follows mixed signals.

Wikipedia Out-Of-Order Execution

Wikipedia Intel Atom CPU's

Regards
8Geee

[ac2011]

Short version: start Linux with pti=on in the kernel line of the bootloader.

I don't see how that will help unless you're running a version of Linux with the patched kernel, which presumably almost none of us are. I can find no mention of the "pti" switch prior to these events, so I'd expect older kernels to simply ignore it. I'd love to be proved wrong.

[Keisha]

Short version: start Linux with pti=on in the kernel line of the bootloader.

I don't see how that will help unless you're running a version of Linux with the patched kernel, which presumably almost none of us are. I can find no mention of the "pti" switch prior to these events, so I'd expect older kernels to simply ignore it. I'd love to be proved wrong.

You're absolutely right, I'm working not with Puppy but with a custom version of Fedora to which the pti/kpti patch had been already applied (using kernel 4.15.rc6). I did not know that pti was not yet already an existing capability. Meanwhile, kernel developers are working on backporting pti/kpti to legacy kernels.

AMD (at https://www.amd.com/en/corporate/speculative-execution) now officially admits that version #1 of Spectre does affect at least some AMD CPU's, though AFAIK Ryzen and Epyc CPU's have not been shown to be affected, only A-series and FX series. Spectre #2 and Meltdown #3 do not affect AMD CPU's. The big performance hits (and a perhaps more important issue, the electricity consumption rise and consequent added heat emission needed to maintain equal performance, problems which have not yet been addressed in the press AFAIK) only occur when combatting #3 by using pti=on, and so only Intel CPU's are hurt.

Therefore I still stand by my speculation that this whole cockup will prove harmful to Intel as far as marketing CPU's to the providers of equipment to cloud providers. Whether this will help AMD in the cloud market, I guess, depends on whether there are other lesser-known CPU makers (Via? IBM? Cavium? Marvell? Sunway? nVidia?) who can step in to compete.

#2 needs a microcode update to fix, and AFAIK at this writing Intel has not yet provided one.

So, my revised opinion as of right now is: patched kernels (***EDITED: and other software components, e.g. llvm***) will be necessary in order to combat #1, #2, and #3 and in addition a CPU microcode update and maybe a BIOS patch are needed in order to combat #2. (***EDITED:The only effect on AMD is, a kernel patch is needed to shield AMD CPU's from #1. This patch will incur no performance hit on AMD CPU's.***)

I have an inquiry in to my own motherboard manufacturer (ASRock) concerning how the BIOS needs to be patched.

Anyone who has new information is certainly welcome to post.

[ac2011]

That sounds right. My current 'solution' is to get hold of a 2009 Atom N280 thin client for sensitive online use. No speculative execution so should be invulnerable to Meltdown and Spectre... I think. It will be slow, though.

I went for the earlier variant as later (but still pre-2013) Atoms presumably come with Intel's ME, and I don't want that. The Venn diagram contains a very thin sliver of acceptable solutions!

I have an AMD machine (A-series) that I can use, but for best security on that I'd have to shift away from my favourite Puppy distro to something with rolling updates to ensure all applications are Spectre-proofed, which would be a shame.

I'm looking at Raspberry Pi longer term, which has neither vulnerability. I'm no longer an avid gamer so that might work for me for everyday use.

Keen to hear what other people have planned, if anything.

[Sailor Enceladus]

It looks like code has been commited today (2018-01-05) to work around the Meltdown problem in kernels 4.4.110, 4.9.75, and 4.14.12 so far (from what I can tell)?
https://git.kernel.org/pub/scm/linux/ke ... 2=v4.4.109
https://www.kernel.org/

I'll try compiling a 4.4.110 when I get home, and compare it with 4.4.109 on my 32-bit Pentium M laptop.

[belham2]

We should also be talking about what users can do to protect themselves. I think the following might help and/or provide some guide:

1) make sure you have 2-4 different bootable systems . System 1 is for general web use. System 2, a backup to System 1. System 3, a system with a browser that is used only for email and sensitive site access (that means going directly to those sites, and nowhere else). System 4, a backup to system 3.

2) Over the past few days, i am hearing this from not only Intel people, but also AMD. Home users, turn off ALL samba. In essence, run isolated systems. A firewall isn't going to help you if you get hit with one of these.

3) In Systems 1 and 2 above, make sure your browser has the ability to disable javascript. It is fairly easy (about:config) in Firefox/Palemoon/Seamonkey, and also easy in Chrome (but more problematic, as evidently, even when you set Chrome to not use javascript, it still does for some things---Google, wakeup for crissakes). And it will be a pain in the butt for most sites. For example, with javascript disabled, on murga when you type out a message, you will be able to do things "Bold" or "italic' or "Underline" and many other things on the murga site. I am facing these things right now writing this message, since all of my machines have javascript disabled (I use heavily "about:config" modded & fortified Firefox as my browser). So, oh well, it's the price to pay until things get clearer.

4) In Systems 3 and 4 above, these systems are with browsers that have javascript enabled (most sensitive sites, i.e. bank, etc won't function without javascript enabled in the browser). Also, these systems are never to be booted up and used for anything else. Install basic minimums to this systems (Ubuntu minimal, or Debian minimal, and Puppy installs are great choices here. My preference: I build minimal Debian installs myself. Also, I trust Barry inside and out---and for a USB stick for a System 3/4 boot, these are the way to go.

5) Use a good ad-blocker in all browsers in Systems (1-4). Today, Ublock Origin is the most recommended by people in the security industry, and with good reason. Ads on the web are malware missiles, and deliver malware payloads.

6) For Systems 1 and 2, you could do a lot worse than installing NoScript. No explanation needed here.

7) Last but not least, in the Systems 3 and 4 that you build, or install and/or whatever, don't install any extra software packages---no new added programs for anything (except a browser, and, if building, only the basics for a DE functioning system). This shouldn't need explained, given what Systems 3 & 4 will be used for.


I believe most of us do the above things to some degree. But we, here on murga, are an absolute minority, and that is in just the general linux world (we don't even register out in the Microsoft/Apple world). Most linux users I know can't save their own butts with just basic functions. I give them a pup, and they just fall flat on their face when something simple happens and they can't fix it because they have no idea how. They are used to just install big Linux OSes, and expect it all to work hunky-dory with no problems whatsoever. These people and types of users scare me, even more than Windows/Apple users. Anyhow, I am not sure how we (the murga gang) can make a difference in what is currently going on.

Please know: I typed all of this out quickly. I do know some of the above would be uber painful for some, especially the Samba part. But, look, overall it can't hurt for a short period of time until things been better understood and mitigated to a greater degree. You get owned on one of Samba-linked computers, you're more likely to get owned on all of them. Equally, nowadays there is no sane mind in the world that would use a general browsing computer for anything done sensitive-wise (banking, health, investments) on the web. If you do engage in this recklessness, you deserve whatever happens.


Lastly, it is my opinion (and I know many are thinking this, Bigpup even joked about it)...but it is my opinion Intel and AMD are absolutely creaming their pants with joy over this. They needed a reason, an end-of-the-world panic to drive the next great chip cycle. This may it, and they may be actively encouraging it. Heck, when the U.S. Department of Homeland Security came out just yesterday that ALL computers on the planet cannot be trusted, and new chips with new designs & software need released, well......I mean, Intel and AMD must have nearly passed out from euphoria while trying to maintain their worried and all-is-still-ok current public face.

[Marv]

A few very quick tests using the 4.14.11 32b PAE kernel (from peebee in the newest LxPupSc) on a second generation i5 laptop. nokaiser is supported as a kernel parameter according to the 4.14.110 changelog and does seem to toggle the kaiser patches. On this hardware (Fujitsu S761 laptop), with the kaiser patches enabled, which is the kernel default, there is about a 1.5% hit on glxgears FPS, a 0.8% hit on the CPU cryptohash, and a 4% hit on the FPU raytracing benchmarks in hardinfo. All based on 5 run averages, performance governor, same machine same OS etc. Temperature monitored and not a factor. The first two are just about down in noise, the FPU delta seems real. I haven't gotten to the core 2 duos or the Bay Trails yet, but mostly use the i5s now and certainly can live with that.

Edit: kaiser is now kpti and the most recent info I've read states that nopti is to be the kernel toggle. Don't know if nokaiser will continue to be supported as an alias or not. This rolling stone really gathers no moss!

[Keisha]

A few very quick tests using the 4.14.11 32b PAE kernel (from peebee in the newest LxPupSc)..."

According to https://access.redhat.com/articles/3311301 it'll have a file
/sys/kernel/debug/x86/pti_enabled
if the pti patch has been applied to the kernel. The slowdowns and thermal throttling on Intel CPU's are seen only on kernels with the pti patch applied.

In ordinary desktop use, my experiments show little or no perceptible slowdown, and only a little thermal problem; for the individual user security is the main issue. Slowdown, excessive electric consumption, and heat are issues equally important to security only in scenarios where there is constant re-filling of the CPU's internal caches (L1, L2, and TLB), and this is not a relevant scenario in most individual desktop or laptop computer use.

Unfortunately however, large-scale data centers with heavy transaction volumes, where we're talking a large roomful of rackmount servers, are exactly this type of scenario. From what I understand and can see, Intel is going to get complaints from these types of customers, e.g. Amazon Web Services, Microsoft Azure, Google, Oracle, all the world's stock exchanges, bitcoin mining centers, etc. etc. etc.

Two law firms (Kaplan Fox & Kilsheimer LLP, and the Law Offices of Howard G. Smith) have already announced investigations into Intel over possible violations of securities law committed by Intel's CEO and other officers in connection with large stock sales done without fanfare in foreknowledge of these security threats; as these cases go forward I foresee that the stonewalling which Intel is evidently engaged in right now, with regard to the added electric and cooling costs of Intel CPU's under the conditions of use in data centers when the AMT/SPS/ME, Spectre, and Meltdown defensive patches have all been applied...will come under consideration before a court.

Worst case scenario, these preliminary lawsuits will then mushroom into gigantic class-actions with the likes of Amazon, Microsoft, Google et al as plaintiffs.

I myself would certainly not be a long-term investor in Intel stock at the moment.

...later (but still pre-2013) Atoms presumably come with Intel's ME, and I don't want that...

Yes, my machine's motherboard is a vintage-2014 socket 2011-3, dual sockets/Xeon's, with a BIOS dated March 31 2016 and yet the SPS (ME equivalent for servers) on the chipset is only version 3, the current version is 4, so I've asked the motherboard manufacturer (ASRock) for guidance. Intel's own security advisory on the AMT/SPS/ME vulnerabilities (https://security-center.intel.com/advis ... geid=en-fr) only covers SPS version 4. No word on what vulnerabilities exist in version 3. *Grrr*.....

I have an AMD machine (A-series) that I can use, but for best security on that I'd have to shift away from my favourite Puppy distro to something with rolling updates to ensure all applications are Spectre-proofed, which would be a shame.

Or wait for a Puppy to appear with an appropriately patched kernel. The final mainline version of 4.16 should work, when it appears. Maybe, just maybe, 4.15. Or a legacy version kernel with the patchset backported.

I myself use an in-house-customized Fedora, which since Fedora is maintained by Red Hat gets needed security updates with minimum delay. Sure hope I don't have to beg my boss for a new Epyc machine though.

[ac2011]

I have an AMD machine (A-series) that I can use, but for best security on that I'd have to shift away from my favourite Puppy distro to something with rolling updates to ensure all applications are Spectre-proofed, which would be a shame.

Or wait for a Puppy to appear with an appropriately patched kernel. The final mainline version of 4.16 should work, when it appears. Maybe, just maybe, 4.15. Or a legacy version kernel with the patchset backported.

That would solve Meltdown but not Spectre, which I believe requires recompilation of applications with 'retpoline' or similar mitigations. Still, perhaps this can be built into the build process for future Puppies?

Admittedly Spectre appears to have the lower risk profile at the moment, but that may change.

[Keisha]

I have an AMD machine (A-series) that I can use, but for best security on that I'd have to shift away from my favourite Puppy distro to something with rolling updates to ensure all applications are Spectre-proofed, which would be a shame.

Or wait for a Puppy to appear with an appropriately patched kernel. The final mainline version of 4.16 should work, when it appears. Maybe, just maybe, 4.15. Or a legacy version kernel with the patchset backported.

That would solve Meltdown but not Spectre, which I believe requires recompilation of applications with 'retpoline' or similar mitigations..

But you said you use an AMD A CPU, which is not susceptible to the Spectre #2 attack which is mitigated by retpoline (see https://www.amd.com/en/corporate/speculative-execution). It only applies to Intel CPU's. Spectre #2 requires either retpoline patches to the kernel and software, or a microcode update (so says https://security.googleblog.com/2018/01 ... cpu_4.html). This implies that if you have the Intel microcode update in force, retpoline is unnecessary.

[8Geee]

I see a reference to the Intel Management Engine with Atom's being suseptible to THOSE things.

The Two Atom Series are;

C3xxx (Silvermont Architecture) AKA x3 series SoFIA
E39xx (Silvermont Archetecture) AKA x5 series Apollo Lake

So the Bonnell Architecture is EXEMPT.

Regards
8Geee

[ac2011]

[ac2011]

I see a reference to the Intel Management Engine with Atom's being suseptible to THOSE things.

The Two Atom Series are;

C3xxx (Silvermont Architecture) AKA x3 series SoFIA
E39xx (Silvermont Archetecture) AKA x5 series Apollo Lake

So the Bonnell Architecture is EXEMPT.

Regards
8Geee

Seems to be, hence the N280. It'll be fun going back a decade in computing power...

[8Geee]

There are some N2800 boxes around. These are 2-core Atoms using the Bonnell archetecture, and the last best processor that is EXEMPT from IME and Meltdown/Spectre.

Cheers
8Geee