Recently, folks in my company suspect if Spectre mitigations are involved once they see a performance bug. So I make a simple summary of different X86 Spectre Variants, performance loss and workarounds. So they can have a general idea if Spectre might be involved or rule out the issue by suggesting the proper workaround to customer when necessory. Some of the kernel command line are for UEK series only, but most of them are compatible with
kernel.
Spectre V1 (Bounds Check Bypass)
can't toggle at runtime
No performance impact
Spectre V2 (Branch Target Injection)
There are two mitigations, Retpoline and IBRS.
Retpoline is the prefered and by default enabled mitigation for most X86 systems, 0%-5% performance loss.
IBRS is intel suggested mitigation for Skylake+ processor and enabled by default for Skylake+, performance
loss is less than 20% in extreme case.
Dynamic switch:
/sys/kernel/debug/x86{retpoline_enabled,ibrs_enabled}
Kernel bootup parameter:
spectre_v2=
on - unconditionally enable
off - unconditionally disable
auto - kernel detects whether your CPU model is
vulnerable (default)
retpoline - replace indirect branches
retpoline,generic - google's original retpoline
retpoline,amd - AMD-specific minimal thunk
ibrs - Use IBRS (if microcode is available).
nospectre_v2
Euivalent to spectre_v2=off
spectre_v2_heuristics=
off - disable all heuristics (see below)
skylake=off - do not use IBRS if present on Skylake
instead of retpoline (this is equivalant
to spectre_v2=retpoline,generic).
ssbd=off - do not use IBRS if mitigating
Speculative Store Bypass.
Spectre V3, Meltdown (Rogue Data Cache Load)
AMD not vulnerable
Mitigated by page table isolation
Intel performance penalty less than 5%
Kernel bootup parameter:
pti=
on - unconditionally enable
off - unconditionally disable
auto - kernel detects whether your CPU model is
vulnerable to issues that PTI mitigates (default)
nopti
Equivalent to pti=off
Spectre V4 (Speculative Store Bypass)
Mitigaged by microcode update and SSBD provided by Intel
No performance impact with default setting, 0-8% if SSBD enabled
Kernel bootup parameter:
spec_store_bypass_disable=
on - Unconditionally disable Speculative Store Bypass
off - Unconditionally enable Speculative Store Bypass
auto - Kernel detects whether the CPU model contains an
implementation of Speculative Store Bypass and
picks the most appropriate mitigation. If the
CPU is not vulnerable, "off" is selected. If the
CPU is vulnerable the default mitigation is
architecture and Kconfig dependent. See below.
prctl - Control Speculative Store Bypass per thread
via prctl. Speculative Store Bypass is enabled
for a process by default. The state of the control
is inherited on fork.
seccomp - Same as "prctl" above, but all seccomp threads
will disable SSB unless they explicitly opt out. (default)
nospec_store_bypass_disable
[HW] Disable all mitigations for the Speculative Store Bypass vulnerability
Spectre-NG (Lazy FP State Restore)
Mitigated by eager fpu restore
No performance impact for most cases when there is little FPU usage
Kernel bootup parameter:
For uek2/3/4:
eagerfpu=
on enable eager fpu restore (default)
off disable eager fpu restore
auto enable eager fpu restore when xsaveopt is
available, disable otherwise.
For uek5:
Can't toggle at runtime, lazy FPU code had been removed since 2016
L1TF (L1 Terminal Fault speculative side channel)
Mitigated by different mitigation combined, such as PTE inversion,
flush L1D, Guest confinement, Interrupt isolation, disable SMT
In default setting, performance impact 0-5%, if SMT is disabled, 0-50%
Kernel bootup parameter:
l1tf=
full
Provides all available mitigations for the
L1TF vulnerability. Disables SMT and
enables all mitigations in the
hypervisors, i.e. unconditional L1D flush.
SMT control and L1D flush control via the
sysfs interface is still possible after
boot. Hypervisors will issue a warning
when the first VM is started in a
potentially insecure configuration,
i.e. SMT enabled or L1D flush disabled.
full,force
Same as 'full', but disables SMT and L1D
flush runtime control. Implies the
'nosmt=force' command line option.
(i.e. sysfs control of SMT is disabled.)
flush
Leaves SMT enabled and enables the default
hypervisor mitigation, i.e. conditional
L1D flush.
SMT control and L1D flush control via the
sysfs interface is still possible after
boot. Hypervisors will issue a warning
when the first VM is started in a
potentially insecure configuration,
i.e. SMT enabled or L1D flush disabled. (default)
flush,nosmt
Disables SMT and enables the default
hypervisor mitigation.
SMT control and L1D flush control via the
sysfs interface is still possible after
boot. Hypervisors will issue a warning
when the first VM is started in a
potentially insecure configuration,
i.e. SMT enabled or L1D flush disabled.
flush,nowarn
Same as 'flush', but hypervisors will not
warn when a VM is started in a potentially
insecure configuration.
off
Disables hypervisor mitigations and doesn't
emit any warnings.
So for default kernel setting, suggesting "nopti spectre_v2=off" is enough for customer to see if performance issue disapper.
Above are only for UEK series, for upstream kernel there are some difference,
e.x. no spectre_v2_heuristics=, no eagerfpu=, no IBRS support but support Enhanced IBRS.
Below are the benchmark related links where performance data come from.
https://access.redhat.com/articles/3307751
https://access.redhat.com/security/vulnerabilities/L1TF-perf
https://www.phoronix.com/scan.php?page=article&item=linux-419-mitigations&num=5
https://www.phoronix.com/scan.php?page=article&item=linux-415-x86pti&num=2
https://www.servethehome.com/intel-publishes-l1tf-and-foreshadow-performance-impacts/
https://www.servethehome.com/intel-offers-enterprise-meltdown-spectre-benchmarks-gift-amd/
