Guard auto-vectorization flag for GCC only

[RawNuke]

Closes #3863

Short description of changes

Wrap the -ftree-vectorize flag in a *-g++* qmake scope so it only reaches GCC compilers.

Context: Fixes an issue?

Fixes #3863

What does this change do?

• MSVC no longer gets the D9002 warning for the unknown -ftree-vectorize flag
• Clang builds are unchanged (it already auto-vectorizes at -O2)
• GCC builds continue to receive the -ftree-vectorize flag where it's needed

Checklist

• I've verified that this Pull Request follows the general code principles
• I tested my code and it does what I want
• My code follows the style guide
• I waited some time after this Pull Request was opened and all GitHub checks completed without errors.
• I've filled all the content above

[ann0see]

Thanks!

[ann0see]

I think this depends on the GCC version. So the comment is not accurate

[softins]

From Google AI:

GCC added an auto-vectorizer starting in GCC 4.0. Historically, automatic loop vectorization (-ftree-vectorize) was only implicitly enabled at -O3 (or manually via -O2 -ftree-vectorize). However, starting with GCC 12, it is enabled by default at -O2 as well.

GCC Auto-Vectorization Milestones

• GCC 4.0 and later: Introduced the tree-ssa auto-vectorizer, available only with -O3 or explicitly via -ftree-vectorize at -O2.
• GCC 12 and later: Enabled automatically at -O2 using a conservative "very cheap" cost model, while -O3 continues to use a more aggressive "dynamic" cost model.

Optimization Level Summary

• -O0 / -O1: Auto-vectorization is off (unless forced manually with -ftree-vectorize at -O1).
• -O2 (GCC 12+): Enabled by default using -fvect-cost-model=very-cheap.
• -O2 (GCC 11 and older): Disabled by default; requires adding -ftree-vectorize manually.
• -O3 (All versions since 4.0): Enabled by default using -fvect-cost-model=dynamic.

[softins]

For reference, I looked up what versions of GCC were used in the main distros:

Distro	GCC default	GCC max available
Ubuntu 20 (CI build for release/3_12)	GCC 9	GCC 10
Ubuntu 22 (CI build for main and 4.x)	GCC 11	GCC 12
Ubuntu 24	GCC 13	GCC 14
Debian 11	GCC 10
Debian 12	GCC 12
Debian 13	GCC 14
RHEL 9	GCC 11
RHEL 10	GCC 14

[RawNuke]

Good catch, thanks. You are right that it depends on the GCC version, so the old comment was wrong to say GCC never vectorizes at -O2. I have corrected it: -ftree-vectorize is on at -O3, and also at -O2 from GCC 12 on. Setting it explicitly keeps vectorization on for GCC builds that would not enable it by default, and scoping to -g++ avoids the MSVC warning and the redundancy on Clang. Happy to adjust the wording further if needed.

[softins]

It looks like there are a lot more considerations to effectively use auto-vectorization with GCC. Here's my conversation with Google AI Mode about it:

---

If -ftree-vectorize is added to -O1, or to -O2 on GCC 11 and older, which cost model is used?

When you explicitly add -ftree-vectorize to -O1 or to -O2 on older compiler setups like GCC 11 and earlier, GCC defaults to using the dynamic vectorizer cost model. [1]

This behavior is dictated by how GCC handles its optimization flags: [2]

The dynamic Cost Model Default

Historically, the dynamic cost model was hardcoded as the global default for the vectorizer whenever it was active. If a lower optimization level like -O1 or GCC 11’s -O2 lacked the vectorizer by default, manually passing -ftree-vectorize awakened the engine using this standard baseline profile. Under the dynamic model, the compiler generates both scalar and vectorized loops alongside runtime checks for data alignment and pointer aliasing, only jumping into the vectorized block if the runtime conditions prove profitable. [1]

How the Cost Model Defaults Evolved

Optimization Setup	Default Cost Model	Behavior Description
-O1 -ftree-vectorize (All Versions)	dynamic	Uses full runtime checks to determine loop profitability.
-O2 -ftree-vectorize (GCC 11 & Older)	dynamic	Inherits the traditional global vectorizer default behavior.
-O3 (All Versions)	dynamic	The standard level for aggressive, runtime-checked vectorization.
-O2 (GCC 12 & Newer)	very-cheap	Changed to prevent code bloat by allowing vectorization only if no runtime checks are needed.

Overriding the Default

If you are compiling code at -O1 or on GCC 11 at -O2, but you want to enforce a different restriction profile, you can bypass this default by pairing the flag with the -fvect-cost-model= switch: [1]

• -fvect-cost-model=cheap: Allows vectorization only if the loop iteration count is high or runtime checks are minimal.
• -fvect-cost-model=very-cheap: Replicates the GCC 12 behavior, vectorizing only if the loop can be optimized cleanly without inserting expensive runtime checks.
• -fvect-cost-model=unlimited: Forces the compiler to vectorize loops ignoring any scalar vs. vector cost formulas entirely (can lead to severe code size bloat). [1, 3]

[1] https://wiki.gentoo.org
[2] https://cvw.cac.cornell.edu
[3] https://stackoverflow.com

Your table above didn't specify the cost model for -O2 -ftree-vectorize for GCC 12 and newer.

That is an important distinction: if you explicitly pass -O2 -ftree-vectorize on GCC 12 and newer, the compiler uses the very-cheap cost model. [1, 2]
Adding -ftree-vectorize explicitly does not elevate the cost model profile. Because -ftree-vectorize is already implicitly active at -O2 starting with GCC 12, adding the flag explicitly becomes redundant and simply inherits the underlying cost model mapped to the -O2 optimization level. [1, 2, 3]
The updated matrix below fills in this specific behavior:

Complete Optimization Flag and Cost Model Matrix

GCC Version	Combination	Effective Cost Model	Resulting Behavior
GCC 11 & Older	-O2	Vectorizer Disabled	Loops remain completely scalar by default.
	-O2 -ftree-vectorize	dynamic	The traditional vectorizer defaults apply, using runtime checks to decide profitability.
	-O3	dynamic	Normal aggressive auto-vectorization.
GCC 12 & Newer	-O2	very-cheap	Vectorizes strictly when code size bloat is low and runtime checks are minimal.
	-O2 -ftree-vectorize	very-cheap	Redundant flag. The compiler treats this exactly like -O2 on its own.
	-O3	dynamic	Normal aggressive auto-vectorization.

Enforcing the Old Behavior on GCC 12+

If you migrate to GCC 12 or newer but want the compiler to optimize exactly like your old -O2 -ftree-vectorize configuration did, you must explicitly change the cost model along with your flags: [4]

gcc -O2 -ftree-vectorize -fvect-cost-model=dynamic main.c

Without appending -fvect-cost-model=dynamic, GCC 12+ will bypass loops that require extensive runtime aliasing or alignment checks to limit binary size. [1, 2]

Are you seeing unexpected vectorization behavior (or a lack thereof) in a project that was recently upgraded to GCC 12 or newer, or are you designing a universal Makefile/CMake template?

[1] https://wiki.gentoo.org
[2] https://developer.arm.com
[3] https://www.phoronix.com
[4] https://github.com

Wanting to add tree vectorization to the existing Jamulus build environment, which might be built on different versions of GCC, depending on the user's build distro. Just trying to understand and catch all permutations.

To safely inject vectorization into the build system of an open-source audio tool like Jamulus across an unpredictable landscape of user distros and compiler versions, you need a highly standardized strategy. [1]
Because Jamulus relies heavily on real-time floating-point audio processing (via the Opus codec and its internal mixing engines), you must account for cost models, target hardware limitations, and IEEE math compliance. [1]

1. The Core GCC Version Mapping

To ensure consistent cost-model behavior and prevent binary bloat on distributions running newer toolchains, use this explicit flag logic:

If the distro builds with an old compiler (GCC < 12) or targets explicit optimization:

-O2 -ftree-vectorize -> Results in: 'dynamic' cost model (Safe, but introduces runtime pointer checks).

If the distro builds with GCC 12 or newer:

-O2 -ftree-vectorize -> Results in: 'very-cheap' cost model (Redundant, won't vectorize if it requires runtime checks).

The Solution: Do not rely on implicit cost models. If you want aggressive, uniform auto-vectorization across all GCC versions at -O2, you must explicitly standardize the cost model profile in the compiler arguments:

-O2 -ftree-vectorize -fvect-cost-model=dynamic

2. The Audio Vectorization Blocker: -ffast-math

Audio mixers typically aggregate, scale, and clear loops of float array data. GCC will refuse to auto-vectorize loops that sum or accumulate floating-point numbers by default. [2, 3]
Vectorization requires changing the order of algebraic operations, which violates strict IEEE 754 precision math guidelines. For Jamulus to actually leverage SIMD lanes on audio buffers, you must pair your vectorization flags with:

-ffast-math

Note: If -ffast-math is too broad or dangerous for the entire codebase, you can use -funsafe-math-optimizations or isolate it to the audio-processing source files.

3. The Portability Trap (-march)

If a package maintainer compiles Jamulus into a generic .deb or .rpm binary for a distribution repository, adding standard vectorization flags without specifying an architecture will limit the compiler to baseline instructions (e.g., standard x86_64 SSE2). [4]

• For Personal Builds: Advise users compiling from source to append -march=native. This unlocks the user's actual hardware vector registers (AVX, AVX2, AVX-512, or ARM NEON).
• For Package Maintainers: Enforce a safe, modern baseline if your users are guaranteed to have modern machines, such as targeting -msse4.1 or -msse4.2. [4, 5, 6]

4. Implementation Blueprint (CMake / Qmake)

Jamulus uses the Qt framework. Depending on whether the build configuration uses Qmake (.pro file) or CMake, use these version-agnostic injection snippets: [1]

For Qmake (Jamulus.pro):

linux-g++* {
    # Force auto-vectorization and stabilize the cost model across GCC versions
    QMAKE_CXXFLAGS_RELEASE += -ftree-vectorize -fvect-cost-model=dynamic -ffast-math

    # Optional: Encourage native instructions if doing a local user build
    # QMAKE_CXXFLAGS_RELEASE += -march=native
}

[1] https://github.com
[2] https://stackoverflow.com
[3] https://stackoverflow.com
[4] https://stackoverflow.com
[5] https://stackoverflow.com
[6] https://developer.arm.com

Jamulus just uses qmake, not cmake.

Since Jamulus strictly utilizes qmake, you will handle everything within its main Jamulus.pro project file. [1, 2, 3]
Unlike CMake, qmake does not provide an elegant native syntax for checking compiler version numbers directly out of the box. However, because you are trying to balance multiple GCC variants across various Linux distributions, you can solve this by executing a quick shell command from directly inside Jamulus.pro to dynamically adapt the build arguments. [4]
The production-ready qmake blocks below safely inject version-stable vectorization while accounting for cross-distro portability.

---

1. Version-Agnostic Setup (Recommended)

If you want to keep the configuration simple and clean, remember that the explicit flag -fvect-cost-model=dynamic works perfectly fine on all versions of GCC (both older versions and modern releases like GCC 12+). By declaring it explicitly, you override the GCC 12 default (very-cheap) and force older versions to behave explicitly, meaning you do not actually need a version check block:

# Apply only to Linux GNU C++ compiler scopes
linux-g++* {
    # 1. Enable vectorization safely across ALL GCC versions
    # 2. Harmonize the cost model to 'dynamic' for consistent loops
    # 3. Allow floating-point reassociation (critical for real-time audio mixers)
    QMAKE_CXXFLAGS_RELEASE += -ftree-vectorize -fvect-cost-model=dynamic -ffast-math
}

---

2. The Advanced Distro-Aware Setup

If you need strict, surgical control over the flags—such as wanting to keep the default very-cheap optimization model on modern systems while aggressively forcing dynamic checks on legacy machines running older distributions—you can pull the GCC major version using system() execution:

linux-g++* {
    # Extract the major version number from the system's active GCC build
    GCC_VERSION = $$system("$$QMAKE_CXX -dumpversion | cut -d. -f1")

    # Target floating-point optimizations needed for audio signal processing
    QMAKE_CXXFLAGS_RELEASE += -ftree-vectorize -ffast-math

    # Condition: Evaluate compiler major version number
    lessThan(GCC_VERSION, 12) {
        # GCC 11 & Older: Uses dynamic by default, but we enforce it for clarity
        QMAKE_CXXFLAGS_RELEASE += -fvect-cost-model=dynamic
    } else {
        # GCC 12 & Newer: Explicitly decide whether to let it sit at 'very-cheap'
        # or force 'dynamic' matching legacy distribution behavior.
        # Uncomment below if you want GCC 12+ to build identical binaries to GCC 11:
        # QMAKE_CXXFLAGS_RELEASE += -fvect-cost-model=dynamic
    }
}

---

3. Handling User vs. Package Maintainer Builds

Injecting vectorization into an open-source project like Jamulus exposes a common issue: if a package maintainer tries to build it for an official repository (like Fedora or Debian), forcing standard vectorization might trigger an unexpected baseline error if target hardware instructions are not managed. [5]
You can add a custom qmake configuration toggle so package maintainers can switch modes cleanly:

linux-g++* {
    # Default behavior targets standard distro compatibility (e.g., SSE2 on x86_64)
    # Allow users building from source to run: qmake CONFIG+=native_simd Jamulus.pro
    CONFIG(native_simd) {
        QMAKE_CXXFLAGS_RELEASE += -march=native
    } else {
        # Generic safe baseline for modern distribution repositories
        # You can specify minimum architecture baselines here if needed
    }
}

[1] https://github.com
[2] https://github.com
[3] https://stackoverflow.com
[4] https://stackoverflow.com
[5] https://jamulus.io

[RawNuke]

Thanks for digging into this. You are right that it is more nuanced than a one line comment can capture, especially the cost model at -O1 and at -O2 on older GCC. I am happy to trim the comment down to just the reason for the change, which is scoping the flag to GCC so it does not warn on MSVC or duplicate what Clang already does, and to drop the claims about specific optimization levels. If you would rather not carry the explicit flag at all given these considerations, I am fine closing this. Whatever you prefer.

[mcfnord]

MY LLM WROTE:

Measured, the matrix above is off in two cells: on GCC ≤ 11, -O2 -ftree-vectorize selects cheap, not dynamic (gcc 11.5.0); and on GCC 12+ it is not redundant — it raises the -O2 cost model from very-cheap to cheap (gcc 13.3.0 and 14.2.0):

gcc -O2 -ftree-vectorize -Q --help=optimizers | grep vect-cost-model

So the explicit flag alone lands every GCC version in the same state (on, cheap) — no -fvect-cost-model= needed. That cost-model step is what vectorizes the mix loops: 0 → 16 in MixEncodeTransmitData on gcc 13.3 x86-64, 0 → 15 on gcc 14.2 aarch64, with no -ffast-math. Full matrix incl. clang/MSVC: #3739 (comment)