Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tinygo fails to build at Arch Linux clean environment #4332

Open
anatol opened this issue Jul 9, 2024 · 10 comments · Fixed by #4338
Open

tinygo fails to build at Arch Linux clean environment #4332

anatol opened this issue Jul 9, 2024 · 10 comments · Fixed by #4338
Labels
bug Something isn't working linux next-release Will be part of next release

Comments

@anatol
Copy link

anatol commented Jul 9, 2024

I am building this project in a Arch Linux clean environment (kinda chroot with sanitized environment) and the build fails:

[158/230] Building CXX object third_party/llvm-project/CMakeFiles/llvm_dwarf.dir/DataExtractor.cpp.o
[159/230] Building CXX object third_party/llvm-project/CMakeFiles/llvm_dwarf.dir/DJB.cpp.o
[160/230] Building CXX object src/wasm/CMakeFiles/wasm.dir/wasm-type.cpp.o
[161/230] Building CXX object src/passes/CMakeFiles/passes.dir/SimplifyLocals.cpp.o
FAILED: src/passes/CMakeFiles/passes.dir/SimplifyLocals.cpp.o 
/usr/bin/c++  -I/build/tinygo/src/tinygo/lib/binaryen/src -I/build/tinygo/src/tinygo/lib/binaryen/third_party/llvm-project/include -I/build/tinygo/src/tinygo/lib/binaryen -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions         -Wp,-D_FORTIFY_SOURCE=3 -Wformat -Werror=format-security         -fstack-clash-protection -fcf-protection         -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -Wp,-D_GLIBCXX_ASSERTIONS -g -ffile-prefix-map=/build/tinygo/src=/usr/src/debug/tinygo -DBUILD_LLVM_DWARF -Wall -Werror -Wextra -Wno-unused-parameter -fno-omit-frame-pointer -fno-rtti -Wno-implicit-int-float-conversion -Wno-unknown-warning-option -Wswitch -Wimplicit-fallthrough -Wnon-virtual-dtor -fPIC -fdiagnostics-color=always -O3 -DNDEBUG -UNDEBUG -std=c++17 -MD -MT src/passes/CMakeFiles/passes.dir/SimplifyLocals.cpp.o -MF src/passes/CMakeFiles/passes.dir/SimplifyLocals.cpp.o.d -o src/passes/CMakeFiles/passes.dir/SimplifyLocals.cpp.o -c /build/tinygo/src/tinygo/lib/binaryen/src/passes/SimplifyLocals.cpp
In file included from /build/tinygo/src/tinygo/lib/binaryen/src/parsing.h:29,
                 from /build/tinygo/src/tinygo/lib/binaryen/src/wasm-builder.h:21,
                 from /build/tinygo/src/tinygo/lib/binaryen/src/ir/literal-utils.h:20,
                 from /build/tinygo/src/tinygo/lib/binaryen/src/ir/bits.h:21,
                 from /build/tinygo/src/tinygo/lib/binaryen/src/ir/properties.h:20,
                 from /build/tinygo/src/tinygo/lib/binaryen/src/ir/iteration.h:20,
                 from /build/tinygo/src/tinygo/lib/binaryen/src/ir/branch-utils.h:20,
                 from /build/tinygo/src/tinygo/lib/binaryen/src/passes/SimplifyLocals.cpp:50:
In constructor ‘wasm::Walker<SubType, VisitorType>::Task::Task(wasm::Walker<SubType, VisitorType>::TaskFunc, wasm::Expression**) [with SubType = wasm::LocalGetCounter; VisitorType = wasm::Visitor<wasm::LocalGetCounter, void>]’,
    inlined from ‘void wasm::SmallVector<T, N>::emplace_back(ArgTypes&& ...) [with ArgTypes = {void (*&)(wasm::LocalGetCounter*, wasm::Expression**), wasm::Expression**&}; T = wasm::Walker<wasm::LocalGetCounter, wasm::Visitor<wasm::LocalGetCounter, void> >::Task; long unsigned int N = 10]’ at /build/tinygo/src/tinygo/lib/binaryen/src/support/small_vector.h:73:7,
    inlined from ‘void wasm::Walker<SubType, VisitorType>::pushTask(TaskFunc, wasm::Expression**) [with SubType = wasm::LocalGetCounter; VisitorType = wasm::Visitor<wasm::LocalGetCounter, void>]’ at /build/tinygo/src/tinygo/lib/binaryen/src/wasm-traversal.h:314:23,
    inlined from ‘void wasm::Walker<SubType, VisitorType>::walk(wasm::Expression*&) [with SubType = wasm::LocalGetCounter; VisitorType = wasm::Visitor<wasm::LocalGetCounter, void>]’ at /build/tinygo/src/tinygo/lib/binaryen/src/wasm-traversal.h:329:13,
    inlined from ‘void wasm::LocalGetCounter::analyze(wasm::Function*, wasm::Expression*)’ at /build/tinygo/src/tinygo/lib/binaryen/src/ir/local-utils.h:37:9:
/build/tinygo/src/tinygo/lib/binaryen/src/wasm-traversal.h:309:59: error: storing the address of local variable ‘ast’ in ‘*(std::array<wasm::Walker<wasm::LocalGetCounter, wasm::Visitor<wasm::LocalGetCounter, void> >::Task, 10>*)((char*)&_2->stack + 8).std::array<wasm::Walker<wasm::LocalGetCounter, wasm::Visitor<wasm::LocalGetCounter, void> >::Task, 10>::_M_elems[<unknown>].wasm::Walker<wasm::LocalGetCounter, wasm::Visitor<wasm::LocalGetCounter, void> >::Task::currp’ [-Werror=dangling-pointer=]
  309 |     Task(TaskFunc func, Expression** currp) : func(func), currp(currp) {}
      |                                                           ^~~~~~~~~~~~
In file included from /build/tinygo/src/tinygo/lib/binaryen/src/passes/SimplifyLocals.cpp:54:
/build/tinygo/src/tinygo/lib/binaryen/src/ir/local-utils.h: In member function ‘void wasm::LocalGetCounter::analyze(wasm::Function*, wasm::Expression*)’:
/build/tinygo/src/tinygo/lib/binaryen/src/ir/local-utils.h:34:44: note: ‘ast’ declared here
   34 |   void analyze(Function* func, Expression* ast) {
      |                                ~~~~~~~~~~~~^~~
/build/tinygo/src/tinygo/lib/binaryen/src/ir/local-utils.h:34:47: note: ‘this’ declared here
   34 |   void analyze(Function* func, Expression* ast) {
      |                                               ^
At global scope:
cc1plus: note: unrecognized command-line option ‘-Wno-unknown-warning-option’ may have been intended to silence earlier diagnostics
cc1plus: note: unrecognized command-line option ‘-Wno-implicit-int-float-conversion’ may have been intended to silence earlier diagnostics
cc1plus: all warnings being treated as errors
[162/230] Building CXX object third_party/llvm-project/CMakeFiles/llvm_dwarf.dir/Dwarf.cpp.o
[163/230] Building CXX object third_party/llvm-project/CMakeFiles/llvm_dwarf.dir/DWARFAbbreviationDeclaration.cpp.o
[164/230] Building CXX object src/analysis/CMakeFiles/analysis.dir/cfg.cpp.o

The binaryen submodule fails to compile SimplifyLocals.cpp because compile complains about unknown options.

The error comes from the fact that the default compiler was picked up, and in the chroot environment that compiler is /usr/bin/c++ that is currently GCC 14.1.1.

binaryen on the other hand does not work with GCC. It looks like it expects to be compiled with clang. In this case GNUmakefile should reflect it. Setting proper CMAKE_CXX_COMPILER cmake variable should help here I think.

@aykevl
Copy link
Member

aykevl commented Jul 10, 2024

The binaryen submodule fails to compile SimplifyLocals.cpp because compile complains about unknown options.

Not exactly. The compiler only warns about them:

cc1plus: note: unrecognized command-line option ‘-Wno-unknown-warning-option’ may have been intended to silence earlier diagnostics
cc1plus: note: unrecognized command-line option ‘-Wno-implicit-int-float-conversion’ may have been intended to silence earlier diagnostics
cc1plus: all warnings being treated as errors

I suspect GCC 14 is just a bit too new for the Binaryen release that we are currently using and they haven't fixed all new warnings yet. So the bug is probably in Binaryen, and a workaround is to use an older compiler.

@anatol
Copy link
Author

anatol commented Jul 11, 2024

GCC 14 is the default compiler version at Arch Linux. Does tinygo even expected to be compiled with GCC?

What would be a proper solution in case if we want to use the system compiler (GCC 14).

@aykevl
Copy link
Member

aykevl commented Jul 11, 2024

Binaryen most certainly works with GCC, because that's how it is usually compiled in TinyGo. In fact you can see here how I just did it (on Fedora 40):

$ make binaryen
mkdir -p build
cd lib/binaryen && cmake -G Ninja . -DBUILD_STATIC_LIB=ON -DBUILD_TESTS=OFF  && ninja bin/wasm-opt
-- The C compiler identification is GNU 14.1.1
-- The CXX compiler identification is GNU 14.1.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- No build type selected, default to Release
-- Building with -DBUILD_LLVM_DWARF
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE  
-- Building with -Wall
-- Building with -Werror
-- Building with -Wextra
-- Building with -Wno-unused-parameter
-- Building with -fno-omit-frame-pointer
-- Building with -fno-rtti
-- Building with -Wno-implicit-int-float-conversion
-- Building with -Wno-unknown-warning-option
-- Building with -Wswitch
-- Building with -Wimplicit-fallthrough
-- Building with -Wnon-virtual-dtor
-- Building with -fPIC
-- Building with -UNDEBUG
-- Building with -fdiagnostics-color=always
-- Building libbinaryen as statically linked library.
-- Configuring done (0.6s)
-- Generating done (0.0s)
-- Build files have been written to: /home/ayke/src/tinygo/tinygo/lib/binaryen
[230/230] Linking CXX executable bin/wasm-opt
cp lib/binaryen/bin/wasm-opt build/wasm-opt

So it looks like this is a problem on Arch Linux specifically.
Unfortunately my system can't run Arch Linux (it's arm64) so I can't reproduce this issue.

@anatol
Copy link
Author

anatol commented Jul 12, 2024

Hi @aykevl thank you for your reply.

I built binaryen separately with your instructions and it compiles fine at Arch Linux x86_64.

So the problem is most likely comes from the way the build environment set up. Among other things it sets compilation flags. I compiled binaryen with those flags and now it fails:

$ export CXXFLAGS="-O2 -pipe -fno-plt -fexceptions         -Wp,-D_FORTIFY_SOURCE=3 -Wformat -Werror=format-security         -fstack-clash-protection -fcf-protection         -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -Wp,-D_GLIBCXX_ASSERTIONS -g"
$ cmake -G Ninja . -DBUILD_TESTS=OFF
-- The C compiler identification is GNU 14.1.1
-- The CXX compiler identification is GNU 14.1.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- No build type selected, default to Release
-- Building with -DBUILD_LLVM_DWARF
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Building with -Wall
-- Building with -Werror
-- Building with -Wextra
-- Building with -Wno-unused-parameter
-- Building with -fno-omit-frame-pointer
-- Building with -fno-rtti
-- Building with -Wno-implicit-int-float-conversion
-- Building with -Wno-unknown-warning-option
-- Building with -Wswitch
-- Building with -Wimplicit-fallthrough
-- Building with -Wnon-virtual-dtor
-- Building with -fPIC
-- Building with -UNDEBUG
-- Building with -fdiagnostics-color=always
-- Building libbinaryen as shared library.
-- Configuring done (0.5s)
-- Generating done (0.0s)
-- Build files have been written to: lib/binaryen
$ ninja bin/wasm-opt -j9
...
[161/230] Building CXX object src/passes/CMakeFiles/passes.dir/SimplifyLocals.cpp.o
FAILED: src/passes/CMakeFiles/passes.dir/SimplifyLocals.cpp.o 
/usr/bin/c++  -I/home/anatol/sources/tinygo/tinygo/lib/binaryen/src -I/home/anatol/sources/tinygo/tinygo/lib/binaryen/third_party/llvm-project/include -I/home/anatol/sources/tinygo/tinygo/lib/binaryen -O2 -pipe -fno-plt -fexceptions         -Wp,-D_FORTIFY_SOURCE=3 -Wformat -Werror=format-security         -fstack-clash-protection -fcf-protection         -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -Wp,-D_GLIBCXX_ASSERTIONS -g -DBUILD_LLVM_DWARF -Wall -Werror -Wextra -Wno-unused-parameter -fno-omit-frame-pointer -fno-rtti -Wno-implicit-int-float-conversion -Wno-unknown-warning-option -Wswitch -Wimplicit-fallthrough -Wnon-virtual-dtor -fPIC -fdiagnostics-color=always -O3 -DNDEBUG -UNDEBUG -std=c++17 -MD -MT src/passes/CMakeFiles/passes.dir/SimplifyLocals.cpp.o -MF src/passes/CMakeFiles/passes.dir/SimplifyLocals.cpp.o.d -o src/passes/CMakeFiles/passes.dir/SimplifyLocals.cpp.o -c /home/anatol/sources/tinygo/tinygo/lib/binaryen/src/passes/SimplifyLocals.cpp
In file included from /home/anatol/sources/tinygo/tinygo/lib/binaryen/src/parsing.h:29,
                 from /home/anatol/sources/tinygo/tinygo/lib/binaryen/src/wasm-builder.h:21,
                 from /home/anatol/sources/tinygo/tinygo/lib/binaryen/src/ir/literal-utils.h:20,
                 from /home/anatol/sources/tinygo/tinygo/lib/binaryen/src/ir/bits.h:21,
                 from /home/anatol/sources/tinygo/tinygo/lib/binaryen/src/ir/properties.h:20,
                 from /home/anatol/sources/tinygo/tinygo/lib/binaryen/src/ir/iteration.h:20,
                 from /home/anatol/sources/tinygo/tinygo/lib/binaryen/src/ir/branch-utils.h:20,
                 from /home/anatol/sources/tinygo/tinygo/lib/binaryen/src/passes/SimplifyLocals.cpp:50:
In constructor ‘wasm::Walker<SubType, VisitorType>::Task::Task(wasm::Walker<SubType, VisitorType>::TaskFunc, wasm::Expression**) [with SubType = wasm::LocalGetCounter; VisitorType = wasm::Visitor<wasm::LocalGetCounter, void>]’,
    inlined from ‘void wasm::SmallVector<T, N>::emplace_back(ArgTypes&& ...) [with ArgTypes = {void (*&)(wasm::LocalGetCounter*, wasm::Expression**), wasm::Expression**&}; T = wasm::Walker<wasm::LocalGetCounter, wasm::Visitor<wasm::LocalGetCounter, void> >::Task; long unsigned int N = 10]’ at /home/anatol/sources/tinygo/tinygo/lib/binaryen/src/support/small_vector.h:73:7,
    inlined from ‘void wasm::Walker<SubType, VisitorType>::pushTask(TaskFunc, wasm::Expression**) [with SubType = wasm::LocalGetCounter; VisitorType = wasm::Visitor<wasm::LocalGetCounter, void>]’ at /home/anatol/sources/tinygo/tinygo/lib/binaryen/src/wasm-traversal.h:314:23,
    inlined from ‘void wasm::Walker<SubType, VisitorType>::walk(wasm::Expression*&) [with SubType = wasm::LocalGetCounter; VisitorType = wasm::Visitor<wasm::LocalGetCounter, void>]’ at /home/anatol/sources/tinygo/tinygo/lib/binaryen/src/wasm-traversal.h:329:13,
    inlined from ‘void wasm::LocalGetCounter::analyze(wasm::Function*, wasm::Expression*)’ at /home/anatol/sources/tinygo/tinygo/lib/binaryen/src/ir/local-utils.h:37:9:
/home/anatol/sources/tinygo/tinygo/lib/binaryen/src/wasm-traversal.h:309:59: error: storing the address of local variable ‘ast’ in ‘*(std::array<wasm::Walker<wasm::LocalGetCounter, wasm::Visitor<wasm::LocalGetCounter, void> >::Task, 10>*)((char*)&_2->stack + 8).std::array<wasm::Walker<wasm::LocalGetCounter, wasm::Visitor<wasm::LocalGetCounter, void> >::Task, 10>::_M_elems[<unknown>].wasm::Walker<wasm::LocalGetCounter, wasm::Visitor<wasm::LocalGetCounter, void> >::Task::currp’ [-Werror=dangling-pointer=]
  309 |     Task(TaskFunc func, Expression** currp) : func(func), currp(currp) {}
      |                                                           ^~~~~~~~~~~~
In file included from /home/anatol/sources/tinygo/tinygo/lib/binaryen/src/passes/SimplifyLocals.cpp:54:
/home/anatol/sources/tinygo/tinygo/lib/binaryen/src/ir/local-utils.h: In member function ‘void wasm::LocalGetCounter::analyze(wasm::Function*, wasm::Expression*)’:
/home/anatol/sources/tinygo/tinygo/lib/binaryen/src/ir/local-utils.h:34:44: note: ‘ast’ declared here
   34 |   void analyze(Function* func, Expression* ast) {
      |                                ~~~~~~~~~~~~^~~
/home/anatol/sources/tinygo/tinygo/lib/binaryen/src/ir/local-utils.h:34:47: note: ‘this’ declared here
   34 |   void analyze(Function* func, Expression* ast) {
      |                                               ^
At global scope:
cc1plus: note: unrecognized command-line option ‘-Wno-unknown-warning-option’ may have been intended to silence earlier diagnostics
cc1plus: note: unrecognized command-line option ‘-Wno-implicit-int-float-conversion’ may have been intended to silence earlier diagnostics
cc1plus: all warnings being treated as errors
[169/230] Building CXX object src/wasm/CMakeFiles/wasm.dir/wat-parser.cpp.o
ninja: build stopped: subcommand failed.

Actually it seems that the real error is

/home/anatol/sources/tinygo/tinygo/lib/binaryen/src/wasm-traversal.h:309:59: error: storing the address of local variable ‘ast’ in ‘*(std::array<wasm::Walker<wasm::LocalGetCounter, wasm::Visitor<wasm::LocalGetCounter, void> >::Task, 10>*)((char*)&_2->stack + 8).std::array<wasm::Walker<wasm::LocalGetCounter, wasm::Visitor<wasm::LocalGetCounter, void> >::Task, 10>::_M_elems[<unknown>].wasm::Walker<wasm::LocalGetCounter, wasm::Visitor<wasm::LocalGetCounter, void> >::Task::currp’ [-Werror=dangling-pointer=]
  309 |     Task(TaskFunc func, Expression** currp) : func(func), currp(currp) {}
      |                                                           ^~~~~~~~~~~~
In file included from /home/anatol/sources/tinygo/tinygo/lib/binaryen/src/passes/SimplifyLocals.cpp:54:
/home/anatol/sources/tinygo/tinygo/lib/binaryen/src/ir/local-utils.h: In member function ‘void wasm::LocalGetCounter::analyze(wasm::Function*, wasm::Expression*)’:
/home/anatol/sources/tinygo/tinygo/lib/binaryen/src/ir/local-utils.h:34:44: note: ‘ast’ declared here

that is probably cause by one of the security fortification flags.

Does the compiler error look legit?

@aykevl
Copy link
Member

aykevl commented Jul 12, 2024

So the problem is most likely comes from the way the build environment set up. Among other things it sets compilation flags. I compiled binaryen with those flags and now it fails:

So the special build environment you have sets those flags and now it fails?
Then the answer is this: don't set those flags. They are what's breaking binaryen. If needed I guess you can use unset CXXFLAGS.

I would suggest you report a binaryen bug instead, because this is a problem in the special way that you are building binaryen and it seems to be flagging a bug in binaryen.

@anatol
Copy link
Author

anatol commented Jul 13, 2024

There is a rabbit hole for this issue. It seems that GCC has some false positives for the binaryen code . Binaryen enables -Werror so the dangling pointer warning becomes an error. This breaks project compilation.

I disabled -Werror as these warnings are useless for us anyway. This let's me compile the project. Here is the patch itself https://gitlab.archlinux.org/archlinux/packaging/packages/tinygo/-/blob/2d17121664515c8e35ec872d3289333e9d76578b/disable_werror_binaryen.patch

It makes sense to add -DENABLE_WERROR=OFF to main branch as well (unless tinygo plans to debug/fix all binaryen warnings in the future).

@anatol
Copy link
Author

anatol commented Jul 13, 2024

It also makes sense to setup a github action to compile the binaryen project against different OS. This will expose the project to different compilers/tools and let's catch this type of compatibility problems. Here is an example how it is done for android-tools project https://github.com/nmeum/android-tools/blob/master/.github/workflows/build.yml

@aykevl
Copy link
Member

aykevl commented Jul 13, 2024

That patch to disable failing on errors is a good idea, can you make a PR out of it?

It also makes sense to setup a github action to compile the binaryen project against different OS.

That would not actually have helped in this case because it only fails in a very specific case with custom flags set (not just on any Arch Linux install). I'm also not convinced it's necessary to be spending the CI time on building binaryen across OSes for very specific issues - we already build binaryen on all OSes that we support.

(Sidenote: I find it somewhat amusing that a "clean" chroot sets some custom build flags).

anatol added a commit to anatol/tinygo that referenced this issue Jul 13, 2024
Compilers like GCC keep adding new checks that produce new warnings.
Sometimes it can be false positives.

Do not treat such warnings in binaryen library as errors. tinygo won't
be able to provide zero warnings in its dependencies.

Closes tinygo-org#4332
anatol added a commit to anatol/tinygo that referenced this issue Jul 13, 2024
Compilers like GCC keep adding new checks that produce new warnings.
Sometimes it can be false positives.

Do not treat such warnings in binaryen library as errors. tinygo won't
be able to provide zero warnings in its dependencies.

Closes tinygo-org#4332
@anatol
Copy link
Author

anatol commented Jul 13, 2024

That patch to disable failing on errors is a good idea, can you make a PR out of it?

#4338

"clean" chroot sets some custom build flags

The purpose of the chroot is to provide a hermetic build environment with predictable dependencies/settings/flags. "clean" here really means the chroot does not contain user-specific package installed and does not have any users envvars. All dependencies/flags are declared and are the same no matter who performs the build. This idea is at the core of reproducible builds.

aykevl pushed a commit that referenced this issue Jul 14, 2024
Compilers like GCC keep adding new checks that produce new warnings.
Sometimes it can be false positives.

Do not treat such warnings in binaryen library as errors. tinygo won't
be able to provide zero warnings in its dependencies.

Closes #4332
@aykevl
Copy link
Member

aykevl commented Jul 14, 2024

Yeah I didn't realize at first this was a package build. It makes sense for distributions to customize the build environment to the distribution preferences (which includes a certain set of build flags).

@deadprogram deadprogram added bug Something isn't working next-release Will be part of next release linux labels Jul 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working linux next-release Will be part of next release
3 participants