incompatible_disable_legacy_crosstool_fields: Disable legacy crosstool fields

[hlopko]

Flag: --incompatible_disable_legacy_crosstool_fields
Available since: 0.22 (January 2019 release)
Will be flipped in: 0.25 (April 2019 release)
Tracking issue: #5883
Rollout doc: https://docs.google.com/document/d/1uv4c1zag6KvdI31qdx8C6jiTognXPQrxgsUpVefm9fM/edit#

Motivation

CROSSTOOL grew much bigger than it was expected in 2011 when it was first introduced. There are many fields that are no longer used (e.g. mao_plugin_header_directory), or there are fields that are superseded by features (e.g. compiler_flag). There are fields that specify toolchain capabilities (e.g. supports_embedded_runtimes) but cannot participate in the feature configuration (and there are user requests for these). All in all, CROSSTOOL needs a revamp, and this change achieves that.

Caveats

This is a big incompatible change, and #5380 will be comparably big one. We advice users to use the migration tool provided for --incompatible_disable_legacy_crosstool_fields as the final step of their CROSSTOOL generating pipeline, and wait with hand-tweaking for the migration tool that will be provided for #5380. --incompatible_disable_legacy_crosstool_fields will not be flipped before #5380 is fixed.

Migration

Please use the legacy_fields_migrator tool provided at rules_cc repository. To verify the correctness of your build, use the command line diffing tool provided at tools/aquery_differ (bazel aquery, that the differ uses, is being improved daily as of January 2019, e.g. 1, so if you don't use bazel > 0.23, we advice to build your own bazel from HEAD for testing the migrated crosstool). As mentioned above, we advice not to hand-tune the CROSSTOOL without also migrating for #5380 (automated migrator to starlark will be provided).

Detailed change description

These features were previously enabled by Bazel, but they will no longer be, so they need to be enabled in the crosstool:

• dependency_file
• random_seed
• module_maps
• module_map_home_cwd
• header_module_compile
• include_paths,
• pic
• preprocessor_defines

This list contains all the legacy fields and how they will be migrated:

• static_runtimes_filegroup will not be used once incompatible_disable_runtimes_filegroups: Remove cc_toolchain.static_runtime_libs and cc_toolchain.dynamic_runtime_libs #6942 is flipped. Use cc_toolchain.static_runtime_lib instead.
• dynamic_runtimes_filegroup will not be used once incompatible_disable_runtimes_filegroups: Remove cc_toolchain.static_runtime_libs and cc_toolchain.dynamic_runtime_libs #6942 is flipped. Use cc_toolchain.dynamic_runtime_lib instead.
• compiler_flag is replaced by features. Flags from compiler_flag should appear before any other legacy field, and should be passed to all compilation actions.
• optional_compiler_flag is not used.
• cxx_flag is replaced by features. Flags from cxx_flag should appear after compiler_flags, and should be passed to all compilation actions except c-compile.
• optional_cxx_flag is not used.
• unfiltered_cxx_flag is replaced by features. Flags from feature named unfiltered_compile_flags are not subject to nocopts filtering.
• optional_unfiltered_cxx_flag is not used.
• linker_flag is replaced by features. Flags from linker_flag should appear before any other legacy field, and should be passed to all linking actions (except the misnamed archiving action c++-link-static-library)
• optional_linker_flag is not used.
• dynamic_library_linker_flag is replaced by features. Flags from dynamic_library_linker_flag should appear after linker_flag fields from linking_mode_flags, and should be only present when linking dynamic libraries.
• optional_dynamic_library_linker_flag is not used.
• test_only_linker_flag is replaced by features. test_only_linker_flag should appear after dynamic_library_linker_flag, and only for tests.
• objcopy_embed_flag is replaced by features.
• ld_embed_flag is replaced by features.
• ar_flag is not used.
• ar_thin_archives_flag is not used.
• gcc_plugin_compiler_flag is not used.
• compilation_mode_flags are replaced by features. Features named opt, dbg, and fastbuild are automatically enabled for given compilation mode. Flags from inner compiler_flag should appear after top-level cxx_flags. Flags from inner cxx_flag should appear after inner compiler_flag. Flags from linker_flag should appear after top-level linker_flag fields, before flags from linking_mode_flags.
• lipo_mode_flags is not used.
• linking_mode_flags are replaced by features. Mode COVERAGE is not used. Mode FULLY_STATIC has been replaced by fully_static_link feature and therefore unused. Mode MOSTLY_STATIC is replaced by static_linking_mode feature and mode DYNAMIC is replaced by dynamic_linking_mode feature. Both are enabled by rules explicitly.
• gcc_plugin_header_directory is not used.
• mao_plugin_header_directory is not used.
• default_python_top is not used.
• default_python_version is not used.
• python_preload_swigdeps is not used.
• supports_gold_linker is only used for --symbol_counts and will be removed.
• supports_thin_archives is not used or user.
• supports_start_end_lib is replaced by feature named supports_start_end_lib that should be enabled by the toolchain or user.
• supports_interface_shared_objects is replaced by feature named supports_interface_shared_libraries that should be enabled by the toolchain or user.
• supports_embedded_runtimes is replaced by feature named static_link_cpp_runtimes that should be enabled by the toolchain or user.
• supports_incremental_linker is not used.
• supports_normalizing_ar is not used.
• supports_fission is replaced by feature named per_object_debug_info that should be enabled by the toolchain or user.
• supports_dsym is not used.
• needsPic is replaced by feature named supports_pic that should be enabled by the toolchain or user. Before Bazel always requrested pic feature, and verified that pic feature is enabled when needsPic was true or when --force_pic option was passed. With this change, Bazel will only request supports_pic when --force_pic is passed, and it verify that supports_pic is enabled only when --force_pic is passed. When --force_pic is not passed, Bazel will not request supports_pic explicitly, and it will assume that when supports_pic is enabled by the toolchain or user, it means PIC is needed for shared libraries. pic and force_pic build variables are still passed in their corresponding configurations.

A really good place to see exactly what is migrated and how is to see the tests for legacy_fields_migrator.

Changelog

• January 3rd: Implemented cmd_line_differ
• January 7th: Bazel 0.22, which contains this flag, was cut
• January 7th: Fixed broken interface library input detection (a4529db), this is unlikely to affect Bazel users.
• January 7th: Discovered the issue where $EXEC_ORIGIN is not used when static_link_cpp_runtimes is disabled, this is unlikely to affect Bazel users. If it does affect you, the fix was to add runtime_library_search_directories to the crosstool and not relying on Bazel to patch it in.
• January 11th: Open-sourced legacy_fields_migrator
• January 15th: Bazel fix to move default_compile_flags feature to the top of the crosstool, to stay backwards compatible. This fix does affect every user that doesn't specify no_legacy_features feature. Migrator fixed accordingly.
• January 17th: Migrator fix for nodeps_dynamic_library.
• February 5th: Migrator fix to replace repeated expand_if_(all|none)_available with nesting (preparation for Crosstool in Starlark migration).
• February 8th: Replaced cmd_line_differ with aquery_differ, which is a more general aquery output diffing tool. (3575797)
• February 13th: Updated the migrator to auto-enable some features that were previously enabled by Bazel

[hlopko]

CC @scentini

[hlopko]

This broke bazel's ndk crosstools, reopening.