Add source manifest proto to recipe_engine.

This adds the initial form of the "source manifest" proto which will
allow recipe_engine to communicate structured data about
stuff-on-disk-that-we-got-from-remote-servers to LUCI services like Milo,
and will also give us a better option than 'got_revision' properties for
exporting this data from recipes (i.e. we'll be able to export all the
data and then query/read it instead of hard-coding a few hacks to export
specific repos).

This only covers git, cipd and isolated for now, but could be expanded
to other data sources, if needed. We don't currently anticipate a need
for that, but the proto is hopefully obviously extensible for this.

Clients consuming this proto (like Milo) may have specialized support
when comparing two manifest (e.g. two entries sharing the same git repo
may render as a git log), but it should also be somewhat easy to show a
useful generic diff when comparing two Manifest protos, even when the
deployment type changes (i.e. local path "foo" used to be a git checkout,
but is now multiple CIPD packages).

R=dnj@chromium.org, hinoka@chromium.org, machenbach@chromium.org
BUG=468053

Review-Url: https://codereview.chromium.org/2998523002
2 files changed
tree: e4296096623690d12ddf0aedff4a26a629849092
  1. bootstrap/
  2. doc/
  3. go/
  4. infra/
  5. misc/
  6. recipe_engine/
  7. recipe_modules/
  8. recipes/
  9. unittests/
  10. .gitattributes
  11. .gitignore
  12. AUTHORS
  13. codereview.settings
  14. CONTRIBUTORS
  15. LICENSE
  16. OWNERS
  17. PRESUBMIT.py
  18. README.md
  19. README.recipes.md
  20. recipes.py
README.md

Recipes

Recipes are a domain-specific language (embedded in python) for specifying sequences of subprocess calls in a cross-platform and testable way.

Files

  • README.md

    This file!

  • bootstrap/

    Bootstrapping and vendoring related code.

  • doc/

    Documentation for the recipe engine (including this file!). Also includes doc/recipes.py, which is a file you can include in your repository to start using recipes. See the design doc for more detailed design information about the recipe engine.

  • infra/

    Chrome infra config files.

  • recipes.py

    The main entry point to the recipe engine. It has many subcommands and flags; run recipes.py -h to see them.

  • recipes/

    Recipes in the recipe engine. These are either example recipes, or recipes which are used to test the engine (see run_test.py to see these run)

  • recipe_modules/

    Built in recipe modules. These are very useful when writing recipes; take a look in there, and look at each of their examples subfolders to get an idea how to use them in a recipe.

  • recipe_engine/

    The core functionality of the recipe engine. Noteworthy files include:

    • package.proto -- The protobuf file which defines the format of a recipes.cfg file.
    • third_party/ -- third_party code which is vendored into the recipe engine.
    • recipe_api.py -- The api exposed to a recipe module.
    • unittests -- Unittests for the engine.

    There are also several files which correspond to a subcommand of recipes.py; depgraph, run, and autoroll are some examples.

  • unittests/

    Somewhat poorly named, these are higher level integration tests.

Contributing

  • Sign the Google CLA.
  • Make sure your user.email and user.name are configured in git config.

Run the following to setup the code review tool and create your first review:

git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git $HOME/src/depot_tools
export PATH="$PATH:$HOME/src/depot_tools"
git checkout -b work origin/master

# hack hack

git commit -a -m "This is awesome\nR=joe@example.com"
# This will ask for your Google Account credentials.
git cl upload -s
# Wait for LGTM over email.
# Check the commit queue box in codereview website.
# Wait for the change to be tested and landed automatically.

Use git cl help and git cl help <cmd> for more details.