Overview
All configuration file paths specified in the pipeline need to be under DD_OP_DATA_DIR/config.
Modifying files under that location while OPW is running might have adverse effects.
Bootstrap the Observability Pipelines Worker within your infrastructure before you set up a pipeline. These environment variables are separate from the pipeline environment variables. The location of the related directories and files:
- Default data directory:
var/lib/observability-pipelines-worker - Bootstrap file:
/etc/observability-pipelines-worker/bootstrap.yaml - Environment variables file:
/etc/default/observability-pipelines-worker
Bootstrap Options
To set bootstrap options, do one of the following:
- Use environmental variables.
- Create a
bootstrap.yaml and start the Worker instance with --bootstrap-config /path/to/bootstrap.yaml.
api_key- env var:
DD_API_KEY - Create a Datadog API key for this environment variable.
pipeline_id- env var:
DD_OP_PIPELINE_ID - Create an Observability Pipelines pipeline ID for this environment variable.
site- env var:
DD_SITE - Your Datadog site (optional, default:
datadoghq.com). - See Getting Started with Sites for more information.
data_dir- env var:
DD_OP_DATA_DIR - The data directory (optional, default:
/var/lib/observability-pipelines-worker). This is the file system directory that the Observability Pipelines Worker uses for local state. tags: []- env var:
DD_OP_TAGS - The tags reported with internal metrics and can be used to filter Observability Pipelines instances for Remote Configuration deployments.
threads- env var:
DD_OP_THREADS - The number of threads to use for processing (optional, default: the number of available cores).
proxy- This option is available for Observability Pipelines Worker 2.1 and later.
- env variables:
DD_PROXY_HTTP, DD_PROXY_HTTPS, DD_PROXY_NO_PROXY - Set proxy servers for the Observability Pipelines Worker. The proxy configuration for the Worker works in the same way as it does for the Datadog Agent.
- The settings are applied to the entire Worker process. The HTTP proxy and HTTPS values are resolved in this order:
1. DD_PROXY_HTTP(S)
2. HTTP(S)_PROXY
3. proxy - An example proxy configuration:
- proxy:
enabled: true
https: https://foo.bar:3128 - Note: The
DD_PROXY_HTTP(S) and HTTP(S)_PROXY environment variables need to be already exported in your environment for the Worker to resolve them. They cannot be prepended to the Worker installation script.
Further reading
Additional helpful documentation, links, and articles:
