Getting Started with APM Tracing

Overview

Datadog Application Performance Monitoring (APM) provides deep visibility into your applications, enabling you to identify performance bottlenecks, troubleshoot issues, and optimize your services.

This guide demonstrates how to get started with APM and send your first trace to Datadog:

  1. Set up Datadog APM to send traces to Datadog.
  2. Run your application to generate data.
  3. Explore the collected data in Datadog.

Prerequisites

To complete this guide, you need the following:

  1. Create a Datadog account if you haven’t already.
  2. Find or create a Datadog API key.
  3. Start up a Linux host or VM.

Create an application

To create an application to observe in Datadog:

  1. On your Linux host or VM, create a new Python application named hello.py. For example, nano hello.py.

  2. Add the following code to hello.py:

    hello.py

      from flask import Flask
      import random
    
      app = Flask(__name__)
      
      quotes = [
          "Strive not to be a success, but rather to be of value. - Albert Einstein",
          "Believe you can and you're halfway there. - Theodore Roosevelt",
          "The future belongs to those who believe in the beauty of their dreams. - Eleanor Roosevelt"
      ]
      
      @app.route('/')
      def index():
          quote = random.choice(quotes)+"\n"
          return quote
      
      if __name__ == '__main__':
          app.run(host='0.0.0.0', port=5050)
      

Set up Datadog APM

To set up Datadog APM without needing to modify your application’s code or the deployment process, use Single Step APM Instrumentation:

Note: Single Step APM Instrumentation is in beta. Alternatively, you can set up APM using Datadog tracing libraries.
  1. Run the installation command:

     DD_API_KEY=<YOUR_DD_API_KEY> DD_SITE="<YOUR_DD_SITE>" DD_APM_INSTRUMENTATION_ENABLED=host DD_ENV=<AGENT_ENV> bash -c "$(curl -L https://install.datadoghq.com/scripts/install_script_agent7.sh)"
    

    Replace <YOUR_DD_API_KEY> with your Datadog API key, <YOUR_DD_SITE> with your Datadog site, and <AGENT_ENV> with the environment your Agent is installed on (for example, development).

  2. Start a new shell session.

  3. Restart the services on your host or VM.

  4. Verify the Agent is running:

    sudo datadog-agent status
    

This approach automatically installs the Datadog Agent, enables Datadog APM, and instruments your application at runtime.

Run the application

When you set up Datadog APM with Single Step Instrumentation, Datadog automatically instruments your application at runtime.

To run hello.py:

  1. Create a Python virtual environment in the current directory:

    python3 -m venv ./venv
    
  2. Activate the venv virtual environment:

    source ./venv/bin/activate
    
  3. Install pip and flask:

    sudo apt-get install python3-pip
    pip install flask
    
  4. Set the service name and run hello.py:

    export DD_SERVICE=hello
    python3 hello.py
    

Test the application

Test the application to send traces to Datadog:

  1. In a new command prompt, run the following:

    curl http://0.0.0.0:5050/
    
  2. Confirm that a random quote is returned.

    Believe you can and you're halfway there. - Theodore Roosevelt
    

Each time you run the curl command, a new trace is sent to Datadog.

Explore traces in Datadog

  1. In Datadog, go to APM > Services. You should see a Python service named hello:

    Service Catalog shows the new Python service.
  2. Select the service to view its performance metrics, such as latency, throughput, and error rates.

  3. Go to APM > Traces. You should see a trace for the hello service:

    Trace explorer shows the trace for the hello service.
  4. Select a trace to see its details, including the flame graph, which helps identify performance bottlenecks.

Advanced APM setup

Up until this point, you let Datadog automatically instrument the hello.py application using Single Step Instrumentation. This approach is recommended if you want to capture essential traces across common libraries and languages without touching code or manually installing libraries.

However, if you need to collect traces from custom code or require more fine-grained control, you can add custom instrumentation.

To illustrate this, you will import the Datadog Python tracing library into hello.py and create a custom span and span tag.

To add custom instrumentation:

  1. Install the Datadog tracing library:

    pip install ddtrace
    
  2. Add the highlighted lines to the code in hello.py to create a custom span tag get_quote and a custom span tag quote:

     from flask import Flask
     import random
     from ddtrace import tracer
    
     app = Flask(__name__)
    
     quotes = [
         "Strive not to be a success, but rather to be of value. - Albert Einstein",
         "Believe you can and you're halfway there. - Theodore Roosevelt",
         "The future belongs to those who believe in the beauty of their dreams. - Eleanor Roosevelt"
     ]
    
     @app.route('/')
     def index():
         with tracer.trace("get_quote") as span:
             quote = random.choice(quotes)+"\n"
             span.set_tag("quote", quote)
             return quote
    
     if __name__ == '__main__':
         app.run(host='0.0.0.0', port=5050)
  3. Run hello.py in the virtual environment from earlier:

    ddtrace-run python hello.py
    
  4. Run a few curl commands in a separate command prompt:

    curl http://0.0.0.0:5050/
    
  5. In Datadog, go to APM > Traces.

  6. Select the hello trace.

  7. Find the new custom get_quote span in the flame graph and hover over it:

    The get_quote custom span displays in the flame graph. On hover, the quote span tag is displayed.
  8. Notice that the custom quote span tag displays on the Info tab.

Further reading