Microsoft Azure Service Profiler

Add Custom Events with Code Instrumentation

Service Profiler collects time-based ETW traces useful for investigating CPU wall clock vs. blocked time. By default, the Service Profiler agent can monitor IIS-hosted ASP.NET web requests, Service Fabric Reliable Actors, and SQL database calls, without requiring any custom code instrumentation. It is relatively easy to augment this set with your own server-side custom events. This tutorial will show you how to:

  1. Define your own EventSource class with Start/Stop events.
  2. Instrument your server-side code with these custom events.
  3. Configure the Service Profiler agent to monitor these events.

Define a Custom EventSource class

Begin by adding a new code file to your project -- MyEventSource.cs -- with the code below. The sample code:

Instrument Server-side Code with Custom Events

Add Start/Stop events to your application code. For example, on each OWIN-based Web API method or Reliable Service method:

        MySource.Log.RequestStart('/someurl');
        << Target code section >>
        MySource.Log.RequestStop(true);

The above code will generate two ETW events: Microsoft-Demos-Activities/Request/Start and Microsoft-Demos-Activities/Request/Stop

(Optional) You can also add nested Start/Stop events, like the snippet below, although this is not required. These will show up as nested activities within Service Profiler's timeline visualizations. In this case, nested activities named Phase1 and Phase2 will be displayed within a timeline for the top-level activity Request.

        MySource.Log.RequestStart('/someurl');
              MySource.Log.Phase1Start('a custom message...');
              // app code...
              MySource.Log.Phase1Stop();
              MySource.Log.Phase2Start('a custom message...');
              // app code...
              MySource.Log.Phase2Stop();
        MySource.Log.RequestStop(true);

A word about how much instrumentation you should consider... ServiceProfiler tries to minimize the number of places you need to instrument. It can do this because stack traces measure CPU and Blocked time, so the tool knows ‘where’ CPU time is being spent even without code instrumentation. This tends to dramatically cut the number of instrumentation points you need in your code. Our recommendation is:

  1. It is still useful to instrument in ‘top level’ items (things that your customer sees directly), if these top level items are not already covered by Service Profiler out of the box. Generally anything you can imagine being part of your service level agreement probably needs direct instrumentation.

  2. Add instrumentation for when the critical data is something more than ‘where’ the code is. For example, often URLs or user names, or other identifiers are really important. Without instrumentation you simply won’t have this data.

Configure the Service Profiler Agent

Once our code is instrumented with custom events, we need to configure Service Profiler to monitor these custom events. Update the EtwMetrics (this is found as a property of the ActivityMonitoring category) property of the Service Profiler configuration. The nested Phase1 and Phase2 activities do not need to be specified here since we don't want to see them on the summary request duration percentile page - but they will show up in detailed timelines as nested child activities.

    "ActivityMonitoring": { 
      "EtwMetrics": [
      {      
        "ProviderName": "Microsoft-Demos-Activities",
        "ProviderKeywords": 72057594037927935,
        "ProviderLevel": "Informational",
        "Event": "Request/Start",
        "EventStop": "Request/Stop",
        "Name": "url"
      }
    ]}

Note: If you manually deployed the agent executable, its configuration can be found in the accompanying Settings.json file. If instead the agent is provisioned via a PowerShell script or Extension Resource Template, look for a section in the script named EtwMetrics.

Update Machine Instances running the Service Profiler Agent

If you have deployed the Service Profiler Agent to multiple machine instances -- such as Cloud Service Worker Roles or a Service Fabric cluster -- execute the updated script again to push the change on your cloud service role instances. As an example, this walkthrough shows how to publish changes to a Service Fabric cluster.

Once the cluster is stable, you should soon be able to view performance data in the Service Profiler web app. Navigate to the Data Cube you're sending data to.

Read about how to explore performance data here: Getting Started - View Performance Data

Additional Resources

Give feedback