Observability User Guide

The Trace Activity page in the Observability tool is where you can:

• View active traces
• Search for specific traces
• Create new trace configurations

To learn more about the information that traces that provide, see trace details.

To navigate to Observability:

1. Log in to your mParticle account.
2. From the overview map, select Observability within the Oversight suite or go to Oversight > Observability > Trace Activity using the left hand nav.
3. This brings you to the Trace Activity page.

You can also jump directly to Observability by clicking the Search icon in the left nav bar (or entering Cmd+K on MacOS or Ctrl+K on Windows) and searching for “Observability”.

View all trace activity

The Trace Activity page displays a list of all recent traces for your development data, and any traces you have configured for your production data. All traces are available for up to 14 days.

To view the details for a specific trace, click the purple ID under the Trace ID column.

Trace status

You can access a trace from Observability as soon as mParticle begins receiving and processing data, but it’s important to note that a trace can’t provide complete information about a data flow until all data in the trace has been fully processed. This typically occurs within 30 minutes.

Traces that are ready to be used for troubleshooting will display a “Complete” Trace Status on the Trace Details Page. Traces for data flows that are still being processed have a Trace Status of “In Progress”.

Sort and filter trace activity

You can filter your results by time frame by clicking the button labeled “Last hour” and selecting one of the predefined date ranges or entering a custom range.

To further sort and filter your results, click Sort and Filters to view the following options:

Sort traces

Use the Order dropdown menu to sort your traces from most recent to oldest, or oldest to most recent.

Filter traces

Under Filters, select any of the following criteria to limit the traces displayed:

• Trace Type:

  • Event: displays only traces for the Events API
  • Identity: displays only traces for the IDSync API

• Result:

  • Success: displays only traces where all data was processed without any issues.
  • Insight: includes traces that experienced an interruption in data flow resulting from a configuration setting (such as a Rule or Filter).
  • Needs Attention: displays only traces that include an error message.
  • Warning: includes traces where an issue was encountered during data processing that could be resolved with a retry.

• Environment:

  • Production: displays only traces for data in your Production environment
  • Development: displays only traces for data in your Development environment
• mPID: filters results based on the MPID associated with a call to the IDSync API
• Trace Configuration ID: displays only traces created for the given trace configuration ID
• Inputs: filters results based on one of your configured data inputs
• Outputs: filters results based on one of your configured data outputs

After selecting your desired sorting and filter options, click Apply. This refreshes the Trace Activity page to display only filters matching your selected criteria.

Customize trace activity page

You can configure the columns that are displayed on the trace activity page by clicking the View Columns button.

To remove a column from the trace activity page, click the toggle switch. Some columns, like “Trace ID”, cannot be removed.

To change the order of the columns, click and drag the handle next to the column name.

Trace details

After opening the details page for a specific trace, you will see the following information:

Trace summary

• Trace ID: the unique ID for the trace.
• Trace Configuration ID: the ID of the trace configuration that the trace belongs to.
• mParticle ID: any MPID that is associated with the trace.
• Trace Status: either “In Progress” or “Complete”. The trace status indicates whether a trace contains enough information for it to be useful when troubleshooting your data flow. Traces that are “In Progress” indicate that the data being traced is still being processed. For more information, see Trace status.
• Trace Start Time: the date and time (in UTC) when the data was initially received by the mParticle platform.
• Duration: the total time elapsed during the trace.

Setup details

• Inputs: the name of the input configuration where the data originated.
• Output Configurations: the configured outputs for the data.
• Data plans: any active data plans that were used during processing.
• Environment: indicates whether the data flowed through your development or production environment.

Trace result

For each trace, you will see either Success, Insight, Needs Attention, or Warning displayed under “Result” along with any applicable messages under “Additional information”. You can use this information to determine whether an issue encountered during the trace was intentional or accidental, and what steps you may need to take to resolve any issues.

Message	Description
Success	Your data was ingested, processed, and forwarded without encountering any issues.
Insight	Your data flow was interrupted due to a configuration setting (such as a Rule or Filter). These interruptions may or may not be intentional, depending on your configuration and desired behavior. Referring to any insights can be helpful in identifying unintended consequences of a configuration setting.
Warning	Indicates that an issue was encountered during data processing that could be resolved with a retry.
Needs Attention	An error was encountered during the trace that cannot be resolved with a retry.

In some cases, you might see additional information with instructions to contact mParticle Support or your mParticle Account Representative, who can help you determine the root cause of an error or issue.

Related traces

It’s possible for one process to trigger other related processes in the mParticle platform. Any related processes that are traced will be listed here.

For example, when mParticle ingests a batch of event data or a request to make a bulk update to your data, each subsequent data flow will have its own unique trace, which you can find and access here.

Timeline view

The Timeline View provides a visual picture of how your data flows through the mParticle platform, broken into different spans, with each span representing a different stage of data processing.

Hover your cursor over the span in the timeline to see its exact start and end times.

To view details for a specific span, click on the span within the timeline and review the information panel at the bottom of the UI.

Not all spans will be presented sequentially, and some will appear to occur at the same time.

This is because mParticle executes different processes in parallel to reduce the amount of time it takes to process your data. Most gaps between spans on your timeline are likely due to networking delays or internal processes that are not represented on the trace timeline.

Span details

The Span Details view provides more granular information about a particular span. The details shown will vary depending on the span category.

To learn more about each span category, view the Span Glossary.

About trace IDs

Each individual trace is uniquely identified by a 36 character Trace ID resembling 66e0c0cd9bb8998a579595e42bae7077.

Trace IDs are essential for pinpointing specific data processing traces. You can find them in two primary ways:

1) Find a trace ID in an API response

Trace IDs are included with all responses to calls to the Events and IDSync APIs.

To find a trace ID in an API response:

1. Search your API response for the header titled X-MP-Trace-Id. The value of this header is the trace ID for the corresponding API call.

2) Find a trace ID in Live Stream

1. Log into your mParticle account and navigate to Activity > Live Stream.
2. Select any event row.
3. In the right hand sidebar, you will find the trace ID associated with that event.

Search for a trace using a trace ID

To search for a specific trace on the Trace Activity page:

1. Find and copy the trace ID from an API response or Live Stream as described in About trace IDs.
2. Log into your mParticle account and navigate to Oversight > Observability > Trace Activity.
3. From the Trace Activity page, enter your trace ID in the search bar and click Search.

Open a trace from Live Stream

You can also view trace details for a data flow directly from the mParticle Live Stream.

1. Log into your mParticle account and navigate to Data Platform > Live Stream.
2. Select the event row you want to view the trace for.
3. In the right hand sidebar, you will find the trace ID associated with that event.
4. Click on the Trace ID to open up the Trace Details page in the Observability suite.

Trace configurations

To view your trace configurations, navigate to Oversight > Observability > Trace Configurations in the left hand navigation.

You define what data you want to trace using a trace configuration. A trace configuration initiates an individual trace with a unique trace ID for each request to the Events or Identity API according to the trace configuration settings you specify.

The trace configurations page displays a list of all configurations, sorted from newest to oldest. You can see the start and end date and times for each configuration, the trace configuration ID, the percentage of data that will be traced, and one of the following trace configuration statuses:

• Active: Active trace configurations will generate a trace for any data ingested from the configured input(s).
• Pending: Pending trace configurations will become active once the selected start date and time is reached.
• Completed: Trace configurations are marked Completed when the scheduled duration for the trace has elapsed.
• Canceled: If you cancel the trace configuration, its status is changed to Canceled.

Create a trace configuration

To create a new trace configuration:

1. Navigate to Oversight > Observability > Trace Configurations, and click Create Trace Configuration.

   

   This opens the Add Tracing Configuration window:

   

2. Under Inputs, select the connected data input you want to trace.

   • You can select one of your configured Warehouse Sync pipelines as the input. Warehouse Sync pipelines will be listed under the Feeds input category.

3. Use the date and time picker to select the Start Date & Time and Duration for your trace. Traces will only be generated after the start time and for the duration you specify.

   • If you select a Warehouse Sync pipeline as your input to trace, you will see a unique trace generated for each event batch within the timeframe you specify.

4. Select the percentage of your data you would like to be traced using the Sample Size drop down menu. You can select either 1%, 3%, or 5% for small sample sizes, or you can select 10% through 100% in increments of 10 for larger sample sizes.

   • You should use large sample size for short-term tracing. For example, when first launching a new mParticle configuration, creating a short-term trace configuration with a large sample size can help you detect problems early on.
   • You should use a small sample size for long-term tracing. For example, once you have a stable configuration but still want to monitor your data, you can create a long-term trace configuration with a small sample size to keep your tracing cost low.
5. Use the date and time picker to select the Start Time (UTC) and Duration for your trace. Traces will only be generated after the start time and for the duration you specify.
6. Click Submit.

After clicking Submit, you will see your new tracing configuration listed on the Trace Configurations page.