Documentation
Developers
API References
Data Subject Request API
Data Subject Request API Version 1 and 2
Data Subject Request API Version 3
Platform API
Platform API Overview
Accounts
Apps
Audiences
Calculated Attributes
Data Points
Feeds
Field Transformations
Services
Users
Workspaces
Warehouse Sync API
Warehouse Sync API Overview
Warehouse Sync API Tutorial
Warehouse Sync API Reference
Data Mapping
Warehouse Sync SQL Reference
Warehouse Sync Troubleshooting Guide
ComposeID
Warehouse Sync API v2 Migration
Bulk Profile Deletion API Reference
Calculated Attributes Seeding API
Data Planning API
Custom Access Roles API
Group Identity API Reference
Pixel Service
Profile API
Events API
mParticle JSON Schema Reference
IDSync
Client SDKs
AMP
AMP SDK
Android
Initialization
Configuration
Network Security Configuration
Event Tracking
User Attributes
IDSync
Screen Events
Commerce Events
Location Tracking
Media
Kits
Application State and Session Management
Data Privacy Controls
Error Tracking
Opt Out
Push Notifications
WebView Integration
Logger
Preventing Blocked HTTP Traffic with CNAME
Linting Data Plans
Troubleshooting the Android SDK
API Reference
Upgrade to Version 5
Cordova
Cordova Plugin
Identity
Direct Url Routing
Direct URL Routing FAQ
Web
Android
iOS
iOS
Initialization
Configuration
Event Tracking
User Attributes
IDSync
Screen Tracking
Commerce Events
Location Tracking
Media
Kits
Application State and Session Management
Data Privacy Controls
Error Tracking
Opt Out
Push Notifications
Webview Integration
Upload Frequency
App Extensions
Preventing Blocked HTTP Traffic with CNAME
Linting Data Plans
Troubleshooting iOS SDK
Social Networks
iOS 14 Guide
iOS 15 FAQ
iOS 16 FAQ
iOS 17 FAQ
iOS 18 FAQ
API Reference
Upgrade to Version 7
React Native
Getting Started
Identity
Unity
Upload Frequency
Getting Started
Opt Out
Initialize the SDK
Event Tracking
Commerce Tracking
Error Tracking
Screen Tracking
Identity
Location Tracking
Session Management
Web
Initialization
Content Security Policy
Configuration
Event Tracking
User Attributes
IDSync
Page View Tracking
Commerce Events
Location Tracking
Media
Kits
Application State and Session Management
Data Privacy Controls
Error Tracking
Opt Out
Custom Logger
Persistence
Native Web Views
Self-Hosting
Multiple Instances
Web SDK via Google Tag Manager
Preventing Blocked HTTP Traffic with CNAME
Facebook Instant Articles
Troubleshooting the Web SDK
Browser Compatibility
Linting Data Plans
API Reference
Upgrade to Version 2 of the SDK
Xamarin
Getting Started
Identity
Web
Alexa
Quickstart
Android
Overview
Step 1. Create an input
Step 2. Verify your input
Step 3. Set up your output
Step 4. Create a connection
Step 5. Verify your connection
Step 6. Track events
Step 7. Track user data
Step 8. Create a data plan
Step 9. Test your local app
iOS Quick Start
Overview
Step 1. Create an input
Step 2. Verify your input
Step 3. Set up your output
Step 4. Create a connection
Step 5. Verify your connection
Step 6. Track events
Step 7. Track user data
Step 8. Create a data plan
Web
Overview
Step 1. Create an input
Step 2. Verify your input
Step 3. Set up your output
Step 4. Create a connection
Step 5. Verify your connection
Step 6. Track events
Step 7. Track user data
Step 8. Create a data plan
Python Quick Start
Step 1. Create an input
Step 2. Create an output
Step 3. Verify output
Server SDKs
Node SDK
Go SDK
Python SDK
Ruby SDK
Java SDK
Guides
Partners
Introduction
Outbound Integrations
Outbound Integrations
Firehose Java SDK
Inbound Integrations
Compose ID
Glossary
Data Hosting Locations
Migrate from Segment to mParticle
Migrate from Segment to mParticle
Migrate from Segment to Client-side mParticle
Migrate from Segment to Server-side mParticle
Segment-to-mParticle Migration Reference
Rules Developer Guide
API Credential Management
The Developer's Guided Journey to mParticle
Guides
Getting Started
Create an Input
Start capturing data
Connect an Event Output
Create an Audience
Connect an Audience Output
Transform and Enhance Your Data
Platform Guide
The New mParticle Experience
The new mParticle Experience
The Overview Map
Introduction
Data Retention
Connections
Activity
Live Stream
Data Filter
Rules
Tiered Events
mParticle Users and Roles
Analytics Free Trial
Troubleshooting mParticle
Usage metering for value-based pricing (VBP)
Analytics
Introduction
Setup
Sync and Activate Analytics User Segments in mParticle
User Segment Activation
Welcome Page Announcements
Settings
Project Settings
Roles and Teammates
Organization Settings
Global Project Filters
Portfolio Analytics
Analytics Data Manager
Analytics Data Manager Overview
Events
Event Properties
User Properties
Revenue Mapping
Export Data
UTM Guide
Query Builder
Data Dictionary
Query Builder Overview
Modify Filters With And/Or Clauses
Query-time Sampling
Query Notes
Filter Where Clauses
Event vs. User Properties
Group By Clauses
Annotations
Cross-tool Compatibility
Apply All for Filter Where Clauses
Date Range and Time Settings Overview
Understanding the Screen View Event
Analyses
Analyses Introduction
Segmentation: Basics
Getting Started
Visualization Options
For Clauses
Date Range and Time Settings
Calculator
Numerical Settings
Segmentation: Advanced
Assisted Analysis
Properties Explorer
Frequency in Segmentation
Trends in Segmentation
Did [not] Perform Clauses
Cumulative vs. Non-Cumulative Analysis in Segmentation
Total Count of vs. Users Who Performed
Save Your Segmentation Analysis
Export Results in Segmentation
Explore Users from Segmentation
Funnels: Basics
Getting Started with Funnels
Group By Settings
Conversion Window
Tracking Properties
Date Range and Time Settings
Visualization Options
Interpreting a Funnel Analysis
Funnels: Advanced
Group By
Filters
Conversion over Time
Conversion Order
Trends
Funnel Direction
Multi-path Funnels
Analyze as Cohort from Funnel
Save a Funnel Analysis
Explore Users from a Funnel
Export Results from a Funnel
Saved Analyses
Manage Analyses in Dashboards
Dashboards
Dashboards––Getting Started
Manage Dashboards
Dashboard Filters
Organize Dashboards
Scheduled Reports
Favorites
Time and Interval Settings in Dashboards
Query Notes in Dashboards
User Aliasing
Analytics Resources
The Demo Environment
Keyboard Shortcuts
Tutorials
Analytics for Marketers
Analytics for Product Managers
Compare Conversion Across Acquisition Sources
Analyze Product Feature Usage
Identify Points of User Friction
Time-based Subscription Analysis
Dashboard Tips and Tricks
Understand Product Stickiness
Optimize User Flow with A/B Testing
User Segments
IDSync
IDSync Overview
Use Cases for IDSync
Components of IDSync
Store and Organize User Data
Identify Users
Default IDSync Configuration
Profile Conversion Strategy
Profile Link Strategy
Profile Isolation Strategy
Best Match Strategy
Aliasing
Data Master
Group Identity
Overview
Create and Manage Group Definitions
Introduction
Catalog
Live Stream
Data Plans
Data Plans
Blocked Data Backfill Guide
Personalization
Predictive Attributes
Predictive Attributes Overview
Create Predictive Attributes
Assess and Troubleshoot Predictions
Use Predictive Attributes in Campaigns
Predictive Audiences
Predictive Audiences Overview
Using Predictive Audiences
Introduction
Profiles
Warehouse Sync
Data Privacy Controls
Data Subject Requests
Default Service Limits
Feeds
Cross-Account Audience Sharing
Approved Sub-Processors
Import Data with CSV Files
Import Data with CSV Files
CSV File Reference
Glossary
Video Index
Analytics (Deprecated)
Identity Providers
Single Sign-On (SSO)
Setup Examples
Introduction
Developer Docs
Introduction
Integrations
Introduction
Rudderstack
Google Tag Manager
Segment
Data Warehouses and Data Lakes
Advanced Data Warehouse Settings
AWS Kinesis (Snowplow)
AWS Redshift (Define Your Own Schema)
AWS S3 Integration (Define Your Own Schema)
AWS S3 (Snowplow Schema)
BigQuery (Snowplow Schema)
BigQuery Firebase Schema
BigQuery (Define Your Own Schema)
GCP BigQuery Export
Snowflake (Snowplow Schema)
Snowplow Schema Overview
Snowflake (Define Your Own Schema)
Developer Basics
Aliasing
Blocked Data Backfill Guide
Backfilling is the process of importing previously blocked data from a Quarantine Output, optionally transforming it, and then uploading it back into mParticle. We provide instructions and helper scripts for you to backfill blocked data to mParticle’s Events API below. You cannot replay blocked data through the UI.
Read more about our Blocking feature here.
Considerations
-
Replaying event attributes requires replaying of events
Replaying event attributes is not possible without replaying their associated events, which can lead to event duplication.
-
Avoid additional MTU charges
If the backfilled MPID and the original MPID do not match, the user will be counted twice and the number of unique MPIDs that determines your mParticle bill will be impacted.
-
Sooner is better than later
We advise replaying data no longer than 2 weeks from the date it was quarantined. Many downstream tools will not accept data over a certain age. The sooner you replay data, the better.
-
Batch and event timestamps
To send data to mParticle via our Events API, events are stored in a batch (see our Events API docs pages for additional detail). Both mParticle batches and events have a timestamp attached to them. To ensure that events are backfilled with the original timestamp, it’s essential to preserve the value stored in the
timestamp_unixtime_msfield that each event object contains (the timestamp attached at the batch level can be ignored). -
Avoid batch deduplication
To avoid batches from being deduplicated in mParticle’s internal data pipeline, make sure to remove the
batch_idfrom the blocked batch before backfilling it to mParticle. -
Backfilling data requires some coding skills
To fix and replay data, you need to know how to code.
Define your backfill strategy
Backfilling blocked data is non-trivial because you typically are interested in backfilling data to several downstream event integrations.
Based on your unique set of target event integrations, you should devise a strategy for your data backfill. The following questions will guide your backfill strategy:
-
Which integrations do I need to backfill?
Different integrations have different limitations when it comes to receiving historical data. Establish the limitation of a target integration by reading their developer docs or by sending a small amount of test data through an mParticle connection.
-
Do I need to backfill unplanned event attributes?
Event attributes cannot be replayed without their associated events. You’ll need a strategy (e.g. deleting previously sent yet incomplete events) to avoid event duplication if you want to replay blocked event attributes.
-
Which mParticle Input should I use to backfill my data?
The cleanest solution is typically to create a new Custom Feed for the purpose of your backfill. You can connect only the integrations that you want to backfill to that feed and then tear it down again once the backfill is complete.
However, some integrations are not available through the Custom Feed Input. In those cases, you will need to either (i) use the keys and secret of the original Input (e.g. Web) in our backfill script or (ii) send data directly to the integration’s API (after transforming it to match their data model).
How to backfill blocked data
Once you have a strategy for your backfill, here are the steps to backfill your data:
- Go to your Quarantine output and find the data you want to transform.
- In your data, find the
contextnode. Within the context node, you will see a node labeledblock_metadata. This node contains the data you have blocked. Reference our sample data below to understand the complete data structure. - Pull out the events you want to fix and replay.
- Apply fixes to your data, then re-upload it to mParticle in its correct format. Use one of our sample scripts provided, or follow steps 5 - 8 for guidance on writing your own replay script.
- Using the JSON data file containing blocked events, get the
block_metadatanode. - Create a new events array and user attribute object containing only the data you would like to backfill.
- Perform any data fixes.
- Using mParticle’s Events API, create a new batch containing the events to be replayed along with any corrected event or user attributes. Send the fixed batch to mParticle’s Events API.
Resources
Sample Scripts for Data Import
Sample Data
Was this page helpful?
- Last Updated: December 20, 2024