{"componentChunkName":"component---src-templates-markdown-template-jsx","path":"/guides/data-platform/blocked-data-backfill-guide/","result":{"data":{"pageMetadata":{"title":"Blocked Data Backfill Guide","path":"/guides/data-platform/blocked-data-backfill-guide/","sourceCode":null,"metadataParent":{"title":"Data Platform","path":"/guides/data-platform/","sourceCode":null,"hasVersionPicker":null,"metadataChildren":[{"title":"Trends","path":"/guides/data-platform/trends/","order":1,"label":null},{"title":"Setup","path":"/guides/data-platform/setup/","order":2,"label":null},{"title":"Live Stream","path":"/guides/data-platform/live-stream/","order":3,"label":null},{"title":"Data Catalog","path":"/guides/data-platform/catalog/","order":4,"label":null},{"title":"Rules","path":"/guides/data-platform/rules/","order":5,"label":null},{"title":"Plans","path":"/guides/data-platform/data-planning/","order":6,"label":null},{"title":"Filters","path":"/guides/data-platform/data-filter/","order":7,"label":null},{"title":"Event Forwarding","path":"/guides/data-platform/event-forwarding/","order":8,"label":null},{"title":"Event Match Quality","path":"/guides/data-platform/event-match-quality/","order":9,"label":null},{"title":"Data Retention","path":"/guides/data-platform/data-retention/","order":10,"label":null},{"title":"Tiered Events","path":"/guides/data-platform/tiered-events/","order":11,"label":null},{"title":"Blocked Data Backfill Guide","path":"/guides/data-platform/blocked-data-backfill-guide/","order":12,"label":null}],"metadataParent":{"title":"Guides","path":"/guides/","metadataChildren":[{"title":"Platform Overview","path":"/guides/platform-overview/","order":0.5,"label":null},{"title":"Getting Started","path":"/guides/getting-started/","order":1,"label":null},{"title":"mParticle Agent","path":"/guides/mparticle-agent/","order":1.5,"label":null},{"title":"Oversight","path":"/guides/oversight/","order":2,"label":null},{"title":"Data Platform","path":"/guides/data-platform/","order":3,"label":null},{"title":"Data Warehouse","path":"/guides/data-warehouse/","order":4,"label":null},{"title":"Customer 360","path":"/guides/customer-360/","order":5,"label":null},{"title":"Analytics","path":"/guides/analytics/","order":6,"label":null},{"title":"Audiences","path":"/guides/audiences/","order":7,"label":null},{"title":"Settings","path":"/guides/settings/","order":8,"label":null},{"title":"Troubleshooting mParticle","path":"/guides/troubleshooting/","order":10,"label":null},{"title":"Default Service Limits","path":"/guides/default-service-limits/","order":11,"label":null},{"title":"Glossary","path":"/guides/glossary/","order":12,"label":null}],"metadataParent":null}}},"markdownRemark":{"html":"

Backfilling is the process of importing previously blocked data from a Quarantine Output, optionally transforming it, and then uploading it back into mParticle. We provide instructions and helper scripts for you to backfill blocked data to mParticle’s Events API below. You cannot replay blocked data through the UI.

\n

Read more about our Blocking feature here.

\n

\n \n \n \n \n \n \n \n Considerations

\n
    \n
  1. \n

    Replaying event attributes requires replaying of events

    \n

    Replaying event attributes is not possible without replaying their associated events, which can lead to event duplication.

    \n
    2. \n
  2. \n

    Avoid additional MTU charges

    \n

    If the backfilled MPID and the original MPID do not match, the user will be counted twice and the number of unique MPIDs that determines your mParticle bill will be impacted.

    \n
    3. \n
  3. \n

    Sooner is better than later

    \n

    We advise replaying data no longer than 2 weeks from the date it was quarantined. Many downstream tools will not accept data over a certain age. The sooner you replay data, the better.

    \n
    4. \n
  4. \n

    Batch and event timestamps

    \n

    To send data to mParticle via our Events API, events are stored in a batch (see our Events API docs pages for additional detail). Both mParticle batches and events have a timestamp attached to them. To ensure that events are backfilled with the original timestamp, it’s essential to preserve the value stored in the timestamp_unixtime_ms field that each event object contains (the timestamp attached at the batch level can be ignored).

    \n
    5. \n
  5. \n

    Avoid batch deduplication

    \n

    To avoid batches from being deduplicated in mParticle’s internal data pipeline, make sure to remove the batch_id from the blocked batch before backfilling it to mParticle.

    \n
    6. \n
  6. \n

    Backfilling data requires some coding skills

    \n

    To fix and replay data, you need to know how to code.

    \n
    7. \n
\n

\n \n \n \n \n \n \n \n Define your backfill strategy

\n

Backfilling blocked data is non-trivial because you typically are interested in backfilling data to several downstream event integrations.

\n

Based on your unique set of target event integrations, you should devise a strategy for your data backfill. The following questions will guide your backfill strategy:

\n
    \n
  1. \n

    Which integrations do I need to backfill?

    \n

    Different integrations have different limitations when it comes to receiving historical data. Establish the limitation of a target integration by reading their developer docs or by sending a small amount of test data through an mParticle connection.

    \n
    2. \n
  2. \n

    Do I need to backfill unplanned event attributes?

    \n

    Event attributes cannot be replayed without their associated events. You’ll need a strategy (e.g. deleting previously sent yet incomplete events) to avoid event duplication if you want to replay blocked event attributes.

    \n
    3. \n
  3. \n

    Which mParticle Input should I use to backfill my data?

    \n

    The cleanest solution is typically to create a new Custom Feed for the purpose of your backfill. You can connect only the integrations that you want to backfill to that feed and then tear it down again once the backfill is complete.

    \n

    However, some integrations are not available through the Custom Feed Input. In those cases, you will need to either (i) use the keys and secret of the original Input (e.g. Web) in our backfill script or (ii) send data directly to the integration’s API (after transforming it to match their data model).

    \n

    \n
    4. \n
\n

\n \n \n \n \n \n \n \n How to backfill blocked data

\n

Once you have a strategy for your backfill, here are the steps to backfill your data:

\n
    \n
  1. Go to your Quarantine output and find the data you want to transform.
    2. \n
  2. In your data, find the context node. Within the context node, you will see a node labeled block_metadata. This node contains the data you have blocked. Reference our sample data below to understand the complete data structure.
    3. \n
  3. Pull out the events you want to fix and replay.
    4. \n
  4. Apply fixes to your data, then re-upload it to mParticle in its correct format. Use one of our sample scripts provided, or follow steps 5 - 8 for guidance on writing your own replay script.
    5. \n
  5. Using the JSON data file containing blocked events, get the block_metadata node.
    6. \n
  6. Create a new events array and user attribute object containing only the data you would like to backfill.
    7. \n
  7. Perform any data fixes.
    8. \n
  8. Using mParticle’s Events API, create a new batch containing the events to be replayed along with any corrected event or user attributes. Send the fixed batch to mParticle’s Events API.
    9. \n
\n

\n \n \n \n \n \n \n \n Resources

\n
\n \n \n \n \n \n \n \n Sample Scripts for Data Import
\n

Quarantine Replay Scripts

\n
\n \n \n \n \n \n \n \n Sample Data
\n

Example Quarantine Batch

","frontmatter":{"title":"Blocked Data Backfill Guide","order":12,"seoTitle":null,"seoDescription":null},"parent":{"__typename":"File","modifiedTime":"October 12, 2026"}}},"pageContext":{"slug":"/guides/data-platform/blocked-data-backfill-guide/"}},"staticQueryHashes":["2950460867"]}