Warehouse Sync API v2 Overview

The Warehouse Sync API is mParticle’s reverse ETL solution that allows you to ingest user profile data or event data into mParticle from a data warehouse.

There are four stages to configuring Warehouse Sync:

  1. Create a connection between mParticle and your database.
  2. Create a data model that specifies what data you want to ingest.
  3. Create a field transformation that maps your source data to fields in mParticle.
  4. Create and configure the pipeline between your database and mParticle using your data model and field transformations. Your pipeline settings determine when, and how frequently, data will sync.

Ingesting user profile data

User profile data includes information like what identities a user has (such as an email address, user name, or account number), custom attributes (such as subscription status), or device and demographic information. User data in mParticle is stored in user profiles which can be used to create segmented audiences that you can engage with using downstream marketing tools.

You can use warehouse sync to ingest user profile data from your database into mParticle, where it can be used to create new, or enrich existing, profiles that describe your users.

Ingesting events data

Event data describes the actions that your users take in your app, website, or product. This could include information about pages your users visit, videos played, products added to wishlists, and more.

Continue reading below for general information about how to access the Warehouse Sync API. For a step-by-step tutorial on how to start creating your first warehouse sync pipeline, go to the Warehouse Sync API Tutorial.

Prerequisites to Accessing the Warehouse Sync API

To authenticate when using the Warehouse Sync API, you will need a new set of API credentials.

  1. Navigate to API Credentials in the mParticle UI, and click Add Credential.
  2. Give your new credential a descriptive name, check the box next to Platform and select Admin from the Permissions dropdown menu.
  3. Click Save, and copy the Client ID and Client Secret. You will use these when fetching an OAuth access token.

Authentication

After creating your new API credential by following the steps above, you can authenticate by issuing a POST request to mParticle’s SSO token endpoint.

https://sso.auth.mparticle.com/oauth/token

The JSON body of the request must contain:

  • client_id - your Client ID that you saved when creating your new API credential
  • client_secret - your Client Secret that you saved when creating your new API credential
  • audience - set to a value of "https://api.mparticle.com"
  • grant_type - set to a value of "client_credentials"

Curl Syntax

curl --request POST \
  --url https://sso.auth.mparticle.com/oauth/token \
  --header 'content-type: application/json' \
  --data '{"client_id":"...","client_secret":"...","audience":"https://api.mparticle.com","grant_type":"client_credentials"}'

Sample Raw HTTP Request

POST /oauth/token HTTP/1.1
Host: sso.auth.mparticle.com
Content-Type: application/json

{
  "client_id": "your_client_id",
  "client_secret": "your_client_secret",
  "audience": "https://api.mparticle.com",
  "grant_type": "client_credentials"
}

Using your Bearer Token

A successful POST request to the token endpoint will result in a JSON response as follows:

{
  "access_token": "YWIxMjdi883GHBBDnjsdKAJQxNjdjYUUJABbg6hdI.8V6HhxW-",
  "expires_in" : 28800,
  "token_type": "Bearer"
}

Subsequent requests to the API can now be authorized by setting the Authorization header as follows:

Authorization: Bearer YWIxMjdi883GHBBDnjsdKAJQxNjdjYUUJABbg6hdI.8V6HhxW-

Versioning

Once you have authenticated, the API resources can be accessed at https://api.mparticle.com/platform/v2/. Subsequent updates to the API that introduce breaking changes will be published with a new version number in the URL.

HTTP Methods

This API uses the HTTP methods GET, POST, PUT, and DELETE.

Headers

This API accepts and sometimes requires the following headers:

Header Required Method Notes
Authorization Required GET, POST, PUT, DELETE
Content-Type Optional GET, POST, PUT, DELETE

Request Bodies

All POST/PUT requests should send JSON as the Request Payload, with Content-Type set to application/json.

Limits

In addition to the standard default service limits, note the following limits specific to the Warehouse Sync API:

Limit Value Notes
Max number of Active Pipelines 5
Historical Record Limit 24 million For new interval based pipelines, there is a 24 million record limit while retrieving records before the schedule start time. See sync mode from and until to filter data to load.
Column limit 100
Record count limit per hourly interval 1 million
Record count limit per daily interval 24 million
Record count limit per weekly interval 40 million
Record count limit per monthly interval 40 million
Record count limit per once request 40 million
Record count limit per on-demand request 24 million Applicable when the trigger API is used

Was this page helpful?