Data Planning API

Overview

Data Master is underpinned by the Data Planning API. This guide covers the underlying data model and describes how you can use Data Master programmatically via an HTTP API. Some example use cases you can achieve with the Data Planning API are:

  • Store your Data Plans in your source code, and use your own SDLC and approval processes to define your data model
  • Create, read, update and delete Data Plans
  • Integrate mParticle’s suite of Data Planning tools to perform compile-time and runtime data quality verification

Data Model

The following diagram shows the relationship structure defined by the Data Planning API:

  • For every mParticle workspace, you may have many Data Plans
  • Data Plans contain Data Plan Versions
  • Data Plan Versions contain Data Points

Data Points

Each Data Point is composed of two key elements, a “criteria” used to match the Data Point within an incoming data stream, and “schema” that is used to validate the contents of the Data Point:

  • Criteria: As data streams into mParticle via the Events API, the Criteria is used to locate the Data Point within the payload, then the schema (described below) is used to validate it.
  • Schema: The validation schema defines the expected syntax of each Data Point. This is a JSON Schema object.

Helper SDKs

The Data Planning HTTP API can be used from one of mParticle’s helper SDKs. These SDKs take care of authentication and implement the HTTP contract for you:

Open API

You can use the Open API specification (also known as Swagger) below to generate helper SDKs (using Swagger Codegen or OpenAPI Generator) for the Data Planning API:

Data Planning Open API Spec

Postman

You can also get started quickly with a Postman collection:

Run in Postman

HTTP API

Authentication

Calls to the Data Planning API are authenticated via mParticle access token, which are based on the JSON Web Token standard. If you’re using one of the helper SDKs, this is taken care of for you.

Parameters

Name Type Description
client_id string Your client ID issued by mParticle
client_secret string Your client secret issued by mParticle
audience string Set to https://api.mparticle.com
grant_type string Set to client_credentials

Example Request

$ curl --location --request POST 'https://sso.auth.mparticle.com/oauth/token' \
--header 'Content-Type: application/json' \
--data-raw '{
    "client_id": "{{client_id}}",
    "client_secret": "{{client_secret}}",
    "audience": "https://api.mparticle.com",
    "grant_type": "client_credentials"
}'

Example Response

{
    "access_token": "YWIxMjdi883GHBBDnjsdKAJQxNjdjYUUJABbg6hdI.8V6HhxW-",
    "expires_in" : 28800,
    "token_type": "Bearer"
}

Subsequent requests to the HTTP API should be authorized by setting the Authorization header to the value of access_token:

Authorization: Bearer YWIxMjdi883GHBBDnjsdKAJQxNjdjYUUJABbg6hdI.8V6HhxW-

Get all Data Plans

Parameters

Name Type Description
workspace_id integer The ID of the workspace containing your data plans

Example Request

curl --location --request GET 'https://api.mparticle.com/platform/v2/workspaces/{{workspace_id}}/plans' \
--header 'Authorization: Bearer {{jwt_token}}'
import { DataPlanService } from '@mparticle/data-planning-node';

const dataPlanService = new DataPlanService({
    workspaceId: '{{workspace_id}}',
    clientId: '{{client_id}}',
    clientSecret: '{{client_secret}}',
});
const dataPlans = await dataPlanService.getDataPlans();

Example Response

[
    {
        "data_plan_id": "mobile_data_plan",
        "data_plan_name": "Mobile Data Plan",
        "data_plan_description": "This is an example data plan description.",
        "data_plan_versions": [
            {
                "version": 1,
                "data_plan_id": "mobile_data_plan",
                "version_description": "This is my first version!",
                "activated_environment": "production",
                "created_on": "2019-11-24T14:45:03.013",
                "created_by": "developers@mparticle.com",
                "last_modified_on": "2019-11-24T18:58:58.233",
                "last_modified_by": "developers@mparticle.com"
            },
            {
                "version": 2,
                "data_plan_id": "mobile_data_plan",
                "version_description": "This is the second version of my plan!",
                "activated_environment": "production",
                "created_on": "2019-11-24T19:02:26.013",
                "created_by": "developers@mparticle.com",
                "last_modified_on": "2019-11-24T19:03:13.913",
                "last_modified_by": "developers@mparticle.com"
            }
        ],
        "created_on": "2019-11-24T14:45:03.007",
        "created_by": "developers@mparticle.com",
        "last_modified_on": "2020-04-07T14:22:57.777",
        "last_modified_by": "developers@mparticle.com"
    },
    {
        "data_plan_id": "foo_bar",
        "data_plan_name": "foo bar",
        "data_plan_description": "foo_bar",
        "data_plan_versions": [
            {
                "version": 1,
                "data_plan_id": "foo_bar",
                "activated_environment": "none",
                "created_on": "2019-11-25T16:53:16.027",
                "created_by": "developers@mparticle.com",
                "last_modified_on": "2020-01-14T14:52:11.43",
                "last_modified_by": "developers@mparticle.com"
            },
            {
                "version": 2,
                "data_plan_id": "foo_bar",
                "activated_environment": "none",
                "created_on": "2020-04-07T14:28:14.903",
                "created_by": "developers@mparticle.com"
            }
        ],
        "created_on": "2019-11-25T16:53:16.023",
        "created_by": "developers@mparticle.com"
    }
]

Create Data Plan

Parameters

Name Type Description
workspace_id integer The ID of the workspace containing your data plans

Example Request

curl --location --request POST 'https://api.mparticle.com/platform/v2/workspaces/{{workspace_id}}/plans' \
--header 'Authorization: Bearer {{jwt_token}}' \
--data-raw '{
    "data_plan_id": "example_data_plan",
    "data_plan_name": "Mobile Data Plan",
    "data_plan_description": "This is an example data plan description.",
    "data_plan_versions": [
        {
            "version": 1,
            "data_plan_id": "mobile_data_plan",
            "activated_environment": "none",
            "version_document": {
                "data_points": [
                    {
                        "description": "My first data point",
                        "match": {
                            "type": "custom_event",
                            "criteria": {
                                "event_name": "My Custom Event Name",
                                "custom_event_type": "other"
                            }
                        },
                        "validator": {
                            "type": "json_schema",
                            "definition": {
                                "properties": {
                                    "data": {
                                        "additionalProperties": true,
                                        "properties": {
                                            "custom_attributes": {
                                                "additionalProperties": false,
                                                "properties": {
                                                    "foo": {
                                                        "type": "number"
                                                    }
                                                },
                                                "required": [
                                                    "foo"
                                                ]
                                            }
                                        },
                                        "required": [
                                            "custom_attributes"
                                        ]
                                    }
                                }
                            }
                        }
                    }
                ]
            }
        }
    ]
}'
import { DataPlanService } from '@mparticle/data-planning-node';
import { ActivatedEnvironment } from '@mparticle/data-planning-models';

const dataPlanService = new DataPlanService({
    workspaceId: '{{workspace_id}}',
    clientId: '{{client_id}}',
    clientSecret: '{{client_secret}}',
});
const plan = {
    data_plan_id: 'example_data_plan',
    data_plan_name: 'My Example Data Plan',
    data_plan_description: 'This is an example data plan description.',
    data_plan_versions: []
}
const version = {
    version: 1,
    version_description: 'This is an example version description.',
    activated_environmment: ActivatedEnvironment.Development,
    version_document: {
        data_points: []
    }
};
const datapoint = {
    description: "My first data point",
    match: {
        type: "custom_event",
        criteria: {
            event_name: "My Custom Event Name",
            custom_event_type: "other"
        }
    },
    validator: {
        type: "json_schema",
        definition: {
            properties: {
                data: {
                    additionalProperties: true,
                    properties: {
                        custom_attributes: {
                            additionalProperties: false,
                            properties: {
                                foo: {
                                    type: "number"
                                }
                            },
                            required: [
                                "foo"
                            ]
                        }
                    },
                    required: [
                        "custom_attributes"
                    ]
                }
            }
        }
    }
};

plan.data_plan_versions.push(version);
version.version_document.data_points.push(datapoint);

const createdPlan = await dataPlanService.createDataPlan(plan)
    .catch(err => {
        console.log(err.status);
        console.log(err.response.data);
    }
);

Example Response

{
    "data_plan_id": "example_data_plan",
    "data_plan_name": "Mobile Data Plan",
    "data_plan_description": "This is an example data plan description.",
    "data_plan_versions": [
        {
            "version": 1,
            "data_plan_id": "example_data_plan",
            "activated_environment": "none",
            "created_on": "2020-04-19T14:45:44.403",
            "created_by": "client_id:rWW4wEjzksyjcCJXh8RGKrIPwPhHCjLF",
            "version_document": {
                "data_points": [
                    {
                        "description": "My first data point",
                        "match": {
                            "type": "custom_event",
                            "criteria": {
                                "event_name": "My Custom Event Name",
                                "custom_event_type": "other"
                            }
                        },
                        "validator": {
                            "type": "json_schema",
                            "definition": {
                                "properties": {
                                    "data": {
                                        "additionalProperties": true,
                                        "properties": {
                                            "custom_attributes": {
                                                "additionalProperties": false,
                                                "properties": {
                                                    "foo": {
                                                        "type": "number"
                                                    }
                                                },
                                                "required": [
                                                    "foo"
                                                ]
                                            }
                                        },
                                        "required": [
                                            "custom_attributes"
                                        ]
                                    }
                                }
                            }
                        }
                    }
                ]
            }
        }
    ],
    "created_on": "2020-04-19T14:45:44.39",
    "created_by": "client_id:rWW4wEjzksyjcCJXh8RGKrIPwPhHCjLF"
}

Get Data Plan

Parameters

Name Type Description
workspace_id integer The ID of the workspace containing your data plans
plan_id integer The ID of the Data Plan to retrieve

Example Request

curl --location --request GET 'https://api.mparticle.com/platform/v2/workspaces/{{workspace_id}}/plans/{{plan_id}}' \
--header 'Authorization: Bearer {{jwt_token}}'
import { DataPlanService } from '@mparticle/data-planning-node';

const dataPlanService = new DataPlanService.DataPlanService({
    workspaceId: '{{workspace_id}}',
    clientId: '{{client_id}}',
    clientSecret: '{{client_secret}}',
});
const dataPlan = await dataPlanService.getDataPlans("example_data_plan_id");

Response

The response will contain the entire Data Plan including all versions. See above for a full JSON example.

Update Data Plan

Parameters

Name Type Description
workspace_id integer The ID of the workspace containing your data plans
plan_id integer The ID of the Data Plan to retrieve

Example Request

curl --location --request PATCH 'https://api.mparticle.com/platform/v2/workspaces/{{workspace_id}}/plans/{{plan_id}}' \
--header 'Authorization: Bearer {{jwt_token}}' \
--data-raw '{
	"data_plan_description": "This is an updated description..."
}'
import { DataPlanService } from '@mparticle/data-planning-node';

const dataPlanService = new DataPlanService.DataPlanService({
    workspaceId: '{{workspace_id}}',
    clientId: '{{client_id}}',
    clientSecret: '{{client_secret}}',
});
//provide any fields you'd like to update
const updatedPlan = {
    data_plan_description: "This is an updated description"
};
const updatedDataPlan = await dataPlanService.updateDataPlan("example_data_plan", updatedPlan)
    .catch(err => {
        console.log(err.status);
        console.log(err.response.data);
    }
);

Response

The response will contain the entire Data Plan including all versions. See above for a full JSON example.

Create Data Plan Version

Parameters

Name Type Description
workspace_id integer The ID of the workspace containing your data plans

Example Request

curl --location --request POST 'https://api.mparticle.com/platform/v2/workspaces/{{workspace_id}}/plans/{{plan_id}}/versions' \
--header 'Authorization: Bearer {{jwt_token}}' \
--data-raw '{
    "version": 2,
    "version_document": {
        "data_points": [
            {
                "description": "My first data point",
                "match": {
                    "type": "custom_event",
                    "criteria": {
                        "event_name": "My Custom Event Name",
                        "custom_event_type": "other"
                    }
                },
                "validator": {
                    "type": "json_schema",
                    "definition": {
                        "properties": {
                            "data": {
                                "additionalProperties": true,
                                "properties": {
                                    "custom_attributes": {
                                        "additionalProperties": false,
                                        "properties": {
                                            "foo": {
                                                "type": "number"
                                            }
                                        },
                                        "required": ["foo"]
                                    }
                                },
                                "required": [
                                    "custom_attributes"
                                ]
                            }
                        }
                    }
                }
            }
        ]
    },
    "activated_environment": "development"
}'
import { DataPlanService } from '@mparticle/data-planning-node';
import { ActivatedEnvironment } from '@mparticle/data-planning-models';

const dataPlanService = new DataPlanService({
    workspaceId: '{{workspace_id}}',
    clientId: '{{client_id}}',
    clientSecret: '{{client_secret}}',
});

const version = {
    version: 1,
    version_description: 'This is an example version description.',
    activated_environmment: ActivatedEnvironment.Development,
    version_document: {
        data_points: []
    }
};
const datapoint = {
    description: "My first data point",
    match: {
        type: "custom_event",
        criteria: {
            event_name: "My Custom Event Name",
            custom_event_type: "other"
        }
    },
    validator: {
        type: "json_schema",
        definition: {
            properties: {
                data: {
                    additionalProperties: true,
                    properties: {
                        custom_attributes: {
                            additionalProperties: false,
                            properties: {
                                foo: {
                                    type: "number"
                                }
                            },
                            required: [
                                "foo"
                            ]
                        }
                    },
                    required: [
                        "custom_attributes"
                    ]
                }
            }
        }
    }
};

version.version_document.data_points.push(datapoint);

const createdPlanVersion = await dataPlanService.createDataPlanVersion("example_data_plan_id", 2)
    .catch(err => {
        console.log(err.status);
        console.log(err.response.data);
    }
);

Response

The response will contain the entire Data Plan Version including all versions. See above for a full JSON example.

Get Data Plan Version

Parameters

Name Type Description
workspace_id integer The ID of the workspace containing your data plans
plan_id integer The ID of the Data Plan to retrieve
version integer The version of the Data Plan to retrieve

Example Request

curl --location --request GET 'https://api.mparticle.com/platform/v2/workspaces/{{workspace_id}}/plans/{{plan_id}}/versions/{{version}}' \
--header 'Authorization: Bearer {{jwt_token}}'
import { DataPlanService } from '@mparticle/data-planning-node';

const dataPlanService = new DataPlanService.DataPlanService({
    workspaceId: '{{workspace_id}}',
    clientId: '{{client_id}}',
    clientSecret: '{{client_secret}}',
});
const dataPlanVersion = await dataPlanService.getDataPlanVersion("example_data_plan_id", 1)
    .catch(err => {
        console.log(err.status);
        console.log(err.response.data);
    }
);

Response

The response will contain the entire Data Plan version. See above for a full JSON example.

Update Data Plan Version

Parameters

Name Type Description
workspace_id integer The ID of the workspace containing your data plans
plan_id integer The ID of the Data Plan to retrieve
version integer The version of the Data Plan to retrieve

Example Request

curl --location --request PATCH 'https://api.mparticle.com/platform/v2/workspaces/{{workspace_id}}/plans/{{plan_id}}/versions/{{version}}' \
--header 'Authorization: Bearer {{jwt_token}}' \
--data-raw '{
    "version": 2,
    "version_document": {
        "data_points": [
            {
                "description": "My first data point, updated",
                "match": {
                    "type": "custom_event",
                    "criteria": {
                        "event_name": "My Custom Event Name",
                        "custom_event_type": "other"
                    }
                },
                "validator": {
                    "type": "json_schema",
                    "definition": {
                        "properties": {
                            "data": {
                                "additionalProperties": true,
                                "properties": {
                                    "custom_attributes": {
                                        "additionalProperties": false,
                                        "properties": {
                                            "foo": {
                                                "type": "number"
                                            }
                                        },
                                        "required": [
                                            "foo"
                                        ]
                                    }
                                },
                                "required": [
                                    "custom_attributes"
                                ]
                            }
                        }
                    }
                }
            }
        ]
    },
    "activated_environment": "development"
}'
const dataPlanService = new DataPlanService.DataPlanService({
    workspaceId: '{{workspace_id}}',
    clientId: '{{client_id}}',
    clientSecret: '{{client_secret}}',
});

const dataPlanVersionUpdate = {
    datapoints: [{
        description: "My first data point",
        match: {
            type: "custom_event",
            criteria: {
                event_name: "My Custom Event Name",
                custom_event_type: "other"
            }
        },
        validator: {
            type: "json_schema",
            definition: {
                properties: {
                    data: {
                        additionalProperties: true,
                        properties: {
                            custom_attributes: {
                                additionalProperties: false,
                                properties: {
                                    foo: {
                                        type: "number"
                                    }
                                },
                                required: [
                                    "foo"
                                ]
                            }
                        },
                        required: [
                            "custom_attributes"
                        ]
                    }
                }
            }
        }
    },
    {
        description: "My second data point",
        match: {
            type: "custom_event",
            criteria: {
                event_name: "My Custom Event Name",
                custom_event_type: "other"
            }
        },
        validator: {
            type: "json_schema",
            definition: {
                properties: {
                    data: {
                        additionalProperties: true,
                        properties: {
                            custom_attributes: {
                                additionalProperties: false,
                                properties: {
                                    foo: {
                                        type: "number"
                                    }
                                },
                                required: [
                                    "foo"
                                ]
                            }
                        },
                        required: [
                            "custom_attributes"
                        ]
                    }
                }
            }
        }
    }]
}
const updatePlanVersion = await dataPlanService.updateDataPlanVersion("example_data_plan_id, 2, dataPlanVersionUpdate)
    .catch(err => {
        console.log(err.status);
        console.log(err.response.data);
    }
);

Response

The response will contain the entire Data Plan version. See above for a full JSON example.

Was this page helpful?