Field Transformations

Use the Field Transformations API to create and manage one-to-one correlations, or mappings, between external data and event attributes in the mParticle JSON schema.

Field transformations format and usage

Each individual field transformation contains an array called mappings. This array contains a list of JSON objects, where each object uses "key":"value" pairs to describe a specific one-to-one map between an external data point and a known field in mParticle. Each field transformation is identified by a unique ID and name (defined as strings) that you set when creating the field transformation.

The keys of a single mapping are:

• destination: the name of the mParticle field as it is listed in the mParticle JSON schema. Follow the JSON path format described below to correctly refer to an mParticle JSON schema field. This must be unset or set to null for ignore mappings.
• source: the name of the column in the source data table. This must be left unset or set to null if mapping_type is set to static.
• mapping_type: valid values: column, static, and ignore.
• value: when mapping_type is set to static, you can set value to any string value you want to map to an mParticle field.

Mapping types

The three mapping type options are:

• column: maps a column in a source database or table to an mParticle field
• static: maps a static value to an mParticle field. source must be set to null for static mappings.
• ignore: excludes an external, source data point from a field transformation

Simplified JSON path format

The mParticle JSON schema defines the structure that must be applied to all event data and user data for it to be ingested and usable by mParticle products.

At the highest level, the mParticle JSON schema contains an events array which represents an event batch. Each object in the events batch represents a single event, and it includes the event data stored in an object called data and the type of event that was logged, or the event_type. The specific event details are then stored in a series of nested JSON objects.

The Field Transformations API uses a simplified path format when mapping external data with individual fields contained in the mParticle JSON schema:

• Top level elements are referred to by their field names:

  • For example, to reference a in {"a": 123}, use a

• Nested elements are seperated by .:

  • For example, to reference b in {"a": {"b": 123}} use a.b

• Fields of objects in nested in arrays are referenced using array indices:

  • For example, to reference b in {"a": [{"b": 123}]} use a[].b

Consider the following simplified example source data table and destination schema:

Example source database table

Example destination JSON schema

{
  "events": [
    {
      "data": {
        "event_id": 1234,
        "session_id": 5678,
        "timestamp_unixtime_ms": 1402521613976,
        "custom_attributes":
        {
          "foo": "bar"
        }
      },
      "event_type": "screen_view"
    }
  ],
  "source_request_id":"7fa67be4-f83a-429f-9d73-38b660c50825",
  "environment": "production",
  "mpid":7346244611012968789,
  "ip": "172.217.12.142"
}

A field transformation mapping each attributes in the database table to the attributes as they exist in the mParticle JSON schema would be:

Example field transformation

{
  "id": "example-field-transformation-id",
  "name": "Example Field Transformation",
  "destination_type": "event_batch",
  "mappings": [
    {
      "mapping_type": "column",
      "source": "$eventId",
      "destination": "events[].data.event_id"
    },
    {
      "mapping_type": "column",
      "source": "$sessionId",
      "destination": "events[].data.session_id"
    },
    {
      "mapping_type": "column",
      "source": "timeStamp",
      "destination": "events[].data.timestamp_unixtime_ms"
    },
    {
      "mapping_type": "column",
      "source": "$ip",
      "destination": "ip"
    },
    {
      "mapping_type": "ignore",
      "source": "$fieldToIgnore"
    },
    {
      "mapping_type": "static",
      "destination": "events[].data.custom_attributes.foo",
      "value": "bar"
    }
  ],
  "created_on": "2023-11-14T21:15:43.182Z",
  "created_by": "developer@example.com",
  "last_modified_on": "2023-11-14T21:15:43.182Z",
  "last_modified_by": "developer@example.com"
}

Authentication

The Field Transformations API can be authenticated with a bearer token.

Authenticate with a bearer token

To create a bearer token, send a POST request to mParticle’s SSO token endpoint at https://sso.auth.mparticle.com/oauth/token.

The JSON body of the request must contain:

• client_id - the client ID, issued by mParticle when creating the API credentials
• client_secret - the client secret, issued by mParticle when creating the API credentials
• audience - set to a value of "https://api.mparticle.com"
• grant_type - set to a value of "client_credentials"

Example cURL request

curl --request POST \
  --url https://sso.auth.mparticle.com/oauth/token \
  --header 'content-type: application/json' \
  --data '{"client_id":"...","client_secret":"...","audience":"https://api.mparticle.com","grant_type":"client_credentials"}'

Example HTTP request

POST /oauth/token HTTP/1.1
Host: sso.auth.mparticle.com
Content-Type: application/json
{
  "client_id": "your_client_id",
  "client_secret": "your_client_secret",
  "audience": "https://api.mparticle.com",
  "grant_type": "client_credentials"
}

A successful POST request to the token endpoint results in a JSON response as follows:

{
  "access_token": "YWIxMjdi883GHBBDnjsdKAJQxNjdjYUUJABbg6hdI.8V6HhxW-",
  "expires_in" : 28800,
  "token_type": "Bearer"
}

Subsequent requests to the API can then be authorized by setting the authorization header to:

Authorization: Bearer YWIxMjdi883GHBBDnjsdKAJQxNjdjYUUJABbg6hdI.8V6HhxW-

Tokens cannot be revoked, but they expire every eight hours. The initial token request can take between one and three seconds, so mParticle recommends that you cache the token and refresh it only when necessary.

Get all field transformations

GET https://api.mparticle.com/platform/experimental/workspaces/{workspaceId}/transformations/fields

Path parameters

Query parameters

Example cURL request

curl --location --request GET 'https://api.mparticle.com/platform/v2/workspaces/{workspaceId}/transformations/fields' \
--header 'Authorization: Bearer <access_token>'

Example JSON request body

No request body.

Response

A successful request receives a 200 response with an array containing each field transformation within a JSON object.

[
  {
    "id": "string",
    "name": "string",
    "destination_type": "event_batch",
    "mappings": [
      {
        "mapping_type": "column",
        "source": "string",
        "destination": "string",
        "value": "string"
      }
    ],
    "created_on": "2023-11-14T21:12:36.246Z",
    "created_by": "string",
    "last_modified_on": "2023-11-14T21:12:36.246Z",
    "last_modified_by": "string"
  }
]

Create a new field transformation

POST https://api.mparticle.com/platform/experimental/workspaces/{workspaceId}/transformations/fields

Example cURL request

curl --location --request POST 'https://api.example.com/users/' \
--header 'Authorization: Bearer <access_token>'

Example JSON request body

{
  "id": "string",
  "name": "string",
  "destination_type": "event_batch",
  "mappings": [
    {
      "mapping_type": "column",
      "source": "string",
      "destination": "string",
      "value": "string"
    }
  ],
  "created_on": "2023-11-14T21:15:43.182Z",
  "created_by": "string",
  "last_modified_on": "2023-11-14T21:15:43.182Z",
  "last_modified_by": "string"
}

Response

A successful request receives a 200 response with a JSON object containing the field transformation you just created.

{
  "id": "string",
  "name": "string",
  "destination_type": "event_batch",
  "mappings": [
    {
      "mapping_type": "column",
      "source": "string",
      "destination": "string",
      "value": "string"
    }
  ],
  "created_on": "2023-11-14T21:15:43.182Z",
  "created_by": "string",
  "last_modified_on": "2023-11-14T21:15:43.182Z",
  "last_modified_by": "string"
}

Get a specific field transformation

GET https://api.mparticle.com/platform/experimental/workspaces/{workspaceId}/transformations/fields/{fieldTransformationId}

Path parameters

Example cURL request

curl --location --request GET 'https://api.mparticle.com/platform/v2/workspaces/{workspaceId}/transformations/fields' \
--header 'Authorization: Bearer <access_token>'

Example JSON request body

No request body.

Response

A successful request receives a 200 response with a JSON object containing the field transformation.

{
  "id": "string",
  "name": "string",
  "destination_type": "event_batch",
  "mappings": [
    {
      "mapping_type": "column",
      "source": "string",
      "destination": "string",
      "value": "string"
    }
  ],
  "created_on": "2023-11-14T21:24:26.511Z",
  "created_by": "string",
  "last_modified_on": "2023-11-14T21:24:26.511Z",
  "last_modified_by": "string"
}

Update a field transformation

PUT https://api.mparticle.com/platform/experimental/workspaces/{workspaceId}/transformations/fields/{fieldTransformationId}

Path parameters

Example cURL request

curl --location --request PUT 'https://api.example.com/users/{userID}' \
--header 'Authorization: Bearer <access_token>'

Example JSON request body

{
  "id": "string",
  "name": "string",
  "destination_type": "event_batch",
  "mappings": [
    {
      "mapping_type": "column",
      "source": "string",
      "destination": "string",
      "value": "string"
    }
  ]
}

Response

A successful request receives a 200 response with a JSON object containing the updated field transformation.

{
  "id": "string",
  "name": "string",
  "destination_type": "event_batch",
  "mappings": [
    {
      "mapping_type": "column",
      "source": "string",
      "destination": "string",
      "value": "string"
    }
  ],
  "created_on": "2023-11-14T21:25:50.947Z",
  "created_by": "string",
  "last_modified_on": "2023-11-14T21:25:50.947Z",
  "last_modified_by": "string"
}

Delete a field transformation

DELETE https://api.mparticle.com/platform/experimental/workspaces/{workspaceId}/transformations/fields/{fieldTransformationId}

Path parameters

Example cURL request

curl --location --request DELETE 'https://api.example.com/users/{userID}' \
--header 'Authorization: Bearer <access_token>'

Example JSON request body

No request body.

Response

A successful request receives a 204 response with an empty body.

Error handling