Event

Amazon Kinesis is a platform for streaming data on AWS, offering powerful services to make it easy to load and analyze streaming data, and also providing the ability for you to build custom streaming data applications for specialized needs.

The Amazon Kinesis Firehose Event Integration supports data from all platforms.

Prerequisites

mParticle’s Event Integration with Amazon Kinesis sends data to a Kinesis Delivery Stream. To set up a Delivery Stream, follow Amazon’s instructions here. To set up the integration, you will need to:

1. Create a Stream

• The Kinesis Stream Name and Kinesis Service Region are required for mParticle setup.
• When you create your stream, be sure to set the Source to Direct PUT or other sources .

2. Create an IAM user

• Be sure to save the credentials file which contains the Access Key Id and Secret Access Key required for mParticle setup.

3. Create a Custom Policy. Use one of the following methods to create the policy:

   1. Manual Policy Creation
   • Select Firehose as the AWS Service
   • Include the PutRecord and DescribeDeliveryStream permissions
   • Enter the ARN of your stream.
   • Click Add Statement
   2. Create Policy from JSON
   • Paste the template policy below into the “Policy Document” field. Be sure to replace the {region}, {account-id} and {stream-name} with your specific values.

   {
      "Version": "2012-10-17",
      "Statement": [
          {
              "Effect": "Allow",
              "Action": [
                  "firehose:DescribeDeliveryStream",
                  "firehose:PutRecord"
              ],
              "Resource": [
                  "arn:aws:firehose:{region}:{account-id}:deliverystream/{stream-name}"
              ]
          }
      ]
   }

4. Assign Customer Policy to User

Data Format

mParticle will forward event data to Amazon Kinesis Firehose as base64 encoded JSON. An example is below (for readability the example has been left unencoded). You can control which events and attributes are forwarded in the Connection Settings. The contents of the ‘Data’ key match the JSON format.

{
    "DeliveryStreamName": "my-stream",
    "Record": {
        "Data": {
           
           "events":[{"data":{"event_name":"Level_Achieved"},"event_type":"custom_event"}],"unique_id":"abc123123","message_id":"521e0ae1-164d-  4495-8e41-95be88fa7e42","message_type":"events","schema_version":1,"device_info":{"ios_advertising_id":"68b7032b-1c2d-4533-92b8-3fb143ebeadf","android_advertising_id":"f9f1b997-f228-415f-99e1-ed7c223e238d","limit_ad_tracking":false,"is_dst":false},"application_info":{"application_version":"1.3","package":"test-app-id-550854415"}}
        }
    }
}

Rate Limits

Amazon Kinesis Firehose imposes standard rate limits that vary depending on your Service Region. If your throughput is close to or in excess of these limits, mParticle will make retries in an exponential backoff pattern. If you continually exceed the limit, you will see 500 - Internal Server Error and Retry Limit Exceeded errors in your System Alerts dashboard. If this occurs, you may need to request a limit increase from Amazon.

Configuration Settings

Connection Settings