Documentation

Developers

API References

Data Subject Request API

Data Subject Request API Version 1 and 2

Data Subject Request API Version 3

Platform API

Platform API Overview

Accounts

Apps

Audiences

Calculated Attributes

Data Points

Feeds

Field Transformations

Services

Users

Workspaces

Warehouse Sync API

Warehouse Sync API Overview

Warehouse Sync API Tutorial

Warehouse Sync API Reference

Data Mapping

Warehouse Sync SQL Reference

Warehouse Sync Troubleshooting Guide

ComposeID

Warehouse Sync API v2 Migration

Bulk Profile Deletion API Reference

Calculated Attributes Seeding API

Data Planning API

Custom Access Roles API

Group Identity API Reference

Pixel Service

Profile API

Events API

mParticle JSON Schema Reference

IDSync

Client SDKs

AMP

AMP SDK

Android

Initialization

Configuration

Network Security Configuration

Event Tracking

User Attributes

IDSync

Screen Events

Commerce Events

Location Tracking

Media

Kits

Application State and Session Management

Data Privacy Controls

Error Tracking

Opt Out

Push Notifications

WebView Integration

Logger

Preventing Blocked HTTP Traffic with CNAME

Linting Data Plans

Troubleshooting the Android SDK

API Reference

Upgrade to Version 5

Cordova

Cordova Plugin

Identity

Direct Url Routing

Direct URL Routing FAQ

Web

Android

iOS

Flutter

Getting Started

Usage

API Reference

iOS

Initialization

Configuration

Event Tracking

User Attributes

IDSync

Screen Tracking

Commerce Events

Location Tracking

Media

Kits

Application State and Session Management

Data Privacy Controls

Error Tracking

Opt Out

Push Notifications

Webview Integration

Upload Frequency

App Extensions

Preventing Blocked HTTP Traffic with CNAME

Linting Data Plans

Troubleshooting iOS SDK

Social Networks

iOS 14 Guide

iOS 15 FAQ

iOS 16 FAQ

iOS 17 FAQ

iOS 18 FAQ

API Reference

Upgrade to Version 7

React Native

Getting Started

Identity

Roku

Getting Started

Identity

Media

Unity

Upload Frequency

Getting Started

Opt Out

Initialize the SDK

Event Tracking

Commerce Tracking

Error Tracking

Screen Tracking

Identity

Location Tracking

Session Management

Xbox

Getting Started

Identity

Web

Initialization

Content Security Policy

Configuration

Event Tracking

User Attributes

IDSync

Page View Tracking

Commerce Events

Location Tracking

Media

Kits

Application State and Session Management

Data Privacy Controls

Error Tracking

Opt Out

Custom Logger

Persistence

Native Web Views

Self-Hosting

Multiple Instances

Web SDK via Google Tag Manager

Preventing Blocked HTTP Traffic with CNAME

Facebook Instant Articles

Troubleshooting the Web SDK

Browser Compatibility

Linting Data Plans

API Reference

Upgrade to Version 2 of the SDK

Xamarin

Getting Started

Identity

Web

Alexa

Quickstart

Android

Overview

Step 1. Create an input

Step 2. Verify your input

Step 3. Set up your output

Step 4. Create a connection

Step 5. Verify your connection

Step 6. Track events

Step 7. Track user data

Step 8. Create a data plan

Step 9. Test your local app

HTTP Quick Start

Step 1. Create an input

Step 2. Create an output

Step 3. Verify output

iOS Quick Start

Overview

Step 1. Create an input

Step 2. Verify your input

Step 3. Set up your output

Step 4. Create a connection

Step 5. Verify your connection

Step 6. Track events

Step 7. Track user data

Step 8. Create a data plan

Java Quick Start

Step 1. Create an input

Step 2. Create an output

Step 3. Verify output

Node Quick Start

Step 1. Create an input

Step 2. Create an output

Step 3. Verify output

Web

Overview

Step 1. Create an input

Step 2. Verify your input

Step 3. Set up your output

Step 4. Create a connection

Step 5. Verify your connection

Step 6. Track events

Step 7. Track user data

Step 8. Create a data plan

Python Quick Start

Step 1. Create an input

Step 2. Create an output

Step 3. Verify output

Media SDKs

Android

iOS

Web

Server SDKs

Node SDK

Go SDK

Python SDK

Ruby SDK

Java SDK

Tools

Linting Tools

mParticle Command Line Interface

Smartype

Guides

Partners

Introduction

Outbound Integrations

Firehose Java SDK

Inbound Integrations

Kit Integrations

Overview

Android Kit Integration

JavaScript Kit Integration

iOS Kit Integration

Compose ID

Glossary

Data Hosting Locations

Migrate from Segment to mParticle

Migrate from Segment to Client-side mParticle

Migrate from Segment to Server-side mParticle

Segment-to-mParticle Migration Reference

Rules Developer Guide

API Credential Management

The Developer's Guided Journey to mParticle

Guides

Getting Started

Create an Input

Start capturing data

Connect an Event Output

Create an Audience

Connect an Audience Output

Transform and Enhance Your Data

Platform Guide

The New mParticle Experience

The new mParticle Experience

The Overview Map

Observability

Observability Overview

Observability User Guide

Observability Span Glossary

Introduction

Data Retention

Connections

Activity

Live Stream

Data Filter

Rules

Tiered Events

mParticle Users and Roles

Analytics Free Trial

Troubleshooting mParticle

Usage metering for value-based pricing (VBP)

Analytics

Introduction

Setup

Sync and Activate Analytics User Segments in mParticle

User Segment Activation

Welcome Page Announcements

Settings

Project Settings

Roles and Teammates

Organization Settings

Global Project Filters

Portfolio Analytics

Analytics Data Manager

Analytics Data Manager Overview

Events

Event Properties

User Properties

Revenue Mapping

Export Data

UTM Guide

Query Builder

Data Dictionary

Query Builder Overview

Modify Filters With And/Or Clauses

Query-time Sampling

Query Notes

Filter Where Clauses

Event vs. User Properties

Group By Clauses

Annotations

Cross-tool Compatibility

Apply All for Filter Where Clauses

Date Range and Time Settings Overview

Understanding the Screen View Event

Analyses

Analyses Introduction

Segmentation: Basics

Getting Started

Visualization Options

For Clauses

Date Range and Time Settings

Calculator

Numerical Settings

Segmentation: Advanced

Assisted Analysis

Properties Explorer

Frequency in Segmentation

Trends in Segmentation

Did [not] Perform Clauses

Cumulative vs. Non-Cumulative Analysis in Segmentation

Total Count of vs. Users Who Performed

Save Your Segmentation Analysis

Export Results in Segmentation

Explore Users from Segmentation

Funnels: Basics

Getting Started with Funnels

Group By Settings

Conversion Window

Tracking Properties

Date Range and Time Settings

Visualization Options

Interpreting a Funnel Analysis

Funnels: Advanced

Group By

Filters

Conversion over Time

Conversion Order

Trends

Funnel Direction

Multi-path Funnels

Analyze as Cohort from Funnel

Save a Funnel Analysis

Explore Users from a Funnel

Export Results from a Funnel

Cohorts

Getting Started with Cohorts

Analysis Modes

Save a Cohort Analysis

Export Results

Explore Users

Saved Analyses

Manage Analyses in Dashboards

Journeys

Getting Started

Event Menu

Visualization

Ending Event

Save a Journey Analysis

Users

Getting Started

User Activity Timelines

Time Settings

Export Results

Save A User Analysis

Dashboards

Dashboards––Getting Started

Manage Dashboards

Dashboard Filters

Organize Dashboards

Scheduled Reports

Favorites

Time and Interval Settings in Dashboards

Query Notes in Dashboards

User Aliasing

Analytics Resources

The Demo Environment

Keyboard Shortcuts

Tutorials

Analytics for Marketers

Analytics for Product Managers

Compare Conversion Across Acquisition Sources

Analyze Product Feature Usage

Identify Points of User Friction

Time-based Subscription Analysis

Dashboard Tips and Tricks

Understand Product Stickiness

Optimize User Flow with A/B Testing

User Segments

APIs

User Segments Export API

Dashboard Filter API

IDSync

IDSync Overview

Use Cases for IDSync

Components of IDSync

Store and Organize User Data

Identify Users

Default IDSync Configuration

Profile Conversion Strategy

Profile Link Strategy

Profile Isolation Strategy

Best Match Strategy

Aliasing

Data Master

Group Identity

Overview

Create and Manage Group Definitions

Introduction

Catalog

Live Stream

Data Plans

Blocked Data Backfill Guide

Personalization

Predictive Attributes

Predictive Attributes Overview

Create Predictive Attributes

Assess and Troubleshoot Predictions

Use Predictive Attributes in Campaigns

Predictive Audiences

Predictive Audiences Overview

Using Predictive Audiences

Introduction

Profiles

Calculated Attributes

Calculated Attributes Overview

Using Calculated Attributes

Create with AI Assistance

Calculated Attributes Reference

Audiences

Audiences Overview

Real-time Audiences

Standard Audiences

Journeys

Journeys Overview

Manage Journeys

Download an audience from a journey

Audience A/B testing from a journey

Journeys 2.0

Warehouse Sync

Data Privacy Controls

Data Subject Requests

Default Service Limits

Feeds

Cross-Account Audience Sharing

Approved Sub-Processors

Import Data with CSV Files

CSV File Reference

Glossary

Video Index

Analytics (Deprecated)

Identity Providers

Single Sign-On (SSO)

Setup Examples

Settings

Debug Console

Data Warehouse Delay Alerting

Introduction

Developer Docs

Introduction

Integrations

Introduction

Rudderstack

Google Tag Manager

Segment

Data Warehouses and Data Lakes

Advanced Data Warehouse Settings

AWS Kinesis (Snowplow)

AWS Redshift (Define Your Own Schema)

AWS S3 Integration (Define Your Own Schema)

AWS S3 (Snowplow Schema)

BigQuery (Snowplow Schema)

BigQuery Firebase Schema

BigQuery (Define Your Own Schema)

GCP BigQuery Export

Snowflake (Snowplow Schema)

Snowplow Schema Overview

Snowflake (Define Your Own Schema)

APIs

Dashboard Filter API (Deprecated)

REST API

User Segments Export API (Deprecated)

SDKs

SDKs Introduction

React Native

iOS

Android

Java

JavaScript

Python

Object API

Developer Basics

Aliasing

CSV File Reference

This topic contains reference information and instructions for advanced techniques when importing data using CSV files.

Configuration Settings

Configuration Name	Data Type	Default	Description
Setting Name	string		If the event type is custom event, and every row in your file has the same event name, specify the name for all events. If also set in column `events.data.event_name`, the data in the file takes precedence.
Custom Event Type	string		If Event Type is Custom Event, and every row in your file is the same Event Type, use this to set the Type for all events. If also set in `events.data.custom_event_type`, the data in the file takes precedence.
Custom Manifest	string		To override the default format, include a custom manifest.
Expect Encrypted Files	Boolean	`false`	Only encrypted OR unencrypted files can be accepted, but not both. You must use PGP encryption with mParticle’s public key. See Drop encrypted files

Use a custom manifest

A custom manifest allows you to use files created by other systems without transformation. In the mParticle UI, when you’re configuring the Custom CSV input, you can provide a JSON manifest to describe how you want to map your CSV data to mParticle’s fields in one of two ways:

With a header row: Map your column names to mParticle’s column names.
Without a header row: Map your columns, which must appear in a specific order, to mParticle’s column names.

In order to guarantee that the new/changed manifest applies to your CSV files, please ensure a gap of about 5 minutes between the manifest change and uploading CSV files.

With a header row

If your CSV has a header row, you can map the column names in your header directly to mParticle fields. The manifest must set "hasHeaderRow": true and contain an array of columns objects, each giving a column name, an action ("keep" or "ignore") and a target mParticle field. The order of entries in the column array doesn’t need to match the order of columns in the CSV file.

Example with a header row

Assume that you drop the following manifest on the SFTP server:

{
  "hasHeaderRow": true, // REQUIRED
  "columns": [
    {
      "column": "Event Name",
      "action": "keep",
      "field": "events.data.event_name"
    },
    {
      "column": "Custom Type",
      "action": "keep",
      "field": "events.data.custom_event_type"
    },
    {
      "column": "Email",
      "action": "keep",
      "field": "user_identities.email"
    },
    {
      "column": "Facebook",
      "action": "ignore",
      "field": "user_identities.facebook"
    },
    {
      "column": "Home City",
      "action": "keep",
      "field": "user_attributes.$city"
    },
    {
      "column": "Category",
      "action": "keep",
      "field": "events.data.custom_attributes.category"
    },
    {
      "column": "Destination",
      "action": "keep",
      "field": "events.data.custom_attributes.destination"
    },
    {
      "column": "Time",
      "action": "keep",
      "field": "events.data.timestamp_unixtime_ms"
    },
    {
      "column": "Environment",
      "action": "keep",
      "field": "environment"
    }
  ]
}

Then assume that you drop a CSV file with the following header row and one row of data:

Event Name,Custom Type,Email,Facebook,Home City,Category,Destination,Time,Environment Viewed Video,other,h.jekyll.md@example.com,h.jekyll.md,London,Destination Intro,Paris,1466456299032,development

The resulting batch will be:

{
    "events" :
    [
        {
            "data" : {
                "event_name": "Viewed Video",
                "custom_event_type": "other",
                "custom_attributes": {
                    "category": "Destination Intro",
                    "destination": "Paris"                   
                },
                "timestamp_unixtime_ms": "1466456299032"
            },
            "event_type" : "custom_event"
        }
    ],
    "user_attributes" : {
        "$city": "London"
    },
    "user_identities" : {
        "email": "h.jekyll.md@example.com"
    },
    "environment" : "development"
}

Without a header row

If your CSV does not have a header row, you can map columns to mParticle fields by the order they appear in the CSV.

The manifest must include "hasHeaderRow": false and contain an array of columns objects, each giving an action ("keep" or "ignore") and a target mParticle field. For this method to work, you must be able to guarantee the same column order in each CSV file you upload.

The following is an example of the same custom manifest as the previous example, but without a header row:

{
    "hasHeaderRow": false, // REQUIRED
    "columns": [{
   		 "action": "keep",
   		 "field": "events.data.event_name"
   	 },
   	 {
   		 "action": "keep",
   		 "field": "events.data.custom_event_type"
   	 },
   	 {
   		 "action": "keep",
   		 "field": "user_identities.email"
   	 },
   	 {
   		 "action": "ignore",
   		 "field": "user_identities.facebook"
   	 },
   	 {
   		 "action": "keep",
   		 "field": "user_attributes.$city"
   	 },
   	 {
   		 "action": "keep",
   		 "field": "events.data.custom_attributes.category"
   	 },
   	 {
   		 "action": "keep",
   		 "field": "events.data.custom_attributes.destination"
   	 },
   	 {
   		 "action": "keep",
   		 "field": "timestamp_unixtime_ms"
   	 },
   	 {
   		 "action": "keep",
   		 "field": "environment"
   	 }
    ]
}

Encrypted files

The Custom CSV Feed can accept files encrypted with PGP, using mParticle’s Public Key. You can use software like GPG Tools for OS X or Windows. Never use web-based tools to encrypt, hash, or encode data.

To enable encryption, set the configuration setting Expect Encrypted Files to true.

It’s most efficient to send multiple files in a zipped, or gzipped format. Encrypt the final ZIP file instead of each individual file.

Use the public encryption key corresponding to your pod:

US1

EU1

AU1

US2

Sensitive data

Use the following public encryption key for sending sensitive data.

Processing speed

For files within recommended size limits, processing speed is consistent with the Events API: about 270 rows per second. To increase the processing speed for your CSV feed, contact mParticle Support.

Processing Behavior

Files aren’t guaranteed to be processed in sequence; files are not linked to one another. Each file is independent and there’s no way to indicate if two files were split from a master file.
You can observe how much data has been processed using Data Master and your outbound connections. There is no notification.
Once dropped, files start processing at any time. Deleting a file from the dropped folder is not a guarantee that it won’t be processed. Overwriting files can lead to partial or incomplete imports, or other errors may occur.
Rows may be batched together for processing. Thus, there may be fewer processed batches than rows. The only fields not considered unique identifiers for batching are event-specific, such as event name, custom event attributes, and the batch-level timestamp. If two rows have the exact same set of attributes and identifiers otherwise, then they may be batched together for processing.
Each file is processed beginning to end. A file is never split or read asynchronously. Header mappings are on a per-configuration basis and are applied to all potential files (if their headers are not valid already). There is no way to associate a mapping with either a filename or filename pattern.
Processed files are deleted within 30 days.

Was this page helpful?

Last Updated: December 20, 2024