When setting up Warehouse Sync, it is possible to encounter problems with one or more components of your configuration:
Following are several troubleshooting guides for each of these problem categories. If you are still encountering issues after following the appropriate steps below, contact mParticle support.
Connectivity issues are often the result of an incomplete or incorrect first-time configuration.
Before troubleshooting, verify the following:
POST {baseURI}/connections
API call. Are they correct for the data warehouse instance you are trying to connect to?Errors or incompatibility in the SQL syntax of your data model will return errors and prevent the sync from succeeding.
Before troubleshooting, run the SQL query outside of mParticle. If it doesn’t run successfully or return the expected results, the issue is likely in your query, independent of Warehouse Sync.
Verify your SQL syntax. While most data warehouses support common SQL syntax, it is possible to encounter exceptions in SQL extension for your warehouse. For example:
SELECT current_timestamp AS \"tstamp\" FROM tableXYZ ... "
in your SQL query will fail if load_timestamp_field_name
is tstamp
in your data model.Workaround 1, Remove the explicit identifier " "
:
SELECT current_timestamp AS tstamp FROM tableXYZ ... "
load_timestamp_field_name": "tstamp"
Workaround 2, force UPPER CASE:
SELECT current_timestamp AS \”TSTAMP\" FROM tableXYZ ... "
"load_timestamp_field_name": "TSTAMP"
If the error is related to the timestamp field in the query, ensure that:
Pipeline issues are typically caused by security problems, the timestamp field provided in the data model, or other factors with the environment.
The inputs/warehouse/pipelines/{{INGEST_PIPELINE_SLUG}}/status API returns some type of error message. For example:
The inputs/warehouse/pipelines/{{INGEST_PIPELINE_SLUG}}/status
API returns 0
for the parameters "records_extracted"
, "records_mapped"
, and "event_batches_generated"
. For example:
{
"pipeline_id": "example-pipeline-2",
"workspace_id": 5328,
"is_faulted": false,
"is_connection_faulted": false,
"is_connection_active": true,
"is_active": false,
"latest_pipeline_run":
{
"pipeline_run_id": 2171,
"id": "example-pipeline-2",
"workspace_id": 5328,
"logical_date": "2022-12-09T21:00:00Z",
"start_date": "2022-12-09T21:00:02Z",
"end_date": "2022-12-09T21:00:02Z",
"data_interval_start": "2022-12-09T21:00:00Z",
"data_interval_end": "2022-12-09T22:00:00Z",
"pipeline_run_type": "scheduled",
"pipeline_run_state": "failed",
"is_externally_triggered": false,
"records_extracted": 0,
"records_mapped": 0,
"event_batches_generated": 0,
"created_on": "2022-12-09T22:00:06.98",
"created_by": "developer@mparticle.com",
"last_modified_on": "2022-12-09T22:00:28.187",
"last_modified_by": "developer@mparticle.com"
}
}
Before troubleshooting, verify the following:
Importing and mapping problems usually result from incorrect mapping between data rows in the warehouse and user profilse or attributes in mParticle.
Before troubleshooting, verify the following:
Provide mParticle support or your account representative with the event batch JSON object from your mParticle Livestream, or the MPID and batch ID for the event, as well as a CSV of the source data
If a table schema changes and validation is still occurring, you may need to wait 24 hours for the cache in BigQuery to clear and reset before trying again.
Was this page helpful?