Skip to main content

Webengage internal tool to migrate other party historical data into webengage ecosystem

Project description

WebEngage Migration Tool

A lightweight tool to migrate historical user or event data into WebEngage ecosystem. Supports large datasets and can handle millions of rows, automatic batching, and adhere to WebEngage API rate limits.


Features

  • Handles large files efficiently (2M+ rows)
  • Batches events or users into groups of 25 adhering API rate limits
  • Supports multiple datacenter routing (in, ksa, us)
  • Built-in logger for tracking batch status - (success / failure)

Instructions

  • Data should always be in CSV format for both events and users.

  • Delete the first column ("converted JSON") before saving the file if you plan to resend the data to WebEngage.

    Sending Users

    • Column 4: Reserve this column for Date of Birth. The header name should be "birthDate"

    • Columns 1 to 11: should contain the following fields: userId, firstName, lastName, birthDate, gender, email, phone, company, city, region, country You can shuffle these columns except for birthDate, which will always remain in column 4

    • birthDate values should follow this format: yyyy-mm-dd (eg: 1999-12-25)

    • Columns 12 onwards will contain custom user attributes.

    Sending Events

    • Columns 1 to 3: should contain the following fields: userId, eventName, eventTime. Shuffling of the columns is allowed within this range.

    • eventTime should follow this format: yyyy-mm-ddTHH:MM:SS-0800 (eg: 1986-08-19T15:45:00-0800).

    • Columns 4 onwards will contain event data.


Installation

To install the package run:

pip install webengage-migration

Usage

To initiate the migration process run the below command:

we --migrate {users/events} -f "{filename.csv}" -d {datacenter} -lc {license_code}

Argument Description Type
{users/events} Type of data you're migrating (users or events) ⚠️ Required
-f "filename.csv" Path to the input CSV file ⚠️ Required
-d {datacenter} WebEngage datacenter (in, ksa, default is us) ⚙️ Optional
-lc {license_code} Your WebEngage license code ⚠️ Required

Example:

Send users: we --migrate users -f "datafile.csv" -d in -lc in~~1234c456

Send events: we --migrate events -f "datafile.csv" -d in -lc in~~1234c456


Once migration is initiated, provide the API key for authorization, API key will be found in webengage dashboard for provided license code under:

Data Platforms → Integrations → Rest API → API KEY

Usage Example


WebEngage API Rate Limits

WebEngage limits bulk event / users API usage to:

  • 25 events / users per API request
  • 500 requests per minute (Total 12,500 events per minute)

This tool automatically respects the above limits by batching and throttling requests.


Legal Notice

This tool is an internal property of WebEngage and is strictly for migration purposes. It is owned by Nipun Patel (Copyright) and any misuse, unauthorized distribution, or external sharing will lead to legal consequences.


© WebEngage. All rights reserved.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

webengage_migration-0.1.0.tar.gz (5.4 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

webengage_migration-0.1.0-py3-none-any.whl (5.9 kB view details)

Uploaded Python 3

File details

Details for the file webengage_migration-0.1.0.tar.gz.

File metadata

  • Download URL: webengage_migration-0.1.0.tar.gz
  • Upload date:
  • Size: 5.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.12.5

File hashes

Hashes for webengage_migration-0.1.0.tar.gz
Algorithm Hash digest
SHA256 97d059d986c300031e147f7fdd1cd75b6613ebd0353a0054a7183e6c93d388cc
MD5 911dad87e9d72053a6e771fe824959cf
BLAKE2b-256 ba35136018c2b0d6cef92b6361bd5aba184fcfee3b74715acaf68e5b053d1c9b

See more details on using hashes here.

File details

Details for the file webengage_migration-0.1.0-py3-none-any.whl.

File metadata

File hashes

Hashes for webengage_migration-0.1.0-py3-none-any.whl
Algorithm Hash digest
SHA256 1f0e8bd04687c6a1d728d55b377821099d56e2eb989bf86e14c0cec7f9adb9c8
MD5 e1198e066f0f8a0a7a9ea0cbe67bf7a4
BLAKE2b-256 f1efd80ae75e194c5c08a0cd9568e1fff4bace8df821b3f810281a2e5afe0021

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page