Skip to main content

Webengage internal tool to migrate other party historical data into webengage ecosystem

Project description

WebEngage Migration Tool

A lightweight tool to migrate historical user or event data into WebEngage ecosystem. Supports large datasets and can handle millions of rows, automatic batching, and adhere to WebEngage API rate limits.


Features

  • Handles large files efficiently (2M+ rows)
  • Batches events or users into groups of 25 adhering API rate limits
  • Supports multiple datacenter routing (in, ksa, us)
  • Built-in logger for tracking batch status - (success / failure)

Instructions

  • Data should always be in CSV format for both events and users.

  • Delete the first column ("converted JSON") before saving the file if you plan to resend the data to WebEngage.

    Sending Users

    • Column 4: Reserve this column for Date of Birth. The header name should be "birthDate"

    • Columns 1 to 11: should contain the following fields: userId, firstName, lastName, birthDate, gender, email, phone, company, city, region, country You can shuffle these columns except for birthDate, which will always remain in column 4

    • birthDate values should follow this format: yyyy-mm-dd (eg: 1999-12-25)

    • Columns 12 onwards will contain custom user attributes.

    Sending Events

    • Columns 1 to 3: should contain the following fields: userId, eventName, eventTime. Shuffling of the columns is allowed within this range.

    • eventTime should follow this format: yyyy-mm-ddTHH:MM:SS-0800 (eg: 1986-08-19T15:45:00-0800).

    • Columns 4 onwards will contain event data.


Installation

To install the package run:

pip install webengage-migration

Usage

To initiate the migration process run the below command:

we --migrate {users/events} -f "{filename.csv}" -d {datacenter} -lc {license_code}

Argument Description Type
{users/events} Type of data you're migrating (users or events) ⚠️ Required
-f "filename.csv" Path to the input CSV file ⚠️ Required
-d {datacenter} WebEngage datacenter (in, ksa, default is us) ⚙️ Optional
-lc {license_code} Your WebEngage license code ⚠️ Required

Example:

Send users: we --migrate users -f "datafile.csv" -d in -lc in~~1234c456

Send events: we --migrate events -f "datafile.csv" -d in -lc in~~1234c456


Once migration is initiated, provide the API key for authorization, API key will be found in webengage dashboard for provided license code under:

Data Platforms → Integrations → Rest API → API KEY

Usage Example


WebEngage API Rate Limits

WebEngage limits bulk event / users API usage to:

  • 25 events / users per API request
  • 500 requests per minute (Total 12,500 events per minute)

This tool automatically respects the above limits by batching and throttling requests.


Legal Notice

This tool is an internal property of WebEngage and is strictly for migration purposes. It is owned by Nipun Patel (Copyright) and any misuse, unauthorized distribution, or external sharing will lead to legal consequences.


© WebEngage. All rights reserved.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

webengage_migration-0.1.1.tar.gz (5.4 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

webengage_migration-0.1.1-py3-none-any.whl (5.9 kB view details)

Uploaded Python 3

File details

Details for the file webengage_migration-0.1.1.tar.gz.

File metadata

  • Download URL: webengage_migration-0.1.1.tar.gz
  • Upload date:
  • Size: 5.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.12.5

File hashes

Hashes for webengage_migration-0.1.1.tar.gz
Algorithm Hash digest
SHA256 98971018085e8f3f0f1fb1bfadac2635ae00a84642fed89a180bfaa30b581674
MD5 dcf630c29f01cd18cbb9cc9a04ddd60c
BLAKE2b-256 d6e4ca362f463fd180e419d92e1aad23d0c9f0feadf74084a7d04f804ee2c73d

See more details on using hashes here.

File details

Details for the file webengage_migration-0.1.1-py3-none-any.whl.

File metadata

File hashes

Hashes for webengage_migration-0.1.1-py3-none-any.whl
Algorithm Hash digest
SHA256 d6b2871742149bd128c28218e8a3f555a7863731118bbbd3451a712a0c543bf4
MD5 64158c8be708bd9313b77b741d4ff6a0
BLAKE2b-256 c4d16df0ee4fdab7093a4342d68dad134c00fce057b70863e348c5cf121ea624

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page