Webengage internal tool to migrate other party historical data into webengage ecosystem
Project description
WebEngage Migration Tool
A lightweight tool to migrate historical user or event data into WebEngage ecosystem. Supports large datasets and can handle millions of rows, automatic batching, and adhere to WebEngage API rate limits.
Features
- Handles large files efficiently (2M+ rows)
- Batches events or users into groups of 25 adhering API rate limits
- Supports multiple datacenter routing (in, ksa, us)
- Built-in logger for tracking batch status - (success / failure)
Instructions
-
Data should always be in CSV format for both events and users.
-
Delete the first column ("converted JSON") before saving the file if you plan to resend the data to WebEngage.
Sending Users
-
Column 4: Reserve this column for Date of Birth. The header name should be "birthDate"
-
Columns 1 to 11: should contain the following fields: userId, firstName, lastName, birthDate, gender, email, phone, company, city, region, country You can shuffle these columns except for birthDate, which will always remain in column 4
-
birthDate values should follow this format:
yyyy-mm-dd(eg: 1999-12-25) -
Columns 12 onwards will contain custom user attributes.
Sending Events
-
Columns 1 to 3: should contain the following fields: userId, eventName, eventTime. Shuffling of the columns is allowed within this range.
-
eventTime should follow this format:
yyyy-mm-ddTHH:MM:SS-0800(eg: 1986-08-19T15:45:00-0800). -
Columns 4 onwards will contain event data.
-
Installation
To install the package run:
pip install webengage-migration
Usage
To initiate the migration process run the below command:
we --migrate {users/events} -f "{filename.csv}" -d {datacenter} -lc {license_code}
| Argument | Description | Type |
|---|---|---|
{users/events} |
Type of data you're migrating (users or events) |
⚠️ Required |
-f "filename.csv" |
Path to the input CSV file | ⚠️ Required |
-d {datacenter} |
WebEngage datacenter (in, ksa, default is us) |
⚙️ Optional |
-lc {license_code} |
Your WebEngage license code | ⚠️ Required |
Example:
Send users: we --migrate users -f "datafile.csv" -d in -lc in~~1234c456
Send events: we --migrate events -f "datafile.csv" -d in -lc in~~1234c456
Once migration is initiated, provide the API key for authorization, API key will be found in webengage dashboard for provided license code under:
Data Platforms → Integrations → Rest API → API KEY
WebEngage API Rate Limits
WebEngage limits bulk event / users API usage to:
- 25 events / users per API request
- 500 requests per minute (Total 12,500 events per minute)
This tool automatically respects the above limits by batching and throttling requests.
Legal Notice
This tool is an internal property of WebEngage and is strictly for migration purposes. It is owned by Nipun Patel (Copyright) and any misuse, unauthorized distribution, or external sharing will lead to legal consequences.
© WebEngage. All rights reserved.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file webengage_migration-0.1.0.tar.gz.
File metadata
- Download URL: webengage_migration-0.1.0.tar.gz
- Upload date:
- Size: 5.4 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.1.0 CPython/3.12.5
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
97d059d986c300031e147f7fdd1cd75b6613ebd0353a0054a7183e6c93d388cc
|
|
| MD5 |
911dad87e9d72053a6e771fe824959cf
|
|
| BLAKE2b-256 |
ba35136018c2b0d6cef92b6361bd5aba184fcfee3b74715acaf68e5b053d1c9b
|
File details
Details for the file webengage_migration-0.1.0-py3-none-any.whl.
File metadata
- Download URL: webengage_migration-0.1.0-py3-none-any.whl
- Upload date:
- Size: 5.9 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.1.0 CPython/3.12.5
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
1f0e8bd04687c6a1d728d55b377821099d56e2eb989bf86e14c0cec7f9adb9c8
|
|
| MD5 |
e1198e066f0f8a0a7a9ea0cbe67bf7a4
|
|
| BLAKE2b-256 |
f1efd80ae75e194c5c08a0cd9568e1fff4bace8df821b3f810281a2e5afe0021
|