Skip to main content

geolens (CLI)

Apache-2.0 command-line interface for the GeoLens API.

Log in, scan local directories of spatial data, publish files, apply manifest-driven catalogs, replace and refresh datasets, inspect source status, run PostGIS analysis operations, export STAC metadata and plan an ArcGIS migration against any GeoLens instance.

Install

pip install geolens-cli      # installs the `geolens` command; Python 3.11 or newer

Quick start

geolens login https://geolens.example.com/api
geolens publish ./data/cities.geojson
geolens status DATASET_ID        # the UUID at the end of the URL publish printed

The CLI wraps the geolens Python SDK package.

Migrate from ArcGIS

Three commands move an ArcGIS Online organization or a self-hosted ArcGIS portal into GeoLens. They are written for an ArcGIS organization administrator, because a portal search only returns items shared with the caller and a non-admin never sees other people's private items. On the GeoLens side the session must be an admin (geolens login as one), since the run reads the user list and hands ownership to the people the plan names.

geolens arcgis inventory --portal-url https://yourorg.maps.arcgis.com --scope org -o ./run
geolens arcgis plan ./run/arcgis-inventory.json --owner-map owners.json
geolens arcgis migrate ./run/arcgis-migration-plan.json
  1. inventory reads the portal and writes arcgis-inventory.json and a readable report.
  2. plan turns it into arcgis-migration-plan.json, which you review and edit. Each item has an action (import_layers, import_file, import_map, export_only or skip), a target owner, visibility, collections and title. owners.json maps ArcGIS usernames to GeoLens usernames or emails; an empty value means no owner yet.
  3. migrate runs the plan one item at a time, in dependency order. Add --dry-run first to check the plan and the owners without importing anything. With --username, the dry run also makes the one ArcGIS sign-in the run starts with, so a refused password exits 3 before the real run; a token isn't checked.

What each action does

  • import_layers: GeoLens imports the feature layers and tables of a hosted or federated service itself; the CLI never contacts the service.
  • import_file: a CSV, GeoJSON, Shapefile or File Geodatabase item is downloaded from the portal and uploaded. It is bounded by the instance's upload size limit; an item over it is skipped with its size.
  • export_only: items GeoLens can't host (dashboards, forms, StoryMaps, Hub pages and so on) are saved under export/<type>/<item id>/ next to the state file, with item.json, the item's data, its resources and, for a story, a readable story.md. export/INDEX.md lists them, and the source files of hosted layers, with what to use instead. Uploaded files keep their type: a CSV is saved as data.csv and a PDF as data.pdf, from the portal's file name or the item type, and data.bin only when neither gives one. A story.md links each map to the GeoLens map made from it at the instance's PUBLIC_APP_URL, or at the instance URL without /api when PUBLIC_APP_URL isn't set, so set it on an install that serves the web app from another origin. A map that hasn't been migrated is linked to its ArcGIS item page instead, which stops working when the portal is retired, so replace those links before cutover. --export-cap-mb (default 200) limits each item. An export can finish with files missing: one over the cap, JSON that can't be redacted, a request the portal refused, or an unsafe resource name. The item notes, the report and INDEX.md (status partial or complete, with errors) say which, so check them before retiring the ArcGIS source. A network or server error while exporting leaves the item failed, and a rerun retries it.
  • import_map: a Web Map becomes a GeoLens map through the server's web map translator. Its operational layers are matched to the datasets the run imported for their service items, so the map waits for those items (depends_on); a layer from another organization or a service that was not imported is left out. Basemaps follow the plan's basemap_map. A matched layer that the web map leaves to its source is drawn with the renderer saved on its layer item, read from the portal, or else its service layer's own renderer, read only from a host the run trusts with its token (--trust-service-host). webmap-loss/<item id>.json next to the state file lists what was not carried over, and the report lists the path under its loss reports and in the follow-ups. The state keeps each ArcGIS layer id's map layer id. The map gets the web map item's thumbnail, read from the portal with the run's credential, and a rerun gives it to a migrated map that has no thumbnail; a thumbnail already on the map is never replaced. A PNG or JPEG over the size GeoLens takes for a thumbnail, any other image type, or a failed read or upload is reported on the item and leaves the map without one. A dry run notes each new map that would get one.

Items are imported as you, then handed to the owner the plan names. Each target user needs a role with upload (datasets) or edit_metadata (maps). --unmapped-owner names an owner for items whose plan owner can't take them yet: the mapping is empty, the user hasn't signed in to GeoLens yet (an OIDC user before first sign-in), is inactive, or has a role without that permission, such as the viewer role new OIDC users get. Those items go to the --unmapped-owner for now and are recorded as waiting; a later run hands each one to its owner once the owner can take it, or to a new --unmapped-owner if you pass another. For a role, the run asks you to promote the user to editor instead of letting the transfer fail, and when GeoLens itself refuses a transfer because of the user's role, the item fails with the same request. Without --unmapped-owner such items stop the run before anything is imported, except a role without the permission, which is a warning and the server decides. When the refusal is an empty owner_map entry and GeoLens already has an account with the same username and the email the inventory read for that owner, the message names it; migrate doesn't map it for you, because an owner gets the item itself, which is more than group membership. A migrated item whose plan owner you later edit to an empty value, a name that matches no user or one that matches two is listed in the output and the report, and stays with its current owner. Folders become collections (defaults.collection_from in the plan: folder or none). A folder whose name another collection holds is filed under <portal name> - <folder>. When that name is held too, the run stops before importing anything, as it does when a group's name and ArcGIS - <name> are both taken; rename the folder in the items' target.collections, or set collection_from to none.

Sharing and groups

ArcGIS sharing maps to GeoLens visibility: private stays private, org becomes internal, public stays public, and an item shared only with groups becomes restricted, shared with a GeoLens group for each of its ArcGIS groups. Under the restrict_public_visibility setting, a public item whose owner isn't a GeoLens admin becomes internal instead; when a run has public items and can't read that setting, it stops before importing anything. Each ArcGIS group's entry in the plan's group_map decides its GeoLens group through geolens_group. Left null, migrate creates a group named after the ArcGIS group (or ArcGIS - <name> when another group already has that name; it never joins a group it didn't create) and adds the members the inventory read. A member resolves like an owner: through owner_map when it lists them, otherwise by the same username or email in GeoLens, but only when the email the inventory read for that ArcGIS user, if any, is the account's. --dry-run and the report list which GeoLens user each ArcGIS member became, so check that a same-named account belongs to the same person. A member who isn't a GeoLens user yet, or whom GeoLens refused to add, stays pending and is added by a later run (the output counts members without a GeoLens account once and lists their names, and names geolens admin users import --from-inventory as the way to create them; the report keeps one line per member and group); a refused add also makes the run exit 1. Set geolens_group to an existing group's exact name to share with that group instead (a name that matches several groups apart from case is refused); its members are left as they are, since adding to it would also give them the rest of what it can read, and the ArcGIS members it lacks are listed. An empty string maps the ArcGIS group to no GeoLens group, and an item none of whose groups maps to one becomes private. plan also writes each entry's collection, the ArcGIS group's title, which names a group migrate creates only when the inventory has no title for it, and visibility, the group's ArcGIS access, which migrate doesn't read. A rerun follows the plan: an item gets the groups its plan entry now names, and loses the groups migrate itself added that the entry no longer names (none while its groups weren't read). Groups shared with an item in GeoLens stay, even when the plan later names them too, and a group someone removes from an item in GeoLens isn't added again or removed later. GeoLens makes a private dataset or map shared with any group restricted, so one the plan keeps private while groups shared in GeoLens remain on it is reported instead of changed. A rerun that can't bring an item up to date leaves its groups as they are and lists the changes still waiting. Sharing replaces an item's whole group list, so a group change someone makes to the same item while migrate shares it can be overwritten; run it while nobody is editing sharing. A map is shared with its groups without widening its layers, which keep the sharing of their own items. A member migrate added to a group it created is removed again on a rerun once the ArcGIS member it was added for has left the group. A member still listed but no longer resolving (renamed, deactivated, or mapped to someone else in owner_map) keeps the account it was added as, nothing is removed while a group's members weren't read, and anyone added or removed in GeoLens stays as they are. The output, --dry-run and the report list each removal. A state file from before version 4 doesn't say which members migrate added, so the next run lists them once for you to check by hand and removes none of them. It doesn't say which groups migrate shared an item with either, so a group the plan no longer names that is still on an item is listed once for you to check, and left. A rerun prints each item it shared with a group or took a group off, and the report lists them under "Item sharing changed by this run".

Only an organization administrator sees every group an item is shared with, and a group can hide its members from everyone but its managers and administrators. plan lists items shared only with groups that a non-admin's inventory may have missed, and the report lists, as under_shared follow-ups, items whose groups weren't all read or carried over. A rerun changes a migrated map's groups the same way while the map is as migrate left it and both it and its plan entry are private or restricted; a map changed in GeoLens since, or one whose plan now gives it another visibility, is reported rather than changed.

Users

Owners and group members have to be GeoLens users before migrate can hand them items or add them to groups. geolens admin users import users.csv creates them from a CSV with a header row naming username, email and role (admin, editor or viewer; empty means viewer), and skips anyone GeoLens already has with that username and email. A row whose username belongs to an account with another email, or whose email belongs to another account, is a conflict: it isn't created, and the run exits 1. --from-inventory arcgis-inventory.json takes the users from an inventory instead: every item owner as an editor, with the email the inventory read, and every member of a group an item is shared with as a viewer. Add --seed-csv users.csv to write those rows to a new file you can review and edit, creating nothing. It refuses to overwrite a file, and a cell starting with =, +, - or @ gets a leading ' so a spreadsheet doesn't run it as a formula. The import drops that ' again, so a name or email that really starts with ' followed by one of those characters can't be given in a CSV. --dry-run says what would be created. The command needs a GeoLens admin session and exits 3 without one. It exits 1 when any user couldn't be created.

Each new user gets a generated 72-character password, the longest GeoLens accepts, so it meets any password policy. --print-passwords shows each one once on standard output; nothing else writes them anywhere, so without it nobody knows them and you set one for each user through the admin password reset (Admin, Users) and pass it on.

An instance that signs in through an identity provider (OpenID Connect, SAML or a sign-in extension) creates its users at their first sign-in, so the import creates nobody there: those owners and members stay pending, and a migrate rerun after they sign in hands them their items and adds them to their groups. --link-by-email creates them anyway, with a random password nobody is shown, so their first sign-in links to the account by email. GeoLens links only when the provider sends email_verified; without it that sign-in is refused. Users without an email stay pending. GeoLens counts an email an admin enters as verified, so whoever signs in through the provider with that address gets the account: check every email in a seeded CSV before importing it with --link-by-email. Ctrl-C stops before the next user and still prints the passwords made so far.

Credentials

Public items need none. For anything else pass --token, ARCGIS_TOKEN or --token-stdin, or --username with the password in ARCGIS_PASSWORD, --password-stdin or a prompt. A token made from a password is renewed before it expires. A token you pasted is not: when it runs out the run stops, writes the state file and tells you to resume with a fresh one. Accounts that sign in with SAML or OpenID Connect have no password to give here, so use a token. Tokens and passwords are never written to the state file, the report or the export folder.

A private service hosted on a federated ArcGIS Server whose host differs from the portal's (or from arcgis.com) only receives the token when you name its host with --trust-service-host <host> (repeatable); without it the item fails before any import and the error names the host to pass.

GeoLens needs REDIS_URL set to take a token for a private service import; without it the instance answers 503 and the run stops with that message.

Updating the plan

Run plan again on a newer inventory to bring an existing plan up to date. The plan records the defaults it gave each item, so the merge can tell your edits apart: a value you edited stays, and one you left alone takes the new default. When both changed, your value stays and plan prints what the inventory now gives. An edit stays an edit until you set it back to its default, even when the portal later changes to match it. New portal items are added at the end with defaults; the rest keep your order. An item gone from the portal stays in the plan with "removed": true; migrate doesn't run it again, and what it already migrated stays in GeoLens. Only a complete --scope org inventory read by an organization administrator marks items removed, since anyone else misses items not shared with them. Any other inventory (partial, truncated, user scope, anonymous, a non-admin account, or one written before the CLI recorded the account's role) marks the items it doesn't list "unlisted": true, and migrate leaves those pending, so the run exits 1 until a merge lists them again. If you never run a complete org inventory as an admin, an item deleted from the portal stays unlisted; set its action to skip, or delete it from the plan, to finish. A non-admin also may not see every group an item is shared with, so its inventory only adds groups to an item's groups; a group it doesn't list is kept until an admin's inventory leaves it out. plan and migrate refuse an inventory of another ArcGIS organization than the plan's, which matters on www.arcgis.com, where every organization shares one URL; a plan from an earlier version records no organization, so only its portal URL is checked. Your owner_map, group_map, basemap_map and plan defaults entries win over the inventory and --owner-map. To leave an item out, set its action to skip: an item deleted from the plan comes back with defaults on the next merge. --force replaces the plan with fresh defaults instead.

Plans written by earlier versions still migrate. They record no defaults and no inventory scope, so their items keep every value as an edit on every merge, depends_on included. Delete an item from the plan, or use --force, to give it defaults that follow the portal.

Resuming and reports

Progress is saved in arcgis-migration-state.json (mode 0600) after every item. Run the same command again to resume: failed items are retried (--no-retry leaves them alone), and layers already imported are never imported twice. --only <item id> runs one item plus the unfinished items it depends on. Only one run can use a state file at a time. A run that finds no state file at its path checks whether GeoLens already holds datasets imported from the plan's items, and when it does it stops before importing anything and names a few of them, since it would import them all again. Point --state at the earlier run's state file, or pass --start-fresh to migrate them again anyway.

A rerun first hands waiting items to their owners, then imports what isn't finished, then brings every item migrated before up to date. Each dataset of a service item is refreshed from its source with the run's ArcGIS credential, so private items need --token or --username on every rerun. A data file item is downloaded again and replaces its datasets. An export_only item is exported again when the portal changed it since its bundle was written, which the new bundle replaces in one step; if that export fails, the earlier bundle stays and the item is reported as not up to date. A service item is probed even when it had nothing to import, so a layer added since is reported. A migrated web map is not translated again: one whose portal item changed since its import is reported as not up to date, so you can update the GeoLens map by hand. The item's metadata is written again on each dataset that a refresh or replacement published, and a field the portal has emptied since is emptied too, unless it was edited in GeoLens. A visibility the portal's sharing changed since the import is applied to each dataset, unless it was changed in GeoLens or the state file can't tell, which is reported instead. Every service item is refreshed, because an item's modified date doesn't move when its features are edited. A refresh that stops for review (live edits, a destructive schema change, an empty result) publishes nothing: the run reports it with the geolens refresh <dataset> --accept-blocked-run <run> command that accepts it, and never accepts it for you. A rerun recognises the run you accept it with, and a refresh GeoLens accepted but whose answer was lost, as this command's own. A migrated web map whose owner was changed in GeoLens is reported, and one waiting for its planned owner is not handed over after such a change. --only limits the rerun to the items it names.

A rerun never overwrites a change made in GeoLens. After each write the state file records the dataset's last update, feature count and data revision, which counts every feature and column edit, your own included. Each dataset is compared right before its own refresh or replacement; one whose values differ (edited, transferred or refreshed in GeoLens) is reported and left alone, and so is one deleted there. The request carries the recorded revision, so GeoLens stops the publication for review if the data is edited after the check, and a file replacement is checked again after its upload. A GeoLens too old to report data revisions gets no refresh or replacement at all; each dataset is reported until you upgrade it. A refresh that doesn't publish leaves the dataset's record as it was, and one edited while its refresh ran is reported and not recorded. Set the item's action to skip to stop bringing it up to date. Change a finished export_only or skip item's action to import_file, import_layers or import_map and the next run imports it (the export bundle stays on disk but is no longer tracked); other action changes on a finished item aren't acted on. A state file from an earlier version has no recorded values, so a dataset is taken as unchanged only when your GeoLens account made its last change, before the item finished. The state file is now version 4, which an earlier CLI refuses to read rather than rewrite.

A run that gets as far as the item loop writes arcgis-migration-report.md and .json next to the state file, also when it is interrupted or stops on a failed request. --dry-run writes none, and neither does a run refused before it starts (an unusable plan, state file or owner mapping, or a failed GeoLens check). The report covers every item of the plan: the manual follow-ups (failed items, skipped files and layers, group-only sharing, partial exports, items that can't run yet or wait on others) and the unsupported items with where each was saved.

Exit codes:

  • 0: no item of the plan is failed, no item the run selected was left waiting or unable to run, and no group membership change failed; or a --cutover run that didn't stop refused nothing, because every check that failed was accepted with its --accept-* flag, and no group membership change failed. Skipped items (a file over the upload limit or with too many layers) and pending items the run did not select, such as items that can't run yet, don't change it, so read the report's counts and follow-ups before treating the migration as complete.
  • 1: an item of the plan is failed (including one this run left out with --only or --no-retry, and a portal download or GeoLens server error once items are running), an item the run selected was left pending because its action can't run yet or it waits on another item, a migrated item couldn't be brought up to date or handed to its owner, a group membership change failed, --cutover refused (any failing check left unaccepted), or the run stopped (for example on an expired pasted token).
  • 2: the plan, state file, owner mapping or credentials couldn't be used, the GeoLens session isn't an admin's, a group or collection name the run needs is taken, there is no state file while GeoLens holds datasets migrated from the plan's items, or the portal URL redirected before any item ran. --dry-run exits 2 on a taken name or those datasets too. A redirect refused once items are running fails that item (exit 1).
  • 3: authentication failed before any item ran (the GeoLens checks, or an ArcGIS password sign-in), or the server rejected a GEOLENS_TOKEN at any point, which stops the run with this code. A stored login that is rejected once items are running fails that item (exit 1).
  • 4: the portal couldn't be reached while signing in, or a GeoLens request couldn't connect or timed out at any point, which stops the run with this code. The exception is a timeout on the status poll of an import job: it is retried until the poll's deadline, and a deadline that runs out fails that item (exit 1). A portal download that fails once items are running also fails only that item (exit 1).
  • 5: a server error from the GeoLens checks, before any item ran.
  • 130: interrupted; run the same command to resume.

For a large migration, run the GeoLens worker with WORKER_CONCURRENCY=2 or more, since each layer import and refresh is a background job.

Cutover

A migration ends with one last run after the portal stops changing, marked with --cutover:

  1. Tell the portal's users that it is frozen, and stop edits to its items and services.
  2. Run inventory --scope org again as an organization administrator, then plan on it, so the plan holds everything the portal has now.
  3. Run migrate --dry-run --cutover to see what the cutover would refuse, and fix what you can: retry failed items, map owners, promote owners to editor, accept or fix refreshes that stopped for review.
  4. Run migrate --cutover with the same credentials as before. It runs like any rerun, then checks what wouldn't carry over and marks the run as the final one only when nothing is left, or when you accept each loss:
    • --accept-unfinished-items: a plan item isn't done (failed, skipped, can't run yet or waits on another item), is unlisted (even a finished one, since a rerun doesn't bring it up to date) or isn't up to date (its refresh failed or stopped for review, or GeoLens changed it). Items set to skip and items gone from the portal don't count.
    • --accept-incomplete-inventory: the plan wasn't last merged from a complete --scope org inventory read by an organization administrator, or the portal has items of the organization changed since that inventory started reading it, by the portal's own clock, so the plan may miss or misstate them. An inventory written by an earlier version, or by a portal that sent no time, records no start and fails this check; run it again. The run asks the portal with its ArcGIS credential, which must be an organization administrator's; without one, or when the portal can't answer, this check fails too.
    • --accept-attachment-loss: a migrated layer has attachments, or its service doesn't say. GeoLens doesn't import attachments yet.
    • --accept-unresolved-users: an item still waits for its owner, its plan owner doesn't name one GeoLens user, or its plan now names another owner than the one it was given; or a member of an ArcGIS group isn't in its GeoLens group yet.
    • --accept-under-sharing: an item shared only with ArcGIS groups may reach fewer people in GeoLens: the inventory wasn't read by an organization administrator, one of its groups maps to no GeoLens group, or a group's members couldn't be read.
  5. Read arcgis-migration-report.md, which records the cutover and the losses you accepted, and keep it with the state file. Then retire the portal. A refresh after that fails with the service's error, and the datasets stay as they are.

A refused cutover exits 1 and leaves the run unmarked; nothing it imported or refreshed is undone. A marked cutover exits 0 even when the items you accepted are still failed or not up to date, and the report lists them. --cutover runs the whole plan, so it can't be combined with --only.

Documentation

The full command reference, manifest schema, exit codes and CI setup are in the CLI guide.

Metadata

Release files for geolens-cli 1.29.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for geolens-cli 1.29.1
File Size Uploaded Provenance Actions
geolens_cli-1.29.1.tar.gz 501.7 kB Without provenance Details for geolens_cli-1.29.1.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for geolens-cli 1.29.1
File Interpreter ABI Platform Actions
geolens_cli-1.29.1-py3-none-any.whl Python 3 none any Details for geolens_cli-1.29.1-py3-none-any.whl

Total release size: 774.2 kB

Release files / geolens_cli-1.29.1.tar.gz

Download URL geolens_cli-1.29.1.tar.gz
Size 501.7 kB
Tags Source
SHA-256 checksum
How to use checksums
8e2b67d3bf390d11798eea155a0e3ad562e8259f942ba7b8609d2093cbd7c2a7
BLAKE2b-256 checksum
How to use checksums
6ba67011ce27a277fb555d607b25323fa6b68309693bed2e9c5b7d6f7d3f72b0
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via uv/0.10.2 {"installer":{"name":"uv","version":"0.10.2","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"26.04","id":"resolute","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Release files / geolens_cli-1.29.1-py3-none-any.whl

Download URL geolens_cli-1.29.1-py3-none-any.whl
Size 272.5 kB
Tags Python 3
SHA-256 checksum
How to use checksums
b42f138b8c3267f0bf18d95b3a87e3b98360fa52c56ff3a9a74b8351801e5da8
BLAKE2b-256 checksum
How to use checksums
02a77a3bb48518205707f76ae8d5405d4a3fbd2e0be92d26d2573d6cae28db70
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via uv/0.10.2 {"installer":{"name":"uv","version":"0.10.2","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"26.04","id":"resolute","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Security for geolens-cli 1.29.1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

1.29.1 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page