geolens (CLI)

Apache-2.0 command-line interface for the GeoLens API.

Login, scan local directories of spatial data, apply manifest-driven catalogs, publish vector or raster files, replace an uploaded dataset's data from a file, refresh remote service datasets, inspect source status, run PostGIS analysis operations, and export STAC metadata against any GeoLens instance.

See docs.getgeolens.com for the full command reference.

Quickstart

pip install geolens-cli
geolens login https://geolens.example.com/api
geolens scan ./data
geolens init
geolens validate geolens.yaml
geolens schema --output geolens-manifest-v1.schema.json
geolens apply --dry-run geolens.yaml
geolens apply geolens.yaml
geolens publish ./data/cities.geojson
geolens replace <dataset-id> ./data/cities-updated.geojson --wait
geolens status <dataset-id>
geolens refresh <dataset-id> --wait
geolens analysis preview <dataset-id> --operation buffer --distance 500 > ring.geojson
geolens analysis materialize <dataset-id> --operation buffer --distance 500 --title "500 m ring"  # waits for the job; --timeout to bound it
geolens export stac <dataset-id> -o cities.stac.json
geolens arcgis inventory --portal-url https://<org>.maps.arcgis.com -o ./inventory

For a one-command quickstart, run geolens publish examples/manifests/first-catalog/city-parks.geojson against a running stack. See the full walkthrough at docs.getgeolens.com.

The CLI consumes the geolens Python SDK package. Manifest apply posts to the generated POST /ingest/manifest/apply contract through the SDK-owned client transport rather than a hand-rolled HTTP client.

Apply, replace, and refresh

geolens apply reconciles declared catalog configuration. It re-imports a manifest entry only when that entry's fingerprint changes; applying an unchanged manifest returns skip_complete and does not re-fetch a remote source whose data changed independently.

An update that removes or retypes a column, or otherwise needs review, ends blocked for review. Applying that entry again unchanged does not queue it a second time: the entry comes back blocked with the run to accept and its reasons, apply prints geolens refresh <dataset-id> --accept-blocked-run <run-id>, and exits 6. A changed entry is queued as usual.

A vector source can carry an optional checksum: sha256:<64 lowercase hex> field. It is declared, not verified: apply never fetches the source bytes to check it, and folds it into the entry fingerprint like any other field. That makes it the way to force a re-import under a stable URI, such as latest.gpkg or a path an ETL job overwrites in place, where the entry itself never changes but the file underneath it does. Bump checksum when the file changes and the next apply reclassifies the entry as an update instead of skipping it. This does not apply to raster_cog sources: manifest raster updates are not supported, so do not set or change checksum on a raster entry. A changed checksum there still reclassifies the entry the same way, but the update then fails with an error result ("Manifest raster updates are not supported; create a new raster dataset instead."), not a skip. An unchanged raster entry, checksum included, still skips normally. Replace raster data by creating a new raster dataset instead.

geolens replace <dataset-id> <file> replaces this dataset's data from a local file, the CLI equivalent of the Re-upload dialog in the web app. It prints the preview (layer, feature count, detected SRID) and what the replacement would change (columns removed, retyped and added, old and new row counts, and each reason it needs review) before committing, then asks for confirmation once. --srid applies to the preview as well as the commit. Pass --yes to skip the prompt; it does not acknowledge the review reasons, so a replacement that needs review still stops as a blocked run. Confirming at the prompt does acknowledge them. --wait follows the run to its end and fails loudly on a bad import. A blocked run prints its reasons and geolens refresh <dataset-id> --accept-blocked-run <run-id>, and exits 6. A file with more than one layer needs --layer, since omitting it would otherwise commit the first layer without telling you. A raster dataset has no layer to preview, so replace uploads and commits it directly and --layer is rejected. replace only accepts a local file. A dataset whose data comes from a remote service origin, or a registered database table, cannot be replaced this way; use geolens refresh for that instead. --json never prompts, so it requires --yes.

geolens refresh <dataset-id> re-pulls data from the origin binding stored by GeoLens. It does not accept a URL, layer, or client-selected trigger. Add --wait to poll the durable refresh run; pass --timeout when you need a finite bound. JSON output includes the verification result. A refresh that ends blocked for review exits 6 and prints the accept command. Unattended refresh is not supported: a blocked run waits for a person to review it. Use apply when the declared source configuration itself changes.

Service refresh compares the staged row count with the source count when the provider supplies one. A mismatch fails without changing live data. A missing source count, an empty replacement for a non-empty dataset, or a removed or retyped column blocks publication for review. After checking the Source panel, use --accept-blocked-run <run-id> to accept that source and staged content once. The retry must match the reviewed attributes and geometries. A later or different result blocks again. Matching counts still do not prove that a mutable provider served every page from one snapshot.

Registered PostGIS refresh measures the registered live relation; it does not copy or preserve that relation. Referenced STAC refresh updates the remote item and asset pointer; it does not copy the asset bytes. A failed or blocked service refresh retains the current managed table. After a successful file replacement or service refresh of a vector dataset, GeoLens keeps the replaced table as the previous version until the next replacement, and POST /api/datasets/{id}/previous-version/restore publishes it again. A registered PostGIS refresh, a partitioned table, rasters, mosaics, tilesets and point clouds keep no previous version, so restoring their earlier data requires a backup or re-import.

Protected services can receive bearer credentials with --token. Use bare --token for a hidden prompt. Use --auth-file with a protected JSON file for bearer, Basic, or named-header credentials. The server needs a reachable shared credential store (REDIS_URL) to hand the single-use secret to the worker; the configuration validation endpoint reports this separately from ordinary cache health. GeoLens does not store the credential in the dataset binding.

geolens status <dataset-id> reports the catalog status together with source origin, freshness, health, and the last successful refresh time. Use --json before the command for a machine-readable status payload.

geolens apply --wait follows each queued job to its end and adds a STATUS column (complete, failed, blocked, cancelled). With --json, each result gains final_status and run_id.

Exit codes

ArcGIS migration inventory

geolens arcgis inventory lists an ArcGIS Online or Portal for ArcGIS organization's content and reports what would carry over to GeoLens. It is read-only: it changes nothing on the portal, needs no GeoLens instance or sign-in, and talks only to the portal URL you give it. It never contacts the services an item points at.

export ARCGIS_TOKEN=...   # or --token-stdin, or --username with a password prompt
geolens arcgis inventory --portal-url https://<org>.maps.arcgis.com -o ./inventory
geolens arcgis inventory --portal-url https://gis.example.com/portal --scope org
geolens --json arcgis inventory --portal-url https://<org>.maps.arcgis.com > inventory.json

Each item is classified as supported (hosted feature layers and views, feature and map services GeoLens can reach), partial (web maps, whose layers import but whose styling and popups need translation; hosted tile layers; OGC services; data files) or unsupported (apps, dashboards, StoryMaps, scenes, image and vector tile services, and anything the CLI doesn't recognize). Web AppBuilder apps and classic Esri Story Maps are flagged with Esri's retirement dates. Web maps and apps are read to record which layers and maps they depend on. Every item also records its snippet, description, tags, credits, license, extent, thumbnail name, spatial reference, folder, the groups it is shared with and its owner's name and email. Folder titles, owner names and group lists come back null where the portal doesn't let the signed-in account read them, and the group list holds only the groups that account can see. An anonymous run skips group reads. --no-groups skips the one-request-per-item group read.

With -o, each Web Map's configuration is also saved as webmaps/<item id>.json and each app's as apps/<item id>.json (StoryMaps, Experiences, Dashboards, Hub sites and pages, Web Scenes, Notebooks and the other app types), mode 0600. Before writing, the value under any key named like a credential (token, password, apiKey, secret, clientSecret, credentials, customParameters) becomes [REDACTED] at any depth. --read-all-item-data reads every other item's data too and records any item ids it names. Hosted views, hosted layers and Survey123 forms also record their source service, source file or results service, at one extra request each. A dependency is marked external when it belongs to another organization, such as Living Atlas content.

• --scope user (the default) lists the signed-in user's folders; --scope org lists every organization item the account can see. Without credentials only --scope org works, and it lists public items.
• Credentials: an existing token through ARCGIS_TOKEN, --token-stdin or --token, or --username for a built-in account. The password comes from ARCGIS_PASSWORD, --password-stdin or a prompt, and is used once to mint a 60-minute token bound to the portal URL as its referer; later requests send that Referer. A token you bring is sent without a Referer. SAML and OpenID Connect accounts need a token. The token travels in the X-Esri-Authorization header, never in a URL, and is never written to the report, the terminal or disk. Redirects are refused.
• Output: Markdown on stdout, JSON with --json, or both files (arcgis-inventory.json, arcgis-inventory.md, mode 0600) with -o/--output-dir. The JSON schema ships in the package as geolens_cli/manifest/schemas/arcgis-inventory-v2.schema.json.
• Limits: --max-items (default 10,000), --concurrency 1 to 4 for item reads, at most ten requests a second, and retries with backoff on 429 and 502 to 504. ArcGIS search returns only the first 10,000 results of a query, so a --scope org listing that reaches that limit is marked truncated (search_ceiling) whatever --max-items is.
• Exit codes: 0 when the report is complete, even if some items could not be read (they are listed under errors; --strict makes that exit 1); 2 for usage errors or a redirecting portal; 3 when the token is rejected; 4 for network errors; 5 for portal server errors. If the token expires mid-run, the partial report is still written, marked "complete": false; rerun with a fresh token.

Planning the migration

geolens arcgis plan turns an inventory into an editable plan. It reads only the inventory file, so it needs no portal or GeoLens access.

geolens arcgis inventory --portal-url https://<org>.maps.arcgis.com -o ./inventory
geolens arcgis plan ./inventory/arcgis-inventory.json
geolens arcgis plan ./inventory/arcgis-inventory.json --owner-map owners.json

It writes arcgis-migration-plan.json (mode 0600) next to the inventory, or into -o/--output-dir, and prints how many items each action covers and which dependencies point at items that are not in the inventory. It refuses a v1 inventory and says to rerun the inventory. Running it again gives the same file; if the file on disk differs, because you edited it, plan stops unless you pass --force.

The fields you are expected to edit:

• items[].action: import_layers, import_file, import_map, export_only or skip. The default follows the inventory: supported feature and map services import as layers, data files import as files, web maps import as maps, and apps, scenes and unsupported types are export_only. Set skip for anything you don't want. A web map whose data could not be read defaults to export_only and the plan warns, because its layer dependencies are unknown.
• items[].target: the visibility (ArcGIS private, org and public become private, internal and public), the collections (the item's folder), the groups (the ArcGIS group ids it is shared with, null when the inventory did not read group sharing), the title and the tags. target.owner is null to use owner_map, or a GeoLens username or email to override it for one item.
• owner_map: every ArcGIS owner, with an empty value to fill in with the GeoLens user who should end up owning the item. --owner-map file.json fills it from a prepared {"arcgis_user": "geolens_user"} object.
• group_map: every group seen, with the collection its items join and that collection's visibility. Items join these collections through target.groups, so renaming a group's collection here renames it for every item.
• basemap_map: every basemap URL used by a web map, to fill in with the GeoLens basemap that replaces it.
• items[].alias_of: set by hand, with action skip, on a hosted view that filters nothing, so web maps using it point at the parent's dataset. The inventory can't tell, because a view's filter lives in the service, which the inventory never contacts.

depends_on lists the items to migrate first. An id that is not among the plan's items is not in the inventory. Items from other organizations, such as Living Atlas layers, get no row and never appear in depends_on. A data file that a hosted layer was published from is export_only with superseded_by naming the layer, so its rows are not imported twice.

The plan and the sidecar files hold the same redacted JSON as the inventory: credential-named keys are replaced, and neither file ever contains a token or password.

Manifest schema distribution

The versioned manifest JSON Schema is intentionally distributed inside geolens-cli, rather than as a separate package. A second artifact would add a release/version-skew surface without a demonstrated independent consumer; the CLI is already the canonical manifest authoring and validation tool. Editors and non-Python tooling can obtain the exact installed schema with geolens schema or geolens schema -o schema.json. Its stable $id identifies manifest v1.

This decision should be revisited if multiple consumers need schema releases on a cadence independent from the CLI. Until then, schema changes and CLI versions ship atomically and the wheel test locks resource inclusion.

Environment variables

The CLI normally stores its active instance through geolens login and keeps tokens in the OS keyring. Ephemeral CI jobs can avoid persistent state with:

For load-test and seed-script variables, use the documented Tooling / Load Tests section in the repository's .env.example.