Mozilla's Glean Telemetry SDK: The Machine that Goes 'Ping!'
Project description
Glean SDK
Documentation
All documentation is available online:
The Glean SDK Book
Overview
Refer to the documentation for using and developing the Glean SDK.
For an overview of Glean beyond just the SDK, see the section in the Firefox data docs.
The code in this repository is organized as follows:
- ./glean-core/ contains the source for the low-level Rust library.
- ./glean-core/ffi contains the mapping into a C FFI.
- ./glean-core/android contains the Kotlin bindings for use by Android applications.
- ./glean-core/ios contains the Swift bindings for use by iOS applications.
- ./glean-core/python contains Python bindings.
Note: The Glean SDK requires at least Rust 1.41.0. Older versions are untested.
Contact
To contact us you can:
- Find us in the #glean channel on chat.mozilla.org.
- To report issues or request changes, file a bug in Bugzilla in Data Platform & Tools :: Glean: SDK.
- Send an email to glean-team@mozilla.com.
- The Glean Core team is: :dexter, :janerik, :mdroettboom, :travis_, :gfritzsche, :chutten, :brizental.
Credits
The Glean logo artwork was contributed by Diana Ciufo. It's licensed under MPL.
License
This Source Code Form is subject to the terms of the Mozilla Public
License, v. 2.0. If a copy of the MPL was not distributed with this
file, You can obtain one at http://mozilla.org/MPL/2.0/
Unreleased changes
v27.0.0 (2020-04-08)
- General:
- Glean will now detect when the upload enabled flag changes outside of the application, for example due to a change in a config file. This means that if upload is disabled while the application wasn't running (e.g. between the runs of a Python command using the Glean SDK), the database is correctly cleared and a deletion request ping is sent. See #791.
- The
events
ping now includes a reason code:startup
,background
ormax_capacity
.
- iOS:
- BUGFIX: A bug where the metrics ping is sent immediately at startup on the last day of the month has been fixed.
- Glean for iOS is now being built with Xcode 11.4.0
- The
measure
convenience function on timing distributions and time spans will now cancel the timing if the measured function throws, then rethrow the exception (#808) - Broken doc generation has been fixed (#805).
- Kotlin
- The
measure
convenience function on timing distributions and time spans will now cancel the timing if the measured function throws, then rethrow the exception (#808)
- The
- Python:
- Glean will now wait at application exit for up to one second to let its worker thread complete.
- Ping uploading now happens in a separate child process by default. This can be disabled with the
allow_multiprocessing
configuration option.
v26.0.0 (2020-03-27)
- General:
- The version of
glean_parser
has been updated to 1.19.0:- Breaking change: The regular expression used to validate labels is stricter and more correct.
- Add more information about pings to markdown documentation:
- State whether the ping includes client id;
- Add list of data review links;
- Add list of related bugs links.
glean_parser
now makes it easier to write external translation functions for different language targets.- BUGFIX: glean_parser now works on 32-bit Windows.
- The version of
- Android:
gradlew clean
will no longer remove the Miniconda installation in~/.gradle/glean
. Thereforeclean
can be used without reinstalling Miniconda afterward every time.
- Python:
- Breaking Change: The
glean.util
andglean.hardware
modules, which were unintentionally public, have been made private. - Most Glean work and I/O is now done on its own worker thread. This brings the parallelism Python in line with the other platforms.
- The timing distribution, memory distribution, string list, labeled boolean and labeled string metric types are now supported in Python (#762, #763, #765, #766)
- Breaking Change: The
v25.1.0 (2020-02-26)
- Python:
v25.0.0 (2020-02-17)
- General:
ping_type
is not included in theping_info
any more (#653), the pipeline takes the value from the submission URL.- The version of
glean_parser
has been upgraded to 1.18.2:- Breaking Change (Java API) Have the metrics names in Java match the names in Kotlin. See Bug 1588060.
- The reasons a ping are sent are now included in the generated markdown documentation.
- Android:
- The
Glean.initialize
method runs mostly off the main thread (#672). - Labels in labeled metrics now have a correct, and slightly stricter, regular expression. See label format for more information.
- The
- iOS:
- The baseline ping will now include
reason
codes that indicate why it was submitted. If an unclean shutdown is detected (e.g. due to force-close), this ping will be sent at startup withreason: dirty_startup
. - Per Bug 1614785, the clearing of application lifetime metrics now occurs after the metrics ping is sent in order to preserve values meant to be included in the startup metrics ping.
initialize()
now performs most of its work in a background thread.
- The baseline ping will now include
- Python:
- When the pre-init task queue overruns, this is now recorded in the metric
glean.error.preinit_tasks_overflow
. - glinter warnings are printed to
stderr
when loadingmetrics.yaml
andpings.yaml
files.
- When the pre-init task queue overruns, this is now recorded in the metric
v24.2.0 (2020-02-11)
- General:
- Add
locale
toclient_info
section. - Deprecation Warning Since
locale
is now in theclient_info
section, the one in the baseline ping (glean.baseline.locale
) is redundant and will be removed by the end of the quarter. - Drop the Glean handle and move state into glean-core (#664)
- If an experiment includes no
extra
fields, it will no longer include{"extra": null}
in the JSON payload. - Support for ping
reason
codes was added. - The metrics ping will now include
reason
codes that indicate why it was submitted. - The version of
glean_parser
has been upgraded to 1.17.3
- Add
- Android:
- Collections performed before initialization (preinit tasks) are now dispatched off the main thread during initialization.
- The baseline ping will now include
reason
codes that indicate why it was submitted. If an unclean shutdown is detected (e.g. due to force-close), this ping will be sent at startup withreason: dirty_startup
.
- iOS:
- Collections performed before initialization (preinit tasks) are now dispatched off the main thread and not awaited during initialization.
- Added recording of
glean.error.preinit_tasks_overflow
to report when the preinit task queue overruns, leading to data loss. See bug 1609734
v24.1.0 (2020-01-16)
- General:
- Stopping a non started measurement in a timing distribution will now be reported
as an
invalid_state
error.
- Stopping a non started measurement in a timing distribution will now be reported
as an
- Android:
- A new metric
glean.error.preinit_tasks_overflow
was added to report when the preinit task queue overruns, leading to data loss. See bug 1609482
- A new metric
v24.0.0 (2020-01-14)
- General:
- Breaking Change An
enableUpload
parameter has been added to theinitialize()
function. This removes the requirement to callsetUploadEnabled()
prior to calling theinitialize()
function.
- Breaking Change An
- Android:
- The metrics ping scheduler will now only send metrics pings while the application is running. The application will no longer "wake up" at 4am using the Work Manager.
- The code for migrating data from Glean SDK before version 19 was removed.
- When using the
GleanTestLocalServer
rule in instrumented tests, pings are immediately flushed by theWorkManager
and will reach the test endpoint as soon as possible.
- Python:
- The Python bindings now support Python 3.5 - 3.7.
- The Python bindings are now distributed as a wheel on Linux, macOS and Windows.
v23.0.1 (2020-01-08)
- Android:
- BUGFIX: The Glean Gradle plugin will now work if an app or library doesn't have a metrics.yaml or pings.yaml file.
- iOS:
- The released iOS binaries are now built with Xcode 11.3.
v23.0.0 (2020-01-07)
- Python bindings:
- Support for events and UUID metrics was added.
- Android:
- The Glean Gradle Plugin correctly triggers docs and API updates when registry files change, without requiring them to be deleted.
parseISOTimeString
has been made 4x faster. This had an impact on Glean migration and initialization.- Metrics with
lifetime: application
are now cleared when the application is started, after startup Glean SDK pings are generated.
- All platforms:
- The public method
PingType.send()
(in all platforms) have been deprecated and renamed toPingType.submit()
. - Rename
deletion_request
ping todeletion-request
ping after glean_parser update
- The public method
v22.1.0 (2019-12-17)
- Add
InvalidOverflow
error toTimingDistribution
s (#583)
v22.0.0 (2019-12-05)
- Add option to defer ping lifetime metric persistence (#530)
- Add a crate for the nice control API (#542)
- Pending
deletion_request
pings are resent on start (#545)
v21.3.0 (2019-12-03)
- Timers are reset when disabled. That avoids recording timespans across disabled/enabled toggling (#495).
- Add a new flag to pings:
send_if_empty
(#528) - Upgrade
glean_parser
to v1.12.0 - Implement the deletion request ping in Glean (#526)
v21.2.0 (2019-11-21)
-
All platforms
-
The experiments API is no longer ignored before the Glean SDK initialized. Calls are recorded and played back once the Glean SDK is initialized.
-
String list items were being truncated to 20, rather than 50, bytes when using
.set()
(rather than.add()
). This has been corrected, but it may result in changes in the sent data if using string list items longer than 20 bytes.
-
v21.1.1 (2019-11-20)
-
Android:
- Use the
LifecycleEventObserver
interface, rather than theDefaultLifecycleObserver
interface, since the latter isn't compatible with old SDK targets.
- Use the
v21.1.0 (2019-11-20)
-
Android:
-
Two new metrics were added to investigate sending of metrics and baseline pings. See bug 1597980 for more information.
-
Glean's two lifecycle observers were refactored to avoid the use of reflection.
-
-
All platforms:
- Timespans will now not record an error if stopping after setting upload enabled to false.
v21.0.0 (2019-11-18)
-
Android:
-
The
GleanTimerId
can now be accessed in Java and is no longer atypealias
. -
Fixed a bug where the metrics ping was getting scheduled twice on startup.
-
-
All platforms
- Bumped
glean_parser
to version 1.11.0.
- Bumped
v20.2.0 (2019-11-11)
- In earlier 20.x.x releases, the version of glean-ffi was incorrectly built against the wrong version of glean-core.
v20.1.0 (2019-11-11)
-
The version of Glean is included in the Glean Gradle plugin.
-
When constructing a ping, events are now sorted by their timestamp. In practice, it rarely happens that event timestamps are unsorted to begin with, but this guards against a potential race condition and incorrect usage of the lower-level API.
v20.0.0 (2019-11-11)
-
Glean users should now use a Gradle plugin rather than a Gradle script. (#421) See integrating with the build system docs for more information.
-
In Kotlin, metrics that can record errors now have a new testing method,
testGetNumRecordedErrors
. (#401)
v19.1.0 (2019-10-29)
- Fixed a crash calling
start
on a timing distribution metric before Glean is initialized. Timings are always measured, but only recorded when upload is enabled (#400) - BUGFIX: When the Debug Activity is used to log pings, each ping is now logged only once (#407)
- New
invalid state
error, used in timespan recording (#230) - Add an Android crash instrumentation walk-through (#399)
- Fix crashing bug by avoiding assert-printing in LMDB (#422)
- Upgrade dependencies, including rkv (#416)
v19.0.0 (2019-10-22)
First stable release of Glean in Rust (aka glean-core). This is a major milestone in using a cross-platform implementation of Glean on the Android platform.
- Fix round-tripping of timezone offsets in dates (#392)
- Handle dynamic labels in coroutine tasks (#394)
v0.0.1-TESTING6 (2019-10-18)
- Ignore dynamically stored labels if Glean is not initialized (#374)
- Make sure ProGuard doesn't remove Glean classes from the app (#380)
- Keep track of pings in all modes (#378)
- Add
jnaTest
dependencies to theforUnitTest
JAR (#382)
v0.0.1-TESTING5 (2019-10-10)
- Upgrade to NDK r20 (#365)
v0.0.1-TESTING4 (2019-10-09)
- Take DST into account when converting a calendar into its items (#359)
- Include a macOS library in the
forUnitTests
builds (#358) - Keep track of all registered pings in test mode (#363)
v0.0.1-TESTING3 (2019-10-08)
- Allow configuration of Glean through the
GleanTestRule
- Bump
glean_parser
version to 1.9.2
v0.0.1-TESTING2 (2019-10-07)
- Include a Windows library in the
forUnitTests
builds
v0.0.1-TESTING1 (2019-10-02)
General
First testing release.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distributions
Hashes for glean_sdk-27.0.0-py3.7-macosx-10.15-x86_64.egg
Algorithm | Hash digest | |
---|---|---|
SHA256 | 16e49a1fde70b9bc925d6e72a6a27a7552f1d0a1d4d82637aab6ad4a9efc8e1d |
|
MD5 | 5f7058f0530e6abe820d37cdea5769d7 |
|
BLAKE2b-256 | 16366e24b141712eae4631ec8f3ddf8677c6d87cffd41a44eefe19b68d4a3eb7 |
Hashes for glean_sdk-27.0.0-py3-none-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c67dc1dd95e3f73c5ecd533a195a4fca6bb05e16edb2a0ac85cd36f5fe75e8b8 |
|
MD5 | 1e358e9701462f98515f5c00377fd24b |
|
BLAKE2b-256 | ec18eb6650e672ed52f78c9f969d1a2fe008008504c3a04cc67dea4bc6d8b30f |
Hashes for glean_sdk-27.0.0-py3-none-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 973fb735659e96a83ff3cd338af8208d3ecdb2aa2f88a088c717d2d02cf7f73f |
|
MD5 | 4a04df2be6e33d5014c7fcc069b4d9fb |
|
BLAKE2b-256 | 63c29137dc6fa37f7bbc8d8a5aa0a4feb6747e3c80d5b1f2fff68c1a8be65b92 |
Hashes for glean_sdk-27.0.0-cp35-abi3-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 92fafdd00c313891d3560e879c7e241541161ba40ed07ae67fd39a4a5bd7758a |
|
MD5 | e6d48333e1262b303134f2c0efbbdbb8 |
|
BLAKE2b-256 | 84fd85a35476f174164ea8cde53b0b12bd57792cd582bf2b15ac6e55effac0ef |
Hashes for glean_sdk-27.0.0-cp35-abi3-macosx_10_7_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f28408047c111d98a6644a6dbd1535df0ab1088904de303923ffe090f897b639 |
|
MD5 | ded934d36c45a0bc2c681c32dd7d5a72 |
|
BLAKE2b-256 | 1d8e355a7590e32063de7c2162c599eb5138b4445a1bf903845cc2c468b77e68 |