galaxy-data 26.1.0

Make database heartbeat more robust by @mvdbeek in #21734

Fix tag duplication when copying datasets/collections between histories by @dannon in #22109

Fix more paired_or_unpaired collection issues by @jmchilton in #22170

Fix Hashicorp Vault 2.0 non-canonical path rejection by @mvdbeek in #22530

Skip missing history item ids in job exports by @guerler in #22563

Merge 26.0 into dev (CI-clean rebase of #22749) by @mvdbeek in #22753

Fix tag creation self-deadlock under SQLite by @mvdbeek in #22846

Handle existing rows in tool_source during db migration by @mvdbeek in #22870

Always attempt to build tool request for new API endpoint by @mvdbeek in #22894

Fix legacy DatabaseVault secrets canonical key migration by @davelopez in #22942

Hide generated headers for tabular previews by @qchiujunhao in #22947

Keep the standalone SSE monitor out of the control-queue routing table by @mvdbeek in #22958

Include hidden intermediates in workflow extraction summary (#22967) by @jmchilton in #22970

Fix storage operation column types (Database migration) by @davelopez in #23032

Update spatialdata datatype by @nilchia in #23087

Bound memory use in h5grove structured content endpoint by @mvdbeek in #23089

Fix Workflow.copy() dropping readme, help, logo_url and doi by @mvdbeek in #23117

Fix fasta preview rendering for sequences larger than 100 KB by @mvdbeek in #23163

Use a thread-safe libmagic handle when sniffing datasets by @mvdbeek in #23200

Show persistent best practice results in workflow editor by @mvdbeek in #19546

Various YAML Tool Hardening and Progress toward Tool State Goals by @jmchilton in #21828

Update Python dependencies by @galaxybot in #21853

Drop use of pkg_resources by @nsoranzo in #21858

Use (also) case-insensitive comparison for user email by @nsoranzo in #21885

Add ASCII Raster (.asc) datatype by @tStehling in #21937

Migrate Python packages to `src` layout and pure namespace packages by @mr-c in #21977

Improve celery rate limit and concurrency handling by @mvdbeek in #22189

Update Python dependencies by @galaxybot in #22206

Test against psycopg3 by @mvdbeek in #22210

Show file size for CRAM datasets and in download tooltip by @dannon in #22273

Add Hashicorp Vault token renewal support by @mvdbeek in #22281

Search and paginate user and roles api by @mvdbeek in #22311

Add sheet_names metadata to XLSX datatype by @Anthony96pi in #22327

Galaxy Notebooks: Persistent Narrative for Human-AI Collaborative Science in Galaxy by @jmchilton in #22361

Clean up legacy SQLAlchemy patterns that emit deprecation warnings by @mvdbeek in #22462

Clean up Pydantic v2 deprecation warnings by @mvdbeek in #22463

Clean up remaining deprecation warnings by @mvdbeek in #22473

Mark History.jobs relationship as viewonly by @mvdbeek in #22474

Server-Sent Events for history + notification updates by @mvdbeek in #22513

Fix display of BAMs with large headers by @wm75 in #22516

Speed up integration tests by @mvdbeek in #22538

Overhaul and synchronize collection type algebra verbs by @jmchilton in #22565

Use now() wrapper instead of datetime.now() by @bgruening in #22579

Add bulk dataset storage migration by @davelopez in #22606

Improve type annotation of workflow management methods by @nsoranzo in #22637

Paginate /api/tools/{tool_id}/build history options by @mvdbeek in #22643

Enhance workflow extraction by IDs with deduplication and UI improvements by @jmchilton in #22706

Respect workflow object store preference for mapped step outputs by @mvdbeek in #22409

Avoid premature commit when copying MetadataFile by @mvdbeek in #22422

Fix testing, publishing and dependencies of packages by @nsoranzo in #22445

Fix `get_structure` when it is passed a DCE with nested elements by @mvdbeek in #22453

Enforce file source access during dataset materialization by @mvdbeek in #22486

Fix sample sheet upload without sample-sheet-specific columns by @mvdbeek in #22488

Truncate dataset name in export arcname to avoid OSError by @mvdbeek in #22518

Catch end of file error when reading corrupted tabular dataset by @guerler in #22523

Fix data manager URL for managers with ‘+’ version suffix by @mvdbeek in #22525

update the binary.py for spatialdata so it also works if the file has… by @nilchia in #22802

Replace per-term joins in workflow search with EXISTS subqueries by @mvdbeek in #22548

add TEI XML datatype by @bgruening in #22718

Drop down revision for drop_job_params migration by @mvdbeek in #21122

Create alembic merge by @mvdbeek in #21447

Provide more accurate step hints for failing subworkflow steps by @mvdbeek in #21514

Fix width and height assignment for image metadata using Pillow by @kostrykin in #21554

Avoid slash duplication in display application urls by @guerler in #21809

Fix redirect upon impersonation by @guerler in #21810

Fix dataset collection tag duplication when copying a history by @mvdbeek in #21984

Fix AttributeError serializing implicit output collections during job prep by @mvdbeek in #22034

Fix preferred_object_store_id not respected for discovered outputs with extended metadata by @mvdbeek in #22056

Fix TypeError when column_names metadata is None by @mvdbeek in #22057

Fail job gracefully when output name exceeds 255 character DB limit by @mvdbeek in #22059

Fix nested collection query performance by @mvdbeek in #22119

Fix CardinalityViolation in nested collection ARRAY walk scalar subqueries by @mvdbeek in #22123

Use skip_locked also when updating collection by @mvdbeek in #22133

Fix redundant empty collection elements by @mvdbeek in #22154

Don’t fail on probable delete race by @mvdbeek in #22183

Back port collection fixes from #22170. by @jmchilton in #22193

Optimize dataset get_edit API by deferring expensive role queries by @mvdbeek in #22232

Batch-prefetch HDCA job_state_summary in history contents listing by @mvdbeek in #22256

Sequence datatypes: only read first character per line for setting metadata by @bernt-matthias in #22276

Fix slow get_private_role_user_emails_dict query by @mvdbeek in #22307

Fix timestamp parsing in job import/export by @mvdbeek in #22372

Ensure x-content-truncated header is string by @mvdbeek in #22386

Fix dataset preview rendering to preserve newlines by @mvdbeek in #22397

Refactor display application handling by @guerler in #15076

Tool Request API by @jmchilton in #20935

Remove Trackster by @guerler in #20974

Move database revision tags to separate file by @guerler in #20998

Update Python dependencies by @galaxybot in #21043

Remove Job.params by @jdavcs in #21075

Remove tabular chunked, binary and large file mako by @guerler in #21106

Add type annotations to job handling code by @nsoranzo in #21171

Reimplement keycloak/cilogon as PSA providers and remove custos by @nuwang in #21234

Slight improvement of mztab2 datatype by @nguilhot in #21258

Move js-client bootstrapping to FastAPI by @guerler in #21264

Implement GA4GH WES API by @jmchilton in #21335

Fix: tool execution typing improvements/fixes by @jmchilton in #21374

Add initial support for the DICOM format by @kostrykin in #21385

Add AI Agent Framework and ChatGXY 2.0 by @dannon in #21434

Enable attaching sample sheet to landing requests by @mvdbeek in #21489

Discriminate decompression tools by @mvdbeek in #21510

Include subworkflow jobs in invocation metrics by @mvdbeek in #21511

Implement subworkflow invocation export by @mvdbeek in #21512

Implement workflow completion monitoring with extensible hooks by @mvdbeek in #21532

Clean up code with pyupgrade by @nsoranzo in #21540

Drop support for Python 3.9 by @nsoranzo in #21583

Apply 2026 black style by @galaxybot in #21618

Allow connecting sample sheet inputs to compatible collection inputs in editor by @mvdbeek in #21859

Add new datatypes: .taf and .taf.gz by @Maed0x in #22242

Add _F/_R auto-pairing filter for forward/reverse datasets by @jmchilton in #22267

Update revision tags for release 25.1 by @arash77 in #21022

Merge 25.1 into dev and create alembic merge revision by @mvdbeek in #21440

Add Database Revision for 26.0 by @guerler in #21675

Fix on_complete column type mismatch by @mvdbeek in #21765

Fix private dataset sharing error handling by @davelopez in #21689

Fix database migration error from 25.0 > 25.1 by @jdavcs in #21737

Make database heartbeat more robust (backport) by @mvdbeek in #21812

Fix recording duplicate workflow_request_input_step_parameter rows by @mvdbeek in #21980

Fix worker segfault in gunicorn preload mode on OSX by @mvdbeek in #21837

Sample sheet column fix by @mvdbeek in #21446

Maintain column definitions on map over by @mvdbeek in #21468

Add missing test file for sig datatype by @SaimMomin12 in #21473

Do not update a user’s update_time when an admin archives (and purges) a history by @natefoo in #21484

Add missing ending newline to test.sig test data file by @nsoranzo in #21485

Usability fixes for sample sheet selection. by @jmchilton in #21503

Fix exception message to enable debugging of missing dataset issue. by @jmchilton in #21504

Add missing dependencies to galaxy-files package by @nsoranzo in #21518

Fix job cache collection copy by @mvdbeek in #21558

Fix #21542 - allow workbook bootstrap generation for sample sheet collection types. by @jmchilton in #21584

Fix race condition in workflow collection populated state check by @mvdbeek in #21613

Fix collection job state not preserved during history export/import by @mvdbeek in #21716

Add Sourmash sig new datatype by @SaimMomin12 in #21469

Add database operation tool to convert sample sheets to list collections by @mvdbeek in #21625

Fix alembic down_revision for trigger replacement migration by @mvdbeek in #20501

Add a merge migration, revert previous edit by @jdavcs in #20507

Add missing cleanup table by @jdavcs in #20594

Fix anndata metadata by @nilchia in #20778

Fix Ipynb datatype sniffer and add unit tests. by @ksuderman in #20811

Bump requirement of directory converters by @bernt-matthias in #20834

Extract: do not use common prefix dir by @bernt-matthias in #20929

Ensure that conversion_key is defined by @guerler in #20984

Fix invocation export dataset exclusion by @davelopez in #21091

Restore job.get_param_values by @mvdbeek in #21130

Improve _touch_collection_update_time_cte performance by @mvdbeek in #21134

Assorted workbook upload fixes. by @jmchilton in #21203

Fix database revision tags by @jdavcs in #21377

Maintain columns on sample sheet map over by @mvdbeek in #21436

Support credentials(secrets/variables) in tool requirements by @arash77 in #19084

Implement Sample Sheets by @jmchilton in #19305

Empower Users to More Pragmatically Import Datasets & Collections From Tables by @jmchilton in #20288

Add short term storage expiration indicator to history items by @davelopez in #20332

Type annotation fixes for mypy 1.16.0 by @nsoranzo in #20424

Implement dataset source requested transformations. by @jmchilton in #20435

Remove deprecated tool document cache by @nsoranzo in #20510

Decode OIDC access token as part of authentication pipeline by @marius-mather in #20547

Implement Data Landing Requests by @jmchilton in #20592

Clean up code from pyupgrade by @nsoranzo in #20642

Update Python dependencies by @galaxybot in #20680

Attempt to reuse previously materialized datasets by @mvdbeek in #20718

Refactor Files Sources Framework for stronger typing using pydantic models by @davelopez in #20728

Hierarchical display collection dataset states by @mvdbeek in #20731

Use a streaming based parser (ijson) when setting metadata for biom1 files. by @ksuderman in #20851

Improve type annotation for job runners and InteractiveToolManager by @nsoranzo in #20871

Update Python dependencies by @galaxybot in #20905

Allow sending and tracking landing request origin by @mvdbeek in #20957

Track landing request with invocation by @mvdbeek in #20960

Optimize /api/invocations/steps/{step_id} by @mvdbeek in #21249

A better way to designate config_watcher by @jdavcs in #21426

Small tweaks and cleanup from sample sheets branch by @jmchilton in #20229

Merge release_25.0 into dev by @davelopez in #20484

Merge release_25.0 into dev by @davelopez in #20490

Sample Sheet Bug Fixes around Preserving Collection Metadata by @jmchilton in #20749

Add tool_id and tool_version column to tool_landing_request table by @mvdbeek in #20795

Add 25.1 migration tags by @ahmedhamidawan in #21017

Restore .get_metadata function by @mvdbeek in #21045

Fix forward merge by @mvdbeek in #21307

Fix db revision tags by @mvdbeek in #21442

Add safetensors datatype by @nilchia in #20754

Skip sam metadata if we have too many references by @mvdbeek in #20820

Backport of #20984 by @davelopez in #20994

Backport fix anndata datatype update by @nilchia in #21111

Create new datasets when creating skipped database operation tool outputs by @mvdbeek in #21142

Fix optional param unset in RO-Crate export by @davelopez in #21192

Add Auspice JSON datatype by @pvanheus in #20466

Add SpatialData datatype by @nilchia in #21000

Use job cache also for implicit conversions by @mvdbeek in #21021

support ZARR v3 for Spatialdata dt by @nilchia in #21265

Ensure that workflow invocations are persisted with state by @mvdbeek in #20784

Fix password reset functionality for lowercase emails by @jdavcs in #20801

Drop eager argument from invocation related methods by @mvdbeek in #20863

Use populated_optimized when serializing collection by @mvdbeek in #20872

Speed up ImplicitCollectionJobs.job_list by @mvdbeek in #20887

Add LDDA purged property by @mvdbeek in #20895

Avoid potential race condition in replacement_for_connection by @mvdbeek in #20909

Fix sessionless tag creation by @mvdbeek in #20925

Fix DOCX detection and add PPTX support by @arash77 in #20827

Add HiC datatype by @abretaud in #20874

Undo accidental push by @bernt-matthias in #20854

Merge 25.0 into dev by @ahmedhamidawan in #20911

Remove num_unique_values tiff metadata element by @mvdbeek in #20464

Propagate cached job output replacement to copies of outputs by @mvdbeek in #20553

Fix deferred datasets in multiple dataset parameters. by @jmchilton in #20650

Add num_unique_values tiff metadata element, fixed by @kostrykin in #20669

Improve workflow monitor loop times by @mvdbeek in #20522

Pairtool updated format and sniffers by @Smeds in #20634

Add support for M4A audio files by @arash77 in #20667

Better handling of public pages and workflows authored by deleted users by @jdavcs in #19394

Alternative format_source fix by @mvdbeek in #19395

Use id entity attribute when setting exampleOfWork property by @nsoranzo in #19396

Fix package test errors: Ignore mypy errors caused by social-auth-core 4.5.5 by @jdavcs in #19620

Fix Galaxy ignoring job object_store_id for quota check by @bernt-matthias in #19854

Fix various mypy issues around mapped attributes by @mvdbeek in #19883

More fixes to FormData drag and drop and typing by @ahmedhamidawan in #19900

Use lower case extension for setting datatype in data discovery by @bernt-matthias in #19954

Add new line to vtpascii test file by @guerler in #20051

Fix certain cases of collection parameter handling during workflow execution. by @jmchilton in #20152

Update tabular_csv.py to use less memory in tsv->csv conversion by @cat-bro in #20187

Fix tool-provided metadata for CONVERTER_tar_to_directory by @mr-c in #20260

Log invalid vault paths by @mvdbeek in #20363

Update triggers by @jdavcs in #20425

Fix copying of job metrics for cached jobs by @mvdbeek in #20436

Dataset Display and Preferred Viz fixes by @dannon in #20439

Recreate triggers by @jdavcs in #20491

Workflow Editor Activity Bar by @ElectronicBlueberry in #18729

Add author and tools details in RO-Crate by @Marie59 in #18820

Extend image metadata by @kostrykin in #18951

Implement tool markdown reports. by @jmchilton in #19054

Let file sources choose a path for uploaded files by @kysrpex in #19154

Calculate hash for new non-deferred datasets when finishing a job by @nsoranzo in #19181

Fix UP031 errors - Part 1 by @nsoranzo in #19194

Fix UP031 errors - Part 4 by @nsoranzo in #19235

Fix UP031 errors - Part 5 by @nsoranzo in #19282

Fix UP031 errors - Part 6 by @nsoranzo in #19314

Misc fixes 202412 by @nsoranzo in #19341

Type annotation fixes for mypy 1.14.0 by @nsoranzo in #19372

Empower Users to Build More Kinds of Collections, More Intelligently by @jmchilton in #19377

SQLAlchemy 2.0 follow-up by @jdavcs in #19388

Set safe default extraction filter for tar archives by @nsoranzo in #19406

Remove transaction helper by @jdavcs in #19407

Prevent users from reusing a banned email after account is purged by @jdavcs in #19413

Add IGB display support for CRAM files by @paige-kulzer in #19428

Add User-Defined Tools by @mvdbeek in #19434

Type annotation improvements by @nsoranzo in #19485

Update Python dependencies by @galaxybot in #19567

Expand workflow metadata for readme. by @jmchilton in #19591

Format code with black 25.1.0 by @nsoranzo in #19625

Type annotation improvements by @nsoranzo in #19642

Generate correct types for Dataset source transformations on backend. by @jmchilton in #19666

Drop support for Python 3.8 by @nsoranzo in #19685

Define simple models for job messages. by @jmchilton in #19688

Use model classes from galaxy.model instead of app.model object - Part 1 by @nsoranzo in #19706

Use model classes from galaxy.model instead of app.model object - Part 2 by @nsoranzo in #19726

Add bigbed to bed converter and tests by @d-callan in #19787

Update Python dependencies by @galaxybot in #19816

Populate image metadata without allocating memory for the entire image content by @kostrykin in #19830

Add mandatory RO-Crate metadata when exporting by @elichad in #19846

Improve type annotations of ModelPersistenceContext and derived classes by @nsoranzo in #19852

Drop old galaxy_session records by @jdavcs in #19872

Clarify is_active method usage for Python Social Auth in Galaxy by @dannon in #19899

Drop old job metrics by @jdavcs in #19936

Allow PathLike parameters in make_fast_zipfile() by @nsoranzo in #19955

Add type hints around collection copying and job things by @mvdbeek in #19961

Job cache allow different names when possible by @mvdbeek in #19962

Improve type annotation of tool parameter wrapping by @nsoranzo in #19991

Implement dataset collection support in workflow landing requests by @mvdbeek in #20004

Enable warn_redundant_casts mypy option and drop redundant casts by @nsoranzo in #20008

Add vitesscejson datatype by @guerler in #20027

Add VTK Visualization Toolkit Plugin by @guerler in #20028

Add DOI to workflow metadata by @jdavcs in #20033

Add sample datasets for visualizations by @guerler in #20046

Add docx datatype by @bgruening in #20055

Add flac audio format by @bgruening in #20057

Improve type annotation of galaxy.util submodules by @nsoranzo in #20104

Add specific datatypes for Cytoscape and Kepler.gl by @guerler in #20117

Visualization-First Display functionality by @dannon in #20190

Improve type annotation of galaxy.model.dataset_collections by @nsoranzo in #20194

Improve performance of job cache query by @mvdbeek in #20319

DatasetView and Card Polish by @dannon in #20342

Merge 24.2 into dev by @ahmedhamidawan in #19933

Fix copying job output from discovered outputs by @mvdbeek in #19999

Add 25.0 migration tags by @ahmedhamidawan in #20265

Rename vitessce_json file_ext to vitessce.json by @mvdbeek in #20473

Reduce default framework tool test timeout to 60 seconds by @mvdbeek in #19819

Try to recover from recurring activation link error by @jdavcs in #19844

Check if index exists before creating by @jdavcs in #19873

Let pysam use extra threads available in job by @mvdbeek in #19917

Handle special charater in raw SQL by @jdavcs in #19925

Use make_fast_zipfile directly by @mvdbeek in #19947

Job cache backports by @mvdbeek in #20078

Conditionally import from galaxy.config in galaxy.model.mapping if TYPE_CHECKING by @natefoo in #20209

Refactor display_as URL generation for UCSC links and fix to remove double slashes in URL by @natefoo in #20239

Make response header values strings by @mvdbeek in #20475

Link workflow invocation outputs upon importing invocation by @mvdbeek in #20485

Make optional edam-ontology in datatypes registry optional by @natefoo in #20492

Bump up max_peek_size to 50MB by @mvdbeek in #19823

Fix private role name performance issue by @jdavcs in #19679

Fix anndata metadata setting for data with integer indexes by @mvdbeek in #19774

Fix parameter model constructions with leading underscores, fixes converter linting by @mvdbeek in #19790

Add rDock prm datatype by @nsoranzo in #19783

Fix recording transform action name. by @jmchilton in #19672

Add bwa_mem2_index directory datatype, framework enhancements for testing directories by @mvdbeek in #19694

Backport #19620 by @nsoranzo in #19725

Drop “Send to cloud” tool and associated cloudauthz code by @jdavcs in #18196

Raise Message Exception when displaying binary data by @mvdbeek in #18503

Fixes for errors reported by mypy 1.11.0 by @nsoranzo in #18608

Fix new flake8-bugbear B039 and mypy type-var errors by @nsoranzo in #18755

Fix migration data fixes tests by @jdavcs in #18885

Fix backend role sharing bug by @jdavcs in #18942

Add merge migration to merge 2 heads by @jdavcs in #19163

Record implicitly converted dataset as input dataset by @mvdbeek in #19301

Fix quota usage with user object stores by @davelopez in #19323

Fix workflows with optional non-default parameter input by @mvdbeek in #19332

Fix importing shared workflows with deeply nested subworkflows by @mvdbeek in #19335

Backport fix from #19396 by @nsoranzo in #19399

Prevent cycling through failing conversion jobs in trackster by @mvdbeek in #19494

Fix extracting workflows from purged and deleted histories by @mvdbeek in #19525

Fix error message when subworkflow input connection missing by @mvdbeek in #19526

Fix admin cancel job message not being displayed to the user by @davelopez in #19537

Add tool_id index on job table by @mvdbeek in #19543

Update 24.2 db revision tags by @jdavcs in #19550

Provide guidance in case of deadlock during db migration by @jdavcs in #19551

Fix job paused on user defined object store by @davelopez in #19578

Handle isatools dependency by @jdavcs in #19582

Experimental galactic wizard by @dannon in #15860

Improve usability of Directory datatype by @wm75 in #17614

Update Python dependencies by @galaxybot in #18226

Allow OAuth 2.0 user defined file sources (w/Dropbox integration) by @jmchilton in #18272

More data access tests, some refactoring and cleanup by @jdavcs in #18312

Update Python dependencies by @galaxybot in #18406

Add pod5 datatype by @TomHarrop in #18419

Prepare for NumPy 2.0 by @nsoranzo in #18422

Add unique constraints to the email and username fields in the galaxy_user table by @jdavcs in #18493

Improvements for K8S deployment (especially ITs) by @almahmoud in #18514

Refactor LibraryDatasetsManager and fix type annotation issue by @nsoranzo in #18773

Handle compressed content in dataset preview for all sequence classes by @PlushZ in #18776

Backend handling of setting user-role, user-group, and group-role associations by @jdavcs in #18777

Workflow Landing Requests by @jmchilton in #18807

Update Mypy to 1.11.2 and fix new signature override errors by @nsoranzo in #18811

Refactor migration testing setup code by @jdavcs in #18886

Allow setting a few global defaults for file source plugin types. by @jmchilton in #18909

Type annotations and fixes by @nsoranzo in #18911

Add filter null collection operation tool by @mvdbeek in #18928

Fix extra files path type hint by @davelopez in #18958

Decouple user email from role name by @jdavcs in #18966

Optimize to_history_dataset_association in create_datasets_from_library_folder by @arash77 in #18970

Workflow landing improvements by @mvdbeek in #18979

Allow recovering a normalized version of workflow request state from API by @jmchilton in #18985

Add some Zarr-based datatypes by @davelopez in #19040

Run installed Galaxy with no config and a simplified entry point by @natefoo in #19050

Annotate DatasetAssociationManager as generic type by @nsoranzo in #19063

Move TRS import into WorkflowContentManager by @mvdbeek in #19070

Allow deferred datasets to behave as URIs by @davelopez in #19077

Better cleanup of sharing roles on user purge by @jdavcs in #19096

Add XML based vtk datatype by @tStehling in #19104

Access public history in job cache / job search by @mvdbeek in #19108

Always validate hashes when provided by @nsoranzo in #19110

Enable specifying dataset hash in test jobs by @mvdbeek in #19113

Enhance UTF-8 support for filename handling in downloads by @arash77 in #19161

Backport of Workflow Editor Activity Bar by @dannon in #19212

Fix default value handling for parameters connected to required parameters by @mvdbeek in #19219

Merge 24.1 by @jdavcs in #18386

Format dev to fix linting. by @jmchilton in #18860

Add 24.2 migration tags by @jdavcs in #19169

Fix type annotations for pysam 0.23.0 by @nsoranzo in #19571

Replace busybox:ubuntu-14.04 image with busybox:1.36.1-glibc by @mvdbeek in #18428

Improve workflow-related exception reporting by @mvdbeek in #18447

Increase API robustness to invalid requests, improve compressed data serving by @mvdbeek in #18494

Prevent job submission if input collection element is deleted by @mvdbeek in #18517

Fix shared caches with extended metadata collection. by @jmchilton in #18520

Also check dataset.deleted when determining if data can be displayed by @mvdbeek in #18547

Fix for not-null in ‘column_list’ object by @hujambo-dunia in #18553

Fix h5ad metadata by @nilchia in #18635

Don’t set file size to zero by @mvdbeek in #18653

Make sure we set file size also for purged outputs by @mvdbeek in #18681

Fix wrong extension on pick data output by @mvdbeek in #18798

Fix copying workflow with subworkflow step for step that you own by @mvdbeek in #18802

Make pylibmagic import optional by @mvdbeek in #18813

Ignore converted datasets in invalid input states by @mvdbeek in #18850

Fix discovered outputs with directory metadata and distributed object by @mvdbeek in #18855

Raise MessageException instead of assertions on rerun problems by @mvdbeek in #18858

Fix wrong final state when init_from is used by @mvdbeek in #18871

Fix history import when parent_hda not serialized by @mvdbeek in #18873

Limit max number of items in dataproviders by @mvdbeek in #18881

Allow cors in biom and q2view display applications by @mvdbeek in #18890

Disable locking when opening h5 files, add missing with by @mvdbeek in #18976

Optimize/fix sqlite hid update statement by @mvdbeek in #19106

Prefer auto-decompressed datatype when picking conversion target by @mvdbeek in #19162

Fix various invocation export issues by @mvdbeek in #19215

Fix bad merge conflict resolution by @nsoranzo in #19297

Backport pod5 datatype by @TomHarrop in #18507

Replace busybox:ubuntu-14.04 image with busybox:1.36.1-glibc by @mvdbeek in #18428

Improve workflow-related exception reporting by @mvdbeek in #18447

Increase API robustness to invalid requests, improve compressed data serving by @mvdbeek in #18494

Prevent job submission if input collection element is deleted by @mvdbeek in #18517

Fix shared caches with extended metadata collection. by @jmchilton in #18520

Also check dataset.deleted when determining if data can be displayed by @mvdbeek in #18547

Fix for not-null in ‘column_list’ object by @hujambo-dunia in #18553

Fix h5ad metadata by @nilchia in #18635

Don’t set file size to zero by @mvdbeek in #18653

Make sure we set file size also for purged outputs by @mvdbeek in #18681

Fix wrong extension on pick data output by @mvdbeek in #18798

Fix copying workflow with subworkflow step for step that you own by @mvdbeek in #18802

Make pylibmagic import optional by @mvdbeek in #18813

Ignore converted datasets in invalid input states by @mvdbeek in #18850

Fix discovered outputs with directory metadata and distributed object by @mvdbeek in #18855

Raise MessageException instead of assertions on rerun problems by @mvdbeek in #18858

Fix wrong final state when init_from is used by @mvdbeek in #18871

Fix history import when parent_hda not serialized by @mvdbeek in #18873

Limit max number of items in dataproviders by @mvdbeek in #18881

Allow cors in biom and q2view display applications by @mvdbeek in #18890

Disable locking when opening h5 files, add missing with by @mvdbeek in #18976

Backport pod5 datatype by @TomHarrop in #18507

Increase API robustness to invalid requests, improve compressed data serving by @mvdbeek in #18494

Prevent job submission if input collection element is deleted by @mvdbeek in #18517

Fix shared caches with extended metadata collection. by @jmchilton in #18520

Also check dataset.deleted when determining if data can be displayed by @mvdbeek in #18547

Fix for not-null in ‘column_list’ object by @hujambo-dunia in #18553

Fix h5ad metadata by @nilchia in #18635

Don’t set file size to zero by @mvdbeek in #18653

Make sure we set file size also for purged outputs by @mvdbeek in #18681

Fix wrong extension on pick data output by @mvdbeek in #18798

Fix copying workflow with subworkflow step for step that you own by @mvdbeek in #18802

Make pylibmagic import optional by @mvdbeek in #18813

Ignore converted datasets in invalid input states by @mvdbeek in #18850

Fix discovered outputs with directory metadata and distributed object by @mvdbeek in #18855

Raise MessageException instead of assertions on rerun problems by @mvdbeek in #18858

Fix wrong final state when init_from is used by @mvdbeek in #18871

Fix history import when parent_hda not serialized by @mvdbeek in #18873

Backport pod5 datatype by @TomHarrop in #18507

Fix syntax for SA2.0 by @jdavcs in #17663

Fix empty usernames in database + bug in username generation by @jdavcs in #18379

Fix input_step_parameters missing values that don’t have a label by @ahmedhamidawan in #18405

Improve workflow-related exception reporting by @mvdbeek in #18447

Enable all-vs-all collection analysis patterns. by @jmchilton in #17366

Visualizing workflow runs with an invocation graph view by @ahmedhamidawan in #17413

Better display of estimated line numbers and add number of columns for tabular by @bernt-matthias in #17492

Enable storage management by object store by @jmchilton in #17500

Set minimal metadata also for empty bed datasets by @wm75 in #17586

Type annotation improvements by @nsoranzo in #17601

Type annotation and CWL-related improvements by @nsoranzo in #17630

Update Python dependencies by @galaxybot in #17653

Code cleanups from ruff and pyupgrade by @nsoranzo in #17654

SQLAlchemy 2.0 by @jdavcs in #17778

Outline use of type_annotation_map to fix mypy issues by @jmchilton in #17902

Add email notifications channel by @davelopez in #17914

Model edits and bug fixes by @jdavcs in #17922

Model typing and SA2.0 follow-up by @jdavcs in #17958

Enable warn_unused_ignores mypy option by @nsoranzo in #17991

Add galaxy to user agent by @mvdbeek in #18003

Refactor galaxy.files plugin loading + config handling. by @jmchilton in #18049

Add stronger type annotations in file sources + refactoring by @davelopez in #18050

Add support for additional media types by @arash77 in #18054

Update Python dependencies by @galaxybot in #18063

Enable flake8-implicit-str-concat ruff rules by @nsoranzo in #18067

Script for deleting userless histories from database + testing + drop unused model testing code by @jdavcs in #18079

Add Net datatype by @martenson in #18080

Empower users to bring their own storage and file sources by @jmchilton in #18127

Harden User Object Store and File Source Creation by @jmchilton in #18172

Update db revision 24.1 release tags by @jdavcs in #18183

Tighten axt sniffer by @martenson in #18204

More structured indexing for user data objects. by @jmchilton in #18291

Chore: remove repetitive words by @tianzedavid in #18076

Fix import broken with forward merge by @mvdbeek in #18152

Downgrade count lines error to warning by @mvdbeek in #18175

Don’t set dataset peek for errored jobs by @mvdbeek in #18231

Transparently open compressed files in DatasetDataProvider by @mvdbeek in #18248

Raise exception when extracting dataset from collection without datasets by @mvdbeek in #18249

Set page importable to false when serializing by @mvdbeek in #18263

Fix first_dataset_element type hint by @mvdbeek in #18284

Do not copy purged outputs to object store by @mvdbeek in #18342

Fix user’s private role can be missing by @davelopez in #18381

Assign default data extension on discovered collection output by @mvdbeek in #18389

Replace busybox:ubuntu-14.04 image with busybox:1.36.1-glibc by @mvdbeek in #18428

Fix for converter tests by @bernt-matthias in #17188

correct dbkey for minerva display app by @hexylena in #17196

Fix invocation serialization if no state was set by @mvdbeek in #17278

Fix quotas ID encoding by @davelopez in #17335

Fix model store exports that include implicit conversions. by @jmchilton in #17346

Fix bug: create new PSAAssociation if not in database by @jdavcs in #17516

Fix social_core methods by @jdavcs in #17530

Fix ancient bug: incorrect usage of func.coalesce in User model by @jdavcs in #17577

Account for newlines in CIF Datatype sniffer by @patrick-austin in #17582

Anticipate PendingRollbackError in check_database_connection by @mvdbeek in #17598

Add basic model import attribute validation by @mvdbeek in #17628

More efficient change_state queries, maybe fix deadlock by @mvdbeek in #17632

Npz sniffing: do not read the whole file by @bernt-matthias in #17672

Assert that at least one file in npz zipfile ends with .npy by @mvdbeek in #17679

Workflow Comment Indexing by @ElectronicBlueberry in #17700

Fix source history update_time being updated when importing a public history by @jmchilton in #17728

Also set extension and metadata on copies of job outputs when finishing job by @mvdbeek in #17777

Defer job attributes that are usually not needed by @mvdbeek in #17795

Fix change_datatype PJA for dynamic collections by @mvdbeek in #17803

Simplify nested collection joins by @mvdbeek in #17817

Fix very slow user data table query by @mvdbeek in #17830

Update db revision 24.0 release tags by @jdavcs in #17834

Minor refactor of query building logic for readability by @jdavcs in #17835

Fix user login when duplicate UserRoleAssociation exists by @mvdbeek in #17854

Make columns types an empty list for empty tabular data by @bernt-matthias in #13918

port invocation API to fastapi by @martenson in #16707

SQLAlchemy 2.0 upgrades (part 5) by @jdavcs in #16932

Python 3.8 as minimum by @mr-c in #16954

Expose more tool information / navigability in UI. by @jmchilton in #17105

Add support for (fast5.tar).xz binary compressed files by @tuncK in #17106

SA2.0 updates: handling “object is being merged into a Session along the backref cascade path” by @jdavcs in #17122

Towards SQLAlchemy 2.0: fix last cases of RemovedIn20Warning by @jdavcs in #17132

Create pydantic model for the return of show operation - get: /api/jobs/{job_id} by @heisner-tillman in #17153

Much simpler default dataset permissions for typical users. by @jmchilton in #17166

Add future=True flag to SA engine by @jdavcs in #17174

Add future=True flag to SA session by @jdavcs in #17179

Vueifiy History Grids by @guerler in #17219

Convert sample object store configuration to YAML and support configuring inline by @natefoo in #17222

Migrate models to pydantic 2 by @mvdbeek in #17262

API endpoint that allows “changing” the objectstore for “safe” scenarios. by @jmchilton in #17329

Enable warn_unreachable mypy option by @mvdbeek in #17365

Fix type annotation of code using XML etree by @nsoranzo in #17367

Add explicit cache_ok attribute to JSONType subclass by @mvdbeek in #17376

More specific type annotation for BaseJobExec.parse_status() by @nsoranzo in #17381

Update to black 2024 stable style by @nsoranzo in #17391

Allow using tool data bundles as inputs to reference data select parameters by @mvdbeek in #17435

UI for “relocating” a dataset to a new object store (when safe) by @jmchilton in #17437

Allow filtering history datasets by object store ID and quota source. by @jmchilton in #17460

Faster FASTA and FASTQ metadata setting by @bernt-matthias in #17462

Feature SBOL datatypes by @guillaume-gricourt in #17482

Display workflow invocation counts. by @jmchilton in #17488

add npy datatype by @astrovsky01 in #17517

Enhance Avivator display app to support regular Tiffs by @davelopez in #17554

Update Python dependencies by @galaxybot in #17580

Add migrations revision identifier for 24.0 by @jdavcs in #17589

Rename to_dict to populate in FormDefintion to fix bug by @jdavcs in #16553

MINERVA display application: enable cors, add for tabular by @hexylena in #16737

Use AlignedSegment.to_string by @mvdbeek in #16803

Ruff and flake8 fixes by @nsoranzo in #16884

prep for updated h5py and typos by @mr-c in #16963

Fix cardinality violation error: subquery returns multiple results by @jdavcs in #17224

Fix Display Application link generation by @mvdbeek in #17227

Display application fixes and tests by @mvdbeek in #17233

Rollback invalidated transaction by @jdavcs in #17280

Set metadata states on dataset association, not dataset by @mvdbeek in #17474

Provide working routes.url_for every ASGI request by @mvdbeek in #17497

Implement default locations for data and collection parameters. by @jmchilton in #14955

Display beginning of non-text files as text instead of triggering a download by @SergeyYakubov in #15447

Tool Shed 2.0 by @jmchilton in #15639

Limit number of celery task executions per second per user by @claudiofr in #16232

Delete non-terminal jobs and subworkflow invocations when cancelling invocation by @mvdbeek in #16252

Towards SQLAlchemy 2.0 (upgrades to SA Core usage) by @jdavcs in #16264

Update Python dependencies by @galaxybot in #16409

Towards SQLAlchemy 2.0 (upgrades to SA ORM usage in /test) by @jdavcs in #16431

SQLAlchemy 2.0 upgrades to ORM usage in /lib by @jdavcs in #16434

Update Python dependencies by @galaxybot in #16436

Implement datatype upload warnings by @jmchilton in #16564

Support new genome browser chain file format by @claudiofr in #16576

Workflow Comments 💬 by @ElectronicBlueberry in #16612

Bump samtools converters by @bernt-matthias in #16668

Misc. edits/refactorings to session handling by @jdavcs in #16712

SQLAlchemy 2.0 upgrades (part 2) by @jdavcs in #16724

Migrate collection elements store to Pinia by @davelopez in #16725

Reset autocommit to False by @jdavcs in #16745

Drop legacy server-side search by @jdavcs in #16755

Optimize iteration in DatasetInstance model + SA2.0 fix by @jdavcs in #16776

Migrate a part of the jobs API to Fast API by @heisner-tillman in #16778

Replace file_name property with get_file_name function by @SergeyYakubov in #16783

Updated path-based interactive tools with entry point path injection, support for ITs with relative links, shortened URLs, doc and config updates including Podman job_conf by @sveinugu in #16795

Allow partial matches in workflow name tag search and search all tags for unquoted query by @ahmedhamidawan in #16860

Vueify Visualizations Grid by @guerler in #16892

Standardize to W3C naming for color. by @dannon in #16949

Move and re-use persist_extra_files by @mvdbeek in #16955

Fix invocation report to target correct workflow version. by @jmchilton in #17008

optimize object store cache operations by @SergeyYakubov in #17025

Use python-isal for fast zip deflate compression in rocrate export by @mvdbeek in #17342

Add magres datatype by @martenson in #17499

Merge 23.1 into dev by @mvdbeek in #16534

Merge release_23.1 into dev by @mvdbeek in #16768

Create db head merge revision. by @dannon in #16838

merge release_23.1 into dev by @martenson in #16933

Minor clarification/typo fix in datatypes.data by @dannon in #16993

Fix .file_name access in merge forward by @mvdbeek in #17097

Backport model store fixes by @mvdbeek in #17439

Fix succces typo by @mvdbeek in #17481

Fix User.current_galaxy_session by @mvdbeek in #17232

Backport #17188: Fix for converter tests by @mvdbeek in #17231

Skip change_datatype things if we’re not actually changing the extension by @mvdbeek in #16931

Fix copying metadata to copied job outputs by @mvdbeek in #17007

Update tar_to_directory dependency by @mvdbeek in #17009

Assert that DatasetCollectioElement has an associated object by @mvdbeek in #17071

Fix input dates in notifications: consider timezone offset by @davelopez in #17088

Allow relative URLs in broadcasts action links by @davelopez in #17093

Improve invocation error reporting by @mvdbeek in #16917

Add support for larch datatypes by @patrick-austin in #17080

Fix bad auto-merge of dev. by @jmchilton in #15386

Merge conflicting db migration branches into one by @jdavcs in #15771

Enable strict_equality mypy option by @nsoranzo in #15808

Fix revision scripts, run migrations in CI, add repair option, improve migrations utils by @jdavcs in #15811

Fix anonymous user uploads when vault is configured by @tchaussepiedifb in #15858

Fix nullable deleted column in API Keys table by @davelopez in #15956

Attempt to fix mypy check by @davelopez in #16103

Ensure session is request-scoped for legacy endpoints by @jdavcs in #16207

Reset autocommit to True (temporarily) by @jdavcs in #16283

Update 23.1 release tags for migration scripts by @jdavcs in #16294

Fix form builder value handling by @guerler in #16304

Fix tags ownership by @davelopez in #16339

Fix disk usage recalculation for distributed object stores by @mvdbeek in #16380

Job cache fixes for DCEs by @mvdbeek in #16384

Fix histories count by @davelopez in #16400

Fix double-encoding notification content by @mvdbeek in #16530

Optimize getting current user session by @mvdbeek in #16604

Fixes for conditional subworkflow steps by @mvdbeek in #16632

Copy the collection contents by default when copying a collection by @mvdbeek in #16717

Fix allowlist deserialization in file sources by @mvdbeek in #16729

Fix workflow output display without label by @mvdbeek in #16749

Fix and prevent persisting null file_size by @mvdbeek in #16855

Fix workflow import losing tool_version by @mvdbeek in #16869

Remove more flushes in database operation tools by @mvdbeek in #16875

Fix join condition for nested collection query by @mvdbeek in #16880

Empower Users to Select Storage Destination by @jmchilton in #14073

Add Storage Dashboard visualizations for histories by @davelopez in #14820

Towards decoupling datatypes and model by @jdavcs in #15186

Add Storage Management API by @davelopez in #15295

OIDC tokens by @SergeyYakubov in #15300

Expose additional beaker caching backends by @claudiofr in #15349

Follow up to #15186 by @jdavcs in #15388

Add support for visualizing HDF5 datasets. by @jarrah42 in #15394

Towards SQLAlchemy 2.0: drop session autocommit setting by @jdavcs in #15421

Update Python dependencies by @galaxybot in #15435

Unify url handling with filesources by @nuwang in #15497

Move TS to Alembic by @jdavcs in #15509

Update Python dependencies by @galaxybot in #15564

Update database_heartbeat for SA 2.0 compatibility by @jdavcs in #15611

Update supports_skip_locked, supports_returning for SA 2.0 compatibility by @jdavcs in #15633

Add Galaxy Notification System by @davelopez in #15663

Drop model mapping unit tests by @jdavcs in #15669

Add transactional state to workflow scheduling manager by @jdavcs in #15683

Remove DELETED_NEW job state from code base by @jdavcs in #15690

Fix/Enhance recalculate disk usage API endpoint by @davelopez in #15739

Drop database views by @jdavcs in #15876

Update Python dependencies by @galaxybot in #15890

FITS data format by @volodymyrss in #15905

Improve display chunk generation for BAMs by @wm75 in #15972

Add History Archival feature by @davelopez in #16003

Add alter_column migration utility by @jdavcs in #16009

Code cleanups from ruff and pyupgrade by @nsoranzo in #16035

Add missing fields to HistorySummary schema model by @davelopez in #16041

Dataset chunking tests (and small fixes) by @jmchilton in #16069

Add create_foreign_key migration utility by @jdavcs in #16077

Refactor models enums to eliminate schema dependency on model layer. by @jmchilton in #16080

Use automated naming convention to generate indexes and constraints in database by @jdavcs in #16089

Add zipped mongodb and genenotebook datatypes by @abretaud in #16173

Drop workarounds for old ro-crate-py and docutils versions by @mvdbeek in #16198

bring grids for (published) pages on par with workflows by @martenson in #16209

Add Visium datatype for squidpy and spatialomics tools by @astrovsky01 in #16255

Filter deleted keys from api_keys relationship by @dannon in #16321

Increase CustosAuthnzToken.external_user_id column size by @davelopez in #16818

Follow up on object store selection PR. by @jmchilton in #15654

Fix Enums in API docs by @davelopez in #15740

merge release_23.0 into dev by @martenson in #15830

Fix linting of FITS datatype code by @mvdbeek in #16169

backport of pysam tostring by @martenson in #16822