udata-front 4.0.3.dev20224

udata customizations for data.gouv.fr

Add a front-end to udata

• Notes on this repo
• Compatibility
• Installation
• Theme development
  • 🚀 Getting started
    • 🐍 Python development
      • 🧱 Installing the python dependencies
      • 🚩 Starting the python development server
    • ☕ Javascript development
      • 🏗 Installing the javascript dependencies
      • 💪 Starting the javascript development server
    • 👀 Other dev commands
  • 🏰 General architecture
    • 🚜 Jinja2 templates
    • 🚲 Javascript
    • 🖼️ Style
  • 🛠️ Build tools
  • 🏭 Javascript architecture
    • 🏔️ Vue mounting
    • 👩‍🔬 Tests
• 📖 Read more

Notes on this repo

This is a new version of udata-gouvfr This is a udata extension, you should read the udata documentation first.

Compatibility

udata-front requires Python 3.7+ and udata.

Installation

Install udata.

Remain in the same Python virtual environment and install udata-front:

pip install udata-front

Create a local configuration file udata.cfg in your udata directory (or where your UDATA_SETTINGS point out) or modify an existing one as following:

PLUGINS = ['front']
THEME = 'gouvfr'

Theme development

The front-end theme for the public facing website, is split into two parts :

• The Jinja templates are located inside udata_front/theme/gouvfr/templates.
• The Less, Vue & other sourcefiles for the front-end are located in theme.

🚀 Getting started

Before you start your developper journey, you have to setup your python and/or javascript development tools.

It is recommended to have a workspace with the following layout:

$WORKSPACE
├── fs
├── udata
│   ├── ...
│   └── setup.py
├── udata-front
│   ├── ...
│   └── setup.py
└── udata.cfg

Modify your local udata.cfg configuration file as following:

PLUGINS = ['front']
THEME = 'gouvfr'

🐍 Python development

🧱 Installing the python dependencies

Prepare a udata development environment.

Note that we're using pip-tools on this repository too.

The following steps use the same Python virtual environment as udata.

Install udata-front in development mode:

cd udata-front
pre-commit install
pip install -e . -r requirements/test.pip -r requirements/develop.pip

NB: the udata.(in|pip) files are used by the CI to stay in sync with udata requirements. You shouldn't need to tinker with them on your local environment, but they might be updated by the CI when you make a Pull Request.

🚩 Starting the python development server

Simply run the udata project with udata-front loaded as a plugin:

cd udata
inv serve

☕ Javascript development

🏗 Installing the javascript dependencies

First, you need to use Node (version 16+) on your platform. You should consider installing NVM which uses the existing .nvmrc.

cd udata-front

nvm install
nvm use

npm install

And voilà ! ✨

🆕 Start the storybook server

If you want to work on Vue or Less files, you can start the storybook server with this command :

npm run storybook

It allows you to work on the front-end only and even mocks the back-end responses.

You can visit their website to learn more about storybook and how to use it.

💪 Starting the javascript development server

Simply run this command in the project directory :

npm start

This will start a development server that will listen to changes and automatically rebuild the project when needed. Note that a webserver is started by Vite (default port is 1234), however we will not be using it as our CSS and JS files will be served by Jinja instead. More on that later.

👀 Other dev commands

Finally, we have a bunch of commands to make your life a tad easier.

You can execute udata-front specific tasks from the udata-front directory with invoke. You can list available development commands with:

inv -l

Example commands:

• i18n: Extract translatable strings
• i18nc: Compile translations
• qa: Run a quality report
• test: Run tests suite

Additionally, you can run javascript-related commands through npm run.

• build: Builds the final CSS/JS files. You should probably use this one in production.
• i18n:report: Generates a report of the i18n missing and unused keys
• i18n:extract: Same as above, but also automatically adds missing keys to translation files
• start: Get to coding with live reload and things. Same as npm run dev
• test: Runs the Cypress tests. More on that in the Tests section of this README.

If you encounter any merge conflict with your package-lock.json, you can fix it with NPM:

npm install --package-lock-only

🏰 General architecture

🚜 Jinja2 templates

Because udata is written in Python, its templating engine is Jinja 2. This means that the HTML received by clients is built at runtime, for each request, using templates with {% block %} tags and includes.

Those template are responsible for building the pages using layouts and blocks. Here are a few to get started (in udata_front/theme/gouvfr/templates), from less specific to more specific :

• raw.html : contains the general html structure exposing a body block where we can write our page's body. This template is also responsible for including the CSS and JS files.
• base.html : contains some extra html structure exposing a content block for our page's content, and wraps it around the header and footer.
• header.html and footer.html : standard header and footer block that will appear on each page
• home.html: the home page template (duh)

🚲 Javascript

In order to add some interactivity to the project, we are using Vue 3 and some good old VanillaJS. The JS assets are compiled in a single index.js file that includes everything for every page. If the bundle size starts to grow a little bit too much, you might need to think about splitting it into separate files for each page.

🖼️ Style

We are using the DSFR to build our front-end components.

In addition we have a nice litle set of CSS Utilities to quickly build custom components, inspired by bootstrap, most of its documentation lives in the css located in theme/less/.

Whenever a components needs some special styling, you can find their corresponding definitions inside theme/less/specific/<component>, it's best if we can avoid having too much specific styling, but sometimes you just really need it.

🛠️ Build tools

This project uses Vite to build and transform our source files into nice bundles for browsers. Its config can be found in the vite.config.js file.

Vile does multiple custom things in this project :

• Transform the .js files into modern Javascript for browsers
• Transform the less files into modern CSS using PostCSS
• Copy the static assets when they change (config is in the vite.config.js)

Vite uses package.json version to name files and udata-front uses its version to load the correct one. If you're udata-front version doesn't match the one loaded in the theme, you may have to do a pip install -e . to update the package information.

🏭 Javascript architecture

🏔️ Vue mounting

We are using the full build of VueJS that includes the compiler in order to compile templates directly in the browser.

There is a single VueJS app (in index.js) that contains every component and plugins. However, this app is mounted multiple times, on each DOM node containing a vuejs class.

This allows us to mount the app only where it's needed, because each subsequent mount is more DOM to compile and thus has an impact on performance. Moreover, mounting to the smallest possible HTML allows us to prevent accidental XSS vulnerability by forbidding users to compile their content with the Vue engine.

In order to allow inter-component communication, a global event bus is available in the global app, under the $bus variable. You can emit events by using $bus.emit('event') and components can listen to events by using $bus.on('event').

👩‍🔬 Tests

Tests are run in a headless browser using Cypress. Test definitions are located in the cypress/integration directory.

Writing tests is very easy thanks to its syntax :

  it("Displays the page title", () => {
    cy.get("h1").should("be.visible");
  });

Then, tests can be run using the following command :

npm run test

Cypress also comes with cypress-axe to allow for accessibility automated testing.

Changelog

Current (in progress)

• Add Organization publishing form #358
• Fix an error that blocks datasets search filters reset #402
• Componentize dataset card #397
• Use udata schema endpoint #336

4.0.2 (2024-04-26)

• Update footer #405

4.0.1 (2024-04-26)

• Update footer

4.0.0 (2024-04-23)

• breaking change Migrate to Python 3.11 #376
• Fix api urls locked on dev.data.gouv.fr #401

3.5.5 (2024-04-16)

• Add beta admin member page #374
• Move Pagination to datagouv-components #365
• Add stories to Resource component #364
• Move Well to datagouv-components #382
• Add markdown editor #351
• Componentize quality component #385
• Add codes and optgroups in Multiselect to display Insee codes and Licence groups [#347] (https://github.com/etalab/udata-front/pull/347)
• Fix datastructure display in ResourceAccordion #389
• Order Organization's reuses by publishing date #390
• Upgrade vue dependency #386
• Fix failing captchEtat responses #392
• Fix release CI #380 #384
• Update constants imports from .models to .constants #375
• Fix rename of VisibleDatasetFactory #387
• Add beta admin member page #374
• Add Administration Panel to datagouv-components [#378] (https://github.com/etalab/udata-front/pull/378)

3.5.4 (2024-03-20)

• Fix footer links and some translations #366
• Show error when dataset discussion from url doesn't exist #367
• Hide quality score on "work in progress" backends #370
• Add a new vite config for development #372
• Fix structure tab not shown #369
• Update sentry configuration to ignore aborted requests #368
• Fix CI not using udata release when needed #363

3.5.3 (2024-02-22)

• Show a map of datasets' spatial coverage #354
• Fix redirect user to auth page #357
• Fix tests factories for HarvestSource, backend is now required udata#2962
• Fix schema is not longer a dict #361 udata#2949

3.5.2 (2024-02-15)

• Show a button to mark as no spam a discussion or a message #352

3.5.1 (2024-02-05)

• Fix vue runtime errors #343
• New featured topic : agriculture #344
• Create data.gouv.fr-components NPM package and move some gouvfr components to it #324#350
• Setup storybook for data.gouv.fr-components #346

3.5.0, 3.4.0 and 3.3.0 (2024-01-23)

Important
These versions contain an invalid js build and are yanked from pypi. The update to the next version is mandatory to have a valid js build.

• Fix dependencies according to udata's Flask-Babel migration #300
• User pages are back #338

3.2.12 (2023-12-15)

• Fix support menu link #335

3.2.11 (2023-12-08)

• Remove parents, children, etc. from territory views #328
• Add publishing form #266 dependabot/36 #331
• New featured topics : culture and education #332
• Fix territory sitemap #334
• Add header-case to validation url #333

3.2.10 (2023-12-01)

• Ignore another ResizeObserver error #321
• Fixed ol not appearing in descriptions #322
• Fix copy discussion link #323
• Add text-transform-none style and use lowercase k as unit #325

3.2.9 (2023-11-21)

• Update build dependencies #309
• Add read-more to discussions #310
• New footer with newsletter #311 #316
• Add metric components and hooks #260 #313 #314
• Add raise_for_status on captchetat request #318

3.2.8 (2023-10-26)

• Add German translations files and French latest translations #307

3.2.7 (2023-10-26)

• Update testing dependencies dependabot/30 dependabot/22 dependabot/23 #297 #298
• Fix translate links in footer #301
• Track participez clicks #302
• Fix datasets search on the organization page #303
• Add discoverability indicator form #304

3.2.6 (2023-09-01)

• Add missing quality criterion (all_resources_available) #287
• Replace linkedin link in footer #292
• Use the word file instead of resource #291
• Update certified icon #288
• Add guides to the menu #290
• Add resource permalink #286 #295
• Update read-more height on details toggle #294
• Fix preview style discrepancies #289

3.2.5 (2023-07-19)

• Fix reuse page padding for discussions and reuses section #270
• Fix dataset oembed links #271
• Hide CaptchEtat icon loader #272
• Update guide links #276
• Redirect pages when missing trailing slash #278
• Add feedback link to header and footer #275
• Fix actions with scroll inside full-page tabs (e.g. dataset page) #281
• Add data search form to search #274
• Upgrade pyyaml dependency to 6.0.1 in deps tree #283
• Set confirmed_at when creating user in MonComptePro auth #284

3.2.4 (2023-06-19)

• Fix word-wrap for dataset description #254
• Fix img folder not copied #255
• Add breadcrumbs to pages #258
• Remove cache around dataset, reuse and organization modify button #256
• Fix display temporal coverage on dataset page #261
• Fix tooltip accessibility #259
• Compute unavailability based on check:available instead of status logic #267
• Fix dependabot/17 and dependabot/18 #264. These aren't udata-front vulnerabilities but only impact vite dev server (unused in gouvfr theme).
• Update DSFR to 1.9 #249
• Fix tab sequence on dataset page #265
• Fix button to show closed discussion #253
• Replace housing venti button by agricultural topic #268
• Fix header search #269

3.2.3 (2023-05-16)

Note
This renames the theme root folder to assets and moves it inside gouvfr theme. This also updates most references of udata_front/theme/gouvfr theme to * to ease the usage of other themes.

• Move theme to udata_front/theme/gouvfr#244 #252
• MonComptePro SSO integration #237:
  • New button on login and register page
  • When loging in, the datastore will seek for a coresponding user on udata. If such user does not exist, she will be created.
  • Created user during SSO will not have a password. To use the user without SSO, a password reset procedure must be used.
• Handle previous format of link to discussions, e.g. from e-mails #241
• Add last_update sort in datasets page #242
• Fix .fr-btn in .markdown #243
• Add Matomo event tracking #246
• New scope for Captchetat piste OAuth #250
• Use datetime.utcnow to make sure to handle utc datetimes #251

3.2.2 (2023-04-18)

• New organization page #230#233
• Update the login form after Flask-Security and WTForms upgrade #224
• Align reuse sidebar with image #234
• Show relative date on dataset and resource cards #231
• Add version in chunk file names for cache invalidation #239
• Fix RGAA criterion 8.9 #174

3.2.1 (2023-03-28)

• Front modifications to display resources schema url field #220
• Update ventti button url #223
• Dataset page fixes #219 #229
• Fix "informations" throughout the repo #218
• fix RGAA criterion 9.3 #222
• New reuse page #210
• Fix dependabot/10 #228
• Fix links color #232

3.2.0 (2023-03-07)

• Upgrading packages following Flask upgrade to 2.1.2 in udata #207
  • Use feedgenerator for AtomFeed instead of Werkzeug deprecated helper
  • contextfunction and contextfilter from Jinja is deprecated and replaced by pass_context
  • Move Flask-Themes2 dep from udata to udata-front and upgrade to 1.0.0
• Fix @background-contrast-blue-cumulus LESS variable #217

3.1.3 (2023-03-02)

Warning Refactor of breadcrumb macro to be easier to use. toolbar_class, breadcrum_class, toolbar_container and breadcrumb_bar options are now removed. Refactor of dataset components names to match new sizes : XS (previously card), SM, MD, LG (previously search-result). dataset.display.after-description hook is now really after the description. Use new dataset.display.after-files hook for previous position.

• Make newsletter url configurable #205
• Show a warning notice when JavaScript is disabled or failed to execute #206
• Update lighthouse to fix security issues #211
• New dataset page #181
• Update vue-toaster dependency to avoid reported vulnerabilities #215

3.1.2 (2023-02-06)

• Use brand color for button style tertiary-no-outline #199
• Use computed dates for resources and datasets #201
• Fix setuptools version used in CI #202
• Move Pagination to @etalab/udata-front-plugin-helpers and add tests #200

3.1.1 (2023-01-20)

• Fix vanilla js scripts #196
• Remove useless published date in resource model #198

3.1.0 (2023-01-18)

Note
This changes the build system from Parcel to Vite. This also adds a way for udata plugin to register their own vue components and to display them in places chosen by the current theme

• Add resource exploration preview #169#180#183
• Fix RGAA criterion 9.2 #178
• Add i18n on size suffix #184
• Add theme view for mail change #192

3.0.1 (2022-12-15)

• Fix Api Representation for media after CaptchEtat integration #173
• Add archived and private badges to dataset search results and update card style #170
• Fix banner links underlined twice #171
• Fix margins for dataset cards #176
• Fix text for datasets button in org page #172
• Add Portuguese translations #167
• Add email address to shared data on oauth authorize page #175

3.0.0 (2022-11-14)

• :warning: Breaking change Use and display harvest metadata introduced in udata 5 #168
• Improve search history #162

2.0.13 (2022-11-02)

• Switch from Flask-restplus to its fork Flask-rest-x #165
• Add CaptchEtat integration #159
  • new environment variables : CAPTCHETAT_BASE_URL, CAPTCHETAT_OAUTH_BASE_URL, CAPTCHETAT_CLIENT_ID and CAPTCHETAT_CLIENT_SECRET
• Fix lighthouse reported errors #158

2.0.12 (2022-10-19)

• Fix i18n errors for /es #156
• Update API card #160
• Create new search results component #157
• Remove quality score banner #163
• Add energy on home venti button #164
• Cache behavior changes #154:
  • Organization and Reuse display page are now cached.
  • Cache keys now embed the last_modified object attribute. This automatically invalidates the cache when modifying the object.

2.0.11 (2022-09-02)

• Fix discussions text wrap #145
• Update Venti buttons #146
• :warning: @blue-470 and @blue-500 are removed
• Fix z-index value in dataset search-result template #153 #155
• Fix RGAA criterion 8.2 #147

2.0.10 (2022-08-11)

• Fix dataset search result link to organization #150

2.0.9 (2022-08-10)

• Fix selected tag with wrong color #149

2.0.8 (2022-08-09)

• Add lighthouse in CircleCI #108
• Fix RGAA criterion 8.2 #130
• Add quality score #135

2.0.7 (2022-07-20)

• Fix window.dsfr.register error #138
• Fix featured toggle #137
• Iterate on search results and cards #136

2.0.6 (2022-07-08)

• Fix RGAA criterion 7.5 #118
• Remove map related stuff #124
• Fix clear button now shown on multiselect #125
• Add aria-current to breadcumbs #121
• Add missing default og:image #127
• Fix 500 on favicon in admin #126
• Update search results #110 #134
• Fix test error with long reuse title #133
• Removed manifest logic #129

2.0.5 (2022-06-14)

• Add new menu items and change edito pages slug #113 #120
• Replace news by release notes in footer #117
• Use DSFR container and remove custom ones #111

2.0.4 (2022-06-09)

• Add accessibility compliance status in footer #114
• Fix SVG display issue #116

2.0.3 (2022-06-03)

• Update search pages #95
• Add support for HTML in posts #106
• Fix RGAA criterion 1.1 #104
• Fix RGAA criterion 10.4 #103
• Add geographical page on home venti button #109
• Fix RGAA criterion 11.10 #102
• Update DSFR to 1.5.1 #107
  • :warning: SVG in JS must use bundle-text: prefix now

2.0.2 (2022-04-11)

• Add harvest catalog view #100
• Add elections on home venti button #101

2.0.1 (2022-04-05)

• Add support for HTML static pages and more DSFR components #96
• Fix mobile bugs after header changes #99
• Fix organizationCertified error when organization is null #98

2.0.0 (2022-03-30)

Breaking change

• :warning: Use refactored search endpoints from udata #60

1.2.5 (2022-03-29)

• Add a transport banner hook #94
• Add button on organization page to see all of its datasets #93
• Format home page numbers #90
• Let browsers decide what cursor to use #89
• Replace see more button on home page with link #91
• Replace Suggest with accessible combobox #88

1.2.4 (2022-03-01)

• Deprecation: Topics are now deprecated and will be removed in upcoming releases.
• Fix <read-more> component height when it contains <img> #65 #85
• Add featured button component back for sysadmin #79
• Update reuse style #52 #81
• Add banner to broken user page #76
• :warning: Button changes #75
  • Remove underline from button hover
  • .btn, .btn-secondary and .btn-secondary are removed. Use DSFR and .fr-btn--secondary-{color} instead.
  • .tags and .tag are removed. Use DSFR ones instead.
  • .dropdown is removed. Use DSFR select instead.
• Fix duplicate request on dataset search #70 86
• Add banner for harvested dataset #73
• Change github footer link to the tickets repository #80
• Remove banner for the new search beta on datasets search page #83
• Fix RGAA criterion 7.3 #82
• Use avatar_url for owner #84
• Update resources style #78

1.2.3 (2022-01-27)

• Fix modals not working #71
• Fix auth messages not shown from query parameter #68
• Fix RGAA criterion 10.14 #72
• Fix thread header wrapped when title is too long #64

1.2.2 (2022-01-21)

• Fix latest modification date for dataset and resources on dataset page #62
• Fix hidden datasets shown on Home and Reuses #67
• Add temporal coverage back to dataset page #63
• :warning: @bg-beige is remove, use @background-contrast-grey instead
• Update colors to fix accessibility issues #56
• Fix missing checkbox using DSFR checkboxes #69

1.2.1 (2022-01-11)

• Change urls in Participate banner to relevant etalab guides #53
• Add topic information on reuse metadata and add a filter by topic on reuse search page #50
• Update DSFR to v1.2.1 #45
• :warning: .btn-tab is removed, use .fr-tag instead 57

1.2.0 (2021-12-10)

• Add a banner for the new search beta on datasets search page #43
• :warning: Remove Issues logic in accordance with udata #42
• :warning: @grey-100 is now #e5e5e5
• Standardize organization page similar to dataset and reuse pages #40
• Fix RGAA criterion 10.7 Each element focusable has a visible focus #46
• Fix Stylemark generation to include JS files and properly include other assets #33
• Redirect about page to "ressources" page in menu #48
• Standardize article discussions and quick fixes to discussions #41 #51
• Fix error on search request cancelation #44

1.1.2 (2021-11-23)

• Standardize reuse page similar to dataset page navigation quickfixes #31
• Move template hook logic to udata and add oauth hooks #29
• Add resources pagination dataset page and use DSFR pagination #30 #37
• Style oauth page #34
• Fix horizontal scroll on mobile #38
• Fix gouvfr static path #39

1.1.1 (2021-10-22)

• Update README to reflect front changes #17
• Add Participate banner in the footer #24
• Fix min-height used in posts images to center them #23
• Update dataset page with navigation quickfixes and add DSFR components #18
• Implement feedbacks on quickfixes #26

1.1.0 (2021-10-12)

• Add Cypress front-end tests stub #9
• Add read only mode back on frontend #10
• Fix RGAA criterion 1.2 Each decorative image is ignored by assistive technologies. #13
• Add a request membership action on organization page #12
• Unset vue delimiters used in html templates to prevent injections #11
• Fix temporal coverage order in search results metadata #14
• VueJS multiple mount points with a global event bus #15 #19
• Fix RGAA criterion 12.6 Block of contents from multiple pages can be reached or skipped #21

1.0.0 (2021-09-16)

• :warning: breaking change: Package renaming and new repository #1:
  • udata-gouvfr is now udata-front
• Update feedparser following setuptools 58.0.2 release that drops support for use_2to3 #6
• Show correct number of latest reuses on homepage #3
• Fix next value on login to prevent infinite loop #4 #8

Previous udata-gouvfr changelog

If you're migrating from udata-gouvfr, see previous changelog here