Skip to main content

A tool to standardize French mobility survey datasets (EMC2, EGT, EMP, etc.) into a unified format.

Project description

Contributors Forks Stargazers Issues GPL v3 DOI

MobiSurvStd

A tool to standardize French mobility survey datasets (EMC², EGT, EMP, etc.) into a unified format.

📚 Documentation 📦 View on PyPI

🇫🇷 Version française

Table of Contents

Introduction

MobiSurvStd (Mobility Survey Standardizer) is an easy-to-use Python command line interface to convert many French mobility surveys (EMC², EGT, EMP, etc.) to a unique standardized and clean format.


Mobility Surveys in France

In France, despite recent efforts by CEREMA to create a standard format for mobility surveys – with the EMC² surveys – various formats co-exist:

  • EMC² (Enquête mobilité certifiée CEREMA): mobility surveys for many French territories (since 2018);
  • EGT H2020 (Enquête Globale Transport, Île-de-France Mobilités): mobility survey for Île-de-France (2018–2020; incomplete due to COVID-19);
  • EMP (Enquête mobilité des personnes, SDES): national mobility survey (2019).
  • EMG (Enquête Mobilité par GPS, Institut Paris Région): mobility survey for Île-de-France (2022-2023) with individuals tracked over several days through GPS.

Also surveys based on previous formats are still in use today:

  • EDVM (Enquêtes Déplacements Villes Moyennes, CEREMA): mobility surveys for medium-size cities (until 2018);
  • EDGT (Enquêtes Déplacements Grands Territoires, CEREMA): mobility surveys for periphery areas (until 2018);
  • EMD (Enquêtes Ménages Déplacements, CEREMA): other mobility surveys from CEREMA (until 2018);
  • EGT 2010 (Enquête Globale Transport, Île-de-France Mobilités): previous version of the Île-de-France mobility survey;
  • ENTD (Enquête nationale transports et déplacements, SDES): former national mobility survey (2008).

Why MobiSurvStd?

The existing formats all have the same drawbacks:

  • Data are sorted in CSV files which are not always straightforward to read (Which separator? Which encoding? What are the variable datatypes?).
  • Variable names and modalities are not always clear (e.g., in the EMC² format, variable "P2" represents the gender of the person, with modality 1 for a man and 2 for a woman).
  • Joining two datasets is hard and not well documented (e.g., in the EMC² format, to join the persons with their household, the variables to use are "METH", "ZFM" and "ECH" for the households and "DMET", "ZFD" and "ECH" for the persons).

Additionally, when working with different territories / periods, it is often necessary to write a similar code multiple times due to the existence of different formats.

MobiSurvStd solves all these issues by being able to convert all survey formats to a well-defined Parquet format.

See this example to understand how MobiSurvStd can simplify your workflow.


Installation

Install the library with

pip install mobisurvstd

Getting Started

Download the EMP 2019 survey (Données individuelles anonymisées (fichiers au format CSV)), then run the following command:

python -m mobisurvstd emp_2019_donnees_individuelles_anonymisees_novembre2024.zip standardized_emp2019 --survey-type emp2019

It will standardize the EMP 2019 survey and save the resulting Parquet files in the standardized_emp2019 directory. These Parquet files can then be analyzed using e.g., polars or pandas in Python or arrow in R.

Note: During standardization, MobiSurvStd may issue warnings when certain variable guarantees are not initially satisfied but are automatically fixed. These warnings can usually be safely ignored, unless they pertain to a large number of observations.

A detailed definition of the Parquet format used by MobiSurvStd is available here.

You can also standardize surveys programmatically in Python with the standardize function:

import mobisurvstd
mobisurvstd.standardize(
    "emp_2019_donnees_individuelles_anonymisees_novembre2024.zip",
    "standardized_emp2019",
)

For more, check the User Guide.

Citing MobiSurvStd

If you use MobiSurvStd in a publication or project, please include the following attribution:

Javaudin, L., 2025. MobiSurvStd. URL: https://doi.org/10.5281/zenodo.17422994,
doi:10.5281/zenodo.17422994

Case Study: Bicycle Use

The following graph represents the share of bicycle trips for EMC², EDGT, EDVM, EMD, and EGT surveys. The circle colors represent the average number of bicycles in the surveyed households. The circle sizes represent the expected number of trips in the surveyed area.

The graph has been generated from the code in analyses/bicycle_shares.py.

Graph of bicycle share by survey

The following map represents the share of bicycle trips within INSEE municipalities. Only municipalities with more than 30 surveyed trips are shown.

The map has been generated from the code in analyses/bicycle_shares_by_insee.py.

Map of bicycle share by INSEE municipality


Supported Surveys

Currently, MobiSurvStd supports the following survey types:

See Survey Types for more.

Other survey types that are planned to be integrated are: entd, emg. If you know another survey format that could be integrated, feel free to open an issue on GitHub.

MobiSurvStd covers only French mobility survey formats. If other countries have similar survey formats, they might be easily integrated into MobiSurvStd (or a variant of it).


Legal Notice

⚠️ MobiSurvStd does not anonymize the data. If you are working with confidential datasets (e.g., EMC² or EGT surveys), you must apply the same confidentiality rules to the standardized data as to the original data. In particular, you must not share the standardized data if your confidentiality agreement prohibits sharing the original data.

Contributing

If you think you found a bug, if you have a suggestion, or if you want to integrate a new format, feel free to open an issue on GitHub or to create a Pull Request.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

mobisurvstd-1.0.0.tar.gz (137.8 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

mobisurvstd-1.0.0-py3-none-any.whl (166.5 kB view details)

Uploaded Python 3

File details

Details for the file mobisurvstd-1.0.0.tar.gz.

File metadata

  • Download URL: mobisurvstd-1.0.0.tar.gz
  • Upload date:
  • Size: 137.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: uv/0.9.11 {"installer":{"name":"uv","version":"0.9.11"},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

File hashes

Hashes for mobisurvstd-1.0.0.tar.gz
Algorithm Hash digest
SHA256 ff0ea0c54140b6ee49adff9007ceef4c297b8f849414a32eab83e9820d67c8d9
MD5 d3890bfe100a563dd20e7add9ee99a83
BLAKE2b-256 975349ec18f07e9ee864ad3a8c7e788ceec7631cb58ca1de0860b3e46b2bd861

See more details on using hashes here.

File details

Details for the file mobisurvstd-1.0.0-py3-none-any.whl.

File metadata

  • Download URL: mobisurvstd-1.0.0-py3-none-any.whl
  • Upload date:
  • Size: 166.5 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: uv/0.9.11 {"installer":{"name":"uv","version":"0.9.11"},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

File hashes

Hashes for mobisurvstd-1.0.0-py3-none-any.whl
Algorithm Hash digest
SHA256 ec9d20aed23fda4a077171cae9157e172f11478f596e376185c634984950e2d7
MD5 7ffeb325c1f22f69085bf6d8a3f4db75
BLAKE2b-256 f9f48209d90ef4d97d48df74caf28d115f2de7752b8504d3ef33d1f873ccb781

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page