csv-detective
ID csvdetective
detector
Python package by the data.gouv.fr team that detects what the columns of a tabular file (CSV, Excel, ODS, Parquet) contain. For each column it combines checks of the values (regular expressions, casting, comparison with official lists) with hints from the header. Version 0.13.0 has 55 formats: generic ones such as dates, emails, URLs, UUIDs and WGS 84 or Lambert-93 coordinates, and many French ones such as SIREN, SIRET, INSEE commune and department codes, postal codes, UAI school codes, RNA association numbers, NAF activity codes, FANTOIR street codes and RNB building identifiers; several of them use the frformat library. udata-hydra runs it on the tabular files published on data.gouv.fr. Formats without a registry datatype (int, float, json, binary, canton names, socio-professional categories and historical association numbers) are not listed. Released under the MIT license.
- Website
- https://github.com/datagouv/csv-detective
- Supported types
- 41
Supported semantic types
| ID | Name | Categories | Countries | Description |
|---|---|---|---|---|
| address | Address (physical location) PII | common, pii, geo | global | Geographic address, commonly consist of postindex, country, area, town, street and building |
| boolean | Boolean | common | global | In computer science, the Boolean data type is a data type that has one of two possible values (usually denoted true and false) which is... |
| country | Country code or name | geo | global | A country is a distinct territorial body or political entity. It may be an independent sovereign state or part of a larger state,[2] as... |
| countrycode_alpha2 | ISO 3166-1 Alpha2 country code pattern | geo | global | ISO 3166-1 alpha-2 codes are two-letter country codes defined in ISO 3166-1, part of the ISO 3166 standard published by the... |
| countrycode_alpha3 | ISO 3166-1 Alpha3 country code pattern | geo | global | ISO 3166-1 alpha-3 codes are three-letter country codes defined in ISO 3166-1, part of the ISO 3166 standard published by the... |
| countrycode_num | ISO 3166-1 Numeric code (country code) pattern | geo | global | ISO 3166-1 numeric (or numeric-3) codes are three-digit country codes defined in ISO 3166-1, part of the ISO 3166 standard published by... |
| date | Date | datetime | global | Calendar date. The regexp covers the ISO 8601 extended form YYYY-MM-DD; other notations, such as DD.MM.YYYY, are patterns of this datatype. |
| datetime | Date and time | datetime | global | Date and time of day. The regexp covers the ISO 8601 extended form YYYY-MM-DDThh:mm[:ss[.fff]] with an optional time zone (Z or an... |
| dayofweek | Day of week draft | datetime | global | Week day name |
| easting | Easting (projected coordinate) draft | geo | global | Eastward coordinate of a point in a projected coordinate reference system, such as a UTM zone or the British National Grid, usually in... |
| Email address PII | common, internet, pii | global | Address of an electronic mailbox in the form local-part@domain, defined in section 3.4.1 of RFC 5322. The local part names the mailbox,... | |
| epcicode | Etablissements publics de cooperation intercommunale (EPCI) | geo | FR | Etablissements publics de cooperation intercommunale (EPCI). French administrative geo code |
| fr_commune | French municipality (commune) draft | geo, government | FR | Name of a commune, the smallest administrative unit of France, for example Paris or Mende. Many names occur in more than one department,... |
| fr_department | French department (département) draft | geo, government | FR | Name of a department of France, the administrative level between the region and the commune, for example Ain or Corse-du-Sud. France has... |
| fr_department_code | French department code (INSEE) pattern draft | geo, government | FR | Code of a French department in the INSEE Official Geographic Code (COG). Departments in metropolitan France use two characters from 01... |
| fr_naf | French activity code (NAF rév. 2 subclass, APE code) draft | industry | FR | Code of a subclass of the French classification of activities NAF rév. 2 (2008), which has the same structure as the European NACE.... |
| fr_region | French region (région) draft | geo, government | FR | Name of an administrative region of France, the level above the department, for example Île-de-France or Bretagne. France has 18... |
| fr_region_code | French region code (INSEE) pattern draft | geo, government | FR | Two-digit code of a French region in the INSEE Official Geographic Code (COG). The 18 current codes are 01 to 04 and 06 for the overseas... |
| fr_rivoli | French RIVOLI street code draft | geo | FR | Code of a street, locality or housing estate within its commune in the FANTOIR file of the French tax administration: the four-character... |
| fr_rna | French association number (RNA) draft | companies | FR | Number of an association in the Répertoire national des associations (RNA), the register of associations declared under the 1901 law... |
| fr_rnb_id | French national building identifier (ID-RNB) draft | geo, realestate | FR | Identifier of a building in the Référentiel National des Bâtiments (RNB), the French national register of buildings. It has 12... |
| fr_uai | French UAI code (school identifier) draft | education | FR | Code of a French educational establishment in the Répertoire national des établissements (RNE): seven digits and a check letter. |
| frpostcode | French postal code PII pattern draft | geo | FR | French postal code |
| gender | Gender PII draft | pii, persons | global | Gender flag and/or any other identification value |
| geojson | GeoJSON geometry draft | geo | global | Geometry, feature or feature collection written in GeoJSON (RFC 7946), the JSON format for geographic data. Positions are longitude... |
| geopoint | Geopoint draft | geo | global | Basic geopoint as coordinates |
| inseecode | INSEE code | geo | FR | The INSEE code is a numerical indexing code used by the French National Institute for Statistics and Economic Studies (INSEE) to... |
| latitude | latitude | geo, common | global | In geography, latitude is a geographic coordinate that specifies the north–south position of a point on the Earths surface. Latitude is... |
| longitude | Longitude | geo, common | global | Longitude is a geographic coordinate that specifies the east–west position of a point on the Earths surface, or the surface of a... |
| money | Money amount draft | values, common | global | Amount of money as numeric value |
| mongodbid | MongoDB unique Object ID | identifiers, common | global | MongoDB ObjectId, the unique id that MongoDB assigns to each document of a collection as the field _id. It is a 12-byte value written as... |
| month | Month draft | datetime | global | Month as number since start of the year. Could be from 1 to 12 |
| northing | Northing (projected coordinate) draft | geo | global | Northward coordinate of a point in a projected coordinate reference system, such as a UTM zone or the British National Grid, usually in... |
| percentage | Percentage value draft | values, common | global | Parcentage float number |
| phone | Phone number PII | pii, common, telecom | global | Telephone number in standard format (RFC3966), without 'tel:' prefix |
| siren | Code SIREN par nom de champ | finances | FR | FR datatype for 'Code SIREN par nom de champ' (siren) from rules in context 'finances'. The SIREN (Système d'Identification du... |
| siret | Code SIRET par nom de champ | finances | FR | FR datatype for 'Code SIRET par nom de champ' (siret) from rules in context 'finances'. The SIRET (Système d'Identification du... |
| url | Uniform Resource Locator (URL) | common, internet | global | Address of a resource on a network that also says how to retrieve it: a scheme such as https, a host, an optional port, a path, a query... |
| username | Username PII draft | pii, persons | global | Name of the user or login, commonly used for authentication or user identification purposes |
| uuid | Universally unique identifier (UUID) | identifiers | global | 128-bit identifier that systems generate without a central registry, defined by RFC 9562 (2024), which replaced RFC 4122. Microsoft... |
| year | Year | datetime | global | A calendar year in western notation like 2022 or 1913 |
Use this record
- JSON: API v1 · legacy shape
- Improve it: edit
data/tools/detectors/csvdetective.yamlon GitHub