Doduo
ID doduo
detector
Doduo is a research model from Megagon Labs, published at SIGMOD 2022, that predicts the semantic type of every column of a table and the relation between pairs of columns. It fine-tunes a BERT language model on the whole table written out as one sequence, so the values of the other columns help to type each column. The repository ships two pretrained models: one trained on WikiTable data from TURL, with 255 Freebase column types, and one trained on VizNet data from Sato, with the same 78 semantic types as Sherlock and Sato, such as birthDate or fileSize. The registry maps the VizNet types through mappings.sherlock, so this list is the same as the one of the sherlock and sato tools. The code is released under the Apache 2.0 license.
- Website
- https://github.com/megagonlabs/doduo
- Supported types
- 30
Supported semantic types
| ID | Name | Categories | Countries | Description |
|---|---|---|---|---|
| address | Address (physical location) PII | common, pii, geo | global | Geographic address, commonly consist of postindex, country, area, town, street and building |
| age | Age PII draft | datetime | global | Age of the person or organization in years |
| birthday | Birthday PII | pii, datetime | global | A birthday is the anniversary of the birth of a person, or figuratively of an institution. Birthdays of people are celebrated in... |
| birthplace | Place of birth PII draft | persons, pii, geo | global | The place where a person was born, usually a city or town, often followed by the region or country, such as "Montreal, QC, CAN".... |
| brand | Brand draft | companies, ecommerce | global | Name of a brand, the name under which a product, a product line, a service or a store chain is sold. A brand can differ from the name of... |
| category | Category draft | common | global | Object or entity category |
| city | City PII | geo, pii, common | global | A city is a large human settlement. It can be defined as a permanent and densely settled place with administratively defined boundaries... |
| comment | Comment / Note | common, texts | global | Comment/note for the object or entity. |
| continent | Continent name or identifier | geo | global | A continent is any of several large landmasses. Related to UN unm49 dataclass |
| country | Country code or name | geo | global | A country is a distinct territorial body or political entity. It may be an independent sovereign state or part of a larger state,[2] as... |
| currency | Currency | finances | global | A currency in the most specific sense is money in any form when in use or circulation as a medium of exchange, especially circulating... |
| dc_creator | Creator (Dublin Core) | dublincore | global | Dublin Core element 'creator': an entity primarily responsible for making the resource: a person, an organization or a service. |
| dc_publisher | Publisher (Dublin Core) | dublincore | global | Dublin Core element 'publisher': an entity responsible for making the resource available. |
| description | Description | common | global | Item, Object or entity description, common for any object |
| duration | Time duration | datetime | global | The amount of time elapsed between two events |
| elevation | Elevation above sea level draft | geo, values | global | Height of a place, such as a mountain summit, a town or a weather station, above mean sea level. Datasets store it as a number in metres... |
| filesize | File size draft | software | global | size of a file |
| gender | Gender PII draft | pii, persons | global | Gender flag and/or any other identification value |
| genre | Genre | common, media | global | Genre of the creative work, broadcast channel or group. |
| industry | Industry (line of business) draft | companies, industry | global | Name of the industry or line of business of an organization, such as Banking, Retail or Telecommunications. Codes of industry... |
| isbn | International Standard Book Number (ISBN) | publishing, objectids | global | ISO 2108 identifier of a book edition, in either of its two forms: the 10-character ISBN used until 2007, whose last character is a... |
| language | Language draft | common | global | Human language written as a name, such as English, Deutsch or allemand (German in French), or as a code. Codes have their own patterns:... |
| name | Name | common | global | Name of the item, object or entity. |
| nationality | Person nationality PII draft | pii, persons | global | Nationality of the person from list of unique code from one of code lists or defined somehow else. |
| orgname | Company or organization name | companies | global | Name of the organization/business/company in any language |
| person_fullname | Person fullname PII draft | persons, pii | global | Full name of the person as combination of last, first, and sometimes middle name |
| religion | Religion PII draft | persons, pii | global | Religion, religious denomination or worldview of a person or a group, such as Catholic, Sunni Islam or Buddhism. Censuses and surveys... |
| species | Species draft | science | global | Name of a biological species, either its scientific name, such as Panthera leo, or a common name, such as lion. Typical columns are in... |
| weight | Weight (mass) draft | values, common | global | Weight of a person, an animal or a product, measured as mass in kilograms, grams, pounds or ounces. Datasets store it as a number, with... |
| year | Year | datetime | global | A calendar year in western notation like 2022 or 1913 |
Use this record
- JSON: API v1 · legacy shape
- Improve it: edit
data/tools/detectors/doduo.yamlon GitHub