{"id":177,"date":"2019-10-23T08:43:41","date_gmt":"2019-10-23T08:43:41","guid":{"rendered":"http:\/\/reacte.ugent.be\/?page_id=177"},"modified":"2021-02-16T17:39:45","modified_gmt":"2021-02-16T16:39:45","slug":"acter-dataset","status":"publish","type":"page","link":"https:\/\/termeval.ugent.be\/nl\/acter-dataset\/","title":{"rendered":"ACTER dataset"},"content":{"rendered":"\n

ACTER Annotated Corpora for Term Extraction Research<\/h2>\n\n\n\n

The data can be downloaded from the Github repository: <\/a>https:\/\/github.com\/AylaRT\/ACTER<\/a> which also contains an elaborate README.md with all necessary information.<\/strong><\/p>\n\n\n\n

The ACTER dataset (currently version 1.3) is publicly available with Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0) (https:\/\/creativecommons.org\/licenses\/by-nc-sa\/4.0\/<\/a>). The dataset can be freely used and adapted for non-commercial purposes, provided any changes made to the data are clearly mentioned and the proper reference is cited: https:\/\/doi.org\/10.1007\/s10579-019-09453-9<\/a><\/p>\n\n\n\n

Background & Numbers<\/h3>\n\n\n\n

The ACTER dataset has been specifically created to address some of the hurdles faced by ATE<\/a>. The corpora cover four domains: corruption, dressage (branch of horse riding), heart failure and wind energy. For each domain, corpora have been collected in three languages \u2013 English, French, and Dutch \u2013 with a more or less equal number of tokens per domain in each language. The corpora within each domain are all comparable, in the sense that they contain original texts of the same types, on the same subject, and they are not parallel translations, so they cannot be aligned (not on sentence -level, or even on document-level). The corpus on corruption also contains a parallel component, with sentence-aligned, trilingual texts. More detailed information about the creation of these corpora, as well as the annotation process, can be found in Rigouts Terryn, Hoste, & Lefever (2018, 2019) and in the overview paper of the TermEval2020 shared task, which will appear in the CompuTerm workshop proceedings. Not all information about the annotations as mentioned in previous work has been publicly released yet, so the remainder of this page will focus only on the information available in ACTER 1.2.<\/p>\n\n\n\n

In each language and domain (for corruption, only the parallel corpus has been annotated), \u00b150k tokens were manually annotated, with a total of 596,058 annotated tokens. This resulted in 119,455 individual annotations or 19,002 unique annotations. The annotation guidelines are freely available online[1]<\/sup>. Table 1 shows a summary of the dataset, showing, for each corpus, the domain, type (comparable or parallel), language, number of tokens, number of annotated tokens and number of unique annotations. For English, there are a total of 1,108,085 tokens, 194,403 of which are annotated[2]. The French corpora count 1,142,575 tokens in total, of which 206,859 are annotated. For Dutch, these numbers are 1,115,264 and 194,796 respectively. In total, the corpora contain 1194 different documents.<\/p>\n\n\n\n\n\n\n\t\n\n\t\n\t\n\t\n\t\n\t\n\t\n\t\n\t\n\t\n\t\n\t\n\t\n\t\n\t\n\t\n\n\t
Type<\/th>Domain<\/th>Language<\/th># Tokens<\/th># Documents<\/th># Annotated Tokens<\/th>\n<\/tr>\n<\/thead>\n
parallel<\/td>corruption<\/td>en<\/td> 176,314 <\/td>24<\/td> 45,234 <\/td>\n<\/tr>\n
fr<\/td> 196,327 <\/td>24<\/td> 50,429 <\/td>\n<\/tr>\n
nl<\/td> 184,541 <\/td>24<\/td> 47,305 <\/td>\n<\/tr>\n
comparable<\/td>corruption<\/td>en<\/td> 468,711 <\/td>44<\/td>-<\/td>\n<\/tr>\n
fr<\/td> 475,244 <\/td>31<\/td>-<\/td>\n<\/tr>\n
nl<\/td> 470,242 <\/td>49<\/td>-<\/td>\n<\/tr>\n
dressage<\/td>en<\/td> 102,654 <\/td>89<\/td> 51,470 <\/td>\n<\/tr>\n
fr<\/td> 109,572 <\/td>125<\/td> 53,316 <\/td>\n<\/tr>\n
nl<\/td> 103,851 <\/td>125<\/td> 50,882<\/td>\n<\/tr>\n
heart failure<\/td>en<\/td> 45,788 <\/td>190<\/td> 45,788 <\/td>\n<\/tr>\n
fr<\/td> 46,751 <\/td>215<\/td> 46,751 <\/td>\n<\/tr>\n
nl<\/td> 47,888 <\/td>175<\/td> 47,888 <\/td>\n<\/tr>\n
wind energy<\/td>en<\/td> 314,618 <\/td>38<\/td> 51,911 <\/td>\n<\/tr>\n
fr<\/td> 314,681 <\/td>12<\/td> 56,363 <\/td>\n<\/tr>\n
nl<\/td> 308,742 <\/td>29<\/td> 49,582 <\/td>\n<\/tr>\n<\/tbody>\n
<\/th>Total<\/th><\/th> 3,365,924 <\/th>1194<\/th> 596,058 <\/th>\n<\/tr>\n<\/tfoot>\n<\/table>\nOverview of the corpora per type, domain, and language; with number of tokens, number of documents and number of annotated tokens.<\/span>\n\n\n\n\n

<\/p>\n\n\n\n

While, for TermEval, ATE is conceived as a binary task (term or not term), there are actually 4 annotation labels: Specific Terms, Common Terms, Out-of-Domain Terms, and Named Entities. For TermEval, all labels are combined, with separate evaluations with and without Named Entities (see Task & Evaluation<\/a>).  No restrictions were placed on the annotations, but rather all lexical items that were considered terms as they were used in the texts, were annotated as such. This means that non-standard terms or wrong spellings are annotated as well, that there is no minimum frequency, that terms can by any length and any POS-pattern. <\/p>\n\n\n\n\n\n\n\t\n\n\t\n\t\n\t\n\t\n\t\n\t\n\t\n\t\n\t\n\t\n\t\n\t\n\n\t
Domain<\/th>Language<\/th># Specific Terms<\/th># Common Terms<\/th># OOD Terms<\/th># Named Entities<\/th># Annotations (total)<\/th>\n<\/tr>\n<\/thead>\n
corruption (parallel)<\/td>en<\/td>278<\/td>644<\/td>6<\/td>248<\/td>1174<\/td>\n<\/tr>\n
fr<\/td>300<\/td>678<\/td>5<\/td>236<\/td>1217<\/td>\n<\/tr>\n
nl<\/td>310<\/td>731<\/td>6<\/td>249<\/td>1295<\/td>\n<\/tr>\n
dressage<\/td>en<\/td>780<\/td>309<\/td>71<\/td>421<\/td>1575<\/td>\n<\/tr>\n
fr<\/td>705<\/td>238<\/td>26<\/td>221<\/td>1183<\/td>\n<\/tr>\n
nl<\/td>1026<\/td>333<\/td>41<\/td>153<\/td>1546<\/td>\n<\/tr>\n
heart failure<\/td>en<\/td>1885<\/td>320<\/td>158<\/td>228<\/td>2585<\/td>\n<\/tr>\n
fr<\/td>1714<\/td>505<\/td>59<\/td>147<\/td>2423<\/td>\n<\/tr>\n
nl<\/td>1561<\/td>450<\/td>66<\/td>182<\/td>2257<\/td>\n<\/tr>\n
wind energy<\/td>en<\/td>781<\/td>296<\/td>14<\/td>444<\/td>1534<\/td>\n<\/tr>\n
fr<\/td>444<\/td>308<\/td>21<\/td>195<\/td>968<\/td>\n<\/tr>\n
nl<\/td>577<\/td>342<\/td>21<\/td>305<\/td>1245<\/td>\n<\/tr>\n<\/tbody>\n
Total<\/th>10361<\/th>5154<\/th>494<\/th>3029<\/th>19002<\/th>\n<\/tr>\n<\/tfoot>\n<\/table>\n\n\n\n\n

Table 2 shows the number of unique annotations (type counts) per corpus, and per label. These are also the number which can be found in ACTER 1.2 The same information is repeated in Table 3, but this time with token counts, i.e. each annotation is counted separately. This information (the annotations as spans in the text) is not yet available in ACTER 1.2.<\/p>\n\n\n\n\n\n\n\t\n\n\t\n\t\n\t\n\t\n\t\n\t\n\t\n\t\n\t\n\t\n\t\n\t\n\n\t
Domain<\/th>Language<\/th># Specific Terms<\/th># Common Terms<\/th># OOD Terms<\/th># Named Entities<\/th># Annotations (total)<\/th>\n<\/tr>\n<\/thead>\n
corruption (parallel)<\/td>en<\/td>1078<\/td>5295<\/td>12<\/td>2373<\/td>8758<\/td>\n<\/tr>\n
fr<\/td>1077<\/td>4842<\/td>11<\/td>2186<\/td>8116<\/td>\n<\/tr>\n
nl<\/td>1041<\/td>4111<\/td>11<\/td>2334<\/td>7497<\/td>\n<\/tr>\n
dressage<\/td>en<\/td>4729<\/td>5997<\/td>163<\/td>970<\/td>11859<\/td>\n<\/tr>\n
fr<\/td>4969<\/td>4389<\/td>39<\/td>467<\/td>9864<\/td>\n<\/tr>\n
nl<\/td>6259<\/td>4891<\/td>57<\/td>295<\/td>11502<\/td>\n<\/tr>\n
heart failure<\/td>en<\/td>7402<\/td>5626<\/td>983<\/td>526<\/td>14537<\/td>\n<\/tr>\n
fr<\/td>5212<\/td>5336<\/td>253<\/td>319<\/td>11120<\/td>\n<\/tr>\n
nl<\/td>5428<\/td>4537<\/td>254<\/td>433<\/td>10652<\/td>\n<\/tr>\n
wind energy<\/td>en<\/td>5151<\/td>4282<\/td>45<\/td>1429<\/td>10907<\/td>\n<\/tr>\n
fr<\/td>3208<\/td>5207<\/td>109<\/td>439<\/td>8963<\/td>\n<\/tr>\n
nl<\/td>2104<\/td>2805<\/td>135<\/td>636<\/td>5680<\/td>\n<\/tr>\n<\/tbody>\n
Total<\/th>47658<\/th>57318<\/th>2072<\/th>12407<\/th>119455<\/th>\n<\/tr>\n<\/tfoot>\n<\/table>\n\n\n\n\n

While these human annotations are unavoidably subject to human error and subjectivity, every care has been taken to obtain high-quality annotations, which have been validated in various studies (Rigouts Terryn, Drouin, et al., 2019; Rigouts Terryn, Hoste, et al., 2019), though some errors may of course still remain. Since the purpose of this shared task is also to provide the community with a practical dataset for ATE, we highly encourage people who use the dataset and spot errors to notify us of these errors so we can continue to improve the data (an email address is provided in the README.md).
<\/p>\n\n\n\n

\n\n\n\n

[1]<\/span><\/sup> <\/span>http:\/\/hdl.handle.net\/1854\/LU-8503113<\/span><\/a><\/p>\n\n\n\n

[2]<\/sup> While the gold standard list of annotated terms may contain many terms that also appear in the unannotated parts of the corpora, the unannotated parts of the corpora will undoubtedly also contain terms that do not appear in the gold standard lists.<\/p>\n\n\n\n

<\/p>\n","protected":false},"excerpt":{"rendered":"

ACTER Annotated Corpora for Term Extraction Research The data can be downloaded from the Github repository: https:\/\/github.com\/AylaRT\/ACTER which also contains an elaborate README.md with all necessary information. The ACTER dataset (currently version 1.3) is publicly available with Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0) (https:\/\/creativecommons.org\/licenses\/by-nc-sa\/4.0\/). The dataset can be freely used and adapted for […]<\/p>\n","protected":false},"author":2,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":[],"translation":{"provider":"WPGlobus","version":"2.8.11","language":"nl","enabled_languages":["en","es","de","fr","nl"],"languages":{"en":{"title":true,"content":true,"excerpt":false},"es":{"title":false,"content":false,"excerpt":false},"de":{"title":false,"content":false,"excerpt":false},"fr":{"title":false,"content":false,"excerpt":false},"nl":{"title":false,"content":false,"excerpt":false}}},"_links":{"self":[{"href":"https:\/\/termeval.ugent.be\/nl\/wp-json\/wp\/v2\/pages\/177"}],"collection":[{"href":"https:\/\/termeval.ugent.be\/nl\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/termeval.ugent.be\/nl\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/termeval.ugent.be\/nl\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/termeval.ugent.be\/nl\/wp-json\/wp\/v2\/comments?post=177"}],"version-history":[{"count":36,"href":"https:\/\/termeval.ugent.be\/nl\/wp-json\/wp\/v2\/pages\/177\/revisions"}],"predecessor-version":[{"id":594,"href":"https:\/\/termeval.ugent.be\/nl\/wp-json\/wp\/v2\/pages\/177\/revisions\/594"}],"wp:attachment":[{"href":"https:\/\/termeval.ugent.be\/nl\/wp-json\/wp\/v2\/media?parent=177"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}