Publicação
Learning Curves Prediction for a Transformers-based Model
| dc.contributor.author | Cruz, Francisco | |
| dc.contributor.author | Castelli, Mauro | |
| dc.contributor.institution | NOVA Information Management School (NOVA IMS) | |
| dc.contributor.institution | Information Management Research Center (MagIC) - NOVA Information Management School | |
| dc.contributor.pbl | Societa Italiana di Istochimica / PAGEPress Publications | |
| dc.date.accessioned | 2023-11-02T22:08:59Z | |
| dc.date.available | 2023-11-02T22:08:59Z | |
| dc.date.issued | 2023-10-01 | |
| dc.description | Cruz, F., & Castelli, M. (2023). Learning Curves Prediction for a Transformers-based Model. Emerging Science Journal, 7(5), 1491-1500. https://doi.org/10.28991/ESJ-2023-07-05-03 | |
| dc.description.abstract | One of the main challenges when training or fine-tuning a machine learning model concerns the number of observations necessary to achieve satisfactory performance. While, in general, more training observations result in a better-performing model, collecting more data can be time-consuming, expensive, or even impossible. For this reason, investigating the relationship between the dataset's size and the performance of a machine learning model is fundamental to deciding, with a certain likelihood, the minimum number of observations that are necessary to ensure a satisfactory-performing model is obtained as a result of the training process. The learning curve represents the relationship between the dataset’s size and the performance of the model and is especially useful when choosing a model for a specific task or planning the annotation work of a dataset. Thus, the purpose of this paper is to find the functions that best fit the learning curves of a Transformers-based model (LayoutLM) when fine-tuned to extract information from invoices. Two new datasets of invoices are made available for such a task. Combined with a third dataset already available online, 22 sub-datasets are defined, and their learning curves are plotted based on cross-validation results. The functions are fit using a non-linear least squares technique. The results show that both a biasymptotic and a Morgan-Mercer-Flodin function fit the learning curves extremely well. Also, an empirical relation is presented to predict the learning curve from a single parameter that may be easily obtained in the early stage of the annotation process. | en |
| dc.description.version | publishersversion | |
| dc.description.version | published | |
| dc.format.extent | 10 | |
| dc.format.extent | 822335 | |
| dc.identifier.doi | 10.28991/ESJ-2023-07-05-03 | |
| dc.identifier.issn | 2610-9182 | |
| dc.identifier.other | PURE: 63539763 | |
| dc.identifier.other | PURE UUID: c547c8b0-44cb-4f1b-9f67-c94a0d4f076c | |
| dc.identifier.other | Scopus: 85174944049 | |
| dc.identifier.other | ORCID: /0000-0002-8793-1451/work/151388751 | |
| dc.identifier.uri | http://hdl.handle.net/10362/159492 | |
| dc.identifier.url | https://www.scopus.com/pages/publications/85174944049 | |
| dc.identifier.url | https://zenodo.org/doi/10.5281/zenodo.6371709 | |
| dc.language.iso | eng | |
| dc.peerreviewed | yes | |
| dc.subject | Dataset Size | |
| dc.subject | Document Data Extraction | |
| dc.subject | Fine-Tuning | |
| dc.subject | Learning Curves | |
| dc.subject | Transformers | |
| dc.subject | General | |
| dc.title | Learning Curves Prediction for a Transformers-based Model | en |
| dc.type | journal article | |
| degois.publication.firstPage | 1491 | |
| degois.publication.issue | 5 | |
| degois.publication.lastPage | 1500 | |
| degois.publication.title | Emerging Science Journal | |
| degois.publication.volume | 7 | |
| dspace.entity.type | Publication | |
| rcaap.rights | openAccess |
Ficheiros
Principais
1 - 1 de 1
A carregar...
- Nome:
- Learning_Curves_Prediction_for_a_Transformers_Based_Model.pdf
- Tamanho:
- 803.06 KB
- Formato:
- Adobe Portable Document Format