{rfName}
In

Indexed in

License and use

Icono OpenAccess

Citations

18

Altmetrics

Analysis of institutional authors

Luna-Jiménez CCorresponding AuthorGil-Martin MCorresponding AuthorKleinlein R.AuthorKleinlein RAuthorSan-Segundo RAuthorFernández-Martinez FAuthor

Share

Publications
>
Proceedings Paper

Interpreting Sign Language Recognition using Transformers and MediaPipe Landmarks

Publicated to:Acm International Conference Proceeding Series. 373-377 - 2023-01-01 (), DOI: 10.1145/3577190.3614143

Authors: Luna-Jiménez, C; Gil-Martín, M; Kleinlein, R; San-Segundo, R; Fernández-Martínez, F

Affiliations

Univ Politecn Madrid, Grp Tecnol Habla & Aprendizaje Automat THAU Grp, Informat Proc & Telecommun Ctr, ETSI Telecomunicac, Madrid, Spain - Author
Universidad Politécnica de Madrid - Author

Abstract

Sign Language Recognition (SLR) is a challenging task that aims to bridge the communication gap between the deaf and hearing communities. In recent years, deep learning-based approaches have shown promising results in SLR. However, the lack of interpretability remains a significant challenge. In this paper, we seek to understand which hand and pose MediaPipe Landmarks are deemed the most important for prediction as estimated by a Transformer model. We propose to embed a learnable array of parameters into the model that performs an element-wise multiplication of the inputs. This learned array highlights the most informative input features that contributed to solve the recognition task. Resulting in a human-interpretable vector that lets us interpret the model predictions. We evaluate our approach on public datasets called WLASL100 (SRL) and IPNHand (gesture recognition). We believe that the insights gained in this way could be exploited for the development of more efficient SLR pipelines.

Keywords

Human-computer interactionInterpretabilityMediapipe landmarksSign language recognitionTransformers

Quality index

Bibliometric impact. Analysis of the contribution and dissemination channel

From a relative perspective, and based on the normalized impact indicator calculated from the Field Citation Ratio (FCR) of the Dimensions source, it yields a value of: 3.96, which indicates that, compared to works in the same discipline and in the same year of publication, it ranks as a work cited above average. (source consulted: Dimensions Jun 2025)

Specifically, and according to different indexing agencies, this work has accumulated citations as of 2025-06-21, the following number of citations:

  • WoS: 4
  • Scopus: 8
  • Google Scholar: 6
  • OpenCitations: 2

Impact and social visibility

From the perspective of influence or social adoption, and based on metrics associated with mentions and interactions provided by agencies specializing in calculating the so-called "Alternative or Social Metrics," we can highlight as of 2025-06-21:

  • The use, from an academic perspective evidenced by the Altmetric agency indicator referring to aggregations made by the personal bibliographic manager Mendeley, gives us a total of: 17.
  • The use of this contribution in bookmarks, code forks, additions to favorite lists for recurrent reading, as well as general views, indicates that someone is using the publication as a basis for their current work. This may be a notable indicator of future more formal and academic citations. This claim is supported by the result of the "Capture" indicator, which yields a total of: 17 (PlumX).

With a more dissemination-oriented intent and targeting more general audiences, we can observe other more global scores such as:

  • The Total Score from Altmetric: 1.85.
  • The number of mentions on the social network X (formerly Twitter): 2 (Altmetric).

It is essential to present evidence supporting full alignment with institutional principles and guidelines on Open Science and the Conservation and Dissemination of Intellectual Heritage. A clear example of this is:

  • The work has been submitted to a journal whose editorial policy allows open Open Access publication.

Leadership analysis of institutional authors

There is a significant leadership presence as some of the institution’s authors appear as the first or last signer, detailed as follows: First Author (LUNA JIMENEZ, CRISTINA) and Last Author (FERNANDEZ MARTINEZ, FERNANDO).

the authors responsible for correspondence tasks have been LUNA JIMENEZ, CRISTINA and GIL MARTIN, MANUEL.