Tri-modal ensemble for enhanced Bangla sign language recognition
This thesis is submitted in partial fulfillment of the requirements for the degree of Bachelor of Science in Computer Science and Engineering, 2023.
| Autors principals: | , , , |
|---|---|
| Altres autors: | |
| Format: | Thesis |
| Idioma: | English |
| Publicat: |
Brac University
2024
|
| Matèries: | |
| Accés en línia: | http://hdl.handle.net/10361/23577 |
| id |
10361-23577 |
|---|---|
| record_format |
dspace |
| spelling |
10361-235772024-06-25T21:01:13Z Tri-modal ensemble for enhanced Bangla sign language recognition Shams, Khan Abrar Reaz, Md. Rafid Islam, Sanjida Rafi, Mohammad Ryan Ur Rahman, Md. Shahriar Rahman, Rafeed Department of Computer Science and Engineering, Brac University Bangla sign language Convolutional neural network Ensemble method Neural networks (Computer science) Computer linguistics This thesis is submitted in partial fulfillment of the requirements for the degree of Bachelor of Science in Computer Science and Engineering, 2023. Cataloged from PDF version of thesis. Includes bibliographical references (pages 49-52). Sign language is the most common method of communication for people with disabling hearing loss. Bangladesh, where BdSL is prominently used among the disabling people, finds communicating with the general mass challenging. Thus, a system to understand BdSL accurately and efficiently has become a popular demand. Deep learning architectures such as CNN, ANN, RNN, and Axis Independent LSTM can interpret Bangla Sign Language into readable digital wording. Commonly, an image-based sign language recognition system contains a recording camera that continuously sends images to a model. The model then provides a prediction based on those images. However, it creates a lot of uncertainty variables, such as the lighting issue, noisy background, skin color, and hand orientations. To this end, we propose a procedure that can reduce this uncertainty variable by considering three different modalities, spatial information, skeleton awareness, and edge awareness. We propose three image pre-processing techniques and integrate three convolutional neural network models. Finally, we tested out nine different ensemble meta-learning algorithms where five of the algorithms are modifications of averaging and voting techniques. As a result, our proposed model achieved higher training accuracy at 99.77%, 98.11%, and 99.30% than any other state-of-the-art image classification architectures except for ResNet50 at 99.87%. We achieved the highest accuracy of 95.13% on the testing set. This research shows that considering multiple modalities can improve the system’s overall performance. Khan Abrar Shams Md. Rafid Reaz Sanjida Islam Mohammad Ryan Ur Rafi B.Sc in Computer Science 2024-06-25T06:48:54Z 2024-06-25T06:48:54Z ©2023 2023-09 Thesis ID 19201052 ID 19201044 ID 20101615 ID 22241152 http://hdl.handle.net/10361/23577 en Brac University theses are protected by copyright. They may be viewed from this source for any purpose, but reproduction or distribution in any format is prohibited without written permission. 61 pages application/pdf Brac University |
| institution |
Brac University |
| collection |
Institutional Repository |
| language |
English |
| topic |
Bangla sign language Convolutional neural network Ensemble method Neural networks (Computer science) Computer linguistics |
| spellingShingle |
Bangla sign language Convolutional neural network Ensemble method Neural networks (Computer science) Computer linguistics Shams, Khan Abrar Reaz, Md. Rafid Islam, Sanjida Rafi, Mohammad Ryan Ur Tri-modal ensemble for enhanced Bangla sign language recognition |
| description |
This thesis is submitted in partial fulfillment of the requirements for the degree of Bachelor of Science in Computer Science and Engineering, 2023. |
| author2 |
Rahman, Md. Shahriar |
| author_facet |
Rahman, Md. Shahriar Shams, Khan Abrar Reaz, Md. Rafid Islam, Sanjida Rafi, Mohammad Ryan Ur |
| format |
Thesis |
| author |
Shams, Khan Abrar Reaz, Md. Rafid Islam, Sanjida Rafi, Mohammad Ryan Ur |
| author_sort |
Shams, Khan Abrar |
| title |
Tri-modal ensemble for enhanced Bangla sign language recognition |
| title_short |
Tri-modal ensemble for enhanced Bangla sign language recognition |
| title_full |
Tri-modal ensemble for enhanced Bangla sign language recognition |
| title_fullStr |
Tri-modal ensemble for enhanced Bangla sign language recognition |
| title_full_unstemmed |
Tri-modal ensemble for enhanced Bangla sign language recognition |
| title_sort |
tri-modal ensemble for enhanced bangla sign language recognition |
| publisher |
Brac University |
| publishDate |
2024 |
| url |
http://hdl.handle.net/10361/23577 |
| work_keys_str_mv |
AT shamskhanabrar trimodalensembleforenhancedbanglasignlanguagerecognition AT reazmdrafid trimodalensembleforenhancedbanglasignlanguagerecognition AT islamsanjida trimodalensembleforenhancedbanglasignlanguagerecognition AT rafimohammadryanur trimodalensembleforenhancedbanglasignlanguagerecognition |
| _version_ |
1814307299570745344 |