Affective social anthropomorphic intelligent system
This thesis is submitted in partial fulfillment of the requirements for the degree of Bachelor of Science in Computer Science and Engineering, 2021.
| Auteurs principaux: | , |
|---|---|
| Autres auteurs: | |
| Format: | Thèse |
| Langue: | English |
| Publié: |
Brac University
2021
|
| Sujets: | |
| Accès en ligne: | http://hdl.handle.net/10361/15324 |
| id |
10361-15324 |
|---|---|
| record_format |
dspace |
| spelling |
10361-153242022-01-26T10:18:14Z Affective social anthropomorphic intelligent system Mamun, Md. Adyelullahil Abdullah, Hasnat Md. Alam, Md. Golam Rabiul Department of Computer Science and Engineering, Brac University IVA NLP SER Emotion Audio-Emotion Personal-Assistant Intelligent control systems This thesis is submitted in partial fulfillment of the requirements for the degree of Bachelor of Science in Computer Science and Engineering, 2021. Cataloged from PDF version of thesis. Includes bibliographical references (pages 48-56). At present, intelligent virtual assistants (IVA) are not only about delivering the functionalities and increasing their performances; they also need a socially interactive personality. As human conversational styles are measured by our sense of humor, personalities, tone of voice, these qualities have become essential for conversational intelligent virtual assistants. Our proposed system is an anthropomorphic intelligent system that can hold a proper human-like conversation with emotion and personality. It can also be able to imitate any person's voice given; voice audio data is available. Initially, the temporal audio wave data will be converted to frequency domain data (Mel-Spectrogram), which contains distinct patterns for audio features like the notes, pitch, rhythm, and melody. A parallel CNN, Transformer-Encoder, is used to predict the emotion from 7 different audio data classes. This audio is also fed to the deep-speech, an RNN model that consists of 5 hidden layers. From the spectrogram, it generates the text transcription. Then the transcript text is transferred to the multi-domain conversation agent, using blended skill talk and transformer-based retrieve-and-generate generation strategy and beam-search decoding an appropriate textual response is generated, which in turn gets synthesized to audio using WaveGlow that is based on WaveNet and Glow. It learns an invertible mapping of data to a latent space that can be manipulated and generates a Mel-spectrogram frame based on previous Mel-spectrogram frames. Finally, from the generated spectrogram, the waveform is generated using WaveGlow. A fine-tuned system can be used in the following but not limited to applications like dubbing, voice assistant, re-creating new movies with old actors. Md. Adyelullahil Mamun Hasnat Md. Abdullah B. Computer Science 2021-10-18T05:05:53Z 2021-10-18T05:05:53Z 2021 2021-01 Thesis ID 20241044 ID 20241047 http://hdl.handle.net/10361/15324 en Brac University theses are protected by copyright. They may be viewed from this source for any purpose, but reproduction or distribution in any format is prohibited without written permission. 56 pages application/pdf Brac University |
| institution |
Brac University |
| collection |
Institutional Repository |
| language |
English |
| topic |
IVA NLP SER Emotion Audio-Emotion Personal-Assistant Intelligent control systems |
| spellingShingle |
IVA NLP SER Emotion Audio-Emotion Personal-Assistant Intelligent control systems Mamun, Md. Adyelullahil Abdullah, Hasnat Md. Affective social anthropomorphic intelligent system |
| description |
This thesis is submitted in partial fulfillment of the requirements for the degree of Bachelor of Science in Computer Science and Engineering, 2021. |
| author2 |
Alam, Md. Golam Rabiul |
| author_facet |
Alam, Md. Golam Rabiul Mamun, Md. Adyelullahil Abdullah, Hasnat Md. |
| format |
Thesis |
| author |
Mamun, Md. Adyelullahil Abdullah, Hasnat Md. |
| author_sort |
Mamun, Md. Adyelullahil |
| title |
Affective social anthropomorphic intelligent system |
| title_short |
Affective social anthropomorphic intelligent system |
| title_full |
Affective social anthropomorphic intelligent system |
| title_fullStr |
Affective social anthropomorphic intelligent system |
| title_full_unstemmed |
Affective social anthropomorphic intelligent system |
| title_sort |
affective social anthropomorphic intelligent system |
| publisher |
Brac University |
| publishDate |
2021 |
| url |
http://hdl.handle.net/10361/15324 |
| work_keys_str_mv |
AT mamunmdadyelullahil affectivesocialanthropomorphicintelligentsystem AT abdullahhasnatmd affectivesocialanthropomorphicintelligentsystem |
| _version_ |
1814308688598401024 |