Interview Bot Development with Natural Language Processing and Machine Learning

Interview for competency assessment takes essential roles in Human Resource Management practices. However, the traditional competency interview process needs considerable time and costs and often requires face-to-face meetings that may endanger both interviewers and interviewees during a pandemic. This study aims to present the development of an interview bot for identifying competency based on the Behavioural Event Interview method by using artificial intelligence technology. It is an automation of the interview process to explore a person’s competencies levels based on past behavioural experiences. The development of the interview bot involved two main activities. The first is the data training process to develop learning models to determine competency levels based on provided valid participant’s responses. The second is the testing and evaluation model for assessment to determine competency levels.  We found that our method can predict a person's competence levels based on their responses. Our approach can make predictions with acceptable accuracy. The interview bot is a valuable and reliable tool to conduct online interviews and support the assessment centre process, especially with conditions of physical and social distancing constraints. It provides flexibility in terms of time and place for participants, and its process is delivered in Indonesia's Language. The interview bot is more cost efficient than traditional interviews with the same behavioural event interview methods, and it would also be preferable for millennials.

Artificial intelligence; Behavioural event interview; Chat bot; Interview bot; Machine learning

Recognizing the importance of competencies for competitive advantages, the Government of Indonesia (GOI), as a policymaker, issued regulations that encourage both governmental and private business organizations to increase employee competencies. One of the government’s regulations is the Decree of the Employment Minister of the Republic of Indonesia, Number 2 of 2016, concerning the National Work Competency Standardization System. The statute contains a comprehensive and synergic arrangement of national work competency standards intended to improve Indonesian human resources competencies. With the issuance of the regulation, Indonesian workers must meet the established competency standards to be able to work in an organization. Therefore, organizations must regularly conduct competency assessments.

One of the techniques that organizations can use to perform competency assessments is a behavioural event interview (BEI). BEI is a systematic interviewing method that is carried out in five stages: introducing and explaining the interview process, extracting information about job responsibilities, extracting information about behavioural events, exploring the characteristics required for the job, and finally, drawing conclusions and summaries. In contrast to ordinary interviews, which are considered less reliable in predicting candidates who will perform well, BEIs can reveal detailed behavioural descriptions of how someone does their job and thus overcome the ineffectiveness of typical interviews. However, although BEI is a suitable method for measuring competencies, it also has some weaknesses. For example, the interview process takes longer, requires many certified interviewers—trained experts are still limited in number—requires high organizing costs, and introduces potential interviewer bias in the assessment. Meanwhile, the need for conducting competency assessment increases in line with the growing awareness of conducting competency assessment in companies and organizations.

In conducting BEIs, interviewers look for clues about the interviewee’s past experiences. In English, these can be identified from past-tense responses. However, Indonesian grammar is simple in the sense that it has no past-tense pattern. Instead, it uses keywords that indicate the time or frequency of one’s actions. Therefore, developing an interview bot that “speaks” Indonesian presents a different challenge and requires a different approach. The fourth Industrial Revolution has impacted automation technology development in various fields, and technological advances are being integrated in every aspect of our lives (Berawi, 2018). Artificial intelligence (AI) facilitates decision-making, creates integrative systems, and simplifies complex mechanisms though automation (Berawi, 2020). There are AI applications in many fields, including human resource management systems. AI refers to machines’ ability to perform tasks usually associated with human thinking, especially by using computer systems. AI allows computers to learn from and make decisions or recommend actions based on available data and helps to solve complex problems. AI applications can support repetitive or patterned processes.

Likewise, human resource management (HRM) is a viable field in which to implement automation technology, for example, in chatbots as part of the interview process. The weaknesses of BEI could be minimized by developing an interview bot as a substitute for or accompaniment to interviewers or appraisers. Moreover, such bots could conduct interviews remotely, reducing interview costs and increasing time flexibility.

Eubanks (2017) reported that experimenting with short message service (SMS)-based interactions aids in interview scheduling provided benefits. Further, the process was undertaken by a bot, not a person. It has also been found that candidates primarily interact with the interview bots outside of regular office hours. Thus, the recruiters do not have to work overtime, as parts of the interview process could be taken over by bots. In a typical automatic talent acquisition process, the candidates interact with an interview bot, allowing the recruiters to engage in other activities simultaneously. Recruitment chatbots cannot respond to questions that are not expected. This could be handled by a program that sends the questions directly to the staff responsible for answering them. Hence, the way in which AI is used today in the recruitment process is intended to leverage the benefits of implementing bots in various ways.

The interview bot application for assessing competency levels is a further development of chatbot technology. A chatbot is a computer system that operates as an interface between human users and software applications, using natural written and oral language to communicate. Some examples of chatbots that have been developed are Siri, IBM Watson, and Google Assistant. A chatbot has several advantages, including ease of access, efficiency, availability, scalability, cost, and insight. Chatbot technology has been applied in various fields, such as handling e-commerce queries (Pricilla et al., 2018), web shopping helpers, hotel reservation agents, and FAQ agents (Siddig & Hines, 2019), and various digital consumers (Rese et al., 2020). However, chatbot applications for supporting HRM practices remain underdeveloped.

A chatbot architecture can be further developed to have an information retrieval function and interactively “generate” questions by applying AI technology. This may work in two ways and can support artificial interviews (Suakanto et al., 2021). AI technology includes machine learning, deep learning, neural networks, and natural language processing (NLP). Cowgill (2018) used machine learning for hiring white-collar workers. The challenge of developing AI and machine learning for HRM is related to the number of data sets, which tends to be relatively small by data science standards (Tambe et al., 2019). As a branch of AI, NLP has been employed in human interview systems. An interactive interview bot system based on NLP was developed to conduct interviews and generate results automatically (Yakkundi et al., 2019). One of the critical benefits of NLP is its ability to process and understand unstructured text data automatically.

This study has successfully developed an interview bot that uses machine learning to determine the competence level of a person. In this research, we use an Indonesian language dataset. To converse with human participants, we use NLP technology. This study demonstrated very good accuracy in various scenarios. The results of this study can be used as the basis for developing an interview bot that is closer to professional interviews. One of the important aspects of this system is datasets. With a more extensive and comprehensive dataset, it is possible that the system would be richer in information and achieve better accuracy. For future works, the system could be enhanced to use voice interaction instead of text-based chat. The frequency, types, and depth of the questions could also be made more adaptive to match the psychological aspects of the interviewee.

Berawi, M.A., 2018. The Fourth Industrial Revolution: Managing Technology Development for Competitiveness. International Journal of Technology, Volume 9(1), pp. 1–4

Berawi, M.A., 2020. Managing Artificial Intelligence Technology for Added Value. International Journal of Technology, Volume 11(1), pp. 1–4

Cowgill, B., 2018. Bias and Productivity in Humans and Algorithms: Theory and Evidence from Résumé Screening. Working paper, Columbia University, New York

Eubanks, B., 2017. Artificial Intelligence for HR Use AI to Support and Develop a Successful Workforce. Kogan Page. Available online at https://books.google.com/books?hl=en&lr=&id=hrN7DwAAQBAJ&oi=fnd&pg=PP1&dq=Artificial+Intelligence+for+HR+Use+AI+to+Support+and+

        Develop+a+Successful+Workforce&ots=jc5-Zm6nbw&sig=rXBsT7SPUpQbJWXMAh6O9gDxsbo

Kim, S., Lee, J., Gweon, G., 2019. Comparing Data from Chatbot and Web Surveys: Effects of Platform and Conversational Style on Survey Response Quality. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, pp. 1–12

Margherita, A., 2021. Human Resources Analytics: A Systematisation of Research Topics and Directions for Future Research. Human Resource Management Review, Volume 32(2) p. 100795

Nawaz, N., Gomes, A.M., 2019. Artificial Intelligence Chatbots are New Recruiters. International Journal of Advanced Computer Science and Applications, Volume 10(9), pp. 1–5

Pereira, V., Hadjielias, E., Christofi, M., Vrontis, D., 2021. A Systematic Literature Review on the Impact of Artificial Intelligence on Workplace Outcomes: A Multi-Process Perspective. Human Resource Management Review, p. 100857

Pricilla, C., Lestari, D.P., Dharma, D., 2018. Designing Interaction for Chatbot-Based Conversational Commerce with User-Centered Design. In: 5th International Conference on Advanced Informatics: Concept Theory and Applications (ICAICTA), IEEE, pp. 244–249

Reilly, P., 2018. The Impact of Artificial Intelligence on the HR Function. Institue for Employment Studies. Available online at https://www.employment-studies.co.uk/system/files/resources/files/mp142_The_impact_of_Artificial_Intelligence_on_the_HR_function-Peter_Reilly.pdf

Rese, A., Ganster, L., Baier, D., 2020. Chatbots in Retailers’ Customer Communication: How to Measure their Acceptance? Journal of Retailing and Consumer Services, Volume 56, p. 102176

Suakanto, S., Siswanto, J., Kusumasari, T.F., Prasetyo, I.R., Hardiyanti, M., 2021. Interview Bot for Improving Human Resource Management. 2021 In: International Conference on ICT for Smart Society (ICISS), IEEE, pp. 1–5

Siddig, A., Hines, A., 2019. A Psychologist Chatbot Developing Experience. In: Proceedings of AICS, pp. 200-211

Tambe, P., Cappelli P., Yakubovich, V., 2019. Artificial Intelligence in Human Resources Management: Challenges and a Path Forward. California Management Review, Volume 61(4), pp. 1–28

Yakkundi, S., Vanjare, A., Wavhal, V. and Patankar, S., 2019. Interactive Interview Chatbot. International Research Journal of Engineering and Technology, Volume 6(4), pp. 2746–2748