SOFTWARE QUALITY EVALUATION USING ISO/IEC 9126 ON THE ITB STIKOM BALI ONLINE ACADEMIC INFORMATION SYSTEM (SION)

 

Ni Nyoman Utami Januhari ¹, Putu Angga Putra Ronda ¹, I Gede Giri Panti Wijaya ¹

 

¹ Computer System, ITB STIKOM Bali, Indonesia

 

1. INTRODUCTION

The need for software in the area of Higher Education is increasing every year. This software is needed to assist in the business processes within the university environment. The success of the software built is seen based on whether or not it aligns with the ongoing business processes. The quality aspect of software is one of the important factors in software development Subedi (2016). Software is not only judged by its end product but also by the development stages of the software itself. Quality assurance is needed in every stage of the software lifecycle. There are several characteristics commonly used to assess software quality, both qualitatively and quantitatively Hilabi (2015).

Currently, many Academic Information Systems (AIS) are built using web-based applications. AIS is crucial in academic institutions because it manages various business processes within a campus or school Yuhana et al. (2014). One of the private universities that has been using software as a tool to provide necessary information to students and other stakeholders, especially in the academic field, is the Institute of Technology and Business (ITB) STIKOM Bali, with the Academic Information System used by students online (SION) and integrated with the Academic Information System (SION). The implementation of the Academic Information System for students online (SION) is used to serve academic activities for the academic community and is generally used by students and academic staff (system administrators). Software quality fulfills documented functional and performance requirements. To ensure good software performance, it is necessary to accurately determine user requirements. A product has good quality if it can satisfy the majority of its users Mulyanto (2016).

The problem at ITB STIKOM Bali is that the Online Information System (SION) as an application software has been implemented for a long time but has not been tested for reliability, so instruments are needed to measure and ensure the quality of the Online Information System (SION). Evaluation of the quality of the Online Information System (SION) software needs to be done by providing a method/model to measure and evaluate it so that the quality of the Online Information System (SION) at ITB STIKOM Bali can be improved in the future. Based on the journal article by Dr. Rafa E. Al-Qutaish (2010) in Amalia & Wijaya (2018), the ISO 9126 quality model created by the International Organization for Standardization (ISO) and the International Electrotechnical Commission (IEC) is the most efficient model because its development is based on international consensus and is approved by all member countries of the ISO organization. This research will measure the quality of the Online Information System (SION) application owned by ITB STIKOM Bali in Renon Denpasar using the ISO 9126 software testing model. The aim of this research is to evaluate the quality of the software based on ISO/IEC9126. The method used for testing the quality of the software with ISO 9126 standards consists of two aspects, which are conducted through questionnaire surveys. With the results of the research, it will be possible to understand the quality of the academic information system software, and it is hoped that the quality improvement of the Online Information System (SION) application at ITB STIKOM Bali can be enhanced in terms of its usability and functionality.

 

2. MATERIALS AND METHODS

To ensure the representativeness and reliability of the findings, a sample of 100 active students from STIKOM Bali was chosen as respondents. The selection criteria included various demographic factors such as age, gender, academic background, and year of study to capture a diverse range of perspectives. The sampling process employed a stratified random sampling technique, where the student population was divided into homogeneous subgroups based on predefined characteristics. From each subgroup, a proportionate number of participants were randomly selected, ensuring that each subgroup was adequately represented in the final sample. Moreover, efforts were made to obtain consent from participants, ensuring their voluntary participation in the research. Ethical considerations were paramount throughout the selection process, with confidentiality and privacy of participants being strictly upheld. By employing a systematic and transparent approach to sample selection, the study aimed to enhance the generalizability and validity of its findings, providing valuable insights into the quality evaluation of the SION system within the context of ITB STIKOM Bali.

The analytical flow conducted by the researcher commenced with a comprehensive literature review, followed by initial data collection, identification of evaluation needs, specification of evaluation criteria, design of evaluation methods, execution of the evaluation process, and ultimately culminated in drawing conclusions and providing recommendations.

Figure 1

Figure 1 Research Analysis

 

The research, titled "Evaluating Software Quality with ISO/IEC 9126 on the Online Academic Information System (SION) at ITB STIKOM Bali," was conducted at the Institute of Technology and Business (ITB) STIKOM Bali, focusing on the Online Academic Information System (SION) located at Jl Raya Puputan No. 86 Renon Denpasar, Bali. The study spanned a duration of 8 months. Primary data was collected directly during the research process. The methodology employed for the analysis and testing stages included observation, interviews, and questionnaires. Observation activities involved direct scrutiny of the academic information system (SION) processes. The findings of these observations were then utilized to determine the system's requirements analysis.

Interviews were conducted directly with academic unit leaders and administrators to gain deeper insights. Questionnaires were utilized for testing the functionality and usability aspects, with questions designed to elicit responses from students, faculty, and administrators. The data collection instrument utilized in this study comprised evaluation sheets in the form of questionnaires. For testing functionality aspects, test cases were employed with criteria tailored to user requirement lists derived from functional system analysis, thus creating research instruments specific to functionality. Usability testing instruments utilized a validated and internationally reliable questionnaire, the USE Questionnaire developed by the STC Usability and User Experience Community by Arnold M. Lund. This questionnaire encompasses four components: Usefulness, Ease of Use, Ease of Learning, and Satisfaction.

 

3. RESULTS AND DISCUSSIONS

The Questionnaire Feasibility Test has also been conducted by the researcher using the SPSS application. After the respondents filled out the questionnaire, at this stage, a feasibility test of the questionnaire responses was conducted prior to the usability processing. The feasibility test conducted included:

Validity test using Pearson product-moment correlation analysis. This validity test aims to determine whether the items answered by the respondents have a valid or invalid level of correlation. The validity test requirement is r-value > r-table Budiastuti & Bandur (2018). The r-table value with a significance level of 5% for 100 respondents is 0.195. The results of the validity test using the Talocraft application usability scale showed that all Pearson Correlation values were greater than 0.195. It can be concluded that all questions distributed in this study are valid.

Reliability test is conducted by comparing the r-table value with the Cronbach's alpha result. The reliability test requirement is alpha > r-table [6]. The r-table value with a significance level of 5% and a number of respondents of 100, as in the validity test, is 0.195. The results of reliability testing using the SPSS application are as follows:

Figure 2

Figure 2 Reliability Test Result

 

The Cronbach's alpha result above shows a value of 0.968, which is greater than the r-table value of 0.195. This indicates that the questionnaire distributed in this study is consistent.

The data processing results conducted by the researcher are as follows:

1)    Functionality Aspect

Functionality testing was conducted by 100 ITB STIKOM Bali students as users. Testing was done using a checklist method based on the functions available in the "SION ITB STIKOM Bali" system. Based on the results of the functionality aspect testing, there were 1991 responses indicating YES and 9 responses indicating NO. The overall functionality aspect testing results were calculated using the formula:

Explanation:

A = Number of functions that do not work correctly/failed/invalid

B = Number of functions evaluated

0 ≤ X ≤ 1, Functionality is considered good if it approaches the value of 1. Therefore, the overall functionality calculation in this study is as follows:

X = 1 – 𝐴/B

   = 1 – 9/2000

   = 1 - 0,0045

   = 0.9955

   = 99.55%

Based on these results, it means that the system operates properly by 99.55%. Therefore, it can be concluded that "SION ITB STIKOM Bali" has fulfilled the functionality aspect with a value of 0.9955, which is close to 1.

The criteria for the functionality aspect are divided into 7 parts, and all of these parts are considered to meet the functionality criteria with the calculation as follows:

Table 1

Table 1 Summary of Functionality Aspects
ASPECT / VALUE	RESULT		CALCULATION	INFORMATION
Suitability (Kelayakan)	YES	200	X = 1 – A/B X = 1 – 0/200 X = 1 (100%)	Fulfill
	NO	0
Accuracy (Akurasi)	YES	297	X = 1 – A/B X = 1 – 3/300 X = 0.99 (99%)	Fulfill
	NO	3
Interoperability (Interoperabilitas)	YES	398	X = 1 – A/B X = 1 – 2/400 X = 0.995 (99.5%)	Fulfill
	NO	2
Security (Keamanan)	YES	100	X = 1 – A/B X = 1 – 0/100 X = 1 (100%)	Fulfill
	NO	0
Conformance (Kesesuaian)	YES	400	X = 1 – A/B X = 1 – 0/400 X = 1 (100%)	Fulfill
	NO	0
Completeness (Kelengkapan)	YES	299	X = 1 – A/B X = 1 – 1/300 X = 0.9967 (99.67%)	Fulfill
	NO	1
Compliance (Kepatuhan)	YES	297	X = 1 – A/B X = 1 – 3/300 X = 0.99 (99%)	Fulfill
	NO	3

 

2)    Usability Aspect

The usability testing yielded the following responses: Strongly Agree (SA) with a total of 1333, Agree (A) with a total of 1568, Disagree (D) with a total of 90, Strongly Disagree (SD) with a total of 2, and Very Strongly Disagree (VSD) with a total of 7. The usability testing results were calculated using the following formula:

Explanation:

SA = Number of respondents who answered Strongly Agree

A = Number of respondents Agree

D = Number of respondents Disagree SD = Number of respondents Disagree

VSD = Number of respondents Strongly Disagree JP = Number of questions

JR = Number of respondents

 

After the score is obtained, then look for the percentage to get an interpretation of the usability test results using the formula:

𝑃 = (𝑆coreCalculate/𝑆𝑘𝑜𝑟𝑚𝑎𝑥) x 100%

 

The results obtained are then compared with the percentage results obtained in table ….. The usability aspect is said to be good if the percentage results show the "feasible" criteria.

Table 2

Table 2 Kategori Penilaian Usability
Percentage Result (%)	Eligibility Criteria
0 – 20	Very Not Feasible
21 – 40	Not Feasible
41 – 60	Feasible Enough
61 – 80	Feasible
81 – 100	Very Feasible

 

So, the overall usability calculation in this research is as follows:

Skortotal = (1333x5) + (1568x4) + (90x3) + (2x2) + (7x1)

                                                                                                  = 6665 + 6272 + 270 + 4 + 7

                                                                                                  = 13218

 

Skormaksimal = 30 x 100 x 5

                             = 15000

P                          = 13218 / 15000 x 100%

                             = 88,12 %

 

The overall usability testing percentage result is 88.12%, which is then compared with Table 1. Therefore, it can be concluded that the overall testing result of the usability characteristics of the Online Academic Information System (SION) application at ITB STIKOM Bali is deemed Very Feasible and fulfills the usability aspect.

The usability calculation is divided into several criteria: Usefulness, Ease of Use, Ease of Learning, and User Satisfaction. All four aspects indicate a rating of Very Feasible, meaning that SION ITB STIKOM Bali is highly usable by users. Below is the calculation for each of these aspects:

Table 3

Table 3 Summary of Usability Aspects
ASPECT / VALUE	CALCULATION	INFORMATION
Usefulness	Skor total = 3575 Skor max = 8x100x5 = 4000 P = 3575/4000 x 100% = 89.37%	Very Feasible
Ease of Use	Skor total = 4807 Skor max = 11x100x5 = 5500 P = 4807/5500 x 100% = 87.4%	Very Feasible
Ease of Learning	Skor total = 1767 Skor max = 4x100x5 = 2000 P = 1767/2000 x 100% = 88.35%	Very Feasible
User Satisfaction	Skor total = 3069 Skor max = 7x100x5 = 3500 P = 3069/3500 x 100% = 87.68%	Very Feasible

 

4. CONCLUSIONS and RECOMMENDATIONS

Based on the various explanations provided above, for this Internal Research Progress Report, it can be concluded that, according to the research conducted by the researcher, the research calculations have been completed and yielded a value of 99.55% with the detailed value of the Functionality aspect are Suitability: 1, Accuracy: 0,99, Interoperability: 0,995, Security: 1, Conformance: 1, Completeness: 0.9967, and Compliance: 0.99. This indicates that "SION ITB STIKOM Bali" has fulfilled the functionality aspect with average value of 0.9955, which is close to 1. Usability testing has also been conducted, resulting in a average score of 88.12% with detailed Usability aspect are Usefulness: 89,27%, Ease of Use: 87,4%, Ease of Learning: 88,35% and User Satisfaction: 87,68%, indicating that it meets the usability aspect with a rating of very feasible.

Recommendations that the researcher suggest are Continued Monitoring: Despite achieving high scores in both functionality and usability, it is essential to continue monitoring the system's performance over time to ensure its continued effectiveness. User Feedback: Regularly gather feedback from users to identify any potential areas for improvement or additional features that could enhance the system's usability and functionality further. Updates and Maintenance: Schedule regular updates and maintenance sessions to address any identified issues, enhance system performance, and keep it aligned with evolving user needs and technological advancements. Training and Support: Provide adequate training and support resources to users to ensure they can fully utilize the system's functionality and features, maximizing its effectiveness in supporting their academic activities.

This study on the software quality evaluation of the STIKOM Bali Online Academic Information System (SION) using the ISO/IEC 9126 framework has some limitations which are the data used for evaluation were collected within a specific timeframe and may not reflect the system's performance under varying conditions or over a longer period and This study does not include a comparative analysis with other similar academic information systems, which could provide additional context and benchmarks for the evaluation.

 

CONFLICT OF INTERESTS

None. 

 

ACKNOWLEDGMENTS

I would like to express my deepest gratitude to all those who have contributed to the successful completion of this research on "Software Quality Evaluation Using ISO/IEC 9126 on the ITB STIKOM Bali Online Academic Information System (SION)." First and foremost, I extend my heartfelt appreciation to the management of ITB STIKOM Bali for granting permission to conduct this research and for providing access to the necessary resources and facilities. I am also indebted to the students of ITB STIKOM Bali who generously volunteered their time and participated in the research activities. Their valuable feedback and input have been integral to the success of this study. Additionally, I would like to acknowledge the support and understanding of my family and friends throughout this journey. Their unwavering encouragement and patience have been a constant source of strength and motivation. Thank you all for your invaluable contributions and support. Your efforts have been instrumental in making this research possible.

 

REFERENCES

Amalia, R., & Wijaya, A. (2018). Evaluasi Kepuasan Pengguna Sistem Informasi Akademik Perguruan Tinggi Menggunakan Standar Iso 9126, J. Ilm. Matrik, 20(2), 151–162. https://doi.org/10.33557/jurnalmatrik.v20i2.117

Budiastuti, D., & Bandur, A. (2018). Validitas Dan Reliabilitas Penelitian. Mitra Wacana Media.  

Hilabi, S. S. (2015). No Analisis Kualitas Perangkat Lunak Terhadap Sistem Informasi Stt Wastukancana Purwakarta,” J. Teknol., 5(1).  

Mulyanto, A. (2016). Pengujian Sistem Informasi Akademik Menggunakan Mccall’s Software Quality Framework,” Jiska, 1(1), 47–57. https://doi.org/10.14421/jiska.2016.11-07

Subedi, B. P. (2016). Using Likert Type Data In Social Science Research: Confusion, Issues and Challenges, International Journal of Contemporary Applied Sciences, 3(2), 36–49.   

Yuhana, U. L., Raharjo, A. B., & Rochimah, S. (2014). Academic Information System Quality Measurement Using Quality Instrument: A Proposed Model, In 2014 International Conference On Data And Software Engineering, Icodse 2014), Institute of Electrical and Electronics Engineers Inc. https://doi.org/10.1109/ICODSE.2014.7062684