Usability Evaluation Using Multi-Method for Improvement Interaction in M-Commerce

. Today, M-Commerce has become one of the common choices as a way of shopping for society. Therefore, the transaction in m-commerce grows significantly. Moreover, the competition of M-Commerce increases and the performance should be improved primarily for the satisfaction of buyers. These conditions are seen from an improvement of the features and service options that can be used by users, even outside the process of buying and selling goods. Increasing amount of features may reduce the usability of the application which may impact on the performance of M-Commerce. Therefore, this study aims to evaluate the usability of M-Commerce service. Our study addressed to evaluate the usability of one famous M-Commerce in Indonesia. The evaluation process was performed by using cognitive walkthrough and heuristic usability method. Data collection is done by using depth interview and questionnaire to collect quantitative and qualitative data. Our findings show some new insight for designers and developers about M-Commerce especially about usability and the potential of usability to improving the satisfaction of users in terms of using the M-Commerce application overall.


Introduction
Usability is one of the important factors in a mobile application. Usability is used as a guide to improve the ease of use of using an application, so users can conveniently and easily understand how the application works. Usability assessment is usually done by userevaluation process from the prototype stage to the final product before it is launched in order to guarantee ease of use for everyone [1]. Online shopping through smartphones or m-commerce is the most widely used and continues to grow today due to the characteristics of smartphones which are simple, efficient and high mobility [2].
In addition to the positive impact, smartphones also provide users to easily switch to various other m-commerce. The two aspects that affecting are customer retention and customer attraction owned by a m-commerce [3]. Therefore m-commerce does not far from the effort to increase usability [4] in order to improve customer retention through increased comfort and user satisfaction [5]. Some aspects related to user satisfaction are interaction quality, environment, and output [6].
Good usability will result in user comfort and satisfaction and improve customer retention of m-commerce. Also increased customer retention will have a positive impact on increasing user consumption and directly increase the amount of trading activity and profit on the m-commerce [7]. Therefore, this research will deepen the experience and opinion from the respondent about usability of the application that support customer retention toward m-commerce T with an interview about interaction, efficiency, effectiveness, learning, and satisfaction toward m-commerce evaluated [8].

Literature Review
There have been several previous research related to Cognitive Walkthrough [9]- [12]and Cognitive Walkthrough was originally developed as an additional tool for evaluation processes in engineering areas, such as giving a preliminary assessment whether users will be in favor to accessing and exploring the system [13]. The goal of the cognitive walkthrough is to assist designers in getting user-side assessments and potentially predicting problems that will arise in the process of interaction with the system [10]. Cognitive Walkthrough also focused on understanding the application learnability for new users by evaluator work through given tasks and answer set of questions [11]. Cognitive walkthroughs can produce deep knowledge from the user's point of view and also help in categorizing and positioning the reasons for the usability problems encountered by respondents.
There have been several previous studies related to usability M-Commerce and various methods applied [13]. One of them is a research on m-commerce in general with a background that m-commerce is more personal than e-commerce. Problems faced are the appearance of issues in the m-commerce application design process are not only from the technical side but also from the end-user point of view which then sought the solution with the evaluation of the respondents [14]. Evaluation of m-commerce usability continues to be profound because m-commerce continues to emerge and develop with its own characteristics to compete in attracting people's attention and trust.

Research Methodology
This research tries to evaluate the usability of one m-commerce in Indonesia with Cognitive Walkthrough method and assisted by the adoption of the Heuristic method and UPT for data retrieval and processing. The outline of this research (refer to Figure 1) is (1) Scenario Making, 2) Data collection 3) Analysis of evaluation data and analysis of usability problem which then proceed with the conclusion and possible solution to solve usability problems [15]. The scenario development used 9 Heuristic Usability points to be included in five scenarios with the composition shown in Table 1. The heuristic points used are adopted from heuristic points and adapted to the conditions m-commerce selected and nine heuristic usability points [16], [17]. These points cover the interaction that can be done with mcommerce. Scenarios initiated from observation of application interfaces, service trials, evaluation of assistance facilities, product purchasing process, assessment of errors, charging of buyer data, and payment processing [18], [19].

Data Collections
Data collected after the respondent conducted the scenario and performed in 2 ways for 2 data types, the first was the questionnaire for the quantitative data and the second was the interactive interview for the qualitative data. The given questionnaire will ask the respondent's assessment of some elements of the M-Commerce application that attempted when doing tasks in the scenario. In each question in the questionnaire will be provided Likert Scale [20]rating that divided into 5 level consisting of Strongly Agree (SA), Agree (A), Neutral (N), Disagree (D), Strongly Disagree (SD). In the qualitative part, the interview process is done by following the flow of respondents or according to the flow scenario to facilitate the respondent. During the interactive interview process, respondents were invited to issue all opinions about m-commerce usability and accompanied by exploring the reason of the respondents to the problems of reusability encountered.

Data Analysis
Quantitative data from the questionnaire will be processed and calculated mean and standard deviation on 5 scenarios from 10 respondents. For qualitative data will be done several steps to analyze and decode qualitative data into structured data. The following steps will be taken to approach the qualitative data.

Framework Development
The initial phase of framework identification is to understand the data collected. Qualitative data obtained from the in-depth interview will be processed with user action data from the video as well as data from the user questionnaire to be aligned and will be processed using the specified framework. This research, leaning toward analysis of qualitative data, therefore, the result will be framed with a table that contains the description of a usability problem, place of occurrence of the problem, and percentage of problem occurrence. Description of the problem of usability is a decoding of raw qualitative data obtained from in-depth interviews with respondents.

Classification of Usability Problems
After determining the framework, the next step is to classify the usability problem using the adoption of a predetermined framework which is also followed by giving severity to each reusability problem. To determine the severity, this study will use the categorization of the [21]Travis skeleton. The framework (referred to in Figure 3) uses three main questions, and then the assessment will be given between 1) Low, 2) Medium, 3) Serious and 4) Critical.

Charting Data
This step is done by making charts based on the usability problem. The chart will be made based on 3 aspects, namely the classification of the problem of reusability, the occurrence of the problem, and the severity (severity) [22].

Result
All selected are undergraduate and postgraduate students and know about online shopping on the smartphone. Age range of respondents are in 18-29 years and the majority over 20 years. Of the 10 respondents are included 1 who have never done online shopping through m-commerce and 9 of them already doing online shopping through m-commerce. Among those who have been shopping through m-commerce, 5 of them on average purchase 1 item per month, and 4 others on average purchase 2 to 3 items per month. For the type of goods purchased the majority are clothing and electronics. Respondents mostly choose a good transaction process as an important characteristic in a m-commerce.

Mean and Standard Deviation Questionnaire Data
The obtained questionnaire data will be used by calculating the mean and standard deviation (refer to Table 2) of the total of all 10 respondents indicating the UP (Usability Problem). The questions order will be sorted by scenario and assessment of scenario projected by 6 questions for each S (scenario). Chart of usability problem (refer to Figure 2) is describing the location of the incident, the number of usability problems and the severity. This is expected to give a clear picture of the comparison of reusability problems at each location/page in m-commerce applications.

Discussion
There were 90 reusability problems from 10 respondents obtained from the whole method, of which 37 were obtained from the questionnaire and 53 were obtained from the interview. The overall problem of usability is composed into 13 unique usability problems with the percentage requirement in interview method above 30% that means the problem often encountered by the respondent. Table 3 is list of usability problems based on the percentage of occurrence of the highest problem (preferred method interview), along with the data on the occurrence of the usability problem and severity. Usability issues with the highest percentage are in the help view and product search results view. The help view is not included in the red route in m-commerce applications and only has an average of severity one so that despite having the most complaints from respondents this problem is not critical or critical but needs to be fixed. For product search result page which is the main feature (red route) and the average severity is 2 or medium; therefore, a solution is needed to overcome the problem of found usability.
Problem with the highest percentage and severity level 2 is about the product suggestion as an alternative in case of problematic search. In interviews, 60% of respondents felt that they did not get a suitable or desirable product suggestion. Suggested products have different brands and even sometimes different types of products with similarity on the category. In addition to that, the suggested products have little amount therefore indirectly make the user must search again using different keywords to find the desired outcome. This usability problem refers to heuristic points of "Help users recognize, diagnose, and recover from errors".
The issue of usability problems with moderate severity (or 2) with a 40% percentage occurring on the home page view with a problem of difficulty reaching or finding the desired application function or service on the main page. The home page location is the reason causes a moderate severity because the home page is the main interface in the mobile app to greet users and should be comfortable and not confusing. Then another reusability is that it is difficult to understand the documentation of goods return assistance. From some help to m-commerce main functions, the item return section has a 50% or high enough reusability problem level even if the severity is only 1 or low.

Discussion& Conclusion
Overall, respondents prefer m-commerce applications that provide comfort to users when searching, purchasing goods, and satisfaction after payment because online shopping process ends when the goods are received. Satisfaction resulted by interaction with application can assured the buyer that the order will arrive at the destination without any problems. According to qualitative and quantitative data as support, the problem of reusability problems with the highest percentage occurring on the search results page is the problem of product proposals as a problematic search solution of a system that has a 60% percentage (moderately high) with moderate severity (or 2) since some respondents may overcome searches with other keywords. Another problem with reusable problems with moderate (or 2) severity with a large percentage is the difficulty of reaching or searching for the desired function on the main page. Based on the results of data analysis, there are three solutions as ideas or suggestions to improve customer retention and customer satisfaction for both new users or old users to remain loyal. The solution includes 1) Create a simple tutorial for the main task process from the main menu to payment. 2) Improve search suggestions or search solutions by adding suggestions generated from each keyword entered so that users can choose. 3) And the addition of structured help documentation with points as a pointer so that users avoid reading all the documentation to find the right solution. For further research, evaluation is expected to be differentiated based on respondent's experience on mcommerce (novice or experienced) so that the usability problem can be categorized to know the priority problem to find the solution in order to improve the usability, convenience, and user satisfaction in online shopping.