Methods of personal information environment evaluation

Current condition of information and communication technology use by general public is suitable for a research of other information phenomena such as personal information environment. As a continuation of the broader study concerning this topic, in this article are discussed various methods of personal information environment assessment and evaluation. Data were collected by the means of survey and supervised tasks. Established differences between selected methods are presented. This study may be seen as suitable preparative study of this field.


Introduction
An increasing trend of information and communication technology (ICT) utilization can be observed. The manifestation of this phenomenon in the area of the Czech Republic can be described as a gradual increase in a number of households equipped with a personalized instance of said technology. The most common devices of such nature are personal computers (including laptops), smart phones, tablets and a growing number of specialized devices, that in some cases may be collectively called a "internet of things". A course said phenomenon was observed by a series of aimed surveys conducted between years 1991 and 2018. In accordance with a global trend surveys show an increase in the number of households in Czech Republic owning a personal computer from 100 thousand household (approximately 4% of Czech Republic population) up to almost 3.3 million households, representing a 76% of a total population. [1,2,3,4,5] Advancement of this phenomenon can be seen as corresponding with ICT structure development among other states of European Union. In an environment of the European Union the most households using some form of personal computer with a high-speed internet connection was found in Netherlands, Luxembourg and Denmark. On the other side of the scale the lowest number of such households was found in Bulgaria, where only 67% of households own and utilize observed forms of ICT. But in accordance with a discussed phenomenon in Bulgaria (as well as in Czech Republic, Slovakia, Romania, Hungary, Poland and Greece) a steep growth of said households can recorded through the observed years 2007 and 2017. [1,2,3,4,5] Technologies of an ICT family in a context of a civilian personal use mainly aim to facilitate and/or simplify information access, inspection, processing and archiving. But with a growing inclusion of various ICT elements into everyday lives a new aim of services can be distinguished. In a contrast to adapting ICT to existing services a new approach has appeared -to create a service, aimed at ICT utilization itself and its further functionality. Among these we can include services as data storage or social networks. In general it can be concluded that manifestation forms of information are spreading through the general population and therefore it can be argued, that the impact of singular information on an organizational unit has increased, depending on the information. This influential function of information can be sometimes described as a value of information and in current social, economic and political environment it can be said that the value on information is increasing in general. Due to a various reasons, for example information extraction or preservation, digital information is considered to be the main medium of information today. [7,8,9] Current development of consumer ICT is mostly focused on speed-up and simplification of the information processes. With a direct technological approach a barrier between developer and user can arise, due to different view of a solved problem. These barriers can be overcome with an application of other science fields. Findings from areas of psychology, librarianship and human-computer interaction (HCI) are widely used. [7,8,9] Throughout both personal and professional life an individual is introduced to a various means of information extraction and processing. User adaptation to introduced methods, processes and other elements of information processing can be seen as a composition of said information processing elements. For the purpose of further research, this composition can be identified as a personal information environment (PIE) of a user. It is a sum of all information sources, tools and methods of information processes concerning an individual user. A manifestation of ICT usage increase can be observed on a PIE phenomenon as well. However, there was only a little or none consideration of an end user so far during ICT development, as the main focus was on a technical advancement in said field. But because a human is still an inseparable part of the whole system, it can be argued that system can fulfill its assignment only as well as its user "allows it", or in other words: a system can beneficial only as much as its user is proficient. The discussion between development and actual application should be open, yet still circumspective. Emphasis on technological aspects only can result in a unused form of beneficial technology, on the other hand too much emphasis on a user or consumer aspects can lead to a highly specialized technology with a field-tied application.
Establishing a certain structure of PIE and its variations can be beneficial for further technological and personal growth with a relation to ICT. There are various existing and described structures that can be used to further ascertain PIE structures and can be used as a relation element for other fields. Various studies were conducted concerning PIE, aimed at a structure and contents. Previously conducted studies were focused on exploring the current situation in the area of PIE concerning usage of ICT elements -what elements of ICT are most commonly used and how. These studies also subsumed usage on non-ICT elements of information processes. For further establishment of PIE categories certain forms of a PIE evaluation are needed. Due to a uniqueness of a PIE problematics, we needed to establish suitability of existing PIE evaluation methods with a possibility of designing a new method for PIE evaluation. [6] Establishing structure or guidelines concerning PIE can have a positive influence on further ICT development and application. Users can be distinguished by their ICT proficiency and ICT requirements, PIE determinants and overall approach to PIE and information. Collected findings may simplify a process of designing complex information systems of both ICT and non-ICT elements aimed at a specific group of users. Newly found strategies and approaches can be applied in any field with an ICT representation, but we can see various security applications as one of the main fields, benefiting from presented PIE studies.

Methods
As a part of the complex PIE study a need to evaluate individual PIEs has appeared. The main aim of such evaluation was to establish an effectivity of individual instance of PIE, or more precisely user proficiency in information processes, notably extraction, processing and archiving. All methods were tested on the same group of individuals. Collected data were evaluated using methods of comparative analysis Testing group consisted of 34 students of Tomas Bata University, distributed between various study programs. Due to the long duration of test, we were unable to gain more participants. Forms of participant motivation should be considered in the future. To be able to compare various methods, all participants were subjected to all chosen methods of evaluation. Following methods were tested:

Self-evaluation questionnaire
This method of PIE evaluation was chosen as a direct continuation of previous studies. The core of this method was a questionnaire. The questionnaire consisted of 12 questions aimed at a various aspects of information processes. Each question had a form of a simple task (for example -finding information of scientific character, finding an original source of information, finding archived information) with three evaluating elements: tools and methods used for this task, an assignment of numeral value according to perceived difficulty (5easy, 1 -difficult) and estimated duration of task completion. Tasks were divided into two categories: tasks concerning newly gained information and task concerning archived information.

Self-evaluation semi-conducted interview
Self-evaluation semi-conducted interview was conducted with a certain time detachment from a selfevaluation questionnaire. Semi-conducted interviews discussed the same tasks as self-evaluation questionnaire with the same evaluating categories. Individual questions had the same evaluating criteria as in previous method. The focus of these semi-conducted studies was to establish a difference between free self-evaluation and assisted evaluation. In some cases user can feel proficient in certain task but further questioning may reveal inadequacy of time perception or perception of task termination.

Sample task
Among discussed methods this method is the most time demanding methods, but it may yield the most accurate results concerning PIE and information processes. This method consisted of so-called sample task -user was given a task of finding, processing and archiving two types of information -one from their professional field and one concerning their personal activities (concerning personal bureaucracy). These tasks were further divided into stages. First stage of the tasks was to find, to process and to archive said information. Second part was performed after a short time-window (more than a week, not more than a month) and was focused on finding and using previously archived information. During both stages the user was observed and the data, concerning used tools, methods and consumed time were recorded. The overall performance of the user was determined and results were discussed with a tested user.

Results
As a first step of method assessment an average value of task difficulty was calculated. By comparing results from all three tested methods it can be established how various method differentiate concerning self-perception and actual task performance. Following figures ( Figure  1, Figure 2, Figure 3) present an percentage of users with an according average evaluation (numbers 1 to 5) of discussed tasks. This average value can be perceived as an assessment of user proficiency concerning information related tasks. By comparing individual figures it can be found that most users not only perceive themselves as intermediate users (values 3 and 4). The biggest difference between self-evaluation and observer evaluation can be found in the most advanced group of users (value 5). In a case of self-evaluation, more than 20 % of asked users considered given tasks to be very easy, but when confronted with additional questions or task itself, their self-evaluation of task difficulty rose considerably. On the other side of spectrum users with a perceived high difficulty of tasks were more precise in assessing their skills.
Next step in comparing selected methods of PIE evaluation was to establish a relation between selfperceived time and actual time user needed for a certain task. Due to a vast amount of time, needed for selected tasks, only two of the tasks included in questionnaire and semi-conducted interview were performed during a testing of "sample task" method. However these tasks were chosen from said questionnaire, allowing a basic comparison of these methods. In the following figures ( Figure 4, Figure 5, Figure 6) average times of users are presented. These averages are shown within their relation to previously established user groups (numbers 1 -5) and the type of task (N -newly gained information, A -archived information).  As can be seen in presented figures, intermediate users (3 and 4) are more precise when approximating time needed for accomplishing a certain task. From the collected data we can conclude, that in general more advanced users tend to underestimate how long will it take them to complete a task and less advanced users are keen to overestimate time needed to accomplish a task.
To better present differences between methods, data collected by the method of sample task were used as objective measurements of said task and following figures (Figure 7, Figure 8, Figure 9) shows the difference between these data and data collected by other methods. In Figure 7 there are shown differences of "sample task" method and both "self-evaluation" method (S-E) and method using semi-conducted interview (S-C).

Fig. 7
Difference of values collected using various methods. [%] Following figures (Figure 8, Figure 9) shows difference between actual time needed to complete a task (data from "sample task" method) and approximated time values of other methods for both tasks -with "newly gained information" (N) and with "archived information" (A) Fig. 8 Difference of values collected using methods "selfevaluation" method (shown) and "sample task" method (compared to).
[minutes] Fig. 9 Difference of values collected using methods "selfevaluation semi-conducted interview" method (shown) and "sample task" method (compared to). [minutes] These data can be interpreted in a way that suggests suitability of all three methods for intermediate users (groups 3 and 4). The further we reach into extremitiesbeginners and very advanced users -the bigger are differences between perception of task and reality. In general it can be said, that semi-conducted interviews yield results more similar to a "sample task" method. This method can be considered more precise than other two methods due to higher involvement of the observer. However, this method is also very time demanding. It requires at least two separated sessions with each user. Their length may vary depending on task type ("newly gained information" or "archived information") and user proficiency.
After close examination of results we can conclude, that method called "semi-conducted" interviews may seem as an optimal solution. While it requires considerably less time, it yields results similar in values to the "sample task" method. In terms of user evaluation percentages, the biggest difference is +/-5 % comparing to almost 20% of self-evaluation method. In the time needed to complete task there is only 1 minute difference for "archived information" tasks and under 4 minute difference for "newly gained information" tasks, comparing to 4 (A) and 6 (N) minute differences in "self-evaluation" method. However these data should only be considered as tentative due to lack of respondents. For a further research a wider sample of population is required.

Discussion
Increasing trend of ICT integration into everyday tasks of modern society and its structures can be observed through its manifestations discussed in this article. Further focus of conducted study was on the means of assessment and evaluation of individual PIE instances. The objective of this part of the study was to ascertain the most suitable method for a mass PIE evaluation and its research. Suitable analysis of gathered data was applied to correctly establish objectivity and accuracy of some methods that can be used for PIE evaluation. Because of a variable character of PIE and its individual instances methods capable of containing such variety had to be used. Study shows, that there may be a definable difference between perception of task difficulty and proficiency in observed tasks. It may seem that more advanced users are prone to overestimating their abilities, whereas less advanced users tend to be more accurate concerning self-evaluation of their PIE. It should be noted that this study has only preliminary character due to a small number of respondents. Further studies of this phenomenon may attract more respondents, partly due to increasing popularity of information and related topics. Security is one of the fields depending on collaboration of information and social sciences that can be also provided by further PIE research and connected phenomena.