Virtual Reality-Based Fuzzy Spatial Relation Knowledge Extraction Method for Observer-Centered Vague Location Descriptions

Abstract

Many documents contain vague location descriptions of observed objects. To represent location information in geographic information systems (GISs), these vague location descriptions need to be transformed into representable fuzzy spatial regions, and knowledge about the location descriptions of observer-to-object spatial relations must serve as the basis for this transformation process. However, a location description from the observer perspective is not a specific fuzzy function, but comes from a subjective viewpoint, which will be different for different individuals, making the corresponding knowledge difficult to represent or obtain. To extract spatial knowledge from such subjective descriptions, this research proposes a virtual reality (VR)-based fuzzy spatial relation knowledge extraction method for observer-centered vague location descriptions (VR-FSRKE). In VR-FSRKE, a VR scene is constructed, and users can interactively determine the fuzzy region corresponding to a location description under the simulated VR observer perspective. Then, a spatial region clustering mechanism is established to summarize the fuzzy regions identified by various individuals into fuzzy spatial relation knowledge. Experiments show that, on the basis of interactive scenes provided through VR, VR-FSRKE can efficiently extract spatial relation knowledge from many individuals and is not restricted by requirements of a certain place or time; furthermore, the knowledge obtained by VR-FSRKE is close to the knowledge obtained from a real scene.

1. Introduction

Many documents, such as travel notes, sightings, and historical records, use natural language text to describe the spatial location at which a person observes a specific object; when the corresponding objects have vanished (e.g., have been destroyed), this spatial information is the key data for reconstructing the objects described by such a document [1,2]. Through geographic information systems (GISs), which have the capabilities of spatial data storage, representation, and analysis, scenes corresponding to documents can be reconstructed with the support of evidential data [3,4]. However, a GIS can process quantitative data, but cannot directly process the qualitative and vague spatial relations contained in location descriptions; therefore, it is necessary to convert the qualitative spatial relation information contained in the document of interest into quantitative coordinates and features [5,6,7].

Currently, the majority of existing vague location extraction methods are based on object-to-object topological relationships. In these methods, spatial relationships are usually expressed based on fuzzy membership functions calculated from the intersecting areas or distances between objects, and this strategy is effective in expressing boundaries and locations with uncertain characteristics [8]. Unfortunately, however, the description mode typical of travel notes, sightings, and historical records has the distinctive characteristic that the observer describes what he or she sees from his or her own perspective, and fuzzy membership functions cannot effectively express such observer-centered descriptions [9,10]. Observer-centered descriptions express observer-to-object topological relationships and reflect highly personal subjective viewpoints; therefore, the key to mastering these relationships is to capture the characteristics of the corresponding observer’s spatial cognition and transform them into spatial knowledge [11]. To this end, this research proposes a virtual reality (VR)-based fuzzy spatial relation knowledge extraction method for observer-centered vague location descriptions (VR-FSRKE). In VR-FSRKE, a VR scene is presented to collect users’ cognitive interpretations of observer-centered spatial relations and, thereby, to obtain fuzzy spatial relation knowledge; this knowledge can then drive the transformation of spatial information from document text to fuzzy spatial regions. In experiments, VR-FSRKE was compared with real scenes and fuzzy spatial relation knowledge collection methods based on 2D software. The results show that the knowledge obtained by VR-FSRKE is closer to the knowledge obtained from a real scene than that obtained using 2D software collection methods, while VR-FSRKE offers higher efficiency than knowledge extraction from real scenes because it is not restricted by the location or environment. Therefore, VR-FSRKE can play an important role in obtaining spatial information from documents that contain observer-centered vague location descriptions.

2. Related Works

2.1. Spatial Relation Extraction from Vague Location Descriptions

Natural language text relating to geographic space poses a challenge for GIS processing owing to vague and context-dependent variations; it usually contains phrases that describe spatial relations and reference objects [12,13,14]. Human cognition of locations is usually related to regions, and the borders of these regions can be either sharp or vague [15]; accordingly, to represent the vague boundaries of regions, fuzzy or logical formalization methods can be adopted in the GIS context [16,17]. Vague regions can be defined by a set of sharp regions combined with features that represent the extent of the vagueness [18]; in this way, topological relations, Voronoi diagrams, and Delaunay triangulations can be used to describe vague locations [19,20,21]. Spatial location and relation terms occurring in natural language can be mapped to geometric formalisms [22,23]. Gazetteer or geographic knowledge can be used for place name recognition [24]. Using knowledge based on empirical data and context, spatial location information can be interpreted from natural language expressions [25]. Travel blogs, social media, housing advertisements, and news articles can all be used as data sources to greatly improve the ability to parse vague place names in GISs [26,27,28,29]. Natural language relating to a geographic location can be used to describe that location or to explain the navigation pathway to reach it [30]. Geometric configurations can be mapped to spatial relation terms by means of random forest and 9-intersection models [31]. With the help of a spatial template, spatial relational terms can be generated from a reference location [32].

Observer-centered descriptions usually reflect a person’s subjective viewpoint; such an underlying viewpoint serving as the basis of spatial terms and features usually does not take the form of a specific fuzzy function, but rather arises from the person’s inner cognitive mode and, accordingly, different individuals or groups will have different viewpoints [33]. Driven by rules, agent-based models can simulate individuals’ or groups’ cognitive results and behavior in a certain location [34,35]. Spatial-temporal accessibility, topological connectivity, the flow between features, and infrastructural connections can be used to quantitatively express spatial distance cognition results [36,37]. To interpret observer-centered vague location descriptions, it is necessary to extract individuals’ cognitive processes and results concerning spatial relationships on a human scale, especially for objects that are “seen” from the perspective of the observer.

2.2. Geographic Scenes Simulated by VR Technology

VR technology can simulate geographic scenes at the human scale and assist in the interactive collection of cognitive results [38,39]. By integrating multidimensional, spatiotemporal, and fully interactive immersion, VR can provide a better understanding of geoprocesses [40]. VR scenes are used to study the influence of environmental factors on the evacuation decisions of individuals, thus evacuation systems can be evaluated and optimized based on user cognitive results [41,42]. Huang proposed a data-driven VR system to provide an immersive experience of forest scenes; the corresponding product can give the public a better understanding of the impact of climate change on the environment [43]. Levin created a VR system for topographical survey learning, thereby reducing the equipment costs and requirements in terms of meteorological conditions for students [44]. Halik adopted a virtual environment of city buildings to collect preferences and behavior regarding participants’ way-finding strategies [45]. As seen from the above results, VR can be used to effectively collect data concerning people’s cognition in a geospatial environment, and this capability is useful for the collection of knowledge regarding observer-centered spatial relations.

2.3. Challenges When Processing Observer-Centered Vague Location Descriptions

At present, although many achievements have been attained in research on fuzzy location representation and fuzzy spatial relationship extraction, these achievements still cannot solve the inference problem for observer-centered vague location descriptions. The main reason is shown in Figure 1 below.

As shown in Figure 1, the existing methods of formulating fuzzy spatial descriptions mainly focus on describing the relationships between objects (“object to object”). These spatial relationships are usually based on the distance between the objects, the intersecting area, or the proportion, and a fuzzy membership function can easily be found to describe these fuzzy characteristics; thus, vague qualitative descriptions can be transformed into quantitative spatial features [46,47]. Conversely, vague location descriptions from the observer perspective (“observer to object”) describe a person’s perception of the spatial relationship between the observer and object; correspondingly, vagueness arises from the observer’s subjective viewpoint, which will be significantly affected by the person’s knowledge, characteristics, and cognitive abilities [48]. A typical example is given as follows:

“There is an ancient statue not far in front of me.”

This sentence contains the observer-centered direction description “in front” and the distance description “not far”. As humans are not strict measuring equipment, the phrase “in front” is unlikely to have the strict meaning that the observer is facing in the direction of the object at a 0 degree angle; a certain range of deviation to the left or right can also be considered to correspond to the “front” direction, leading to fuzziness of this spatial description. This fuzzy attribute does not originate from a specific membership function, and different people’s viewpoints and cognitions will lead to different angular ranges associated with the “front” direction. In addition, the distance description will also be affected by the observer’s personal characteristics; for a basketball player with a height of 2 m and an older person in poor health, the phrase “not far” may have very different meanings in terms of absolute distance. Traditional fuzzy membership functions have difficulty expressing such subjective viewpoints. To infer the interpretation of the above vague description from the observer perspective, we need to apply the following strategies:

(1) Obtain fuzzy spatial relation knowledge corresponding to a specific vague description.

Once the relationship between a specific vague location description and the corresponding representable fuzzy region is established, this relationship can be stored as fuzzy spatial relation knowledge. Through this knowledge, vague descriptions can be converted into fuzzy regions that can be represented in a GIS.

(2) Associate fuzzy spatial relation knowledge with the personal characteristics of specific groups.

For example, people of different heights, different ages, and different health conditions may have different views on spatial relationships. Obtaining different knowledge for specific groups can enable more accurate interpretation of their descriptions of spatial relationships.

For the application of the above two strategies, the method adopted in our team’s previous research was as follows: users manually drew the spatial regions corresponding to vague location descriptions in a 2D GIS, thereby creating corresponding knowledge [48]. This method is simple and easy to implement; however, drawings on a 2D map obviously exhibit large differences from the perception of spatial relations in the real world, thus the information obtained in this way cannot be guaranteed to reflect the true viewpoint of an individual.

To accurately extract subjective spatial viewpoints concerning the relations between the observer and objects, the most direct method is to conduct physical on-site experiments, in which objects and observers are placed at certain locations in a large area, the observers’ opinions on whether an object’s location matches a given vague description are continuously collected, and the relative positions between the observers and objects are synchronously recorded. After many such placement and recording processes, the corresponding spatial knowledge can be obtained. This method can ensure that the acquired knowledge truly corresponds to an individual’s subjective spatial viewpoint, but some difficulties will be encountered in practical application:

(1) The experiment requires a large area.

It is difficult to find a sufficiently large and undisturbed area for object placement and description test recording. As the scope of the descriptions of interest increases, the requirements for the area of the site, the funding, and the number of participants will increase accordingly. Consequently, it is difficult to collect the necessary knowledge from a real environment owing to location and funding restrictions.

(2) The knowledge collection efficiency is low.

In each spatial knowledge collection process, it is necessary to place objects, measure coordinates, and then record them; it takes considerable time to perform these tasks in a real scene. Because of the vague nature of the location descriptions, characterizing even the simplest description requires the collection of a very large amount of data; for multiple groups and multiple descriptions, it is difficult to complete collection in a reasonable time.

For the above reasons, it is difficult to collect fuzzy spatial relation knowledge from real scenes. VR technology provides us with a new way to solve this problem. We can use VR to build a virtual test scene based on a certain location description; the tester can easily place objects anywhere in the scene and automatically record the spatial relationships between the observer and objects. In this virtual scene, there are no location, funding, time, or participant restrictions, thus the data collection efficiency is sufficiently high. Therefore, this research aims to introduce VR technology for fuzzy spatial relation knowledge extraction for observer-centered object vague location descriptions.

3. Methodology

3.1. Overall Process of the Method

This research proposes VR-FSRKE. VR-FSRKE can handle vague location descriptions from the observer’s perspective; the goal of the method is shown in Figure 2.

As shown in Figure 2, VR-FSRKE is mainly used to extract spatial relationships corresponding to location descriptions for objects close to the observer. These relationships include eight directional terms, Direction = {Front, Left front, Right front, Left, Right, Left rear, Rear, Right rear}, and three distance terms, Distance = {Adjacent to, Not far from, Slightly far from}. Although these descriptions can express the spatial relationship between the observer m and a specific location, they are nonquantitative and cannot be expressed and analyzed in a GIS. The strategy of VR-FSRKE is to convert these descriptions into corresponding regions. These regions have corresponding spatial coordinates and degrees of membership and can be expressed in a GIS. The regions corresponding to different location descriptions constitute fuzzy spatial relation knowledge. VR-FSRKE needs to extract this knowledge and use it to infer the spatial relations contained in vague location descriptions.

To extract the spatial knowledge depicted in Figure 2, the associations between vague location descriptions and regions need to be established. Specifically, Direction × Distance = 8 × 3 = 24 spatial relations need to be collected, and each relation requires iterative testing with an observer m and an object (to determine whether the spatial relationship between m and the object is consistent with the location description) within a certain scene. In a real scenario, this will be a very time-consuming task. To improve efficiency, in VR-FSRKE, a VR environment is created to allow knowledge extraction to be performed through an interactive VR process. The main process of VR-FSRKE is shown in Figure 3.

As shown in Figure 3, VR-FSRKE has four steps.

(1) Generation of the virtual interactive scene

To describe the content of the VR environment, VR-FSRKE uses the raster dataset V_raster = {v₁, v₂, …, v_n}, where v_i indicates the color and object placed in the corresponding raster area and contains detailed information on regions and objects. Through a virtual interactive scene generation method, VR-FSRKE converts the content in V_raster into the VR scene E_scene. This step is described in detail in Section 3.2.

(2) Acquisition of individual spatial knowledge through virtual interactions

The observer m uses VR devices to interact in E_scene. For a specific location description d, m places virtual objects in E_scene such that the spatial relationships between these objects and m conform to the spatial relations described in d. When multiple objects are placed, these objects form a spatial region R_ind that reflects the individual’s subjective view of the spatial relationship. Thus, R_ind, m, and d can be combined to obtain the corresponding individual spatial knowledge k_ind. After performing multiple k_ind collections, VR-FSRKE can obtain a dataset of individual spatial knowledge K_ind = {k₁, k₂, …, k_n}. This step is described in detail in Section 3.3.

(3) Fuzzy spatial relation knowledge extraction

The knowledge set K_ind obtained in the second step only reflects the spatial knowledge of a specific individual and does not have the property of generalization. To obtain truly representative fuzzy spatial relation knowledge, further statistical analysis and fusion needs to be performed on K_ind. The contents of different K_ind sets are grouped based on the characteristics of the observers (e.g., height and age) and the location descriptions. Each group corresponds to a particular subset of fuzzy spatial relation knowledge, and all groups together can yield a general fuzzy spatial relation knowledge set K_knowledge = {k₁, k₂, …, k_n}, where each k_i contains fuzzy regions corresponding to a specific observer group and location description. Based on these k_i, VR-FSRKE can infer spatial relations from descriptions. This step is described in detail in Section 3.4.

(4) Vague location description inference

For an input description d, K_knowledge is used to infer the corresponding spatial region. In this inference process, the first task is to find the knowledge k_i in K_knowledge that matches the location description and observer associated with d, and then the R_ind corresponding to this k_i can be obtained; subsequently, in accordance with the location coordinates and orientation of the observer, R_ind is transformed into real GIS location coordinates to obtain the result region. This step is described in detail in Section 3.5.

3.2. Generation of the Virtual Interactive Scene

VR-FSRKE first constructs a raster dataset V_raster = {v₁, v₂, …, v_n} describing the content of the virtual scene. The structure and characteristics of this raster dataset are shown in Figure 4.

As shown in Figure 4, the width of each raster area in V_raster is P_scale, and each raster area in V_raster is described by a tuple v_i = (coordinate, renderstyle, content, membership) containing four key elements:

(1) coordinate

The coordinate field of v_i describes its position in V_raster. In VR-FSRKE, a raster area v_center is selected as the location of the observer, where coordinate = (0,0); v_center is the reference point for the coordinates along the x- and y-axes, and for each v_i, the x- and y-coordinates of coordinate are set in accordance with the grid distance to v_center.

(2) renderstyle

The renderstyle property indicates the style with which v_i needs to be displayed in the virtual scene. This property determines the height, color, and placement of the block in the virtual scene.

(3) content

The attribute content is a list of objects that need to be placed on v_i. When content = ø, no objects are placed on v_i; when content ≠ ø, the corresponding objects will be placed on v_i.

(4) membership

In each interaction between m and the VR environment, v_i may be assigned to a certain region, and membership stores its membership value for the corresponding region. membership takes values in the range of [0, 1], where 0 means that v_i has no correspondence with the corresponding location description and 1 means that this raster area unambiguously belongs to this location description.

The above four elements are divided into two groups. In the virtual scene content group, coordinate and renderstyle are used to generate (or modify) the VR environment E_scene = {e₁, e₂, ..., e_n}. Each e_i = {box, rendercontent} directly corresponds to the respective v_i. Here, box indicates the position and range of v_i in the VR environment; its width is P_scale and its position is (x, y) = (v_i.coordinate.x × P_scale v_i.coordinate.y × P_scale). The rendercontent property of e_i corresponds to the renderstyle property of v_i, which directly determines the color of the box and the objects to be placed. In the interactive modification group, the content and membership attributes of each v_i will be modified during the interaction between the observer m and E_scene. The VR-FSRKE method will modify the renderstyle of v_i based on content and membership, and coordinate and renderstyle will cause further changes in E_scene. Through this mechanism, a cycle of “generation -> interaction -> generation” is formed. The corresponding process is shown in Algorithm 1.

Algorithm 1 Interactive VR scene generation algorithm (IVRSG)

Input:Vraster, Escene, m, d Output: Vraster, Escene

Begin Initialize all ei in Escene = {e1, e2, …, en} based on Vraster; while (m needs to make further changes in the region)   for i = 1 to length(Vraster)    vi = Vraster[i]; ei = Escene[i];    if (the content and membership of vi have changed)      vi.renderstyle = Determined based on the content and membership of vi;       ei.rendercontent = Determined based on the renderstyle of vi;       Update ei in Escene;   m places objects in Escene that conform to the spatial relations in d;   Modify the content and membership of each element in Vraster based on the placed objects; Return Vraster, Escene; End

IVRSG forms a cyclical process: V_raster affects the display content of E_scene and the VR users interact with E_scene to further modify the content of V_raster. At this time, an observer’s viewpoint regarding the vague location description d can be displayed in the VR environment by means of objects and regions, so the observer can intuitively verify the consistency of his or her viewpoint regarding the spatial relations and E_scene and can continue to improve it.

3.3. Acquisition of Individual Spatial Knowledge through Virtual Interactions

For a specific location description d, an observer m can interact with E_scene in the VR environment and continuously modify V_raster. From these interactive modifications, corresponding spatial knowledge can be extracted. The process is shown in Figure 5.

As shown in Figure 5, based on a specific description d, the observer m places objects in E_scene that conform to the subjective viewpoint of m regarding the spatial relationship expressed by d. Based on the positions at which the objects are placed, the smallest convex polygon that can contain them all is constructed. Then, the boxes in E_scene are intersected with the convex polygon to determine the boxes corresponding to the fuzzy region. To express the fuzziness of the subjective viewpoint of m, each object to be placed also possesses a membership attribute; a high membership value for an object means that it is definitely in the region described by d, and a low membership value means it may or may not be in the region described by d. Two difficulties may be encountered when using VR to collect data: (1) VR equipment is relatively expensive, so most laboratories cannot use many devices in parallel to collect data. (2) Some participants will experience severe dizziness and nausea when using VR for a long time; therefore, we need to reduce the time and difficulty of the VR collection process as much as possible. According to experimental experience, people often easily understand the difference between membership degrees with larger differences, but find it difficult to grasp more subtle differences. Therefore, by default, an object in VR-FSRKE may be assigned one of three levels of membership:

(1) membership = 1.0: the object is certainly in the spatial region described by d.

(2) membership = 0.5: the object is likely to be in the spatial region described by d.

(3) membership = 0.1: the object may possibly be in the spatial region described by d.

m interactively places objects in the virtual environment to modify the content of V_raster, and the corresponding process of obtaining the individual spatial knowledge k_ind is described by Algorithm 2.

Algorithm 2 Individual spatial knowledge acquisition algorithm (ISKA)

Input:Vraster, Escene, m, d Output:kind

Begin olist= According to the subjective perception of d, m places objects in Escene; groups = Group olist based on membership and sort the groups by membership value in ascending order; for i = 1 to length(groups)   group = groups[i];   convex = Establish a minimum convex polygon based on the positions of the objects in the group;   blocks = Find all ei that intersect with convex in Escene;   rlist = Find the rasters in Vraster corresponding to blocks;   All membership in rlist = membership value of the group; Update rlist in Vraster; Rind = All raster areas in Vraster such that membership≠0; kind = (m, d, Rind); Return kind; End

The output result of ISKA is the individual spatial knowledge k_ind = (m, d, R_ind), where the content of m is the typical attribute information of the observer, such as height, age, and gender, and d is the location description corresponding to Direction and Distance. ISKA first groups all objects, and all members of the same group are considered to have the same membership. A convex polygon is constructed based on the locations of all objects in the same group; then, the convex polygon for each group is intersected with E_scene to obtain R_ind.

When there exist multiple observers and multiple descriptions, multiple instances of individual spatial knowledge can be obtained by repeatedly executing the ISKA method, thus a dataset of individual spatial knowledge K_ind = {k₁, k₂, ..., k_n} can be formed. Each k_i in K_ind is closely related to a specific individual’s knowledge; consequently, because individuals often have a certain degree of subjective bias, further summarization and statistics are needed to improve the inference ability that can be achieved on the basis of this knowledge.

3.4. Fuzzy Spatial Relation Knowledge Extraction

The knowledge set K_ind obtained in the second step of VR-FSRKE contains individual characteristics and the corresponding regions R_ind for certain location descriptions; these items are bound to specific individuals and have low generalization ability. To improve the generalization ability and obtain more broadly useful spatial knowledge, VR-FSRKE should group different K_ind sets by their characteristics and further perform region fusion to obtain more general fuzzy spatial relation knowledge. The specific process is shown in Figure 6.

As shown in Figure 6, all location descriptions form a set of n_desc elements D = {d₁, d₂, ..., d_ndesc}, and all observers can be partitioned by their personal characteristics (such as height) to form a set of n_fth elements S = {s₁, s₂,..., s_nfth}; then, D × S can form a set of n_desc × n_fth elements kg_i = {k₁, k₂, ..., k_n}, where each k_i can be used to select individual spatial knowledge in K_ind and its corresponding combined spatial region R_cmb = {r₁, r₂,..., r_n} can be expressed as follows:

$$R_{\mathrm{cmb}}=\frac{{\displaystyle \sum _i^n{k}_i.R_{ind}}}{n}.$$

(1)

The values of the elements of R_cmb are also in the range of [0, 1]; to reduce the differences between the R_cmb obtained for different k_i and to improve the interpretability of the extracted spatial knowledge, VR-FSRKE needs to set discrete values of R_cmb. Accordingly, the value of each element r_i is mapped to 0, 0.1, 0.5, or 1:

$$r_i^{\prime }=\{\begin{array}{cc}\begin{array}{c}0\\ 0.1\\ 0.5\\ 1\end{array}& \begin{array}{c}{r}_i=0\\ 0<r_i\le 0.1\\ 0.1<r_i\le 0.5\\ r_i>0.5\end{array}\end{array}.$$

(2)

Through Formula (2), we can obtain the mapped region R_knd = {r₁′, r₂′, …, r_n′}. The process of obtaining R_knd is described in Algorithm 3.

Algorithm 3 Fuzzy spatial relation knowledge extraction algorithm (FSRKE)

Input: Kind, D, S Output:Kknowledge

Begin Kknowledge = ø;   for i = 1 to length(D×S)     (di, si) = (D×S)[i];     kgi = Select elements in Kind based on di and si;     Rcmb = Calculate all Rind in kgi using Formula (1);     Rknd = Calculate Rcmb using Formula (2);     Kknowledge←(di, si, Rknd); End

Through FSRKE, we can obtain a fuzzy spatial relation knowledge set K_knowledge = {k₁, k₂, …, k_n}, where k_i is expressed as (d_i, s_i, R_knd); that is, for a group of specific descriptions and observers (conforming to d_i and s_i), the corresponding fuzzy region R_knd can be obtained. Therefore, k_i can also be expressed in rule form as follows:

“IF description contains d_i AND observer belongs to s_i THEN EXPORT R_knd;”

K_knowledge represents a group’s common point of view on spatial relations; it can be used to infer the meanings of vague location descriptions and express the results in a GIS.

3.5. Vague Location Description Inference

Suppose that an observer p gives a vague location description d that describes the particular location or spatial relation of an object o from her perspective. The spatial region associated with d can be inferred from K_knowledge. The inference process is shown in Figure 7.

Figure 7 shows the process for an observer in a real GIS environment p = (l, s), where l is the location and direction information of the observer p and s is p’s inherent characteristics. The spatial description d contains the fuzzy description of the object seen by p. Using d and s, the corresponding knowledge k_i = (d_i, s_i, R_knd) can be found in K_knowledge, where d ∈ d_i, s ∈ s_i, and R_knd is a fuzzy region corresponding to d. The regions in R_knd are expressed using coordinates relative to the observer; based on the position and direction of l, all coordinates of R_knd can be transformed into real coordinates. Thus, the possible locations of the object o can be expressed as a grid region Region = {r₁, r₂, ..., r_n}. In the initial state, the membership values of r_i are all 0, which means that the location of o is not known at all. When object o is seen by multiple observers P = {p₁, p₂, ..., p_n} (e.g., as in the witness reports for a legal case) and each observer provides a vague location description D = {d₁, d₂, ..., d_n}, a knowledge matching process can be applied to obtain the corresponding regions R = {r₁, r₂, ..., r_n} and, through coordinate transformation, the corresponding area set R′ = {r′₁, r′₂, ..., r′_n} in a GIS can be generated. We use Region = {r₁, r₂, ..., r_n} to export the results of this inference process, where the membership of r_i is used to express the degrees of matching with the different descriptions; membership = 0 means that the corresponding r_i is not at all related to the descriptions, and the more an r_i′ in R’ intersects with r_i, the higher the corresponding membership value of r_i will be, indicating that the corresponding region has a higher probability of being covered by the location described by the descriptions. For r_i in Region, the accumulated result of the intersection with multiple regions r_i’ is shown in Formula (3):

$$r_i=r_i+{\displaystyle \sum _{j=1}^n(r_j^{\prime }\times \frac{area(r_i\cap r_j^{\prime })}{area(r_i)})}.$$

(3)

when all the r^′_i values have been accumulated, the number of descriptions is used to normalize each r_i in Region:

$$r_i=\frac{r_i}{\mathrm{number} \mathrm{of} \mathrm{descriptions}}.$$

(4)

Elements in Region that are too small (locations with few supporting descriptions or membership values that are too low) need to be filtered out:

$$r_i=\{\begin{array}{c}\begin{array}{cc}0& r_i\le \alpha \end{array}\\ \begin{array}{cc}{r}_i& r_i>\alpha \end{array}\end{array},$$

(5)

where a is the minimum support threshold and a = 0.1 by default. Algorithm 4 describes the above inference process.

Algorithm 4 Fuzzy region inference (FRI)

Input: P, D, Kknowledge Output:Region

Begin Region = Initialize the inference result grid; R’ = ø;   for i = 1 to length((P, D))    (pi, di) = (P, D)[i];    ki = Find knowledge in Kknowledge in accordance with pi.s and d;    R’← Spatially transform ki.Rknd based on pi.l;  for i = 1 to length(Region)    ri= Region[i];    ri ← Accumulate R’ using Formula (3);   Region = Normalize Region and filter out grids whose support is too low using Formulas (4) and (5); Return Region; End

The output Region of the FRI algorithm contains the inference results for P and D. When the membership value of r_i in Region is higher, o is more likely to be in the location corresponding to r_i; when this value is lower, the possibility of the object being in this region will be lower. The FRI algorithm is used to finally infer the possible location of o based on K_knowledge.

4. Experiments

4.1. Implementation and Execution of the Method

The implementation of the VR-FSRKE method consists of two components: (1) VR interaction and knowledge collection: C++ and Unreal Engine are used to realize the VR environment, and the individual spatial knowledge sets K_ind are collected from the VR environment and saved in a PostgreSQL database. (2) Spatial extraction and inference: Python 3.7 and ArcGIS Pro 2.6.0 are used to perform fuzzy spatial knowledge extraction, location inference, and result output, and all output (knowledge and results) can be stored in an ArcGIS project. All programs are run on a computer equipped with an Intel i9-9900K CPU, a GeForce RTX 2080 Ti 11 GB GPU, and 64 GB of memory, and for the VR equipment, the HTC VIVE is adopted.

For the VR environment, we built a VR scene E_scene to perform interactive data collection from observers. V_raster in VR-FSRKE corresponds to a 101 × 101 raster grid with a scale parameter of P_scale = 0.5 m; the details of the VR environment are shown in Figure 8.

Figure 8a shows a top view of E_scene. The red square at the center on the ground is V_center, at which the observer is placed. Large trees are placed directly in front of, behind, and to the left and right of the square to assist the observer in judging the direction. The virtual scene that the user can see through the VR glasses is close to the corresponding real scene in size and distance. Figure 8b shows the virtual objects that can be manipulated and the display result for E_scene after objects are placed on the ground.

We invited 20 students from Changchun Institute of Technology to participate in the test of the VR-FSRKE algorithm. These 20 students were randomly selected, and they included 10 male students and 10 female students; everyone could use the ISKA algorithm to place virtual items in E_scene through interaction for the acquisition of individual spatial knowledge k_ind. Through data collection, VR-FSRKE could obtain K_knowledge.

4.2. Comparison of Spatial Knowledge Collection Methods

To verify the spatial knowledge collection effectiveness of VR-FSRKE, three methods were chosen for comparison in this study:

(1) Fuzzy spatial region extraction model for the vague object location descriptions from the observer perspective (FSREM-OP): FSREM-OP utilizes 2D graphical interface software that allows users to draw their own cognitive regions or spatial relations. These regions can be combined and converted into spatial knowledge [48].

(2) Manual spatial knowledge collection in real scenes (MSKC): A real 0.5 × 0.5 grid is drawn at a real site, and the positions of real objects manually placed at the site are recorded. The recorded positions are transformed into spatial regions using the ISKA algorithm, and the FRI algorithm is used to obtain spatial knowledge.

(3) VR-FSRKE: The proposed method based on interaction in a VR environment for the automatic collection of spatial region knowledge.

For the 24 descriptions of Direction × Distance given in Section 3.1, grouped by Distance, the time needed to obtain spatial knowledge is shown in

Table 1. The time required to collect data at different distances using the three methods.

Method	Total Time Required to Collect 8 Directions for 20 People (Minutes)
	Distance = Adjacent	Distance = Not far	Distance = Slightly far
FSREM-OP	321	335	367
MSKC	1040	1631	Unable to collect
VR-FSRKE	568	579	603

.

As seen from Table 1, FSREM-OP is based on drawing directly on the 2D plane of the software interface. This method is the simplest to use, so it takes the least time to collect spatial knowledge. For MSKC, as our team could not provide a sufficiently large and open ground space, it was not possible to collect corresponding knowledge for Distance = Slightly far; moreover, because manually placing objects and recording positions takes considerable time, the time required for MSKC is the longest. VR-FSRKE is based on interactions with objects and virtual environments in VR and is not limited by the real environment. Therefore, knowledge collection can be achieved no matter how great Distance is. Because determining the positions of virtually manipulated positions is more difficult than drawing a polygon with a mouse, the time consumption of VR-FSRKE is longer than that of FSREM-OP; however, because it is significantly easier than manual placement and recording, the time consumption of VR-FSRKE is markedly less than that of MSKC.

When Distance = Adjacent and the observer characteristics = ø (there is no partitioning based on the observer’s characteristics), the spatial knowledge content obtained using the three methods is as shown in Figure 9.

As shown in Figure 9, because real scenes are used to collect knowledge in MSKC, the content it yields is the closest to the observers’ real cognition and viewpoints; therefore, the results of MSKC can be used as verification data for FSREM-OP and VR-FSRKE. As seen from the results circled in purple, (1) in terms of asymmetry, the results produced by FSREM-OP are symmetrical, whereas the MSKC results are asymmetrical between left and right, and the VR-FSRKE and MSKC results are more consistent; (2) the area corresponding to FSREM-OP is significantly larger than those corresponding to VR-FSRKE and MSKC. This shows that the spatial knowledge obtained by VR-FSRKE is closer to the knowledge obtained in the real environment.

Focusing on Distance = Adjacent and Direction = Left, we choose the observer characteristics = {ø, male, female}. The corresponding comparison of the obtained knowledge is shown in Figure 10.

As shown in Figure 10, for FSREM-OP, the difference between characteristics = male and female is not obvious. For MSKC and VR-FSRKE, however, the results obtained for different observer groups are slightly different, indicating that VR-FSRKE can describe the characteristics of a group’s spatial knowledge cognition in more detail. On the one hand, this again proves that the vague location descriptions based on the observer’s perspective are subjective, as they are related to specific groups and individual characteristics. The consistency of the trends of MSKC and VR-FSRKE shows that an observer’s behavior during spatial knowledge collection in the VR environment is very close to that in the real environment, thus VR-FSRKE can obtain spatial knowledge more accurately than FSREM-OP.

4.3. Inference of Spatial Locations in a GIS

To test the value of the spatial knowledge collected using these methods, GIS data for Changchun Institute of Technology were used in this study to perform inference on spatial locations in a real area. The research area and the observers’ information are shown in Figure 11.

Figure 11 shows the location of Changchun Institute of Technology and an object Object1 on a weedy playground. There are five observers near Object1, and they describe the location of Object1. The corresponding descriptions are listed in

Table 2. Observers’ vague location descriptions for Object1.

Observer	Description
Observer1	Object1 is adjacent to and in front of me.
Observer2	In the adjacent area to the left in front of me, I saw Object1.
Observer3	At a distance not far from me, Object1 is to the right and in front ofme.
Observer4	At a distance slightly far from me, Object1 is to the left and in front of me.
Observer5	Object 1 is in front of me, and the distance is slightly far.

.

Table 2 provides an appropriate case to prove the effectiveness of the proposed method; Object1 has been seen by a limited number of observers, and related methods can be used to extract the possible locations of Object1 from this limited number of descriptions. For all descriptions in Table 2, because there are no quantitative spatial coordinates or ranges, it is obviously impossible to directly express these descriptions in a GIS. However, through the spatial knowledge obtained as described in Section 4.2, these descriptions can be converted into fuzzy spatial regions. The outputs of the three methods are shown in Figure 12.

As shown in Figure 12, with MSKC, the contents for Observers 4 and 5 cannot be inferred owing to the lack of knowledge for Distance = Not far from or Slightly far from. For FSREM-OP, as described in the Section 4.2 experiment, the knowledge it yields is larger than that of MSKC in both distance and area; however, although the high-membership area contains Object1 for Descriptions 1 and 2, for Descriptions 3 to 5, Object1 is in the area of low membership or even beyond the scope of the inference result. In comparison, VR-FSRKE yields the best inference results. For Descriptions 1 and 3, the output is similar to that of MSKC. For Descriptions 4 and 5, on the one hand, as this method is able to obtain corresponding spatial knowledge, VR-FSRKE can be used to infer the location; on the other hand, Object1 is contained in the high-membership area of the inference results. These findings indicate that the spatial knowledge collected by VR-FSRKE can indeed meet the needs of vague location description inference.

The location descriptions from the different observers can be superimposed on each other using the FRI algorithm. This superposition process is necessary for the reconstruction of location information from many witness reports. For Object1, the superimposed results for the knowledge obtained using the three methods are shown in Figure 13.

As shown in Figure 13, owing to the relatively large areas associated with the knowledge acquired via the FSREM-OP method, it cannot completely match individuals’ specific cognition, so the area indicated by FSREM-OP is also the largest (least accurate). For MSKC, the high-membership area of the superimposed region contains Object1, which is very helpful for the determination of Object1’s location. Unfortunately, however, Descriptions 4 and 5 cannot be used for inference owing to the lack of corresponding spatial knowledge, which leads to a much larger area around the observer becoming a possible location for Object1. The superposition results of VR-FSRKE are better than those of FSREM-OP; the results cover a relatively small area, and the high-membership area of the superimposed region contains Object1. The above results show that, for an observed object, the knowledge obtained by VR-FSRKE can be more accurate in describing the location information.

4.4. Analysis of the Spatial Knowledge Collection Strategies of the Three Methods

MSKC is the most direct way to collect knowledge of a person’s cognition of a specific vague location description from a first-person perspective. MSKC requires interaction with a variety of real locations and descriptions in a real scene; with the introduction of more observers and more objects for verification, spatial cognition information can be extracted as fuzzy spatial relation knowledge. However, this method has obvious drawbacks: it requires a large ground space, the participation of many people, and considerable interaction and recording time. As seen in Table 1, this method already has a high time cost when collecting knowledge for only 20 people. For descriptions of locations sufficiently far from the observer, knowledge cannot be collected via MSKC owing to the inability to provide a correspondingly large area. Therefore, MSKC is not efficient in practice, and it is difficult to collect knowledge from larger groups or larger scenes.

Regarding FSREM-OP, as it involves drawing polygons on a 2D plane on a computer, it is the easiest to implement; consequently, FSREM-OP also requires the shortest time to complete knowledge collection for 20 people. However, there is a large difference between the results of drawing on a 2D plane and those of manual collection in a real scene. Figure 9 shows that the area obtained by FSREM-OP is much larger than the area obtained by MSKC. As seen in Figure 12 and Figure 13, in the process of inference, as the distance increases and the number of descriptions increases, the knowledge acquired by FSREM-OP will accumulate larger errors, which shows that the reliability of the spatial knowledge extracted by FSREM-OP is not high.

In VR-FSRKE, VR scenes are introduced to collect spatial knowledge. It can be seen from Figure 9 that the VR-FSRKE method produces results very similar to those of the manual method; at the same time, owing to the introduction of VR technology, it is not restricted by place and time, and VR-FSRKE can be used to collect individual spatial knowledge in a shorter time. Figure 12 and Figure 13 show that VR-FSRKE can yield more comprehensive spatial knowledge with content close to that collected through MSKC. This verifies that VR-FSRKE performs well in terms of spatial knowledge collection ability and efficiency.

5. Conclusions

Vague location descriptions from the observer’s perspective contain much spatial location information. However, because they have no quantitative coordinates, they cannot be directly expressed or analyzed in a GIS. To convert vague location descriptions into areas that can be expressed in a GIS, spatial knowledge corresponding to such vague location descriptions is necessary. However, this knowledge is an expression of the observer’s subjective viewpoint in a real scene. Collecting this knowledge requires interacting with real objects in real scenes and, in most cases, it is not possible to meet the collection needs in terms of place and time.

To the best of our knowledge, VR-FSRKE is the first method to introduce the use of VR scenes into research on fuzzy spatial relation knowledge extraction. In VR-FSRKE, an interactive VR scene is constructed in which the user can interactively establish a connection between a given description and a spatial location, thereby enabling the extraction of fuzzy spatial knowledge. The main advantages of VR-FSRKE are as follows:

(1) It introduces a VR environment to collect spatial knowledge

With the help of a VR scene, on the one hand, users can quickly and intuitively establish relationships between specific descriptions and locations, directions, and distances for the acquisition of spatial knowledge; on the other hand, owing to the use of a VR environment, the knowledge collection process is not limited by temporal or spatial factors. It can be seen from the reported experiments that the knowledge collection speed of VR-FSRKE is far superior to that of MSKC; moreover, for some knowledge involving larger spatial distances, VR-FSRKE can be used to easily collect knowledge that is impossible to collect through MSKC because it is limited by the space available in the real scene.

(2) It provides relatively accurate spatial knowledge

In the process of interacting with the VR environment, the user can experience the environment in a way that is close to the corresponding real spatial positions and directions. Thus, the spatial knowledge obtained based on this experience is close to that obtained from the real scene. In the reported experiments, VR-FSRKE yielded better spatial knowledge than FSREM-OP, and the inference results obtained using this knowledge were more accurate.

Based on the above advantages, VR-based knowledge can build a bridge between fuzzy location descriptions and a GIS and can provide assistance in expressing and drawing inferences regarding the spatial locations of specific objects (especially historic sites that have disappeared or the circumstances described by sighting reports); moreover, the spatial knowledge obtained through VR-FSRKE is not specific to VR and can be generalized and reused. The reusability of this knowledge makes it easier to obtain reasonable returns for the high design costs of VR scenes. VR-FSRKE expands the application scope of VR technology to GISs, and it has good application value in extracting spatial regions from vague location descriptions from the observer’s perspective.