Detecting Similarity Between the Texts of Posts of Virtual Communities for the Formation of Software Documentation

Abstract

The field of information technologies consists of two significantly different parts: the production of information technologies (machines, equipment, programs, etc.) and the production of the information itself, which must be documented. Today, the demand for software development is at a record high and even exceeds the market supply. An important feature of software is the availability of proper accompanying documentation, which is necessary for both developers and end users. Information sources for the formation of software documentation can be virtual communities, which are the most visited resources among Internet users. The advantages of using virtual communities are given in the article. An important characteristic of software documentation is the provision of unique information content. To fulfill this requirement, it is necessary to check their content after uploading the publications to the data repository. It should be noted that virtual communities for the formation of software documentation should be thematic. As a result, a method was developed that displays the cosine similarity between all available posts. The research conducted showed that most of the posts contain unique content, but some may have similar texts. The advantage of using the similarity detection method is that after its execution among all pairwise comparison posts, the positions of pairs of posts can be determined. In the future, we will choose posts whose values will be of interest to the researcher and will allow us to conduct analysis using other methods. Next, cases of post similarity tests were investigated and actions to solve them were described, one of which is to joint similar posts and save all comments. It is recommended to use the N-gram method for posts that received a high value of the similarity measure using the cosine similarity method.