Semantical Math Editor for Wikipedia: Deep UX tribute to OpenSource

The Problem

Development of Next-Gen math tools is blocked by a complexity of semantic markup languages. To make features like automatic hand-written calculus and physics solution correctness checking or interactive tooltips with each number meaning on formulas and equations, search documents in the internet with specific equation or formula, we need to have a simple and easy to use tool to make semantic math expressions can be created by anyone without a single line of code.

Only at the level of complete semantic markup, the computer understands the physical meaning of the expression, and the true and complete meaning of the expression. To understand the benefits of the complete semantic markup, you can take a look at several concept services designed by me.

The first example is the possible use of semantic markup in Wikipedia, which allows you to get the title, description and links to detailed articles, and for constants also their meaning in pop-up windows, while you hover mouse cursor over symbols, without having to navigate to other pages.

A second example is use in interactive math environments and math learning systems.
Such systems can reduce the time for students to study the exact sciences, and also save teachers' time on checking homework, since problems, even those solved on paper using character recognition, can be checked automatically.

The third example is the implementation of searching for pages and documents on the Internet that contain a specific mathematical expression, provided that they contain semantic markup.

Purpose

The purpose of my work was to make guidelines for creating a tool in which you can quickly and painlessly create, edit and view mathematical expressions with full semantic markup. To make this possible, existing solutions were analyzed, their advantages were found, prototypes of editors were created and tested, the frequency of using mathematical symbols and the procedure for studying symbols in the school curriculum were analyzed, and the recommendations themselves were written.

Work plan

• Explore the UX of existing methods for entering mathematical expressions;
• Find competitive advantages in UX in each;
• Build UI prototypes and figure out the configuration that achieves the best performance
• Analyze the frequency of use of mathematical symbols and the order of studying symbols in the school curriculum for their ergonomic arrangement in the interface
• Write recommendations for building the most effective interface

Solution principle

The principle of operation of full semantic markup implies the use of a partial markup syntax with the use of an additional identifier for elements, which is associated with a dictionary of values of elements of mathematical expressions. The so-called Gold Standard was adopted a year ago by a group of researchers, with the participation of representatives of the W3C consortium and the developers of Wikipedia. It uses Wikidata as an open and community-extensible reference, DLMF as an expert-only extensible reference, and arXiv as a reference for information on new scientific discoveries.

Existing solutions

Today, the only example of full semantic markup implementation is the Wikipedia component, which launched in January 2020.
As a reference, the Wikidata knowledge base is used, built on the basis of content from the Wikipedia blocks in the upper corner of the page, it is possible to view the values, which, however, requires a transition between pages. The solution is not convenient enough for regular use, therefore it has a perspective for the implementation of the work results. Like the content of Wikipedia itself, its engine is developed free of charge. I contacted the developers from the working group and they expressed interest in my work and now I am translating it into English so that they have the opportunity to familiarize themselves with it.

Wikipedia semantic editor prototype (2020)

The next most developed tool for working with semantic markup, however, is partially the Gradarius training system, the essence of which is to teach students mathematical analysis by solving problems on a computer screen in the same way as if they were solved manually on paper and provides opportunities as a step-by-step check indicating errors in the solution process, as well as a batch final check on the result of the control task with an assessment, freeing the teacher from this work.

Gradarius / Semantic Math Editor (WIP)

In scientific circles, the most common editor for texts containing mathematical notations is TeXmacs, which has a rather convenient system of hotkeys when typing mathematical expressions, however, it supports only partial semantic markup.

TeXmacs, partial semantics support since 2013

A Pilot Experiment

To begin with, a pilot experiment was carried out, where respondents were asked to type the same mathematical expression in 4 of the most popular online formula editors: Mathcha, Wikipedia, Tutorialspoint and Wiris Mathtype.

After each one, respondents were asked to express an opinion on the advantages and disadvantages and assess the convenience on a ten-point scale. Editors order were random to neutralise the effect of consistency. Respondents were wearing eye-trackers and the time to complete each task was measured. The experiment was done in one day in total of 8 hours with 10 respondents in the Usability Lab of ITMO University. It helped to understand the best performing practices in math editor UX design and formulate 4 experimental hypotheses about some notable features of the editor interface, that require testing on real prototypes:

• Categories are best done in user's local language;
• Categories are best done with icons;
• The tabbed interface is more convenient than the categories;
• Grouping items into small categories is more convenient than into large ones.

Experimental Prototypes

For the main experiment, 6 different prototypes of semantic formula editors were developed using the Typescript language. Between themselves, in pairs, these prototypes had one difference in design. At least 10 respondents for each pair of prototypes consistently entered a mathematical expression into each editor of the pair.

Study results

• The language is not very important because users just look at the content of the categories without reading the names
• Using logical icons instead of category names can speed up work by 10%
• The respondent spends 2.6 times less time to complete the task with the prototype, where the categories are divided into tabs
• Half of respondents refused to use a prototype with too much category content

The study was carried out among students from different countries and specialities at the Tomas Bata University in Zlin (Czech Republic). In order to eliminate the effect of the sequence, the order of prototypes in the test was changed. Statistical significance was verified using Wilcoxon's T-test.

Experiments

Experiment 1

Influence of the native language in the interface on the performance of the assignment on Russian-speaking students was investigated. The difference was not significant.

Experiment 2

Names of the categories were replaced with icons of the most recognizable symbols from the corresponding sections of mathematics. There is a funny fact here that a quarter of the respondents did not notice the difference, but when they were using icons they completed the task faster.

Experiment 3

There was a significant gain in time when using tabs in the symbol insertion button bar, compared to using a scrolling grouped list.

Experiment 4

When the symbols were divided into too broad categories, respondents gave up and asked to stop the experiment.

Based on the results of the study and feedback from the respondents, conclusions were drawn on how best to design the editor interface:

• Categories are more efficient with icons
• Placing input fields with a vertical line reduces eye movements to only one axis
• Number indices should be clearly visible in the WYSIWYG view
• Need contextual learning
• The most frequently used elements should be at the beginning

Symbols learning order and use quantity research

Then an analysis of the frequency of using mathematical symbols in articles on arXiv was carried out and 9 Russian school textbooks on mathematical sciences were analyzed by the order of introduction of symbols and, according to the results, the symbols were divided into 7 categories.

Tool's Interface

The most common usage pattern is typing a mathematical expression and then semantic markup. The meaning of operators is recognized automatically, while the meaning of numbers is specified by the user. However, the system tries to suggest the most accurate option.

Alternative input scenarios are also designed: for example, right in the WYSIWIG editing window by pressing the backslash, you can open the insert window and write the name of the symbol or its latex code. And also, the tool has an autocorrect function and, for example, three dots in ranges, rows and matrices will be replaced with a regular or middle ellipsis, respectively.

Awards

• "Best Science Paper" of "XV Congress of Young Scientists" (Конгресс Молодых Учёных), Saint-Petersburg, 2019

Publications

• Article at "Scientific and technical bulletin of information technologies, mechanics and optics" (Научно-технический вестник информационных технологий, механики и оптики), 2020
• "Design of an application interface to work with semantic markup of mathematical expressions", Master's Degree Thesis, 2020