Abstract:

In this paper we present a brief study on the capability of LLM to edit mathematical expressions by means of natural language instructions. Mathematical notation used in the study is encoded in the LaTeX-based EQED format, developed by the authors within the original math editing tool: Equation wizard (Polish “Edytor wzorów”) (Bier and Sroczyński 2015, 2019). We evaluate the repeatability and precision of EQED code alterations made by GPT-4o in reference to natural language instructions on required formula edition. For the three distinguished classes of editing commands: inserting, deleting, or replacing parts of mathematical expressions, we run a series of experiments and analyze the produced EQED code and its compilation obtained by prompting the LLM. We demonstrate the consistency and repeatability of LLM output based on the test requests.