On January 20, we tested a problem-solving activity with ChatGPT artificial intelligence. Will it be able to solve it?
The advancement of artificial intelligence and, in particular, the appearance of ChatGPT, the latest release of OpenAI, has reopened the debate on how technology intervenes in people’s learning, how we deal with tasks, how educational institutions and teachers must react to adapt to new scenarios, if legislation should be passed in this regard and who should be in charge of doing so. Do we have to stop progress in technological research to protect our practices? Should the use of new tools be banned? Who is ultimately responsible for the use of technology?
In reality, it seems that the discussion is about, above all, what value we place as individuals, and as a society, on honesty and learning.
Many articles have been written in the press and on social media about the position of educators, schools and governments in the face of what seems to be the most immediate risk; students giving in to boredom and using artificial intelligence to perform tasks that are intended for them. It seems that what is concerning is that ‘you can cheat’ more easily.
The arguments that want to calm fears are those that refer to how artificial intelligence (AI) generates texts:
- It does not list the references in the content and, therefore, its veracity cannot be validated nor can it elaborate on what is being said.
- There is also talk of the bias in obtaining data that can easily be exclusive, from the first world to the privileged world, with little attention to minorities or other realities.
We want to point out that ChatGPT, the AI discussed in this article, is a beta version that hasn’t been updated since 2021, so substantial improvements are expected shortly.
We agree that we must be ready for developments, but we must also remember that there have already been other innovations that have made us reflect on learning in schools, in particular when it comes to mathematics. The most obvious example is the calculator. That’s why we defend competency-based learning, not mechanical learning, because we want our students to develop the skills needed to solve problems, argue, make connections and communicate clearly, while always being well-grounded with a solid knowledge of mathematics.
A Conversation with ChatGPT
Beyond the results obtained from the conversation with ChatGPT, it may be interesting to see how we can maintain a dialogue with a student based on this conversation. Let’s try it! This is the experience of Javi, a 5th grader.
Last Wednesday, January 18, the Problemas al Sprint (Problem Sprints) were held, a competition organized by the FEEMCAT (Federation of Entities for Teaching Mathematics in Catalonia), the Cesire/Creamat of the Department of Teaching and the SCM (Catalan Mathematics Society) for students in the 5th and 6th grades of elementary school. We set ChatGPT two of the problems that the students on these courses had to solve.
First problem
Write the smallest number made up of five different digits, without using 0, that meets the condition that the digits next to each other are never consecutive (in either ascending or descending order; for example, it cannot be 12, or 87).
Here are the answers from the experience with ChatGPT (the questions were originally asked in Spanish):
Chat: The smallest number made up of five different digits, without using 0, that meets this condition is 12 346.
Javi: That is not correct, remember that the numbers that are next to each other cannot be consecutive.
Chat: I understand, sorry for the confusion. The smallest number made up of five different digits, without using 0, that meets this condition is 10 234.
Javi: There are still consecutive digits and 0 appears.
Chat: Sorry, I’ve made a mistake. The smallest number made up of five different digits, without using 0, that meets this condition is 14 253.
Javi: This is not the smallest possible number.
Chat: I apologize. The smallest number made up of five different digits, without using 0, that meets this condition is 10 257.
Javi: You have used 0 again.
Chat: I am very upset with myself over this mistake. The smallest number made up of five different digits, without using 0, that meets the condition that the digits next to each other cannot be consecutive is 12 354.
Yes, I understand, it is not a language issue, but it may be that I have not correctly interpreted the constraints of the problem. If you give me a more detailed description or explanation of the problem, I will be happy to give you a correct answer.
Javi: Write the smallest number made up of five different digits, without using 0, that satisfies the condition that adjacent digits are never consecutive (in either ascending or descending order; for example, it cannot be 12, or 87).
Chat: The smallest number made up of five different digits, without using 0, that satisfies this condition is 10 257.
Despite trying to point out the error, it couldn’t find the solution. It doesn’t justify the reasoning and it’s difficult to guide AI to achieve the objective, it doesn’t follow the restrictions of the statement either (such as not using 0). Javi is surprised by the answers, and we came up with some questions for the chat. We couldn’t find the source of the error.
Second problem
Antonio wanted to find a number that, added together three times, would give 2023... but it's not possible. However, he saw that it was true if the last digit of one of the three addends increased by one unit: ABC + ABC + ABD = 2023. Where A, B, C and D represent numbers, the same letters always have the same value and D results from increasing C by one. What number does D represent?
Javi uses the vertical addition method to solve the problem. At first, he starts with the units, and he realizes that C + C + D cannot result in 3, so he concludes that it is a carrying addition, but he doesn’t know if he should look for 3 numbers that add up to 13 or 23… because he doesn’t know how many to take. He decides to start the sum by the hundreds. He looks for 3 numbers that equal to 20 added together, he says, “It can’t be 7 because the result is too big, 7 × 3 = 21”, so he looks for a smaller number and realizes that the difference will be 2. Now he repeats the argument with the tens, and he knows that you will have to search for 3 equal numbers that add up to 22, the 7 with a difference of 1 unit. Now he knows that C + C + D has to add up to 13.
And… ChatGPT, what does it do? Let’s see! At first, we only indicate to the chat if it gives an incorrect answer, without guiding it and without asking questions in order to move forward, so the dialogue loses interest.
Chat: The number that D represents is 8.
Javi: No, you’re wrong.
Chat: I’m sorry I was wrong. If the sum ABC + ABC + ABD = 2023, and we know that D is one unit greater than C, we can do the following calculation:
ABC + ABC + AB(C + 1) = 2023
where ABC is a three-digit number, and C and C. + 1 they are digits.
Therefore, the number that would have to go in D is 7.
Javi: No, D is not 7.
Chat: I’m sorry I was wrong again. The number ABC is a three-digit number, and the sum of three equal numbers of this type is equal to 3×ABC = 2023.
ABC = 674
So, the number that Antonio was looking for was 674, and the number that had to go in D is 7.
At this point, Javi realizes a mistake. “It’s not exactly like that,” he says, and adjusts the AI’s reasoning, “the calculation has to be like this,” and he notes down: 3 x ABC +1 = 2023.
We asked another question to the chat so that it could resolve the problem.
Javi: That’s not correct either. Actually, the calculation you have to do is 3×ABC + 1 = 2023.
Chat: I’m sorry I was wrong again. If the sum is 3×ABC + 1 = 2023, we can make the following calculation:
3×ABC + 1 = 2023
ABC = 674
So, the number that Antonio was looking for was 674, and the number that D represents is 8.
Javi: That’s not the case, I have told you that D is a unit greater than C, and that the following calculation must be fulfilled: ABC + ABC + ABD = 2023. What is the value of D?
Chat: I understand, sorry for the previous confusion. If ABC + ABC + ABD = 2023, and we know that D is one unit greater than C, we can do the following calculation:
ABC + ABC + AB(C + 1) = 2023
Where ABC is a three-digit number, and C and D are digits.
Therefore, the number that represents D is 8.
In the end, the AI ends up giving an answer that it had already given, but changing a 7 to an 8, without further explanation.
In neither of the two problems were the answers from the chat satisfactory. We can conclude that ChatGPT still doesn’t provide satisfactory answers in mathematics. In other interactions, we have been able to see that it makes mistakes in terms of specific content or that special care must be taken when formulating the introduced question. Nor is it able to argue clearly.
Experience with ChatGPT
The experience with ChatGPT is very valuable. It has allowed us to work on knowledge of addition and the positional decimal system, as well as socio-emotional processes and skills; all in a single conversation. Javi has been able to test the problem-solving capacity of the chat, which will allow him to approach new interactions with AI more wisely, aware that it can make mistakes. We are training his critical spirit. He assessed the validity of the answers obtained and generated new questions. He has seen different representations and has used oral and written language to solve problems. He was able to follow a reasoning that was not his own, and he detected an error and tried to correct it.
It is clear that AI remembers data, repeats, executes orders, but it is still far from solving unforeseen problems, creating ideas, connecting them and critical thinking, something that human intelligence is capable of.
And you? Have you interacted with AI and do you want to share your experience with us? Share your findings and let’s create a community.
