Students who do mathematics with Innovamat show up to 11% better performance on standardized mathematical-competency tests. This is demonstrated by the EQUAL study (Longitudinal Evaluation Notebook Study) promoted by Vertex, our mathematics education research group.
From the very beginning, we have developed our entire curriculum based on research from prestigious educational institutions such as Utrecht University, Cambridge University, OECD, and the National Council of Teachers of Mathematics in the United States (NCTM). In addition, we have a team of researchers to assess our educational project. The validation of educational curriculums through standardized tests is essential to ensure that the ways of teaching and learning are effective and beneficial for students.
One way of assessing educational curriculums is through standardized tests. For this reason, at Innovamat, we wanted to analyze the results obtained by sixth grade students in the basic competency tests, tests designed by the Department of Education of the Generalitat de Catalunya. These are standardized tests which serve as an indicator of the level of acquisition of competencies students complete elementary education with.
The results of the study, which you can consult in more detail below, show that students who take Innovamat show a performance between 6 and 11% higher in all the content strands evaluated in the tests: “Numbers and Operations”, “Space, Measurement and Graphical Representation of Data”, and “Change and Relationships”. This demonstrates that our curriculum, based on a competency-based approach to mathematics, in a practical and participatory way, is effective in terms of improving mathematics performance, without sacrificing any content.
EQUAL (Longitudinal Evaluation Notebook Study)
The objective of the EQUAL Study is to analyze the results of the basic competencies tests in sixth grade, designed by the Department of Education of the Generalitat de Catalunya to determine variables that influence competency assessment and content acquisition.
To conduct the study, our research team compared the difference in performance between students who do math with Innovamat and those who do not. The results of approximately 1,000 students from 20 schools were analyzed and the sample was divided into two groups: 10 schools that use Innovamat in the sixth grade and 10 that do not.
In the following graph, and in the accompanying table, we can see that children who use Innovamat perform between 6% and 11% better in all the content strands of the basic competency tests. These differences are significant in the case of Numbers and Operations, and with moderate magnitudes of the differences (measures of effect). Let’s take a look at the data and discuss them below.
Technical analysis of data
We performed a bilateral Mann-Whitney U hypothesis test, with type I error at 5%. The p-values in the table indicate the probability that the observed difference, according to our data, is caused by chance. Since this value is less than 0.05 (type I error) in Numbers and Operations, we can accept the hypothesis that the observed differences are not the result of chance. In addition, we also see a strong trend in the strands of Space, Measurement and Representation, and Change and Relationships. The measure of effect measures the magnitude of the difference and has been calculated using Cohen's d: a value greater than 0.5 is considered to correspond to a moderate measure of effect. In summary, with the p-value, we see whether the differences may be due to chance and, with the measures of effect, how large these differences are.
In summary, with the p-value we see whether the differences can be due to chance (p-value > 0.05) and, with the effect sizes (if Cohen’s d is between 0.2 and 0.5, it is a small effect size; if Cohen’s d is between 0.5 and 0.8, it is a moderate effect size, and if Cohen’s d is greater than 0.8, it is a large effect size), how large these differences are. So, in our case, we can state that they are not the product of chance, and that the differences in results are relevant.
Finally, it should be noted that the choice of the sample was not random. All schools were given the opportunity to participate in the study, and 20 schools (10 that use Innovamat in 6th grade and 10 that do not) wanted to participate. That is why there may also be uncontrolled factors that have influenced the results, such as, for example, teacher’s individual motivation.
The results of these standardized tests are divided into three content strands: Numbers and Operations, Space, Measurement and Graphical Representation of Data, and Change and Relationships. As a score, we have used the average number of students’ correct answers in each item of the test, and we have weighted it according to the school’s official demographic. To make this correction, we have used the results provided by the Department of Education of the average number of correct answers for each demographic and for Catalonia as a whole.
The study shows that students using the Innovamat curriculum obtain better results in mathematics when measured with this standardized test in Catalonia. Although the cross-sectional design of the study does not allow us to attribute these differences to the use of Innovamat, they do suggest that working with mathematics in a competency-based way contributes to the construction of knowledge.
Research at Innovamat
Since its inception, Innovamat has been based on academic research on mathematics education. To develop our curriculum, we have drawn from prestigious sources such as the Freudenthal Institute of the University of Utrecht, the NRICH Project of the University of Cambridge, the OECD’s PISA Framework, and the National Council of Teachers of Mathematics (NCTM) of the United States.
In addition, we have an internal team of researchers made up of experts and advisors who are in charge of carrying out analyses and studies to evaluate the effectiveness of our educational program and the resources we offer. But that’s not all, we also have Vèrtex schools, educational schools with high scientific standards that, regardless of whether they use Innovamat or not, participate in projects and studies promoted by the Pirámide Group; and they have access to the results and personalized conclusions about their students. The goal is to share findings through scientific publications and to continue learning about mathematics education and learning.
