本研究旨在探討MagicSchool AI數位對話工具應用於國小學生數學素養題學習活動後，學生數學學習成效，學習成效與年級、性別是否有差異之初探性研究。本研究於暑期課後照顧班中進行，以國小四、五年級學生共13名為研究對象，其中1名學生因於實驗期間請假未完成完整三週的實驗活動，及另一名學生未正確使用MagicSchool AI數位對話工具進行數學素養題練習，故將兩名學生於學習成效統計分析中予以排除，最終納入11名學生進行前測、後測與延宕後測之學習成效量化分析。 本研究採單組前測、後測與延宕後測之前實驗研究設計，介入活動為期三週，共進行五節數學素養題練習課程。研究工具包括MagicSchool AI數位對話工具、數學素養題題庫與測驗卷。使用數學素養題題目與答案為題庫，透過提示詞設定，於 MagicSchool AI平台設置一個數學AI小老師，使其與學生互動，過程中以提問、提示、步驟拆解、不直接給予答案的方式提供鷹架支持協助學生理解與學習。數學素養題庫與測驗題取自國家教育研究院素養導向試題，內容涵蓋生活情境判讀、數量關係理解與多步驟推理。 數學素養題測驗卷評分方式採用人工評分與AI評分，並通過ICC評分一致性驗證，最終分數取兩者平均值進行後續分析。 資料分析方面，本研究以年級與性別做為群組，使用Friedman 檢定與 Conover 事後檢定的量化分析方式分析學生於前測、後測與延宕後測之學習成效差異。另外挑選四年級、五年級學習成效進步與持平共4位學生，四年級、五年級學習保留效果不佳共2位學生，針對測驗卷與練習題的數學概念進行關聯分析，以理解學生的學習歷程，其中 1 位五年級學生同時納入兩種分析，所以共5位學生進行學習歷程分析。 研究結果顯示，學生於數學AI小老師介入活動後，全體學生組、四年級組與男性組，後測與前測分數的差異達到統計上的顯著，顯示介入活動期間觀察到立即學習表現的提升伴隨現象。五年級組與女性組前後測分數差異則呈現大效果量但未達統計顯著，可能受限於樣本數過小。後測與延宕後測分數變化未達顯著差異，顯示學習成效具有一定的保留效力。;This exploratory study examined the effects of using the MagicSchool AI digital conversational tool to support elementary school students in learning mathematical literacy problems. It also explored whether students’ learning outcomes differed by grade level and gender. The study was conducted in a summer after-school program with 13 fourth- and fifth-grade students. Two students were excluded from the learning effectiveness analysis: one did not complete the full three-week intervention due to absence， and the other did not use the MagicSchool AI tool properly during the practice activities. As a result， 11 students were included in the final quantitative analysis of pretest， posttest， and delayed posttest results. A one-group pretest–posttest–delayed posttest pre-experimental design was adopted. The intervention lasted for three weeks and included five sessions of mathematical literacy practice. The research instruments included the MagicSchool AI digital conversational tool， a mathematical literacy problem bank， and test papers. Based on selected mathematical literacy problems and their solutions， a “Math AI Tutor” was created on the MagicSchool AI platform through prompt design. The tutor interacted with students by asking guiding questions， giving hints， breaking complex problems into smaller steps， and avoiding direct answers. In this way， it provided scaffolded support to help students understand and solve mathematical literacy problems. The problem bank and test items were selected from competency-oriented assessment items developed by the National Academy for Educational Research， covering real-life context interpretation， quantitative reasoning， and multi-step problem solving. The mathematical literacy tests were scored using both human scoring and AI scoring. The consistency between the two scoring methods was examined using the intraclass correlation coefficient， ICC. The final test scores were calculated by averaging the human and AI scores for subsequent analysis. For data analysis， students were grouped by grade level and gender. The Friedman test and Conover post hoc test were used to examine differences in learning outcomes across the pretest， posttest， and delayed posttest. In addition， students were selected for further learning process analysis based on their learning performance and retention outcomes. Specifically， four fourth- and fifth-grade students whose performance either improved or remained stable， and two fourth- and fifth-grade students whose retention was relatively weak， were selected for analysis. Since one fifth-grade student was included in both categories， a total of five students were ultimately included in the learning process analysis. Their test responses and practice records were analyzed to identify connections among mathematical concepts and to better understand their learning processes. The results showed that， after the intervention with the Math AI Tutor， the overall group， the fourth-grade group， and the male group showed statistically significant improvement from pretest to posttest. This suggests that students demonstrated immediate gains in mathematical learning during the intervention. For the fifth-grade group and the female group， the pretest–posttest differences showed large effect sizes but did not reach statistical significance， possibly due to the small sample size. No significant differences were found between the posttest and delayed posttest scores， indicating that the learning gains were retained to a certain extent.
