Espirales. Revista multidisciplinaria de investigación científica, Vol. 8, No. 48

January - March 2024. e-ISSN 2550-6862. pp 79-92

DOI https://doi.org/10.31876/er.v47i7.860

Enhancing Structural Engineering Education: Integrating

Artificial Intelligence for Continuous Improvement

Mejora Continua en la Educación de Ingeniería Estructural: Aplicación

Práctica del uso de Inteligencia Artificial

Diego Hernán Hidalgo Robalino*

Jessica Paulina Brito Noboa**

Nelson Estuardo Patiño Vaca***

Alexis Iván Andrade Valle****

Received: October 19, 2023

Approved: November 02, 2023

Abstract

The study aims to analyze the impact of artificial intelligence (AI)

usage on structural learning through student-developed

programming in open-source software languages: Python, Octave,

and OpenSees. The research collaborates with 90 undergraduate

students in the early courses of civil engineering at the Universidad

Nacional de Chimborazo. The ADDIE methodology is employed in

the initial phase for planning, development, and monitoring. A survey

on students' perceptions regarding effectiveness, satisfaction,

recommendation, and feedback is conducted, followed by academic

performance evaluation using a grading rubric to verify the

achievement of set objectives. An analysis of factors contributing to

AI-focused learning is then performed. Initial results revealed outliers,

some deviating from study parameters and others discarded for a

comprehensive view of study behavior. Regarding the survey data

analysis, efficiency and satisfaction exhibited the highest reliability.

Subsequently, variables were correlated considering their normality,

showing a relationship between effectiveness and satisfaction;

however, a strong connection cannot be guaranteed for these or

other variables. Therefore, ANOVA tests, indicating positive linear

relationships, and hypothesis testing were employed, demonstrating

that students achieved objectives with a moderately high degree of

effectiveness and satisfaction. The use of technological options and

consideration of innovative learning methods can positively enhance

the learning experience, contingent on prior education. Exploring

artificial intelligence may prove challenging without guided

information search based on predefined criteria and constraints.

Keywords:

Artificial intelligence, Artificial intelligence in education,

STEM education, General system theory, educational system

Madroñal-Ortiz, M., Cuartas-Ramírez, D.,

Escobar-Mora, N., Osorio, M. (2024).

Enhancing Structural Engineering Education:

Integrating Artificial Intelligence for

Continuous Improvement. Espirales Revista

Multidisciplinaria de investigación científica,

8 (48), 79-92

* Magister en Ingeniería Estructural, Docente investigador

Universidad Nacional de Chimborazo,

dhhidalgo@unach.edu.ec, https://orcid.org/0000-0003-1341-

8206

** Master of Science in Water Resources (Practical Research

Track), Docente investigador Universidad Nacional de

Chimborazo, jessica.brito@unach.edu.ec,

https://orcid.org/0000-0001-5550-5688

*** Máster Universitario en Hidrología y Gestión de Recursos

Hídricos. Magister en Administración Ambiental. Docente

investigador Universidad Nacional de Chimborazo,

npatino@unach.edu.ec, https://orcid.org/0009-0006-3492-

7092

**** Máster universitario en planificación y gestión en

ingeniería civil, Docente investigador Universidad Nacional de

Chimborazo alexis.andrade@unach.edu.ec,

https://orcid.org/0000-0003-1543-4381

Diego Hernán Hidalgo Robalino, Jessica Paulina Brito Noboa, Nelson Estuardo Patiño Vaca, Alexis Iván Andrade Valle

Espirales. Revista multidisciplinaria de investigación científica, Vol. 8, No. 48

January - March 2024. e-ISSN 2550-6862. pp 80-93

Introduction

In the realm of education within Science, Technology, Engineering, and Mathematics

(STEM) disciplines, artificial intelligence (AI) has demonstrated positive educational

effects, including enhancements in academic performance, stimulation of critical

thinking and problem-solving skills, and increased student interest and motivation. AI

enables personalized and adaptive learning environments, providing supportive tools

for autonomous learning. Technologically, AI has proven effective and precise in various

applications, such as predicting student performance, automated assessment of open-

ended responses, and improving predictive models. AI-based systems, including

educational robots and virtual agents, have been developed to interact with students

and offer personalized feedback. However, recent reviews Xu & Ouyang (2022)

underline the invaluable potential of AI in higher education and STEM, emphasizing the

need for continued research and advancements in this field.

In the context of online higher education, recent research Ouyang et al (2022) highlights

the pivotal role of artificial intelligence in offering various beneficial functions, including

Resumen

El estudio pretende analizar el impacto del uso de la inteligencia artificial

(IA) en el aprendizaje estructural a través de la programación desarrollada

por los estudiantes en lenguajes de software de código abierto: Python,

Octave y OpenSees. En la investigación colaboran 90 estudiantes de

pregrado de los primeros cursos de ingeniería civil de la Universidad

Nacional de Chimborazo. Se emplea la metodología ADDIE en la fase

inicial para la planificación, desarrollo y seguimiento. Se realiza una

encuesta sobre las percepciones de los estudiantes en cuanto a

efectividad, satisfacción, recomendación y retroalimentación, seguida de

una evaluación del desempeño académico utilizando una rúbrica de

calificación para verificar el logro de los objetivos planteados. A

continuación, se realiza un análisis de los factores que contribuyen al

aprendizaje centrado en la IA. Los resultados iniciales revelaron valores

atípicos, algunos desviados de los parámetros del estudio y otros

descartados para obtener una visión global del comportamiento del

estudio. En cuanto al análisis de los datos de la encuesta, la eficiencia y la

satisfacción mostraron la mayor fiabilidad. Posteriormente, se

correlacionaron las variables teniendo en cuenta su normalidad, mostrando

una relación entre la eficacia y la satisfacción; sin embargo, no se puede

garantizar una conexión fuerte para estas u otras variables. Por lo tanto, se

emplearon pruebas ANOVA, que indicaron relaciones lineales positivas, y

pruebas de hipótesis, demostrando que los estudiantes alcanzaron los

objetivos con un grado moderadamente alto de eficacia y satisfacción. El

uso de opciones tecnológicas y la consideración de métodos de

aprendizaje innovadores pueden mejorar positivamente la experiencia de

aprendizaje, en función de la formación previa. La exploración de la

inteligencia artificial puede resultar difícil sin una búsqueda guiada de

información basada en criterios y restricciones predefinidos.

Palabras clave:

Inteligencia artificial, Inteligencia artificial en la educación,

educación STEM, Teoría general de sistemas, sistema educativo

Enhancing Structural Engineering Education: Integrating Artificial Intelligence for Continuous Improvement

Espirales. Revista multidisciplinaria de investigación científica, Vol. 8, No. 48

January - March 2024. e-ISSN 2550-6862. pp 80-93

predicting learning progress, student performance, and satisfaction, resource

recommendations, automated assessment, and enhancing the overall learning

experience. These AI applications have gained widespread acceptance in educational

settings and demonstrated positive outcomes, such as accurate AI-backed predictions,

high-quality recommendations based on individual student characteristics, improved

academic performance, and increased online engagement. The research suggests three

key implications for future research: integrating educational theories into online learning

with AI, adopting advanced AI technologies for real-time data analysis, and conducting

empirical research to confirm the effects of AI applications in online higher education.

While previous studies Cosmes Aragón & Montoya Delgadillo (2021); Ghoniem &

Ghoniem (2022) have explored mathematical modeling in engineering education and

the need for technology-driven learning strategies in structural engineering, a gap

remains in understanding the impact of AI on structural engineering education. Despite

the proliferation of AI applications in Educational AI (AIED) seeking to enhance the

learning process, previous research Chichekian & Benteux (2022) has indicated a lack of

significant progress from a pedagogical and theoretical perspective in the AIED field.

This suggests an unawareness of previous review findings that may have impeded its

progress, emphasizing the necessity for a new review to address these limitations and

contribute to advancing learning theories and the effective implementation of AI

applications in the educational environment (Doblada & Caballes, 2021).

Therefore, promoting collaboration in researching and developing AI-based

educational technologies is crucial for improving engagement in educational settings.

This involves educators' involvement in decision-making and implementing AI

applications in classrooms, focusing on creating learning experiences that consider

student interaction with AI, including motivation and engagement. It is also essential to

clearly define the role of teachers in the classroom and consider their perspectives in

the development of AI-based educational technologies. Exploring the integration of

interdisciplinary perspectives for AI use in educational environments becomes essential,

emphasizing the importance of educators' involvement in assessing the pedagogical

and ethical implications of implementing AI applications in classrooms. This contributes

to the advancement of learning theories through a suitable and coherent conceptual

framework.

The overarching research question is then posed: What is the impact of using artificial

intelligence (AI) in structural learning through programming in open-source software

languages developed by early undergraduate civil engineering students, including

student perception, academic performance, and factors contributing to AI success in

the learning process?

Materials and methods

To conduct the study, the ADDIE methodology (Analysis, Design, Development,

Implementation, and Evaluation) is employed, proven to be effective in the teaching

and learning process (Almelhi, 2021). Within the evaluation phase, the Four Levels of

Learning Evaluation by Donald Kirkpatrick (Kirkpatrick & Kirkpatrick, 2005) are utilized.

Diego Hernán Hidalgo Robalino, Jessica Paulina Brito Noboa, Nelson Estuardo Patiño Vaca, Alexis Iván Andrade Valle

Espirales. Revista multidisciplinaria de investigación científica, Vol. 8, No. 48

January - March 2024. e-ISSN 2550-6862. pp 80-93

Results are derived from summative evaluation and a survey designed and administered

to students. The components of each phase are outlined in the flowchart of the initial

methodology in

¡Error! No se encuentra el origen de la referencia.

Figure 1.

ADDIE Methodology Applied to the Case Study

A correlational research design and a questionnaire serve as data collection tools.

Convenience sampling is applied, with the sample consisting of active students enrolled

in the courses of rational mechanics and strength of materials in the second and third

semesters, respectively, of the Civil Engineering program at the Universidad Nacional

de Chimborazo.

Students developed programs for structural analysis and design using three open-

source software: Python, Octave, and OpenSees, incorporating artificial intelligence

applications to achieve this objective. The category of Artificial Intelligence Tutoring

Systems (ITS) in the STEM education context is considered. ITS are AI-driven systems

providing personalized instruction and feedback, fostering an adaptive and

personalized learning approach. This includes various subcategories such as the delivery

of instructional content, recommendation of personalized learning paths, and

suggestion of learning resources. It's noteworthy that the group comprises students with

varying levels of programming knowledge.

Before the information gathering phase using the survey, assurance of information

confidentiality was provided to mitigate biases, particularly regarding expectations

satisfaction towards the instructor. The survey was conducted before the evaluation and

assignment of grades to explore relationships between satisfaction and effectiveness

irrespective of the obtained grade.

To analyze the impact of the research, summative evaluations indicating goal attainment

or non-attainment are employed. Additionally, to assess effectiveness and satisfaction,

a survey was designed, developed, and administered. A quantitative approach is

utilized for the questionnaire, comprising fifteen statements analyzing responses in

terms of effectiveness and satisfaction. The Likert scale is employed with options (1)

Enhancing Structural Engineering Education: Integrating Artificial Intelligence for Continuous Improvement

Espirales. Revista multidisciplinaria de investigación científica, Vol. 8, No. 48

January - March 2024. e-ISSN 2550-6862. pp 80-93

Strongly Agree = 5, (2) Agree = 4, (3) Disagree = 3, (4) Disagree = 2, and (5) Strongly

Disagree, adapted to each question.

The hypothesis verification is carried out through variable correlation. The obtained data

are first analyzed using various statistical tests: outlier test, Cronbach's reliability test,

normality test, conventionally true value, and correlation tests, following a methodology

similar to that proposed by Nurmayanti & Suryadi (2023).

Figure 2.

Statistical Data Analysis Methodology

Results

To verify the validity and reliability of the results, the statistical analysis specified in the

methodology was conducted. The student group corresponds to N = 90 before the

removal of outliers, and the analysis is performed with a 5% significance level.

Outlier Test Using Boxplots

The first step involved analyzing the data to identify outliers. 3 displays the results after

this procedure for the variables of Effectiveness, Satisfaction, Recommendation, and

student grade.

Diego Hernán Hidalgo Robalino, Jessica Paulina Brito Noboa, Nelson Estuardo Patiño Vaca, Alexis Iván Andrade Valle

Espirales. Revista multidisciplinaria de investigación científica, Vol. 8, No. 48

January - March 2024. e-ISSN 2550-6862. pp 80-93

Figure 3.

Boxplot depicting the knowledge level against the studied variables

Cronbach's Alpha Reliability Test

This test assesses the internal consistency or reliability of a set of data or questions on

a measurement scale. The resulting value, known as "Cronbach's alpha," indicates how

consistent and reliable the dataset is in terms of measurement. A higher Cronbach's

alpha value is generally interpreted as higher reliability in measurements. The

questionnaire is considered moderately reliable if the Cronbach's alpha value > 0.6. The

test is not dependent on whether the data follows a normal or non-normal distribution,

making it applicable to all data (Taber, 2018).

Figure 4

. Results of Cronbach's Alpha test

Normality Test

After removing outliers and conducting the Cronbach's test, the normality curves are

examined using the Anderson–Darling test. Results are shown in Figure 5.

Enhancing Structural Engineering Education: Integrating Artificial Intelligence for Continuous Improvement

Espirales. Revista multidisciplinaria de investigación científica, Vol. 8, No. 48

January - March 2024. e-ISSN 2550-6862. pp 80-93

Figure 5.

Anderson–Darling Normality Test

Correlation Test

Pearson correlation is performed for variables with a normal distribution, i.e.,

Satisfaction and Effectiveness. For the remaining data, Spearman correlation is applied

(Shestakov et al., 2022).

Figure 6: Pearson correlation between Effectiveness and Satisfaction and Spearman

correlation between all variables

Diego Hernán Hidalgo Robalino, Jessica Paulina Brito Noboa, Nelson Estuardo Patiño Vaca, Alexis Iván Andrade Valle

Espirales. Revista multidisciplinaria de investigación científica, Vol. 8, No. 48

January - March 2024. e-ISSN 2550-6862. pp 80-93

ANOVA Test

ANOVA assumes that populations from which samples are drawn should follow a normal

distribution. However, ANOVA is robust to moderate deviations from this assumption,

especially when the sample size is large. (Zhao et al., 2021) The ANOVA test is

conducted between Effectiveness and Satisfaction, and between variables, as depicted

in Figures 7 and 8.

Figure 7.

ANOVA test between Effectiveness and Satisfaction

Enhancing Structural Engineering Education: Integrating Artificial Intelligence for Continuous Improvement

Espirales. Revista multidisciplinaria de investigación científica, Vol. 8, No. 48

January - March 2024. e-ISSN 2550-6862. pp 80-93

Figure 8.

ANOVA test between variables

Conventional True Value

Considering that Effectiveness and Satisfaction have a normal distribution, a one-

sample t-test is conducted. The assumption is that the exercise is perceived as effective

and with a high degree of satisfaction. The graph indicates that the effectiveness is

perceived as moderate, and satisfaction ranges from moderate to high, as shown in

Figure 9.

The Recommendation and Grade do not follow a normal distribution, leading to the

application of the non-parametric Wilcoxon test (Fay & Proschan, 2010). The median,

with a 95% confidence level, is 4 for Recommendation and 4.75 for Grade.

Figure 9.

Hypothesis Test

Chi-Square Test for Association

Considering that Effectiveness and Satisfaction have a normal distribution, a chi-square

test for association is performed. Table 1 presents the relationship between

Effectiveness and the variables (Doblada & Caballes, 2021).

Diego Hernán Hidalgo Robalino, Jessica Paulina Brito Noboa, Nelson Estuardo Patiño Vaca, Alexis Iván Andrade Valle

Espirales. Revista multidisciplinaria de investigación científica, Vol. 8, No. 48

January - March 2024. e-ISSN 2550-6862. pp 80-93

Table 1.

Relationship between the Efectiveness and variables.

Effectiveness

Chi-

square

value

p-value

Decision on

Remarks

Knowledge

2.536

0.864

Failed to

reject

Not

significant

Satisfaction

2.873

0.238

Failed to

reject

Not

significant

Student

Grade

0.078

Failed to

reject

Not

significant

(Note: df = degrees of freedom)

Starting with the main finding, after the analysis, it was deemed necessary to exclude

outlier cases, including students who couldn't complete the activity due to reasons

beyond the study's scope. Exploratory graphs were employed to identify these and

other outliers to ensure they did not distort the results.

Upon obtaining data post-outlier removal, the Cronbach's alpha test was conducted.

The values obtained were 0.8691 for effectiveness and 0.7667 for satisfaction,

establishing the test's reliability for these parameters.

Normality tests were also performed on the variables. It was observed that effectiveness

and satisfaction follow a normal distribution, while student grades, recommendations,

and student grades do not. The asymmetry in the variable distributions suggests that

linear patterns might not be followed, prompting the consideration of non-parametric

statistical methods. Hence, in addition to the Pearson correlation test, the Spearman

correlation test is proposed.

In the correlation tests, the strongest correlation is found between effectiveness and

satisfaction. Relationships between the other variables range from moderate to low,

indicating a lack of strong associations.

The ANOVA test aids in better understanding relationships between variables. The

students' programming knowledge was related to effectiveness, satisfaction, student

grades, and their recommendation of this learning methodology. Linear ascending

relationships were noted, suggesting a positive linear relationship between variables. In

other words, as students' knowledge increases, their perception of effectiveness,

satisfaction, recommendation, and student grades also increases. This implies that

teaching positively influences the use of artificial intelligence.

Hypothesis tests reveal that the perceived effectiveness is above the mean value,

suggesting that students generally believe the objectives were achieved within the

required timeframe. Satisfaction is very close to the mean. Regarding recommendation,

Enhancing Structural Engineering Education: Integrating Artificial Intelligence for Continuous Improvement

Espirales. Revista multidisciplinaria de investigación científica, Vol. 8, No. 48

January - March 2024. e-ISSN 2550-6862. pp 80-93

results indicate that students generally recommend the use of the methodology, while

student grades, excluding outliers, show that students achieved the exercise objectives.

The association between responses in a chi-square test on students' knowledge and

exercise effectiveness (p-value of 0.864) suggests no significant evidence to reject the

null hypothesis that there is no association between these two variables. A similar

situation arises with student satisfaction and effectiveness (p-value of 0.238) and the

relationship between student grades and effectiveness (Pearson Chi-Square of 0.078).

During the study, it became apparent that the use of AI does not accurately reflect the

true learning state of students. Perhaps a shift towards an intelligent tutoring system

could yield better results (Francisco & Silva, 2022). It's also noteworthy that students

who have not yet received formal education in programming consider themselves to

have a higher level of knowledge than those who have completed that level. This

reflects a statistical phenomenon rather than a reflection of human nature (the Dunning-

Kruger effect) (Magnus & Peresetsky, 2022).

Conclusions

Despite the statistical data not revealing a strong relationship between variables, a

notable correlation is observed between students' effectiveness and satisfaction. This

suggests that as results are achieved within the expected timeframe, students feel their

expectations are fulfilled. Furthermore, it appears that students with less programming

knowledge show lower levels of satisfaction and effectiveness, while those with more

programming knowledge exhibit higher levels of satisfaction and effectiveness.

Regarding the use of open-source software packages, it is evident that individuals with

limited programming knowledge encounter difficulties with all packages. In contrast,

those with greater knowledge find Python easier, followed by Octave and, finally,

OpenSees. There are outliers in the moderately knowledgeable programming group—

some find Python very easy, while others find it challenging. The relationship between

effectiveness and knowledge seems proportional. The perception of the utility of open-

source software programs is similar for all three options, with those who believe they

have more knowledge finding them more useful than those who think they lack

programming knowledge.

The initial level of knowledge doesn't impact as much as the motivation of the students.

Qualitative analysis of questions suggests that while artificial intelligences are helpful

for problem-solving, they function to the extent that tools can be managed and are

accessible. Explanations by professionals seem to yield better results, emphasizing that,

for now, artificial intelligence cannot replace the role of teachers but is necessary for the

proper and complementary use of these tools.

The pursuit of knowledge is crucial as it allows exploration of fields that benefit effective

learning. As Alexander Pope stated, "A little learning is a dangerous thing."

Diego Hernán Hidalgo Robalino, Jessica Paulina Brito Noboa, Nelson Estuardo Patiño Vaca, Alexis Iván Andrade Valle

Espirales. Revista multidisciplinaria de investigación científica, Vol. 8, No. 48

January - March 2024. e-ISSN 2550-6862. pp 80-93

..........................................................................................................

References

Almelhi, A. M. (2021). Effectiveness of the

ADDIE Model within an E-Learning

Environment in Developing Creative

Writing in EFL Students. English

Language Teaching, 14(2), 20.

https://doi.org/10.5539/elt.v14n2p20

Chichekian, T., & Benteux, B. (2022). The

potential of learning with (and not from)

artificial intelligence in education.

Frontiers in Artificial Intelligence, 5.

https://doi.org/10.3389/frai.2022.90305

Cosmes Aragón, S. E., & Montoya Delgadillo, E.

(2021). Understanding Links Between

Mathematics and Engineering Through

Mathematical Modelling—The Case of

Training Civil Engineers in a Course of

Structural Analysis (pp. 527–537).

https://doi.org/10.1007/978-3-030-

66996-6_44

Doblada, J. C. L., & Caballes, D. G. (2021).

Relationship of Teachers’ Technology

Skills and Selected Profile: Basis for

Redesigning Training for Online

Distance Learning Modality. Instabright

International Journal of Multidisciplinary

Research, 3(1), 17–22.

https://doi.org/10.52877/instabright.00

3.01.0044

Fay, M. P., & Proschan, M. A. (2010). Wilcoxon-

Mann-Whitney or t-test? On assumptions

for hypothesis tests and multiple

interpretations of decision rules.

Statistics Surveys, 4(none).

https://doi.org/10.1214/09-SS051

Francisco, R., & Silva, F. (2022). Intelligent

Tutoring System for Computer Science

Education and the Use of Artificial

Intelligence: A Literature Review.

Proceedings of the 14th International

Enhancing Structural Engineering Education: Integrating Artificial Intelligence for Continuous Improvement

Espirales. Revista multidisciplinaria de investigación científica, Vol. 8, No. 48

January - March 2024. e-ISSN 2550-6862. pp 80-93

Conference on Computer Supported

Education, 338–345.

https://doi.org/10.5220/001108440000

3182

Ghoniem, A., & Ghoniem, E. (2022). Inducing

competence-based assignment in

traditional structural engineering

education: A case study. Computer

Applications in Engineering Education,

30(3), 907–916.

https://doi.org/10.1002/cae.22493

Kirkpatrick, Donald. L., & Kirkpatrick, J. D.

(2005). Transferring Learning to

Behavior: Using the Four Levels to

Improve Performance.

https://api.semanticscholar.org/CorpusI

D:107017451

Magnus, J. R., & Peresetsky, A. A. (2022). A

Statistical Explanation of the Dunning–

Kruger Effect. Frontiers in Psychology,

13.

https://doi.org/10.3389/fpsyg.2022.840

180

Nurmayanti, N., & Suryadi, S. (2023). The

Effectiveness Of Using Quillbot In

Improving Writing For Students Of

English Education Study Program. Jurnal

Teknologi Pendidikan : Jurnal Penelitian

Dan Pengembangan Pembelajaran, 8(1),

32.

https://doi.org/10.33394/jtp.v8i1.6392

Ouyang, F., Zheng, L., & Jiao, P. (2022). Artificial

intelligence in online higher education: A

systematic review of empirical research

from 2011 to 2020. Education and

Information Technologies, 27(6), 7893–

7925. https://doi.org/10.1007/s10639-

022-10925-9

Shestakov, V. N., Yakunin, Y. Yu., Lyksonova, D.

I., & Pogrebnikov, A. K. (2022).

Assessment of the relevance of the

student survey results in the educational

process. Perspectives of Science and

Diego Hernán Hidalgo Robalino, Jessica Paulina Brito Noboa, Nelson Estuardo Patiño Vaca, Alexis Iván Andrade Valle

Espirales. Revista multidisciplinaria de investigación científica, Vol. 8, No. 48

January - March 2024. e-ISSN 2550-6862. pp 80-93

Education, 56(2), 641–656.

https://doi.org/10.32744/pse.2022.2.38

Taber, K. S. (2018). The Use of Cronbach’s Alpha

When Developing and Reporting

Research Instruments in Science

Education. Research in Science

Education, 48(6), 1273–1296.

https://doi.org/10.1007/s11165-016-

9602-2

Xu, W., & Ouyang, F. (2022). The application of

AI technologies in STEM education: a

systematic review from 2011 to 2021.

International Journal of STEM Education,

9(1), 59.

https://doi.org/10.1186/s40594-022-

00377-5

Zhao, L., Liu, X., & Su, Y.-S. (2021). The

Differentiate Effect of Self-Efficacy,

Motivation, and Satisfaction on Pre-

Service Teacher Students’ Learning

Achievement in a Flipped Classroom: A

Case of a Modern Educational

Technology Course. Sustainability, 13(5),

2888.

https://doi.org/10.3390/su13052888