JRSSEM 2023, Vol. 02 No. 8, 1652 – 1660

E-ISSN: 2807 - 6311, P-ISSN: 2807 - 6494

DOI: 10.36418/jrssem.v2i08.403 https://jrssem.publikasiindonesia.id/index.php/jrssem

SCHOOL ENGAGEMENT MEASURE INSTRUMENT ADAPTATION

Masyitha Nur Ramadhani

Urip Purwono

Magister Profession Faculty of Psychology, Padjadjaran University, Indonesia

Faculty of Psychology, Padjadjaran University, Indonesia

e-mail: masyitha2100[email protected]pad.ac.id, urippurwono@gmail.com

*Correspondence: masyitha2100[email protected]npad.ac.id

Submitted

: 24

February 2023

Revised

: 09

Maret 2023

Accepted

: 13

Maret 2023

Abstract: School engagement is a multidimensional construct that refers to student involvement in

schools in three dimensions, namely involvement in behavioral, emotional and cognitive aspects.

School Engagement Measure is an instrument to measure student engagement in schools that has

been widely used in various countries. In order to use the instrument in a culturally different way,

an adaptation process is needed so that the adapted instrument is valid and reliable. The purpose

of this study was to obtain a standardized version of the School Engagement Measure in Indonesia.

The International Test Commission Guidelines (2017) were used in the adaptation process. Based

on the results of the CFA analysis using the JASP 0.15.0.0 program, it can be concluded that the

overall School Engagement Measure model with indicators consisting of three dimensions, namely

behavioral engagement, Emotional engagement and cognitive engagement, has a good fit model.

The null hypothesis means that the SEM model is acceptable. This explains that the resulting model

can describe the actual conditions.

Keywords: School Engagement Measure; adaptation.

Masyitha Nur Ramadhani

Urip Purwono

| 1653

INTRODUCTION

(Fredricks et al., 2004) shed light on the

forms of student involvement in schools. In

this regard, (Fredricks et al., 2004) state that

school engagement is a multidimensional

construct that refers to student

involvement in school in three dimensions,

namely involvement in behavioral,

emotional and cognitive aspects. Behavior

engagement includes positive behavior,

obeying the rules, not doing various

disruptive behaviors at school,

concentrating on following learning, doing

assignments, persevering, giving questions,

focusing on learning, actively participating

in discussions and providing active

participation in school activities. Emotional

engagement involves students' emotions

towards teachers, friends or academic and

school activities. Cognitive engagement

includes the willingness and attention

exerted by students as an effort to

understand learning material or a certain

skill.

All dimensions in School engagement

are believed to affect student achievement

in school. This is supported by various

research results. (Fredricks et al., 2004)

stated that students who were able to be

optimally involved in school showed

positive academic presentation results

while students who were disengaged in

school would actually experience more

dropouts. In addition, (Wonglorsaichon et

al., 2014) in their research also showed a

relationship between the variable School

engagement and student achievement. In

this case it was found that school

engagement of students had a significant

relationship with student learning

achievement. Not only that, (Arlinkasari &

Akmal, 2017) found that academic efficacy

is positively correlated with many studies

on School engagement conducted, ranging

from the relationship between School

engagement and Academic Self, Peer

support, academic achievement, and many

other studies. School engagement itself is a

multidimensional construct that refers to

student involvement in the school. In this

case, the involvement is reflected in three

dimensions, namely behavior, emotional

and cognitive. Mandernach (Christenson et

al., 2012) states that school engagement is

the result of intrinsic motivation or

individual needs that make students have

positive feelings, foster perseverance and

confidence in themselves. In addition,

Jimerson, Campos and Greif (Christenson et

al., 2012) also state that school

engagement includes diverse dimensions,

namely the behavioral, emotional and

cognitive dimensions. (Klimstra, 2013)

states that one of the factors influencing

the process of forming adolescent self-

identity is how much adolescents are able

to find components from various sources to

help shape their identity. Therefore, it is

important for adolescents as a student to

be involved in various activities at school as

a forum that can facilitate the exploration

process carried out. There are several

measuring tools used to measure School

engagement, especially in vulnerable

children to adolescents, namely the School

engagement Measure (SEM), Student

School engagement (SSES), Motivation and

Engagement Scale (MES), 4-H Study for

Positive Youth Development: School

engagement Scale, High School Survey of

Student Engagement (HSSE) and School

1654 | School Engagement Measure Instrument Adaptation

engagement Scale (SES). But these

measuring instruments are still rarely

translated into Indonesian. Because it is still

very rare for these measuring instruments

to be translated into Indonesia, so in

Indonesia itself there is still very little

research on School engagement. After

some consideration, the author determined

the topic of the measuring instrument to be

made was the adaptation of the School

engagement Measure (SEM). SEM is a

measuring tool developed by Phyllis

Blumenfeld and Jennifer Fredricks in 2004,

consisting of 19 items and measuring three

dimensions, namely behavioral

engagement, Emotional engagement and

cognitive engagement. All items are scored

using a five-point likert scale from never to

always. SEM also has good internal validity

(α=0.83). To date, SEM has been widely

used in various studies on careers abroad

as well as in Indonesia, however, there has

been no research on its validity test other

than that conducted by Phyllis Blumenfeld

and Jennifer Fredricks (2004). Based on this,

this study aims to adapt to SEM so that

valid and reliable SEM is obtained for use in

Indonesia.

MATERIALS AND METHODS

Research Procedure

The measuring tool used to measure

the level of School engagement of students

is the School engagement Measure (SEM)

scale by adapting the scale developed by

Fredricks, J.A, Blumenfeld, O., Friedel, J, and

Paris, A. (Fredricks et al., 2004), will measure

three dimensions, namely Behavior

engagement includes the presence of

positive behavior, obeying the rules, not

doing various disruptive behaviors in

school, Concentrate on following learning,

doing assignments, persevering, giving

questions, focusing on learning, actively

participating in discussions and providing

active participation in school activities.

Emotional engagement involves students'

emotions towards teachers, friends or

academic and school activities. Cognitive

engagement includes the willingness and

attention exerted by students as an effort

to understand learning material or a certain

skill.

Guidelines for translating and adapting

test kits are also issued by the International

Test Commission (Gregoire, 2018) which are

outlined in The ITC Guidelines for

Translating and Adapting Tests. The

following are the stages of adaptation

carried out by the researcher:

a. Pre-condition stage. At this stage, the

researcher communicated via email and

obtained permission from the second

researchers, Phyllis Blumenfeld and Jennifer

Fredricks, to adapt to the scale of career

commitments to Indonesian.

b. Test development stage. The second

stage carried out by the researcher is to

translate the scale of the School

Engagement Measure (SEM) into

Indonesian with the context of Indonesian

culture. Translation from English to

Indonesian is done by two translators. The

criteria for a translator is a person who has

adequate knowledge of (1) the language

involved, (2) culture, (3) test content, and (4)

test principles in general. From these

criteria, researchers describe the criteria for

translators to be Indonesians who have an

IELTS score of at least 7.00 or TOEFL IBT of

at least 80, have a background in

Masyitha Nur Ramadhani

Urip Purwono

| 1655

psychology, especially educational topics,

and have lived in an English-speaking

country for more than two years. The

process of translating is carried out

independently by each translator.

c. The third stage is to synthesize by

discussing the results of the translation of

the two translators facilitated by the

researcher. One whole translation is

obtained based on the agreement of two

translators. This stage received results in

the form of a draft translation of SEM

Indonesian.

d. The fourth stage is to review the

translation results. A review is carried out to

test the equality of translations by experts.

Experts in this stage are psychology

lecturers who have translation experience

and have lived in an English-speaking

country for at least two years. The expert

gives an assessment of the SEM draft,

whether the existing translation is

appropriate and has the same meaning as

the original language.

e. The fifth stage is the readability test. New

sentences in Indonesian consulted

linguists. After the statement item was

compiled into a scale, it was then tested on

10 ordinary people, namely State High

School Students and conducted cognitive

interviews. This is done to see whether the

instructions and content of the aitem can

be understood by the subject or not. At this

stage it produces a semi-final scale of SEM

Indonesian (attached).

f. The sixth stage carried out by the

researcher is to carry out the lay-out of the

instructions and aitems of the SEM

statement so that it is ready to be

presented to the research subject.

g. The next stage of the researcher tests the

suitability of the measurement model for

each dimension of the career commitment

scale version of Indonesian through a

process of confirmatory factor analysis

(CFA) and reliability analysis using the

classic test theory (CTT) method, with the

help of SPSS 24 software. to facilitate the

calculation of reliability by calculating

Cronbach Alpha, to see the consistency

between items in measuring the construct

same. This study involved 213 participants.

This scale consists of 19 items and

measures three dimensions, namely

behavioral engagement, emotional

engagement and cognitive engagement.

All items are scored using a five-point likert

scale from never to always.

RESULTS AND DISCUSSION

The results of the initial CFA analysis on

the School Engagement Measure variable

are presented in table 3. Table 3 shows that

in general the measurement model is still

not fit. The value of the fit p-value, GFI,

RMSEA, NFI, IFI, CFI, TLI, and AGFI still does

not match the established criteria. The SEM

model and loading factor of each aitem can

be seen in figure 1. Because the model is

not yet fit, the researcher made

modifications to the model to obtain a

better model. Model modification is done

by removing items that have a loading

factor below 0.5. From the initial model

analysis, it is known that there are three

items that have a loading factor below 0.5,

namely B2 (item 2), C1 (item 12), C7 (item

18) B1 (item 1), E1 (item6), E6 (item 11), C6

1656 | School Engagement Measure Instrument Adaptation

(item 17), E3 (item 8), and C8 (item 19).

Thus, the final model will not include all

three aitems. The final model accuracy

parameters after deleting nine items can be

seen in table 1.

Table 1. Model accuracy parameters before modification

Parameter fit

Output

Criterion

Information

Chi square P-Value

0.025

≤ 0.05

Tidak Fit

Comparative Fit Index

(CFI)

0.818

≤ 0.90

Tidak Fit

Tucker-Lewis Index

(TLI)

0.791

≤ 0.9

Tidak Fit

Bentler-Bonett

Normed Fit Index (NFI)

0.730

≤ 0.9

Tidak Fit

Root mean square

error of approximation

(RMSEA)

0.082

≤ 0.08

Tidak Fit

Goodness of fit index

(GFI)

0.837

≤ 0.9

Tidak Fit

Table 1. shows the final model after

modification of the model. In the final

model, it can be seen that in Chi-square the

pvalue value is already above 0.05. This

shows that the model has been fit, where

there is no significant difference between

the ideal model and the proposed model

based on observational data. The GFI, NFI,

CFI, TLI, and RMSEA values also match the

criteria set for obtaining a fit model. Thus

this final model is already fit, which means

that the proposed model is fit with

empirical data. The complete model and

loading factor of each item on the final

model can be seen in figure 1.

Figure 1. Initial measurement model School

Engagement Measure

Variable

Masyitha Nur Ramadhani

Urip Purwono

| 1657

Table 2. Model se accuracy parametershave been modified

Parameter fit

Output

Criterion

Information

Chi square P-Value

0.025

≤ 0.05

Tidak Fit

Comparative Fit Index

(CFI)

0.949

≥ 0.90

Fit

Tucker-Lewis Index

(TLI)

0.908

≥ 0.9

Fit

Bentler-Bonett

Normed Fit Index (NFI)

0.912

≥ 0.9

Fit

Root mean square

error of approximation

(RMSEA)

0.074

≥ 0.08

Fit

Goodness of fit index

(GFI)

0.955

≥ 0.9

Fit

After that, a reliability test is carried

out. The reliability method used to

measure this student engagement measure

is classic test theory (CTT). The author uses

SPSS 24.0 software to facilitate reliability

calculations by calculating cronbach alpha,

to see the consistency between items in

measuring the same construct. The criteria

used to determine the level of reliability of

the measuring instrument in this study are

based on the criteria from (Kaplan &

Saccuzzo, 2017), namely α ≥ 0.70 means

that the measuring instrument is reliable,

and if α < 0.70 indicates that the measuring

instrument is not reliable. Based on table

5 it is known that all dimensions in SEM

instruments have excellent reliability can be

seen in table 6

Figure 2. Final Variable

School Engagement Measure

measurement

model

1658 | School Engagement Measure Instrument Adaptation

Table 4. Subscale Reliability At Every SEM Dimension

Based on the table above, it is known that

two of the three dimensions have reliable

reliability, it can be seen from each

dimension that they have a reliability value

in the >0.70 range, except behavioral

engagement dimenation which has a range

of < 0.70, but is still in the moderate

category or is still categorized as reliable

(Surjaweni, 2014). A discriminant test has

been carried out, a item nalisis performed

by the discriminant item method is an

analysis that reveals how much the test

item can distinguish between subjects who

fall into the high group and subjects who

fall into the low group. The discriminantt

item of the measuring instrument is derived

from the item-correlated value through

calculating the reliability of the test tool

using the help of SPSS 24.0 for Macintosh

software. According to (Azwar, 2016) as a

criteria for selecting items based on total

item correlation. Usually used rix limit ≤

0.30. Therefore, all items that reach a

correlation coefficient of at least 0.30, the

difference is considered satisfactory. From

the results of the analysis using SPSS on the

student engagement measure scale, results

were obtained as in table 5.

Table. 3 RallyA Billiarity Scale On The School Engagement Measure Questionnaire

Scale

Number of Items

Cronbach’s Alpha