EXAMINING THE RELATIONSHIP BETWEEN STUDENTS’ CREATION OF SPECIFIC DIGITAL PRODUCTS AND THEIR TECHNOLOGY ASSESSMENT SCORES

Mingyuan Zhang; Tiffany Sutter

doi:10.29121/granthaalayah.v8.i9.2020.1342

Authors

Mingyuan Zhang Central Michigan University Tiffany Sutter
Tiffany Sutter Central Michigan University, USA https://orcid.org/0000-0003-4761-6307

DOI:

https://doi.org/10.29121/granthaalayah.v8.i9.2020.1342

Keywords:

technology, digital media, Technology & Engineering Literacy (TEL), NAEP, student computer use

Abstract [English]

This study presented a secondary analysis of the National Assessment of Educational Progress (NAEP) dataset. The paper examined the impact of eighth grade students’ specific digital product creation on their Technology & Engineering Literacy ICT scores. In order to gain a better understanding of the impact of computer use technology achievement of eighth-grade students, this study used a quantitative descriptive research design to analyze secondary data extracted from the 2018 NAEP data set. The findings include (1) using computers to create, edit, or organize digital media both for school work and activities beyond school increased overall ICT scores. (2) The average score of students who used computers to create presentations in school increased while those who did this activity after school time saw a decrease in scores. (3) Students who reported that they create spreadsheets mostly saw a decrease in their average ICT score no matter the frequency or purpose for the activity. These findings may indicate that there is an ideal frequency for digital product creation in school, but that these types of activities may not be indicative of real-world use which is how students are assessed.

Downloads

Download data is not yet available.

References

Albion, P. R., & Ertmer, P. A. (2002). Beyond the foundations: The role of vision and belief in teachers' preparation for integration of technology. TechTrends, 46(5), 34-38. DOI: https://doi.org/10.1007/BF02818306

Badia, A., Meneses, J., Sigalés, C., & Fàbregues, S. (2014). Factors affecting school teachers’ perceptions of the instructional benefits of digital technology. Procedia - Social and Behavioral Sciences, 141(C), 357-362. DOI: https://doi.org/10.1016/j.sbspro.2014.05.063

Bandekar, B., Yitbarek, E. M., Lopez, K., Tran, C., & Momoh, E. (2018). Social Media: Effect, Affect, Teens, and Possible Addiction. Collin College Undergraduate Interdisciplinary Student Research Conference. 2. https://digitalcommons.collin.edu/ccuisrc/2018/thursday/2

Barron, A. E., Kemker, K., Harmes, C., & Kalaydjian, K. (2003). Large-scale research study on technology in K–12 schools. Journal of Research on Technology in Education, 35(4), 489-507. https://doi.org/10.1080/15391523.2003.10782398 DOI: https://doi.org/10.1080/15391523.2003.10782398

Beland, L. P., & Murphy, R. (2016). Ill communication: Technology, distraction & student performance. Labour Economics, 41, 61-76. DOI: https://doi.org/10.1016/j.labeco.2016.04.004

Biagi, F., & Loi, M. (2013). Measuring ICT use and learning outcomes: Evidence from recent econometric studies. European Journal of Education, 48(1), 28-42. DOI: https://doi.org/10.1111/ejed.12016

Bond, J., & Zhang, M. (2017). The Impact of Conversations on Fourth Grade Reading Performance - What NAEP Data Explorer Tells?. European Journal of Educational Research, 6(4), 407-417. doi:10.12973/eu-jer.6.4.407 DOI: https://doi.org/10.12973/eu-jer.6.4.407

Carter, S. P., Greenberg, K., & Walker, M. S. (2017). The impact of computer usage on academic performance: Evidence from a randomized trial at the United States Military Academy. Economics of Education Review, 56, 118-132. DOI: https://doi.org/10.1016/j.econedurev.2016.12.005

Chen, C. (2008). Why do teachers not practice what they believe regarding technology integration? Journal of Educational Research, 102(1), 65-75. DOI: https://doi.org/10.3200/JOER.102.1.65-75

Claro, M., Preiss, D. D., San Martín, E., Jara, I., Hinostroza, J. E., Valenzuela, S., Cortes, F., & Nussbaum, M. (2012). Assessment of 21st century ICT skills in Chile: Test design and results from high school level students. Computers & Education, 59(3), 1042-1053. DOI: https://doi.org/10.1016/j.compedu.2012.04.004

Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155-159. DOI: https://doi.org/10.1037/0033-2909.112.1.155

Davis, J. H. (2015, December 10). President Obama signs into law a rewrite of No Child Left Behind. The New York Times. Retrieved January 28, 2019, from https://www.nytimes.com/ 2015/12/11/us/politics/president-obama-signs-into-law-a-rewrite-of-no-child-left-behind.html.

DeBell, M., & Chapman, C. (2006). Computer and Internet Use by Students in 2003. Statistical Analysis Report. NCES 2006-065. National Center for Education Statistics.

Erstad, O. (2015). Educating the digital generation: Exploring media literacy for the 21st century. Nordic Journal of Digital Literacy, 10, 85-102.

Ertmer, P. (2005). Teacher pedagogical beliefs: The final frontier in our quest for technology integration? Educational Technology Research and Development, 53(4), 25-39. DOI: https://doi.org/10.1007/BF02504683

Eshet-Alkalai, Y. (2004). Digital literacy: A conceptual framework for survival skills in the digital era. Journal of Educational Multimedia and Hypermedia, 13(1), 93-106.

Eshet-Alkalai, Y. (2012). Thinking in the digital era: A revised model for digital literacy. Issues in Informing Science and Information Technology, 9, 267–276. DOI: https://doi.org/10.28945/1621

Ezquerra, A., Manso, J., Burgos, M., & Hallabrin, C. (2014). Creation of audiovisual presentations as a tool to develop key competencies in secondary-school students: A case study in science class. International Journal of Education and Development Using Information and Communication Technology, 10(4), 155-170.

Firman, H., Rustaman, N. Y., & Suwarma, I. R. (2015, November). The development of technology and engineering literacy through STEM-based education. In 2015 International Conference on Innovation in Engineering and Vocational Education. Atlantis Press. DOI: https://doi.org/10.2991/icieve-15.2016.45

Fraillon, J., Ainley, J., Schulz, W., Friedman, T., & Gebhardt, E. (2014). Preparing for life in a digital age: The IEA international computer and information literacy study international report. Springer. DOI: https://doi.org/10.1007/978-3-319-14222-7

Fraillon, J., Schulz, W., & Ainley, J. (2013). International computer and information literacy study: Assessment framework. Springer.

Hatlevik, O. E., Guðmundsdóttir, G. B., & Loi, M. (2015). Digital diversity among upper secondary students: A multilevel analysis of the relationship between cultural capital, self-efficacy, strategic use of information and digital competence. Computers & Education, 81, 345-353. DOI: https://doi.org/10.1016/j.compedu.2014.10.019

Hatlevik, O. E., Throndsen, I., Loi, M., & Guðmundsdóttir, G. B. (2018). Students’ ICT self-efficacy and computer and information literacy: Determinants and relationships. Computers & Education, 118, 107-119. DOI: https://doi.org/10.1016/j.compedu.2017.11.011

Helsper, E., & Eynon, R. (2010). Digital natives: Where is the evidence? British Educational Research Journal, 36(3), 503-520. DOI: https://doi.org/10.1080/01411920902989227

Hohlfeld, T., Ritzhaupt, A., & Barron, A. (2010). Development and validation of the student tool for technology literacy (ST2L). Journal of Research on Technology in Education, 42(4), 361-389. DOI: https://doi.org/10.1080/15391523.2010.10782556

Kaminski, K., Seel, P., & Cullen, K. (2003). Technology literate students. Educause Quarterly, 3, 34-40.

Kirschner, P., & De Bruyckere, P. (2017). The myths of the digital native and the multitasker. Teaching and Teacher Education, 67(C), 135-142. DOI: https://doi.org/10.1016/j.tate.2017.06.001

Krupczak, J., & Disney, K. A. (2013). Technological literacy: assessment and measurement of learning gains. In Makalah disajikan dalam at the 120th American Society for Engineering Education Annual Conference & Exposition.

Layton, L. (2015, December 10). Obama signs new K-12 education law that ends No Child Left Behind. The Washington Post. Retrieved January 28, 2019, from https:// www.washingtonpost.com/local/education/obama-signs-new-k-12-education-law-that-ends-no-child-left-behind/2015/12/10/c9e58d7c-9f51-11e5-a3c5-c77f2cc5a43c_story.html?noredirect=on&utm_term=.9c2e0eb231c0.

Lei, J., & Zhao, Y. (2007). Technology uses and student achievement: A longitudinal study. Computers & Education, 49(2), 284-296. DOI: https://doi.org/10.1016/j.compedu.2005.06.013

Lenhart, A. (2015, April 9). Teens, social media & technology overview 2015. Pew Research Center. https://www.pewresearch.org/internet/2015/04/09/teens-social-media-technology-2015/

Li, Y., Garza, V., Keicher, A., & Popov, V. (2019). Predicting high school teacher use of technology: Pedagogical beliefs, technological beliefs and attitudes, and teacher training. Technology, Knowledge and Learning, 24(3), 501-518. DOI: https://doi.org/10.1007/s10758-018-9355-2

Lu, J., Hao, Q., & Jing, M. (2016). Consuming, sharing, and creating content: How young students use new social media in and outside school. Computers in Human Behavior, 64(C), 55-64. DOI: https://doi.org/10.1016/j.chb.2016.06.019

McEuen, S. (2001). How fluent with information technology (FIT) are our students? Educause Quarterly, 24(4), 8-17.

The Nation’s Report Card. (2018a). About the TEL assessment: Assessment framework and design. Retrieved February 18, 2020, from https://www.nationsreportcard.gov/tel/about/ assessment-framework-design/

The Nation’s Report Card. (2018b). About the TEL assessment: Sample scenario-based tasks and discrete questions. Retrieved March 20, 2020, from https://www.nationsreportcard.gov/tel/tasks/

The Nation’s Report Card. (2018c). About the TEL assessment: Samples, inclusion, and participation. Retrieved February 18, 2020, from https://www.nationsreportcard.gov/tel/about/ samples-inclusion-participation/

National Assessment of Educational Progress. (2014). 2014 Technology & Engineering Literacy (TEL). The Nation’s Report Card. https://www.nationsreportcard.gov/tel_2014/#

National Assessment of Educational Progress. (2018). NAEP Technology & Engineering Literacy (TEL) Report Card. The Nation’s Report Card. https:// www.nationsreportcard.gov/tel/

National Center for Education Statistics. (2019). Focus on NAEP: Sampling. The Nation’s Report Card. Retrieved February 18, 2020, from https:// www.nationsreportcard.gov/focus_on_naep/files/ sampling_infographic.pdf

National Center for Educational Statistics. (2018, May 24). About NAEP. Retrieved February 18, 2020, from https://nces.ed.gov/nationsreportcard/about/

National Research Council. (1999). Being fluent with information technology. Washington, DC: The National Academies Press. https://doi.org/10.17226/6482 DOI: https://doi.org/10.17226/6482

Niederhauser, D., Lindstrom, D., & Strobel, J. (2007). Evidence of the NETS*S in K-12 classrooms: Implications for teacher education. Journal of Technology and Teacher Education, 15(4), 483-512.

No Child Left Behind Act of 2001, PL 107-110, 107th Cong., § 2401-2404, 115 Stat. 1425 (2002)

Northrup, D. A. (Fall 1996). The problem of the self-report in survey research. Institute for Social Research, 11(3). Retrieved from http://www.math.yorku.ca/ISR/self.htm

O’Dwyer, L. M., Russell, M., Bebell, D., & Tucker-Seeley, K. R. (2005). Examining the relationship between home and school computer use and students’ English/language arts test scores. Journal of Technology, Learning, and Assessment, 3(3).

Park, B. J., Broer, M., & Bohrnstedt, G. W. (2017, April 28). The relationship between students' contextual factors related to technology and Technology and Engineering Literacy performance. Paper presented at the 2017 annual meeting of the American Educational Research Association. Retrieved March 2, 2020, from the AERA Online Paper Repository.

Prensky, M. (2001). Digital natives, digital immigrants part 1. On the Horizon, 9(5), 1-6. https:// doi.org/10.1108/10748120110424816 DOI: https://doi.org/10.1108/10748120110424816

Ritzhaupt, A., Dawson, K., & Cavanaugh, C. (2012). An investigation of factors influencing student use of technology in K-12 classrooms Using Path Analysis. Journal of Educational Computing Research, 46(3), 229-254. DOI: https://doi.org/10.2190/EC.46.3.b

Schmid, R., & Petko, D. (2019). Does the use of educational technology in personalized learning environments correlate with self-reported digital skills and beliefs of secondary-school students? Computers & Education, 136, 75-86. DOI: https://doi.org/10.1016/j.compedu.2019.03.006

Sert, H., Taskin Yilmaz, F., Karakoc Kumsar, A., & Aygin, D. (2019). Effect of technology addiction on academic success and fatigue among Turkish university students. Fatigue: Biomedicine, Health & Behavior, 7(1), 41-51. DOI: https://doi.org/10.1080/21641846.2019.1585598

Spiezia, V. (2011). Does computer use increase educational achievements? Student-level evidence from PISA. OECD Journal: Economic Studies, 2010(1), 1-22. DOI: https://doi.org/10.1787/eco_studies-2010-5km33scwlvkf

Taimalu, M., & Luik, P. (2019). The impact of beliefs and knowledge on the integration of technology among teacher educators: A path analysis. Teaching and Teacher Education, 79, 101-110. DOI: https://doi.org/10.1016/j.tate.2018.12.012

Tort, F., Blondel, F. M., & Bruillard, É. (2008, July). Spreadsheet knowledge and skills of French secondary school students. In International Conference on Informatics in Secondary Schools-Evolution and Perspectives (pp. 305-316). Springer, Berlin, Heidelberg. DOI: https://doi.org/10.1007/978-3-540-69924-8_28

U.S. Department of Education, Office of Planning, Evaluation and Policy Development. (2009). Evaluation of the National Assessment of Educational Progress, final report. https://www2.ed.gov/about/offices/list/opepd/ppss/reports.html

Uzun, A. M., & Kilis, S. (2019). Does persistent involvement in media and technology lead to lower academic performance? Evaluating media and technology use in relation to multitasking, self-regulation and academic performance. Computers in Human Behavior, 90, 196-203. DOI: https://doi.org/10.1016/j.chb.2018.08.045

Wang, H., Zhou, X., Lu, C., Wu, J., Deng, X., & Hong, L. (2011). Problematic Internet use in high school students in Guangdong Province, China. PloS one, 6(5). DOI: https://doi.org/10.1371/journal.pone.0019660

Wang, K., Wang, X., & Li, J. (2018, April 16) Relationship between students' experiences and Technology and Engineering Literacy performance. Paper presented at the 2018 annual meeting of the American Educational Research Association. Retrieved March 2, 2020, from the AERA Online Paper Repository.

Wang, S., Hsu, H., Campbell, T., Coster, D., & Longhurst, C. (2014). An investigation of middle school science teachers and students use of technology inside and outside of classrooms: Considering whether digital natives are more technology savvy than their teachers. Educational Technology Research and Development, 62(6), 637-662. DOI: https://doi.org/10.1007/s11423-014-9355-4

Wastiau, P., Blamire, R., Kearney, C., Quittre, V., Van de Gaer, E., & Monseur, C. (2013). The Use of ICT in Education: a survey of schools in Europe. European Journal of Education, 48(1), 11-27. DOI: https://doi.org/10.1111/ejed.12020

Wilson, M. L., & Zietz, J. (2004). Systematic bias in student self-reported data. Journal for Economic Educators, 4(4), 13-19.

Zhang, T., Xie, Q., Park, B. J., Kim, Y. Y., Broer, M., & Bohrnstedt, G. (2016). Computer familiarity and its relationship to performance in three NAEP digital-based assessments. Washington, DC: American Institutes for Research.