EXAMINING THE RELATIONSHIP BETWEEN STUDENTS’ CREATION OF SPECIFIC DIGITAL PRODUCTS AND THEIR TECHNOLOGY ASSESSMENT SCORES

This study presented a secondary analysis of the National Assessment of Educational Progress (NAEP) dataset. The paper examined the impact of eighth grade students’ specific digital product creation on their Technology & Engineering Literacy ICT scores. In order to gain a better understanding of the impact of computer use technology achievement of eighth-grade students, this study used a quantitative descriptive research design to analyze secondary data extracted from the 2018 NAEP data set. The findings include (1) using computers to create, edit, or organize digital media both for school work and activities beyond school increased overall ICT scores. (2) The average score of students who used computers to create presentations in school increased while those who did this activity after school time saw a decrease in scores. (3) Students who reported that they create spreadsheets mostly saw a decrease in their average ICT score no matter the frequency or purpose for the activity. These findings may indicate that there is an ideal frequency for digital product creation in school, but that these types of activities may not be indicative of real-world use which is how students are assessed.


INTRODUCTION
With the advent of the term digital native by Prensky (2001), many are led to believe that students today are actively using technology in a meaningful way simply because of the era into which they were born. While generation is certainly a factor in a user's comfort with technology, it is not the only factor that decides if someone should be considered a digital native nor should age automatically mean proficiency in technology (Helsper & Eynon, 2010). According to Kirschner and De Bruyckere (2017), understanding that digital natives are not necessarily sitting in our classrooms will help teachers "avoid the pitfall of assuming that their students possess talents and abilities that they do not actually have. The skills and competences attributed to this generation of students…need to be properly taught and acquired before they can be applied" (p. 137). It is through purposeful instruction and inclusion of technology in curricula that students can increase their abilities.
In 2001, the Enhancing Education Through Technology Act required districts to not only demonstrate technological integration into curriculum but to prove that all students had met a specific level of mastery by the completion of eighth grade (Barron et al., 2003; No Child Left Behind Act of 2001, Title II, Part D, § 2402Part D, § , 2002. This incentive-based program and its rewards and consequences directed some of the technology curricula of K12 schools for numerous years. However, once the federal aspects of No Child Left Behind were eliminated and the states were given authority regarding curriculum and governing standards, many states reverted back to a time when they did not assess students' overall technological abilities since it was not required (Davis, 2015;Layton, 2015).
In order to gauge students' abilities regarding technology, the National Assessment of Educational Progress (NAEP, 2014) designed and administered a national assessment to eighth-graders involving technology and engineering challenges. In 2014, only 43% of tested students performed at or above the proficient level (NAEP, 2014). While the number of proficient students grew significantly in 2018, to 46%, there is still a clear majority of students who are not meeting basic levels of abilities (NAEP, 2018). In order to discern characteristics that may positively impact scores, this research is concerned with whether a relationship exists between students' achievement in the Technology and Engineering Literacy (TEL) assessment and how often and in what ways they use a computer for school work and activities unrelated to school. In particular, computer use will constitute creating and organizing digital media and creating presentations and spreadsheets.
Despite a lack of basic proficiency, our digital native students are utilizing technology both in and out of school more than previous generations. Teachers are integrating the creation of digital media, presentations, and spreadsheets into their classroom instruction which not only gives students hands-on experience but provides a variety of ways to utilize technology in a constructive way to meet standards. In fact, Helsper and Eynon (2010) found that experience with technology helps to create a competent digital user as does the breadth of that use. Therefore, it is these factors that should be looked at when evaluating users' abilities with technology. This study seeks to bridge the gap between the scores and some of the factors which may positively contribute to overall proficiency. The data of computer use in and out of school and its relationship to TEL scores may be a helpful reference for teachers to understand what kind of computer-created products to include in their lessons. This information may also serve as a catalyst for encouraging students to create their own products beyond the classroom.
The purpose of this study is to explore the relationship between eighth-grade students' 2018 Technology and Engineering Literacy (TEL) scores and specific computer-created products for school work and outside of school activities. In order to explore this relationship, this study focused on these research questions: Does using computers to create media, presentations, and spreadsheets for school work have an impact on eighth-grade TEL scores?
Does using computers to create media, presentations, and spreadsheets outside of school for activities unrelated to their school work have an impact on eighth-grade TEL scores?
Our theoretical framework for this research adopts a scientific inquiry-based approach. The framework was described in great details in The Impact of Conversations on Fourth Grade Reading Performance -What NAEP Data Explorer Tells? (Bond & Zhang, 2017). In summary, the research methods combined the inquiry process with scientific knowledge, reasoning, and critical thinking. We started with an extensive exploration of the dataset, and that led to the designing of the research questions. The research questions further guided us to mine the data with great in-depth.

LITERATURE REVIEW
Digital literacy is no longer a way to differentiate oneself from others who do not have that skill; it is imperative that all graduating students reach at least a basic, proficient level in their abilities with technology. In fact, the ability of students to be digitally literate is no longer optional but has become a necessary survival skill for all regardless of curricular requirements (Eshet-Alkalai, 2004; Eshet-Alkalai, 2012). Even though technology will constantly change, and current tools will shift and evolve, the concepts of digital citizenship and literacies will not only sustain but expand (National Research Council, 1999). Through direct instruction, independent discovery, and hands-on experiences, students need opportunities to learn about and with technologies (National Research Council, 1999, p. 55). Often, students can be introduced to digital tools in the classroom which may prove useful for them beyond the walls of the school as well.

TECHNOLOGY USE IN SCHOOL
Since the choice to use technology or not within a lesson is a personal one, it is important to understand that teachers' beliefs and perceptions impact the overall design of their classrooms. The decisions of what types of activities occur in a classroom each day are largely dependent on teachers' pedagogical choices and knowledge of their content. Additionally, the correlation between teachers' beliefs about technology and how and when they choose to integrate it into their lessons has been established by several researchers (Albion & Ertmer, 2002;Badia et al., 2014;Chen, 2008;Ertmer, 2005;Taimalu & Luik, 2019). Taking it one step further, Ritzhaupt et al. (2012) noted that how teachers choose to integrate technology into the classroom directly impacted the frequency with which students used technology for learning.
More than a decade ago, findings were already showing an encouraging trend where teachers had students use more advanced, creation software rather than basic answer-input technologies (Niederhauser et al., 2007). For example, students can create digital presentations, spreadsheets, and concept maps and 76% of the activities Niederhauser et al. (2007) witnessed included these types of activities (pp. 495-496). Furthermore, in the case study by Ezquerra et al. (2014), the researchers found that by working with digital media in class, students not only reported an increase in their overall abilities, but also an increase in their potential to replicate the activity on their own. More recently, Schmid and Petko (2019) found that a clear majority of students used computers in their classes regularly. Combined, these data show that teachers are choosing to increase technological experiences for students in school, and they are utilizing higher-level computer use and creation rather than using the technology solely as a productivity tool or for instructional purposes.

TECHNOLOGY USE OUT OF SCHOOL
Today's students not only use technology in school environments, but most have better access to technology outside of their formal learning as well. Wang et al. (2014) found that a vast majority of students have regular use of smartphones, laptops, and desktops at home and tend to use technology more at home than in school. Research has found, however, that students are more likely to be creators of content in school whereas outside of school they may be more of a consumer; the difference in technology use may be attributed to students' personal entertainment purposes during their free time as well as requirements from their instructors to produce rather than share others' work (Lu, Hao, & Jing, 2016). In school, students were much more likely to engage in the use and creation of spreadsheets and presentations than outside of school where other media engagement surpassed the in-school use of the same tools (Wang et al., 2014). According to DeBell and Chapman (2006), a majority of students use home computers to play games and be entertained and 47% use the home computer to complete school work with word processors being the most common tool. Therefore, while students have more home access to technology and computers than ever before, they do not appear to be using it productively most of the time.

STUDENTS' TECHNOLOGY ABILITIES
With the prevalence of technology today, students regularly use digital and social media tools both within and outside of the classroom. However, Erstad (2015) noted that this "can easily be misleading and give the impression that all young people today are super-users and highly competent in their use of different media" (p. 85). In fact, often, the opposite is true as teachers report that students do not always know how to use new and emerging technologies without devoting school instructional time to navigating the tool alone (Wang et al., 2014). Furthermore, the types of assessment necessary to judge students' abilities cannot be done with typical standardized testing scenarios (Hohlfeld et al., 2010). This means that most of the data about students' abilities are tied to selfreported surveys and observations done by teachers and researchers.
A survey by Kaminski et al., (2003) found that students had relatively low software proficiencies despite having technology access and only about half of the respondents, or less, reported having some basic skills in hardware and information gathering. Additionally, they found that students preferred to increase their understanding through classroom instruction designed at improving their skills rather than seeking out to advance on their own (Kaminski et al., 2003). This was in contrast to the survey findings of McEuen (2001) where a majority of students stated that they troubleshoot problems and learn new technologies on their own. The McEuen (2001) survey also found that a majority of students self-reported average to expert capabilities in word processing, media creation, spreadsheeting, and using the internet.
Overall, students who use integrated technology in school will be better tech users than those who are not exposed to digital production and creation mandated by teachers. Moreover, students who have technologies available at home and choose to use them beyond consumerism will continue their learning in authentic ways. Just as classroom use of technology can impact home computer use, students who increase their abilities beyond the classroom may try to incorporate that knowledge into their school work. Whether students are interacting with digital tools formally or informally, there should be an overall impact on their general capabilities when it comes to technology.

METHODOLOGY
The National Center for Educational Statistics (NCES) designed the National Assessment of Educational Progress (NAEP), also known as The Nation's Report Card, in 1969 to measure what U.S. students knew and could do within various subject areas. NAEP is a congressionally mandated project administered by the NCES within the U.S. Department of Education and the Institute of Education Sciences (NCES, 2018). The present study will examine a subcomponent of the eighth-grade Technology & Engineering Literacy (TEL) scores of students at the national level. Of the three content classifications within the TEL score, the data used in this study is from the Information and Communication Technology (ICT) segment. This score ranges from 0-300 and was chosen because it "measures students' knowledge of the software and systems used for accessing, creating, and communicating information, and for facilitating creative expression" (The Nation's Report Card, 2018a, para. 5). This definition was most closely related to the variables needed for the research questions and aligned with the categorization of the same variables in research completed by Park et al. (2017).

PARTICIPANTS AND SCHOOL SELECTION
Using a probability sample design, participants for the assessments are selected randomly to ensure accurate, proportional representation of the greater population. According to a NAEP sampling infographic, NAEP uses a primary sampling unit (PSU) to identify single counties or groups of contiguous counties which are a representative sample of the nation at large (NCES, 2019). The national sample tests, such as the TEL, include public and non-public schools. Then, schools are chosen within PSUs to ensure that there is an equal representation of location type, race or ethnicity, and student achievement. Finally, students are then selected randomly from the appropriate grade level and students are assigned to a test. NAEP provides accommodations for special education students, students with disabilities, and English-language learners (NCES, 2019). Given every four years, the most recent TEL assessment was administered between January and March of 2018 to approximately 15,400 students from 600 schools (The Nation's Report Card, 2018c).

NAEP SAMPLING AND DATA COLLECTION
The TEL assessment is given completely via computer and does not break down collected data by state. According to The Nation's Report Card (2018a), the complete TEL assessment contains 15 scenario-based tasks and 77 standalone questions; however, students do not answer each question. Students test for a total of 60 minutes which is broken into two 30-minute sessions that vary based on the participant; the assessment is accompanied by a student questionnaire where participants have an opportunity to respond about their education and experiences both in and out of school (The Nation's Report Card, 2018a). The present study uses the average ICT scaled scores for the 2018 TEL assessment and there were six variables selected for analysis. Results were restricted to national public schools which exclude scores from the Bureau of Indian Education schools and Department of Defense Education Activity schools but include charter schools.

DATA ANALYSIS
The NAEP website includes a Data Explorer which allows users to access state and national results from all the assessments given. By inputting selected criteria, researchers can access data tables and charts and run basic statistical analyses. The Data Explorer was used to choose the appropriate TEL sub-score for eighth-graders and the desired variables before running statistical tests.
The six variables chosen for this study were all student-reported. Three of the variables asked students to consider the instruction received within the classroom while the other three focused on their time outside of school. The specific variables are presented in Table 1. The possible answers to all six of these student questions were a 5-point Likert-type scale indicating frequency from "never/almost never" to "almost every day." After selecting the proper sub-score and these variables, statistical significance tests were run to determine differences between groups.

RESULTS AND DISCUSSION
The NAEP Data Explorer provided summary means and standard deviations of TEL scores in relation to each category answer for the variables. The average ICT scaled score for all eighth-grade participants was 151 (SD=37) in 2018. Presented below are the mean scores for each variable related to the research questions as well as significance testing results. When significance was found, Cohen's d was calculated by hand using an effect size calculator from the University of Colorado (https://www.uccs.edu/lbecker/).

RESEARCH QUESTION #1 -COMPUTER USE IN CLASS
During the student questionnaire, students were asked about their computer use in the classroom and the ways in which they interact with specific technologies during instruction and assessment. To address the first research question, three variables were chosen for how students work during classroom activities. The average ICT scaled score, as well as the standard deviations, by question and response are shown in Table  2. Each variable is then broken down into its significance in the sections that follow.

CREATE, EDIT, OR ORGANIZE DIGITAL MEDIA
The highest overall scores for students using computers to create, edit, or organize digital media for school work were from the students who used technology for this purpose once or twice a week or almost every day. The overall lowest score earned in the ICT portion was from the students who never or almost never used a computer for this purpose. The results of multiple t-tests for the first variable, taken from Data Explorer, are shown in Table 3. Students reporting using a computer to create, edit, or organize digital media for school work a few times a year (M=153, SD=36), 1-2 times a month (M=152, SD=37), once or twice a week (M=156, SD=37), and almost every day (M=155, SD=38) had significantly higher average scale scores than those in the never or almost never group (p<0.01). The mean of the once or twice a week group was also significantly higher (M=156, SD=37) than the mean of the 1-2 times a month group (M=152, SD=37). To report the effect sizes of the significant variable responses, Cohen's d was calculated and is presented in Table 4. According to Cohen (1992), an effect size of 0.2 is considered small, a 0.5 is medium, and a 0.8 is large. For this variable, the effect sizes ranged from 0.11 to 0.24 indicating relatively small significance.

CREATE PRESENTATIONS
In looking at how often students create presentations on the computer for their school work, the highest average scores for the ICT portion was earned by students who did these activities 1-2 times a month or once or twice a week. The lowest overall scaled score was earned by those students who never or almost never use a computer in this way. The results of multiple t-tests for the second variable, taken from NAEP, are shown in Table 5. Students reporting using a computer to create presentations for school work a few times a year (M=152, SD=36), 1-2 times a month (M=157, SD=36), once or twice a week (M=154, SD=37), and almost every day (M=146, SD=38) had significantly higher average scale scores than those in the never or almost never group (p<0.001). The mean of the 1-2 times a month group was also significantly higher (M=157, SD=36) than the mean score of 152 (SD=36) of the few times a year group (p<0.001). Significantly lower scaled scores were found between the group who responded almost every day (M=146, SD=38) and those in the few times a year (M=152, SD=36), 1-2 times a month (M=157, SD=36), and once or twice a week (M=154, SD=37) groups (p<0.001). Finally, the 3-point scaled score difference between the group who created presentations 1-2 times a month (M=157, SD=36) and those who did it once or twice a week (M=154, SD=37) was also a significant difference (p<0.05). To report the effect sizes of the variable responses, Cohen's d was again calculated and is presented in Table 6. The range for the effect size from this variable was from 0.14 to 0.70. Five of the effect size comparisons were on the small end while the largest significance was found between the students who use a computer to create presentations for school 1-2 times a month and those who never do it (0.702). A medium effect size was found between once or twice a week and never or almost never (0.613) and a few times a year and never or almost never (0.567).

CREATE SPREADSHEETS
Students who used a computer to create spreadsheets for school a few times a year yielded the highest scaled score on the ICT portion of the TEL while those who did it almost every day had the lowest scores. The results of multiple t-tests for the third variable, taken from the NAEP Data Explorer, are shown in Table 7.  Students reporting using a computer to create spreadsheets for school work a few times a year (M=158, SD=36) had significantly higher average scale scores than those in the never or almost never group (p<0.05). The mean of the 1-2 times a month group was significantly lower (M=148, SD=36) than the mean score of the never or almost never (M=155, SD=37) and the few times a year (M=158, SD=36) groups (p<0.001). Significantly lower scaled scores were found between the group who responded once or twice a week (M=145, SD=36) and those in the never or almost never (M=155, SD=37), few times a year (M=158, SD=36), and 1-2 times a month (M=148, SD=36) groups (p<0.05). Finally, students who use computers in class for spreadsheet creation almost every day (M=139, SD=38) had significantly lower scaled scores than those in the never or almost never (M=155, SD=37), a few times a year (M=158, SD=36), 1-2 times a month (M=148, SD=36), and once or twice a week (M=145, SD=36) groups (p<0.01). To report the effect sizes of the variable responses, Cohen's d was calculated and is presented in Table 8. The overall range on these calculations was from 0.08 to 0.51 with the majority of the effect sizes at the small level. A medium effect size was found between the responses of a few times a year and almost every day (0.513) noting that the lower creation frequency yielded the higher score.

RESEARCH QUESTION #2 -COMPUTER USE BEYOND SCHOOL
During the student questionnaire, students were asked about their computer use outside of the classroom and the ways in which they interact with specific technologies for activities unrelated to their school work. To address the second research question, three variables were chosen for how students use their personal computer time beyond school. The average ICT scaled score, as well as the standard deviations, by question and response are shown in Table  9. Each variable is then broken down into its significance in the sections that follow.

CREATE, EDIT, OR ORGANIZE DIGITAL MEDIA
When looking at product creation on the computer for non-school activities, the highest scores on the ICT portion came from students who created, edited, or organized digital media almost every day. The lowest scaled score came from the group of students who did this activity a few times a year. The results of multiple t-tests for the first variable, taken from the NAEP Data Explorer, are shown in Table 10. Students reporting using a computer to create, edit, or organize digital media for activities beyond school almost every day (M=157, SD=38) had a significantly higher average scale score than those in the never or almost never (M=152, SD=36), a few times a year (M=150, SD=37), and 1-2 times a month (M=151, SD=38) groups (p<0.01). To report the effect sizes of the variable responses, Cohen's d was calculated and is presented in Table 11. They range from 0.14 to 0.19, indicating a small significance.

CREATE PRESENTATIONS
When evaluating the variable which asked about students' use of computers to make presentations unrelated to school, the lowest ICT scores came from those students who did this almost every day while the highest scores were students who never or almost never did this activity. The results of multiple t-tests for the second variable, taken from the NAEP Data Explorer, are shown in Table 12.  Students reporting using a computer to create presentations for activities beyond school almost every day (M=136, SD=38) had a significantly lower average scale score than those in the never or almost never (M=160, SD=35), a few times a year (M=153, SD=38), and 1-2 times a month (M=144, SD=36) groups (p<0.001). The means of the once or twice a week group (M=141, SD=38) and the 1-2 times a month group (M=144, SD=36) were both significantly lower than the means of the few times a year (M=153, SD=38) and the never or almost never (M=160, SD=35) groups (p<0.001). Finally, the 153 mean scaled score (SD=38) for the students who created presentations outside of school work a few times a year was significantly lower than those who never or almost never (M=160, SD=35) did this activity (p<0.001). To report the effect sizes of the variable responses, Cohen's d was calculated and is presented in Table 13. For this variable, mostly small effect sizes were found, ranging from 0.19 to 0.45; however, a medium effect was found for responses of never or almost never and once or twice a week (0.520) as well as never or almost never and almost every day (0.657) with the never or almost never group having the highest score over the other two groups of students.

CREATE SPREADSHEETS
The final variable looked at how often students use computers beyond school to create spreadsheets. The highest scaled score was earned by students who never or almost never did this activity while those who did it almost every day had the lowest scores on the ICT portion of the assessment. The results of multiple t-tests for the third variable, taken from the NAEP Data Explorer, are shown in Table 14.  Students reporting using a computer to create spreadsheets for activities beyond school almost every day (M=134, SD=39) had a significantly lower average scale score than those in the never or almost never (M=160, SD=34), a few times a year (M=149, SD=37), and 1-2 times a month (M=139, SD=37) groups (p<0.05). The means of the once or twice a week group (M=136, SD=36) and the 1-2 times a month group (M=139, SD=37) were both significantly lower than the means of the few times a year (M=149, SD=37) and the never or almost never (M=160, SD=34) groups (p<0.001). Finally, the 149 mean scaled score (SD=37) for the students who created spreadsheets outside of school work a few times a year was significantly lower than those who never or almost never (M=160, SD=34) did this activity (p<0.001). To report the effect sizes of the variable responses, Cohen's d was calculated and is presented in Table 15. The overall range of effect sizes was from 0.13 to 0.71. While many of the significance were on the small side, three medium effect sizes were found between never or almost never and 1-2 times a month (0.591), never or almost never and once or twice a week (0.685), and never or almost never and almost every day (0.711). The strength in these frequency changes is negative as scores decreased with more use rather than increased.

DISCUSSION
This study explored students' scores from the TEL assessment on the NAEP website as they related to specific uses of digital technologies both in and out of the classroom. O'Dwyer et al. (2005) noted the importance of using a subscale test score when researching standardized assessments in order to more closely align the data to specific instrument variables; therefore only the ICT scaled score was used as it included "...computers and software learning tools...and other technologies for accessing, creating, and communicating information and for facilitating creative expression" (Firman et al., 2015). The selected variables measured the frequency of students' ICT use of digital media, presentations, and spreadsheets for school work and personal projects.

INTEGRATION OF TECHNOLOGY INTO THE CLASSROOM
One challenge to this research is that students are not in control of their use of technology in the classroom. Involvement with and use of digital technologies such as media, presentations, and spreadsheets requires teachers to integrate them into their pedagogy which is a deeply personal instructional choice affected by many factors (Ertmer, 2005; Taimalu & Luik, 2019). Teachers' knowledge and implementation of specific technologies can hinge on many circumstances including their personal experience and comfort level, their teacher training program, and if any professional development has continued (Li et al., 2019).
This study found that students who responded that they used computers in school to create, edit, or organize digital media at any frequency greater than never achieved better scores on the TEL assessment. Additionally, a use for this purpose once or twice a week yielded the highest score and a significant (albeit small) gain over those who only did it 1-2 times a month. This is similar to the findings of Wang et al. (2018) who noted a positive association between in-class creation of digital media and higher ICT assessment scores.
Furthermore, this research found that students who responded that they used computers in school to create presentations at any frequency greater than never achieved better scores on the TEL assessment. The work by Lei & Zhao (2007) found that creating presentations was one of the most popular technology uses in school. The best frequency, according to the NAEP data, to create presentations for school work was 1-2 times a month as it had a medium to large effect size over other frequencies and students in this category demonstrated the highest ICT score on the TEL assessment. This means that teachers do not need to integrate this type of activity regularly to realize its benefit. This contradicts the work by Park et al. (2017) who found almost no relationship between students' TEL scores and their ICT use for school-related work and Carter et al. (2017) who noted a negative association between school computer use and content assessment scores.
One paradox was the use of spreadsheets which generally found that the more often a student used this tool, the lower their ICT scores. This is despite the fact that spreadsheet knowledge may be considered an important vocational tool and a necessary component of ICT instruction and knowledge (Tort et al., 2008). The highest score, however, was earned by students who did this type of activity a few times a year, but there was only a small effect size over those who never created spreadsheets. Without knowing specific numbers of students in each category, it is hard to know just how many students created spreadsheets in the classroom at all meaning that a small number of participants could have dictated the scaled score. Regardless, these findings mimic those of Wang et al. (2018). This type of activity may not really be integrated into teaching and learning opportunities or may have very specific uses in math, science, or STEM courses. Moreover, students and teachers may not fully understand the capabilities and options offered with the use of a spreadsheet which may impact the frequency and depth of their use for classwork (Tort et al., 2008).
Overall, the increased use of computers at school does not necessarily translate into higher abilities with computers as the frequency of ICT use itself is not enough to develop the higher-level quality addressed on technology assessments (Hatlevik et al., 2018;Wastiau et al., 2013). Teachers cannot control the technology students utilize at home and must integrate important ICT components into the classroom in order to best impact achievement (Hatlevik et al., 2015). This study showed that occasional use of these digital tools and media may impact students' technology assessment scores but there is other research that has not found the same correlations (Beland & Murphy, 2016;Biagi & Loi, 2013;Claro et al., 2012;Spiezia, 2011). Therefore, as with most in-school pedagogical choices, moderation in a breadth of areas may be the key to helping students slowly and steadily improve their technological abilities.

UTILIZING TECHNOLOGY BEYOND SCHOOL
When asking students to self-report their use of technology for projects unrelated to school, surveys rely on them to be forthcoming. Researchers have documented that perceptions do not always reflect reality, especially for sensitive topics, and may be biased or dishonest (Northrup, 1996;Wilson & Zietz, 2004). It is not likely that students taking the TEL assessment seek to be misleading with their answers to the questionnaire; however, they may not consider some of what they do independently as creation of digital media, presentations, or spreadsheets if it does not appear identical to in-class uses. It is with this in mind that some interesting correlations were found between students' ICT scores and their use frequency of these types of products.
Students who responded that they used home computers to create, edit, or organize digital media once or twice a week or almost every day saw higher scaled scores than those who never did these activities beyond school. Significantly higher scores, albeit a small effect size, for those who did it almost every day are similar to findings by Wang et al. (2018) regarding ICT experiences out of school. This also correlates to research which found that students who use a home computer in addition to having regular access in school had greater confidence in their ICT competencies which can account for higher overall scores (Wastiau et al., 2013).
In looking at the scores for students who create presentations or spreadsheets beyond school, the data showed that those who never do these two activities scored significantly higher than those who completed these tasks at any frequency. In fact, the participants who did these two tasks almost every day yielded the smallest ICT score of all the frequencies. Wang et al. (2018) also found a negative association between ICT score and more frequent creation of presentations and spreadsheets out of school, and this is also in alignment with the findings of Zhang et al. (2016) when looking at a previous administration of the TEL assessment. These types of tasks are less challenging and may not adequately match scenarios offered by the TEL assessment for real-life problems. Further, the repetition of basic tasks such as creation of presentations and spreadsheets may distract students by taking time away from other TELrelated learning which could better impact scores (Zhang et al., 2016). Wang et al. (2018) found that many of the out of school variables negatively impacted TEL scores. It is possible that the items on the TEL assessment which contribute to the students' ICT score are not strongly aligned with the types of activities mentioned in the questionnaire especially considering the ways in which students engage with technology at home. Wastiau et al. (2013) found students to be more competent with social media if they had online access at home. However, Uzun & Kilis (2019) found a significant relationship between heavy media usage at home and lower levels of academic performance. This association may explain some of the lower ICT scores on the TEL assessment as students do not practice media, presentation, and spreadsheet product creation at home, yet they spent an immense amount of social time in the digital world.
According to a study conducted by the Pew Research Center, at least 92% of students are accessing the Internet daily with approximately 24% who responded that they are online almost constantly (Lenhart, 2015). Students are regularly accessing social media, video games, and text-based communication; in other words, they are using the Internet for entertainment and communication (Lenhart, 2015;Wang et al., 2011). This work confirms that it is not a meaningful occurrence for students to be creating, editing, or organizing digital media, presentations, or spreadsheets in their personal time. However, negative relationships between students' use of home technology and their standardized achievement scores have been documented in areas of and beyond technology (O'Dwyer et al., 2005). Therefore, it is important to recognize how students' behaviors outside of the classroom will impact their achievement on standardized assessments such as the TEL in order to combat it with proper classroom instruction involving technology.

CONCLUSIONS AND RECOMMENDATIONS
Students today are generally not proficient in technology, as evidenced by the TEL scores overall and the low scores exhibited in the ICT subset; some students are "independent and critical users of ICT," but there are many who are not (Fraillon et 2017) who noted higher content assessment scores from students without regular use of technology. The main consideration, therefore, is that technology must be an appropriate pedagogical choice to enhance content and not distract from it or be taught in isolation as it is the quality and not the quantity that impacts scores (Lei & Zhao, 2007). With this in mind, there are three principles to pull from the findings.

IDEAL FREQUENCIES FOR IN-CLASS PRODUCT CREATION
This study showed that it is best to have students create, edit, or organize digital media once or twice a week to help improve their ICT scores. Additionally, having them create presentations once or twice a month can increase standardized assessment achievement. It is these two classroom activities which increased scores on the ICT portion of the TEL a minimum of 5 points but up to an astounding 26 points. The implication for teachers is to find ways to meaningfully incorporate the creation of digital media and presentations into all curricular areas so students can use their higher-order processing abilities to see a better connection to real-life applications of these digital tools. After all, students who are highly engaged in technology-based lessons and passionate about the learning at hand are more likely to learn faster and retain it better (Bandekar et al., 2018).

OUT OF SCHOOL TECHNOLOGY IMPACTS SCORES
The results of this research indicated that what students do outside of school mostly has a negative impact on their standardized assessment ICT scores. Therefore, students need to be aware of how their media usage can impact their academic achievement and the risks of being too involved with technology (Sert et al., 2019; Uzun & Kilis, 2019). Since the only way to positively impact scores is to create, edit, or organize digital media, students should be encouraged to do this at home although research shows they do this type of behavior already (Lenhart, 2015). This can help students to see transference between the topics and tools learned in school and how those types of products can have authentic, real-world applications and uses.

INCORPORATION OF REAL-LIFE APPLICATIONS FOR TECHNOLOGY
There are marked differences in how technology is used in class, out of class, and what is asked on an assessment. Several of the technology assessments available both nationally and internationally are boiled down to multiple choice questions or may ask for constructed responses about highly abstract concepts, and often find that a vast majority of students are not proficient (Fraillon et al., 2014;Fraillon et al., 2013;Krupczak & Disney, 2013; The Nation's Report Card, 2018b). These types of assessments may not match how instruction is usually delivered in class and how students utilize technology outside of school which can account for the lower proficiency scores overall and within the ICT subset. Therefore, when considering technology integration, it is important to go beyond simple uses of software programs and encourage scenario-and simulation-based tasks which better encourage problem-solving and allow for better transference of skills to an assessment.
Overall, the results of this study imply that teachers need to be utilizing technology in the classroom in order to see an increase in students' technology scores. Since student scores can improve when they create, edit, or organize digital media within the classroom, then teachers need to be provided proper professional development to know what this looks like in their specific content area. Specifically, teachers need to know how to include more authentic uses of technology for their subject area rather than having students artificially working with technology. With this information, administrators can make sure their staff are knowledgeable about some uses of authentic learning and assessment as well as problem-based learning. Additionally, presentation creation is one way to see scores rise, so teachers should include this type of digital activity in their classes although it does not need to be done daily. In this way, teachers can better equip students with proper technical knowledge for the 21st century.

LIMITATIONS
This research had several limitations. The first, already mentioned, is that students provided responses on questionnaires after completing the TEL assessment and may provide skewed answers that do not match reality (Northrup, 1996). The ambiguity of the questions asked, and thus used to analyze results, may also contribute to this problem and may not best match how students accurately use technology both in and out of school. A second limitation is the proficiency bands created by NAEP for use with their assessments to determine student achievement; many of the scores presented here show a lack of student abilities with regard to ICT when the cut scores set by NAEP may simply be too high (U.S. Department of Education, Office of Planning, Evaluation and Policy Development, 2009). This could mean that evaluating scores provided by NAEP is artificial and does not provide adequate information for stakeholders to draw meaningful conclusions. Finally, the NAEP website allows for some limited statistical analyses with the data but does not allow researchers to download original datasets or know N values (NAEP, 2014). This limited scope of use restricts findings to simple correlations and does not account for any regressions or predictions to make impactful changes.

RECOMMENDATIONS FOR FUTURE STUDY
Since a student's home use of computers cannot be regulated or monitored, it is necessary for administrators and educators to focus on what is being done in classrooms regarding technology use and instruction. Future research with ICT scores from the TEL assessment should look at some of the other variables related to in-class uses of technology such as the frequency of students designing computer programs or using digital tools to test hypotheses. These other variables may provide greater insight into what specifically helps to improve student scores. Additionally, some of the areas which found higher effect sizes may direct future research. For example, looking at the frequency of students creating presentations for the optimal success in addition to looking at what constitutes presentation-making with technology tools available. Ideally, researchers should strive to find classroom technology uses that significantly contribute to higher scores overall as the nation continues to look for ways to increase the technology abilities of our students.

SOURCES OF FUNDING
This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.