1. Introduction

The global outbreak of COVID-19 prompted widespread closures of educational institutions, resulting in significant disruptions to the educational process and scheduled evaluations. Since the onset of the COVID-19 pandemic in March 2020, concerns have grown among educators, parents, and policymakers regarding the potential educational setbacks caused by these persistent interruptions. Initial predictions, based on research regarding summer learning loss, weather-related school closures, and absenteeism, suggested that students could miss out on 30% to 50% of their expected learning due to the spring 2020 school closures [1]. Furthermore, it was anticipated that the pandemic would exacerbate existing educational inequalities, particularly along the lines of race and income. This expectation included significant increases in achievement gaps between low- and high-income elementary school students [2]. These gaps were projected to widen due to various factors, encompassing disparities in educational resources within schools and external factors like elevated COVID-19 exposure risks and limited access to high-quality healthcare coverage for disadvantaged families. A survey of education researchers predicted substantial growth in achievement disparities among elementary school students from different income backgrounds [2].

During the COVID-19 pandemic, the educational landscape underwent substantial upheaval, and the repercussions of these disruptions are expected to persist over an extended period. Ongoing research into the pandemic's impact on education has revealed that disruptions to learning extended well into the 2021–2022 academic year [3−5]. Notably, test scores for students in grades 3–8 during this period declined significantly compared to the pre-pandemic, surpassing the disruptions seen during Hurricane Katrina [5]. Of particular concern is the widening gap in test scores observed between low-poverty and high-poverty elementary schools [5]. While there are signs of improvement in the 2021–2022 data, historically marginalized students and those in high-poverty schools continue to grapple with significant disparities in mathematical achievement [3-5]. Consistently, the data highlights varying impacts of the pandemic on math and reading achievement in grades 1–8, exacerbated by factors such as race/ethnicity and school poverty classification. An analytical study estimated the time required to close these achievement gaps, based on initial progress [5]. For students in grades 5–8, it is projected to take more than six years to fully bridge the mathematics gap, indicating that certain students may not fully recover from the educational disruptions caused by COVID-19 before their high school graduation. The consequences of this incomplete learning, particularly as it affects high school students transitioning to college, raise significant concerns [5].

The pandemic's impact on student readiness in higher education has been the focus of investigation in numerous research studies. These studies have aimed to explore and gain a comprehensive understanding of how the pandemic has influenced the preparedness of students for various educational challenges and experiences within the higher education context [6,9].

High school students in their sophomore, junior, or senior years during the pandemic encountered a markedly altered educational landscape, deviating substantially from the traditional norms, especially when it came to their preparations for the transition to college. This disparity became particularly evident in their experiences with general chemistry courses, where laboratory skills and mathematics skills were notably affected. The pandemic-induced disruptions had a significant impact on the way these students engaged with these core academic skills, challenging their preparedness for higher education [10-15].

Borough of Manhattan Community College (BMCC) is among the seven community colleges in the City University of New York (CUNY) system, which also encompasses 11 senior colleges and 7 graduate and professional schools across the five boroughs of New York City. Recognized as a Minority and Hispanic-Serving Institution, BMCC attracts students from more than 155 countries, each contributing diverse linguistic and cultural perspectives, establishing BMCC as a prominent community college in the United States.

In Spring 2020, as New York City emerged as the global epicenter of the COVID-19 pandemic, BMCC, like many institutions worldwide, rapidly transitioned to fully online remote learning. An important number of BMCC students live within New York City's five boroughs and commute to BMCC using public transportation. The commuter students often juggle multiple commitments, including family, employment, and other responsibilities, alongside their coursework. Many of them are first-generation, underrepresented, and/or minority students who may occasionally miss classes due to health/personal issues or job commitments [16-17]. As a result of BMCC's open-admission policy and the diversity of students' prior educational experiences, there is substantial variation in college readiness, leading to notable disparities in class performance.

In this study, we investigated the laboratory grades of General Chemistry students before and after the onset of the COVID-19 pandemic. Our objective was to assess the impact of the pandemic on chemistry education within the context of an urban community college.

2. Study Design

CHE121, the introductory course in the General, Organic, and Biological (GOB) chemistry sequence, is recommended for students intending to transfer into bachelor's degree programs in nursing and allied health sciences. It provides a foundational understanding of the fundamental principles in general, organic, and biological chemistry. Recognized for its substantial size and high enrollment figures, CHE121 occupies a prominent position within the Science Department at Borough of Manhattan Community College (BMCC). Each semester, this course draws approximately 700 students distributed across about 30 sections, establishing it as one of the department's largest offerings. Significantly, CHE121 serves as a gateway course for students intending to enter the nursing program, frequently serving as their initial science course. CHE121 is a 4-credit, 6-hour course that meets twice a week, 3 hours for the lecture and 3 hours for the lab. For this study, data were gathered from two sections in the pre-COVID Fall 2019 semester, two sections in Spring 2022 when students completely returned to campus for fully in-person labs, two sections in Fall 2022, and two sections in Spring 2023. In each section, there were 25-27 students, resulting in a total of 201 students across all sections. For CHE 121 Lab, there are typically 15 lab reports, 15 pre-lab quizzes, and 15 post-lab quizzes in a standard semester. Average grades for students' lab reports, including their lab work and in-class questions, average grades for online pre-lab and post-lab quizzes, percentages of pre and post-lab online quiz completions, and grade averages of lecture test 3, the test most closely related to the lab experiments where students faced the most difficulties and experienced most COVID-related learning loss, were analyzed for both the periods before (Fall 2019) and after the onset of COVID-19 (Spring 2022, Fall 2022, and Spring 2023).

Following the observation of significant learning setbacks in laboratory skills and the required math skills in Spring 2022, proactive measures were implemented. Starting in Fall 2022, students were provided with instructional YouTube videos accessible on Blackboard, covering the upcoming lab experiments to enhance their understanding and visualize the experiments before the hands-on work. Also, at the beginning of each lab session, a comprehensive math review was conducted, focusing on the formulas necessary for lab report calculations and in-class questions.

The pre-lab and post-lab assignments consisted of five multiple-choice questions each, and they were made available as online assignments on Blackboard. Students had two attempts for each assignment, and the average grade for each attempt was calculated. To ensure fairness, the order of the questions was randomized for each student and each attempt. Grading for these assignments was automated through Blackboard. To discourage copying, students could access the answers only after the assignment deadline had passed. The third lecture test, which covered topics correlating with the lowest average lab report grades (Mole Calculations, Stoichiometric Relationships, Balancing Chemical Reactions, Gas Formulas, and Solution Chemistry), consisted of 33 multiple-choice questions. These tests were administered in the traditional paper-and-pencil format during class, with students marking their answers on scantron sheets. The same set of Test 3 questions, though with different ordering of questions and answers, was used for Fall 2019, Spring 2022, Fall 2022, and Spring 2023. Lab report sheets were collected at the conclusion of each lab session.

3. Results

Following the return to fully in-person labs in Spring 2022, student lab performance dropped significantly due to COVID-19's impact on chemistry skills. On average, lab report grades fell by 16% compared to the pre-COVID Fall 2019 averages of 81%. However, a helpful change was made starting in Fall 2022. To assist students in recovering their lab skills after the pandemic disruptions, they were assigned YouTube videos on Blackboard. These videos gave students a visual preview of the lab experiments, helping them become familiar with the procedures beforehand. This initiative proved effective in bridging the gap caused by COVID-19. The results became evident in subsequent semesters. Spring 2022 saw an average lab report grade of 68%, reflecting the initial challenges. However, by Fall 2022, there was an improvement, with the average grade at 73%. This positive trend continued, with an average lab report grade of 79% in Spring 2023 (Figure 1). These findings indicate a substantial recovery in students' chemistry laboratory skills, showing their adaptability and determination to overcome the pandemic's disruptions. The data suggests a promising path toward full restoration in lab performance.

Figure 1. Average lab report performance before COVID-19 (Fall 19) and after COVID-19 (Spring 22, Fall 22, and Spring 23)

The evidence of COVID-related learning loss in lab skills was particularly pronounced in labs related to Moles and Chemical Formulas, Gas Laws, and Solution Concentration. Students faced substantial challenges in these experiments, as they required stronger lab skills and math proficiency for calculations in lab reports and in-class questions. These labs demanded a higher level of hands-on precision and specific lab skills, including precise measurements, the proper use of a Bunsen burner, and effective collaboration with lab partners. These findings align with lecture experiences, where mathematical skills are crucial for grasping topics like Mole Calculations, Stoichiometric Relationships, Balancing Chemical Reactions, Gas Laws, and Solution Chemistry. These topics involve the application of math skills for rearranging chemical formulas and understanding algebraic equations, all of which are tested in the lecture portion of Test 3. To assess the impact of COVID-19 on math skills-related lab performance loss and the correlation with related lecture topics, we analyzed mean Test 3 scores before the pandemic (Fall 2019) and after the pandemic (Spring 2022, Fall 2022, and Spring 2023). Figure 2 illustrates that average Test 3 score results exhibit a very similar pattern to lab grade performance both before and after COVID-19.

Figure 2. Average Test 3 scores in percentage before COVID-19 (Fall 19) and after COVID-19 (Spring 22, Fall 22, and Spring 23)

On average, students' performance in Test 3 saw a decline of approximately 13% when compared to the pre-COVID averages recorded in Fall 2019, where the average Test 3 score stood at 72%. In Spring 2022, the average Test 3 score experienced a dip, reaching 63%. However, there has been a consistent and positive upward trend in Test 3 score averages in the subsequent semesters, following the implementation of detailed math review sections related to chemistry formulas at the beginning of each lab session. In Fall 2022, there was a noticeable improvement, with the average score climbing to 66%. This positive trend continued, reaching an average Test 3 score of 67% in Spring 2023. This data aligns with the trends observed in lab report performance before and after the onset of COVID-19, highlighting the crucial role of mathematical abilities in chemistry lab performance.

On a positive note, the submission rates for weekly online pre-lab and post-lab quizzes exhibited an upward trend in response to the challenges posed by COVID-19 (Figure 3).

Figure 3. The submission rates for weekly online pre and post-lab quizzes before COVID-19 (Fall 19) and after COVID-19 (Spring 22, Fall 22, and Spring 23)

Before the pandemic, the average submission rate for online pre and post-lab quizzes stood at 84%. However, after the initial full return to campus for in-person labs in Spring 2022, following COVID-19, there was a notable increase in these weekly completion rates, which rose to 92%, reflecting a substantial about 10% increase. This shift indicated that COVID-19 had made students more conscientious about their weekly online assignments, motivating them to engage more actively with these tasks. Furthermore, this heightened sense of responsibility persisted, with completion rates consistently ranging from 88% to 90% throughout Fall 2022 and Spring 2023, highlighting the positive impact of the pandemic on students' commitment to online assignments.

The increase in pre-lab and post-lab submission rates aligns with an upward trend in average scores for online pre-lab quizzes (Figure 4) and post-lab quizzes (Figure 5). While we acknowledge that access to additional resources may have contributed to enhanced student success, it's important to note that the administration of pre-labs and post-labs remained unchanged before and after the pandemic. Prior to the pandemic, the average pre-lab quiz score was 77%. However, in Spring 2022, after the initial return to full in-person labs, there was a notable improvement, with the average score rising to 84%, marking a 9% increase. This positive trend continued throughout Fall 2022 and Spring 2023, with average scores maintaining a range between 83% and 85%.

Figure 4. Average pre-lab scores before COVID-19 (Fall 19) and after COVID-19 (Spring 22, Fall 22, and Spring 23)

Figure 5. Average post-lab scores before COVID-19 (Fall 19) and after COVID-19 (Spring 22, Fall 22, and Spring 23)

A similar pattern is seen when examining the average scores for post-lab quizzes. Before the pandemic, the average post-lab quiz score stood at 79%. In Spring 2022, following the initial return to campus for in-person labs, there was a significant improvement, with the average score reaching 87%, marking a 10% increase. This positive trend persisted into Fall 2022 and Spring 2023, with average scores consistently ranging between 84% and 86%.

4. Discussion

The study presents significant insights into the impact of the COVID-19 pandemic on General Chemistry education at an urban community college. Following the return to fully in-person labs in Spring 2022, a notable decrease in laboratory performance was observed. This decline, amounting to an average of 16% in lab report grades compared to pre-pandemic levels, highlighted the challenges students faced in adapting to the hands-on aspects of chemistry after a period of remote learning. Specifically, experiments involving Moles and Chemical Formulas, Gas Laws, and Solution Concentration posed greater difficulties due to their demands for precise lab skills and mathematical proficiency.

To mitigate the adverse effects of COVID-19 disruptions on laboratory skills, proactive measures were introduced in Fall 2022. Students were provided with instructional YouTube videos available on Blackboard, offering them a visual preview of upcoming experiments. This initiative allowed students to familiarize themselves with lab procedures in advance, enhancing their preparedness. Moreover, at the beginning of each lab session, comprehensive math review sections were conducted, focusing on the formulas required for lab report calculations and in-class questions. The positive outcomes of these interventions became evident in subsequent semesters, with the average lab report grade reaching 79% by Spring 2023. This demonstrates students' resilience and adaptability in regaining their chemistry laboratory skills.

The study identified a parallel trend in Test 3 scores, which assess mathematical abilities related to chemistry concepts. While initially experiencing a decline following the onset of the pandemic, Test 3 score averages showed consistent improvement in subsequent semesters. This positive trajectory can be attributed to the incorporation of detailed math review sections related to chemistry formulas at the beginning of each lab session. The correlation between laboratory performance and math skills underscores the critical role of mathematical abilities in chemistry education.

In addition to the notable recoveries in laboratory performance and math skills, the study identified a significant increase in submission rates for weekly online pre-lab and post-lab quizzes. This shift towards greater engagement with online assignments contributed to enhanced academic performance and reflected students' increased comfort with and adaptability to online learning tools. The challenges brought about by the pandemic encouraged students to embrace online resources, resulting in higher completion rates for these online assignments. This shift in behavior underscores the lasting impact of remote learning, which prompted students to become more proficient in navigating online platforms and adopting a more adaptable approach to education. It suggests that the lessons learned during the pandemic, including the proficiency gained in using online learning tools like YouTube videos to visualize experiments before lab sessions, can still benefit students in their academic journeys.

To address the COVID-19-induced learning loss in laboratory skills and related math skills crucial for success in chemistry courses, institutions may consider the implementation of focused workshops or boot camps during summer and winter sessions. These workshops would aim to equip students with essential laboratory competencies, including proper equipment usage, proficiency in measurements, adherence to lab protocols, and mastery of relevant mathematical skills. By offering these intensive workshops as a preparatory step before students undertake their chemistry laboratory courses, institutions can effectively expedite the process of skill recovery. These proactive measures not only empower students to overcome pandemic-related challenges but also ensure they enter their chemistry labs well-prepared, enhancing their overall learning experience and success in the subject.

The study underscores the resilience and adaptability of students in the face of disruptive events such as the COVID-19 pandemic. It highlights the effectiveness of proactive measures, including instructional videos and math review sessions, in aiding students' recovery of laboratory skills and mathematical proficiency. To further enhance chemistry education, it is imperative to continue supporting students in these areas and consider the integration of innovative teaching strategies. This study offers valuable insights into addressing educational setbacks caused by unforeseen disruptions and provides a foundation for ongoing improvements in chemistry education at the community college level.