Contributor: Kimberly Benien, Wharton County Junior College (Wharton, Texas).
I trusted, I thought, I assumed. For years, I would review student test scores on exams and when they were scoring 80% or above, I believed they understood the material. Next I would look more closely at the problems missed, whether students completed the review, and with what they may have struggled. Then for each student I would look more closely at their homework and whether or not they utilized the practice assignments.
I was always hoping to find some correlation between what they had done on previous assignments and what they may not have done. I wanted to know how it related to their work on the exam. Was there a pattern of students avoiding a particular concept? Essentially I was looking to equate correct on a problem with a demonstration of mastery. Demonstration of mastery is a multi-faceted strategy where students must make changes in behavior, be willing to wrestle with problems, and increase learning if they are to achieve success.
Why do students struggle with newly presented content? As an educator I looked for other reasons why students would struggle with new material. Were they simply not practicing until they felt confident with the material? After all, practice makes perfect; unless you are practicing it “wrong.”
Technology allows instructors to peer into a student’s work at the question level. We can then make comparisons to performance on problems exactly alike or, at least, similar enough to the ones they missed. The information gleaned leads to another question, why can they do it on one assignment but not on the next? There is an apparent inconsistency with some students, a lack of transference of learning from one problem to the next or from one day to the next.
Student feedback has shown that they are not always struggling with the newly presented content but that they are struggling with prerequisite skills. In interviewing students on this subject, they explained a circle of “getting through” the math without really understanding it. They say they are just cramming for an exam, only to move on and quickly forget what they just “learned.”
Algebraic and math skills are the foundation required for trigonometric equations, limits, and break-even points. In essence, too many students may be trying to learn the material being presented in class and the algebra that supports it.
1. Review: Analyzing and assessing what is “correct”
In recent years, I have passively fought the notion that a “correct” answer correlated to a demonstration of mastery. Whether I allow students five attempts
or ten, “correct” could mean correct on the first attempt, or the tenth. Assuming that correct on the tenth attempt is equivalent to correct on the first attempt; can we now make the connection that scoring correct on the tenth attempt equates to mastery? I remember professors stating, “Do not assume what a student knows.” That phrase is worth repeating.
Until now, if I wanted to determine how many attempts a student used to achieve the correct answer, I would have to look at each student’s assignment and then into each question. While the information would have been valuable, it was an impossible task when serving 150+ students in five courses.Enhanced WebAssign’s new Class Insights takes that idea to the moon and back. These real-time analytics allow me to view, at the assignment level, what percentages of students have received a “correct” on 1st attempt, 2nd attempt, or any attempt; in addition, questions are flagged for review based on the percentage of students that are answering them incorrectly on the first try (see Example 1).
The next level is by topic or the concept. I can choose to look specifically at a topic in one assignment category or in all assignment categories containing that topic. The opportunity to look at a topic across multiple assignments saves me time while providing me with a large amount of data.
For a flagged question, I can view the question, dig deeper into the question performance, and in Question Statistics see a list of students answering correct on 1st attempt, 2nd attempt, not correct, and even a list of the students who never attempted to answer the question. The ability to identify at-risk students on one screen and in one pop-up window would have previously taken me dozens of screens and over an hour to compile. The data collected from Class Insights can drive the review in the first ten minutes of class to only those problems with which students may struggle the most (see Example 2).
2. Reflect: Considerations that support further conclusions
For students turning in paper homework, it may be days before it is returned, if at all, leaving students to believe that they understand the concept better than they actually do. How many times have we heard our students say, “I thought I knew what I was doing”? Before I began utilizing technology, I heard that phrase often when returning an assignment to students. Persistent and prolonged practice does not always make perfect. Especially if the practice is misguided.
As we leverage technology in providing opportunities for all students, we are also leveraging the feedback. With Enhanced WebAssign, students receive immediate and enhanced feedback that guides them or redirects their process and allows them to try the problem again while it is fresh for them. Beyond the help while students are learning and practicing the concepts, students need to reflect on their abilities and their understanding. Displaying “correct on 1st attempt” data to students creates a reality check and further reflection on merely “correct for a grade” or “correct for understanding.”
Why do some students score well on assignments and struggle on exams?
Most students do not know how to begin making major changes in their behavior and forming new habits. Whether that’s through organization, the way they study, where they study, the priority of class attendance, or completing all assignments. Their priorities need to include “completing assignments on time.”
Instructors need to encourage students to make changes that will help them “learn” how to learn and how they learn. New habits should increase learning and success in the current course and be transferable to the next course.
How can instructors teach the required course content and assure student success?
When we think about our students, there are two things all instructors know: we will have students, and they will have varying experiences and ability levels. Students need opportunities that foster effortful learning and make new connections to concepts they have learned previously. Those that develop their own learning have a deeper and stronger understanding of the material encountered.
These learning experiences and assessments must align with the student learning outcomes of the course if all students are to accomplish the goals. Teaching for success must be planned and purposeful…it does not just happen!
3. Redesign: Assuring assignments and exams correlate outcomes to improve exam performance
To this point we have reviewed student data, introduced student feedback, and drawn conclusions from our experiences. Topics to be addressed are prerequisite skills, retrieving tangible connections to previous learning, and purposeful practice as a part of a learning path, aligning course outcomes, and behavior modification.
Enhanced WebAssign allows instructors to incorporate a just-in-time review as a part of the assignment design and that encourages students to retrieve what they know about the content while refreshing prerequisite skills. If I am introducing limits, I want to review factoring polynomials; before introducing integration by parts I review substitution. Retrieval provides opportunity for students to correct a previous misunderstanding, while instant feedback provides purposeful practice and assurance that students understand the concepts and methods.
As technology has taken on a greater role in teaching and learning, Enhanced WebAssign gives instructors more control over the tools, resources, and multi-media. Incorporating embedded video instruction throughout a course leverages learning for all students.
Student feedback has stated that they leave class with more questions than answers. So often, instructors believe that if students are taking notes then they are engaged. Providing opportunities outside of class to engage students with the content is most beneficial in the learning process.
Student study habits are just as important as the content they learn from. When designing a course to incorporate necessary components for learning, encourage students to adopt new study habits that develop deeper and stronger learning. Returning once again to Class Insights, instructors should see concepts consistent through the learning path, including in the unit review and in the exam.
If questions from linear programming are only present in your exam but not within other assignments, there exists a break in the student learning outcomes that must be addressed.
As an instructor, I believe that it is my job to ignite a student’s passion to discover, to impart a desire to learn. Getting a student to truly master a concept involves learning and altering their behavior. Ask not just if a student got the answer correct, but how many attempts did it take them? What are their study habits? Are they truly mastering the content?
Enhanced WebAssign allows the instructor to identify where students are struggling and provides the tools to tailor the program to their needs. Assuring student success in a course is believed by many to ultimately be the responsibility of the student; but that’s only one-half of the story.
ABOUT CLASS INSIGHTS
Kimberly tested Enhanced WebAssign’s Class Insights in six of her courses during the spring of 2015. The data it provided were unique and allowed her to think “outside-the-box” when triangulating and analyzing the data. Kimberly will continue to mine the data and look for new ways that Class Insights can help her raise the bar for students. Class Insights is expected for full release in fall of 2015. Learn more about WebAssign’s Class Insights.
Kimberly Benien is a mathematics instructor; course builder of all distance education courses and coordinator of course management systems for the department at Wharton County Junior College. She received her B.A. in Educational Studies and in Mathematics and her M.A. in Mathematics Education from Western Governors University in Salt Lake City, Utah. Kimberly’s experience includes Secondary Mathematics, Intermediate Algebra, College Algebra, Trigonometry, Finite Mathematics, Business Calculus, Elementary Statistics, Precalculus, and Calculus. Her experience with technology integration and six different course management systems made Kimberly the choice to pilot Enhanced WebAssign in her online course. Due to its ease of use, innovative platform, and unique options that encourage freedom of instruction, she now trains instructors to integrate Enhanced WebAssign into their courses. Kimberly has also integrated Aplia into her Elementary Statistics course because it was intuitive to use and provided unique feedback in the form of thought-provoking explanations.