WebAssign is a flexible and fully customizable online instructional system that puts powerful tools in the hands of teachers, enabling them to deploy assignments, instantly assess individual student performance, and realize their teaching goals.
Students using WebAssign enter their answers to exercises using traditional mathematical notation. This means that students communicate with the WebAssign system by using the language of mathematics. Similar to the English language, the language of mathematics can be somewhat ambiguous at times. For example, consider the following sentence.
“I saw a man with a telescope.”
In this sentence, it is not clear whether the telescope is located with the man, or if the man was observed using a telescope.
Statements written in the language of mathematics can be as equally imprecise. Consider the relatively simple statement “sin x y”. Does this statement mean “(sin(x)) y” or “sin(x y)”?
“WebAssign grades more like a real professor, and less like a computer program.”
The WebAssign grading engine is complex, but the end result is simple: WebAssign grades more like a real professor, and less like a computer program. The software behind the grading system is structured to understand and interact with the mathematical language instead of forcing answer statements to fit a specific and inflexible framework.
WebAssign provides an intuitive experience so the student’s focus remains on learning the course material, not on learning a homework system.
The Challenge
See the WebAssign grading engine in action
One of the major advantages and applications of an online grading system like WebAssign is the ability to grade “free response” questions. Take, for example, the following exercise and corresponding answer key.
Exercise: Find the indefinite integral. 
Answer Key: 
In order to deliver and grade this question, the software must be able to accept many different variations on the answer key that are all mathematically correct. For instance, if a student wrote the response,
he should be awarded credit. After all, the student has only applied the laws of logarithms to rewrite the expression in a mathematically valid way. Notice also that the student has not used parentheses to explicitly delimit the argument to the natural log function. If the student wanted to use parentheses, such as
his answer should still be accepted. In general, there will be many ways to write equivalent forms of the required expression, and the homework system cannot predict what form the student’s answer will take.
One traditional workaround for this problem was to force students to enter their answers in a specific form. Unfortunately, this introduces many new problems. Foremost, restricting responses to a specific form destroys the “free response” aspect of the question that is so pedagogically important. Also, a homework system that accepts only specific forms of a response may cause usability issues. Finally, this restriction may mislead students into believing their equivalent response is mathematically incorrect.
Taking the idea of equivalent and correct responses even further, we wonder if responses like
should be accepted as well. After all, this answer is no less general than the one shown in the answer key, since C is assumed to be any arbitrary real number. If answers like these are to be accepted, then
should also be accepted, since any real number can be written as lnC for some number C. Therefore, even answers like
should be accepted for the original answer key of
In situations like this, the student’s response should be considered correct if and only if it satisfies some specific mathematical conditions. The original answer key is merely one example of many correct answers. Questions requiring this sort of grading occur frequently in all areas of mathematics instruction.
A similar problem arises when numerical approximations are permitted. Take, for example, the following question and answer.
Exercise: Use a graphing utility to approximate a sinusoidal function with period 4 which has a peak at (1, 3.7), and a valley at (3, 1.7).
Answer key: 
Students who use a calculator to approach this problem may obtain answers like
Or even
Exercises involving long numerical calculations are common, especially in academic disciplines such as physics. The ability to accept expressions that are approximately correct is crucial for an online homework system.
The challenge is to provide a general framework for effectively delivering and automatically grading these types of “open-ended” questions.
Strategy and Implementation
After considering these and other real-world examples, WebAssign’s first goal was to accept student input and process its mathematical meaning. The input could then be algorithmically tested for certain mathematical criteria. Those test results would let WebAssign output immediate assessment, grades, and feedback.
The critical step of this strategy was building a robust evaluator for traditional mathematical notation for WebAssign. After studying the notation styles used in various mathematics textbooks and publications, it was apparent that the language of mathematics is fundamentally different from a computer programming language. A programming language is purposefully designed and unlikely to change. In contrast, the language of mathematics has continuously evolved over the course of human history. As a result, mathematical notation is extremely flexible and expressive; however, with that flexibility and power comes the possibility of ambiguity.
Ambiguity in this context is usually the result of different overlapping notational constructs applying to the same portion of an expression. For example, multiplication between two terms is commonly notated by separating the two terms by a space, as in
However, spaces are also commonly employed as argument separators when using functions without parentheses. The following example clearly shows both uses of the space character.
To let students write notation in this way, it was necessary to permit the computer to interpret the space character in either way, whenever appropriate. Now, imagine a student enters the following expression, sin x y.
It is not clear whether the student meant to use the second space as a separator, meaning sin(x) y or as multiplication between x and y, meaning sin (xy).
Ambiguities like these have the potential to appear in many different mathematical constructions. As another example, consider an expression like “x + 1,234 + y.” This expression could mean either “x + 1,234 + y” or “a two-element answer consisting of the element ‘x+1’ and the element ‘234+y’.”
Since both of these interpretations could represent valid answers in the right context, it is difficult for a computer to decide which one is “better” for a given situation.
Permitting all academically accepted mathematical notations meant the possibility of ambiguous notation had to be permitted by WebAssign as well. Even though the language of traditional math notation has the potential for ambiguity, we still need a computer to evaluate it in a robust fashion.
In the interest of preserving students’ natural notation styles as much as possible, WebAssign made the decision to simply deal with the ambiguity programmatically rather than force the computer to choose one interpretation over another.
Additionally, the mathematical syntax itself can help eliminate spurious interpretations before they are formed. For example, the character “i” has multiple meanings including the imaginary unit √(-1) and the 3-dimensional unit vector <1,0,0>. But when a student inputs i+2, we know to ignore the vector interpretation because one cannot add a scalar to a vector. In this way, many spurious interpretations are avoided while still considering all salient interpretations.
Implementing the methods to account for ambiguous responses was a major accomplishment, but there was still the crucial step of actually grading the student input. For example, consider a question that requires the derivative of the response to exactly match the derivative of the answer key. In this scenario, the student response and the answer key can be passed to an auxiliary computer algebra system and differentiated symbolically.
Conclusion
The WebAssign answer evaluation and grading system offers solutions to many of the most difficult challenges associated with automatic homework assessment. Most importantly, the system helps WebAssign achieve two main goals: to help students learn and to help teachers teach.
“The WebAssign answer evaluation and grading system ensures students receive immediate and accurate feedback.”
From a student perspective, online homework systems like WebAssign are deemed effective only if the software is at least as easy to use as the traditional means of doing homework: pencil and paper. Well-designed input tools like mathPad are the first step in crafting a strong learning environment, but the “behind the scenes” processing that occurs is critical as well. By permitting and processing all types of natural mathematical input, students answering homework exercises in WebAssign have an uncomplicated and intuitive experience. This design ensures students are not spending time learning a homework system, but instead are focused properly on the real course material.
For instructors, the WebAssign design results in a trustworthy and pedagogically accurate assessment tool. The automatic grading attached to each WebAssign exercise is customized to be as strict or as lenient as the exercise’s instructions and definitions demand, enabling the system to grade much more like an actual instructor than a computer.
The WebAssign answer evaluation and grading system ensures students receive immediate and accurate feedback. Overall, WebAssign provides the tools for students to develop and demonstrate concept mastery. With these robust methods in place, WebAssign frees instructors to focus on teaching the course material.