I started working for SSVH (Swedish National School for Adults in Härnösnad) on a regular basis in 1996. (I had worked for them off and on from 1990.) I was teaching blended distance courses in Physics and Math. The students would come for a week and go home and continue studying and snding in exercises. Then they would come for another week at SSVH which usually ended with a final exam.
At about that time we were experimenting with using Internet to complement our offerings. The early offerings were pretty much ”book på burk” (canned books) but they could have some simple interactive questions (the usual multichoice or rather inflexible fill in the blank types).
The things on Internet that attracted my teacher’s eye were:
- some Java applets that could show many things quite dynamically and flexibly
- a kind of ”tutorial questions” that could give a more helpful feedback
Java applets, simulations
For my physics classes I started collecting java applet links. I would use them when we had school weeks and show them with a projector. Being able to show wave interference to explain diffraction patterns or how resonant tones in tubes work was wonderful. A number of students were starting to get Internet connection so I started collecting the applets themselves to our server. Then I could translate the surrounding page and put in things they should try to do and follow-up questions. When I used the applets I often found that the original instructions were often incomplete and that many could have some idiosyncrasies that could give a wrong impression about some things they would seem to demonstrate.
The quality of applets could vary immensely. Some things were so oversimplified I wouldn’t even like to use them in kindergarten! The ones I liked best were often very powerful. That (usually!) wasn’t a problem when I was using them to demonstrate for a class, but one often had to setup a lot of parameters to show something specific. When I would want to make a page an online student could use, this would mean often including a list of 5-10 things the student would have to setup to see one effect. This could be quite daunting to even try and time consuming and error prone. The student would then often be unaware of the error and could be lead to wrong conclusions.
Fortunately I discovered that many applet constructors would often gladly implement features that I missed, or fix bugs. The first case was with an applet by very productive physics teacher, named Hwang, about the doppler effect. I saw that it was very similar to a question on a National Exam and was thinking of making some exercises around it. One needed to measure some distances in the simulation. In many applets one can drag the mouse and get the distance and/or delta x, delta y for the dragged line. But when I tried to do that in the applet, the display changed from xy to angles as soon as I pressed the mouse button. That was a nice feature for some calculations but not the one I wanted to do. Before going home from work, I wrote an e-mail to Hwang wondering if it were possible to show both.
The next morning I found a reply in my inbox. He, living in Taiwan on the other side of the Earth, had already added the feature I wanted and wondered if I had any other suggestions! He has made many other changes and additions to his applets after contacting him. He is not the only case. Contact them!
Tutorial questions, especially numerical
The tutorial questions I found were based on a suite of PERL scripts. They could handle multiple choice, numerical and algebraic questions. The really appealing one for me was the numerical question type. All the questions could portion out hints, one at a time. The nice thing about numerical questions was that it could, in one question, accept a student formulated answer, give specific feedback for many different numbers or intervals. This meant that the classic problem ”how can we help the student that doesn’t have the ”right” answer?” could be solved. We could give specific feedback for the kind of mistakes that could lead to that kind of question. So I started ”collecting mistakes” when I was hand correcting student assignments. I would have a wrong answer and know HOW they had gotten there. It is very important to note cases where the final answer gotten in the WRONG WAY looks the same as the properly rounded correct answer. In these cases it is best to adjust the question so that kind of mistake would give an answer in a different interval. (The number 2 can be something to avoid: 2+2=2*2=2^2 are all 4 in spite of totally different operations! 1 and 0 can have their problems too. Forgetting *1 or +0 won’t make any difference.)
I started making tutorial questions to the first assignment in the math and physics courses that I had running. There weren’t so many students that had Internet, but those that did weren’t difficult to get to try the questions. Their responses encouraged me to continue. I changed the scripts so the presentation would be in Swedish and made the layout more compact. The questions were very good at correctly interpretting numerical answers entered in a lot of different ways. 3.65 3.6500 3+0.65 3+6.510-1 3+6.5E-01 would all be considered correct. In Sweden we use , as decimal and that wasn’t accepted. So I dove into the world of regular expressions and added some filters so that ,->. ^-> %->0.01 ”rot(”->”sqrt(” and even the disputable ”½”->”+1/2″
Each tutorial question is one HTML page. The page is marked up with some specific tags so the PERL script can sort out what to show and what to skip. I put a lot of work into the hints. Most questions would have between 5 and 20 hints. I wanted the students to first try to answer the question and if they didn’t get it right or feedback that helped them know what to do, they should check some hints. I would usually have a hint ending with a BOLD text: ”Don’t read any more hints before really trying to answer the question!” After that would be hints that could maybe go almost all the way to the final answer. I also put in a way to get to see ALL hints at one time. It is a small link until they have the right answer. After that I encourage them to check ALL the hints. I would have liked to prevent them looking at any hints before submitting an answer, but I never implemented that.
In the sciences we try to get students to express numbers with a proper degree of precision. For most questions I would have one ”correct answer” that was properly rounded off and one ”almost correct interval” that would capture the ”exact answer” and the answers that weren’t rounded off enough. For these I wanted to give a very encouraging feedback though asking them to resubmit after checking significant digits. ”THAT’S A VERY GOOD ANSWER! You’ve almost certainly calculated right. But you should check your significant digits and round off and resubmit to get a fully correct answer.” Students tend to be discouraged when an answer isn’t considered right, so I try to be very positive and include links to pages about significant digits and rounding off. But it is good to try and ”nag” it in, right from the beginning. Since this question type can even accept numerical expressions like 1.23*(5.4+1.2)^2, I usually include an unrounded answer that they can round off without having to feed the expression into a calculator.
The script gives automatic feedback when the student enters something that is neither recognized as the right answer or one of the other prepared expected answers. ”That was incorrect.” I have had a template which included the specified ”incorrect” message: ”We haven’t recognized your answer as the correct one or one of the common mistakes. Please give us feedback telling how you got your answer. You can thereafter, try checking the many hints if you haven’t done that already.” It’s important to not be overbearing in the feedback. Sometimes a proper answer may not be recognized. Sometimes the teacher’s answer is oversimplified (or downright wrong!). Sometimes the question itself can be interpretted in different ways. I believe that Internet should be used to collect feedback that can improve the content on every page. This is especially important in educational interactive questions.
Feedback to improve content
I made tutorial questions for all the questions in the first assignment and tried on a few volunteers. After some adjustments I decided to make it obligatory. Almost all students by this time could get onto Internet somewhere. Some embraced the chance. Some were very reluctant. One reluctant kind was the Internet geek that had seen too many ”interactive questions” that just said ”RIGHT” if you happened to write exactly like the question creator had, and ”Wrong” if you wrote ”.5” rather than ”1/2”. Those students didn’t want to get annoyed. The other kind was the computer newbie that just expected the worst from computers. I told both these groups that you only have to try to do the first assignment this way. ”If you have problems with getting it to work, you must either contact me or write down what you did and what happened and send it to me with your solutions.” Even these students wanted to continue with the tutorial questions after trying them! The only negative reaction was from one student that was afraid that mistakes (I think it was just significant digits mistakes) done here would lower her grade. But I have never used the interactive questions for setting grades. I don’t know who or how it is done when on the web. All students must first interactively correct the first assignment and thereafter send me their solutions. Because the web process almost always helped them get the right answer, correcting their paper solutions was easy and fast. It was mainly a question of seeing how they expressed themselves: can I follow their line of thought, are expressions written correctly, are the units right? For adult students with well done solutions I said that they could in the following assignments chose if they wanted extra feedback on their written solutions. For the ones that had sketchy solutions, formal mistakes, missing units, I said they must continue to send in their paper solutions after web processing.
Other question types
The tutorial algebraic question type could only give CORRECT or INCORRECT feedback, though one could write the feedback as one wanted. It lacked the ability to give feedback for expected mistakes. It was good at checking equivalent answers: a+b/c or (ac+b)/c or b/c + a .
Some questions were problematical to make interactive. Quadratic equations with 2 answers. Graphs and other drawings.
I first had a java applet that could allow a student to draw in a webpage (even on a background picture) and that was automatically sent to me. Then I found PAD (Physics applets for drawing). These were similar to Physlets (and can interact with them). They were also similar to tutorial questions. Students could draw (graphic curves, vectors, lines, bars). Their characteristics were compared with various conditions and could give feedback for each condition as well as a CORRECT if all conditions were met. Though initially made for physics, it is useful in maths and other subjects. Draw arrows in a water cycle picture.
I discovered Moodle in connection with trying to import learning objects from our repository. When I tried out their quiz questions I was ”pleasantly surprised” to say the least. Almost all question types had possibilities for rich feedback even for answers other than the expected. At the same time an active Moodler presented his Regular Expression Shortanswer question type. That was something that I had been thinking of trying to make for several years. And he had done it so well too. You could give feedback for various patterns and for patterns that were missing.
There were numerical questions that were in some ways good (had some support for units, could recognize various answer formats including decimal sign =”,” and intervals). But it lacked the possibility to write in other answers for other feedback and the unit support was not consistent and feedback ”all or nothing”. I started using Moodle anyhow, though I used the Embedded questions numerical type, because there one could have feedback for several intervals. The embedded questions are a ”bit geeky” to work with, but have a lot of possibilities. Anyhow the nice things about working in Moodle were: the great statistics, the ease with which one could update things online plus all the exciting other possibilities in other modules of Moodle. The things I miss(ed):
Being able to easily make feedback for different intervals. (From version 1.7[?] the Moodle numerical question interface supports this.)
Inviting page/question feedback is difficult. My feedback JS can be made to work, but the link back to the page is useless probably because it is session dependent. So to get something usable one would have to make some PHP/JS? module that hooks into Moodle.