Incentive systems and how they are immediately gamed

I was annoyed recently with my two boys, ages 8 and 11, who were bickering non-stop about trivialities. So I came up with a "system" on the spur of the moment. The system was, that for every negative thing they said to one another, they had to say 5 positive things.

Well, that worked about as well as I might have imagined, had I thought about it for more than 5 seconds.

Kenny:  "Peter, you're beautiful, you're beautiful, you're beautiful, you're beautiful, you're beautiful!
Okay, now I get to say something mean!"

Bottom line - any time a system is created, the VERY FIRST thought that most people subject to this system have, is how to game it.

Another example - I was reading some forum posts recently on the Mr Money Mustache site, which I check out frequently. There's lots of interesting stuff there.  I read a post recently about a workplace that instituted a reward program to get people to stop smoking. The reward for stopping smoking was substantial - around $700. The forum discussion was about smoking for a day, then quitting, in order to be able to claim the $700 reward for quitting smoking. The consensus appeared to be - look very carefully at the details, and make sure your health insurance rates wouldn't go up, etc., etc., but once you've done that, then go for it.

I had a similar situation come up recently. A company that I worked for offered a $1000 wellness bonus, that could be used for health club membership or exercise classes. I'm generally in good shape, and am also diligent about exercising on my own, without a health club or exercise classes. It's far more convenient to exercise that way. But because I didn't want to do group classes, there was no way for me to get that $1000. And that really pissed me off!

UPDATE - Why I'm not longer interested in solving challenges for Innocentive

I wrote a blog post a few months ago about entering problem solving competitions on the crowd-sourcing website Innocentive - Exercise your brain by solving problems for Innocentive. I was psyched about working on the challenges back then, and entered a bunch of Innocentive competitions. I didn't win any of them - so you could potentially call this a case of sour grapes - but in any case, I'm considerably less enthusiastic now. There's a few problems that I see with the whole setup.

The most recent challenge I thought seriously about entering for was this one: Seeking Improved Document Identification and Verification Methods. It's about automatically identifying documents that are submitted by customers (usually financial records that are scanned in and uploaded).

It sounded interesting, and I put many hours of thought into it. But when reviewing the challenge, there were many, many unanswered questions. There's a forum for asking questions on Innocentive, so I did. But the answers I received were perfunctory and had no depth. It struck me that for a question of this complexity, involving all kinds of internal company processes and systems, you can't write a single page document presenting a problem. Solving this kind of problem requires you have full and complete access to the details of a problem - potentially sitting at a table with people who understand the problem, or experiencing it yourself. Then you make proposals, set up trials, do experiments, do some tinkering. That's how you come up with more ideas to test, and more potential solutions.

The limited information that you have about the challenge, the lack of interaction with the people that have presented the challenge, and the lack of feedback on the actual challenge - and the fact that you hear nothing back about your own solution, or the solution that was chosen - that makes the whole experience less than satisfying.

Interesting - I found this post on LinkedIn when I was doing some research, which goes into the same points that I made above. The writer actually received a few Innocentive awards, though.

My first android app with MIT App Inventor - what a blast!

I'd been thinking for quite some time that I'd like to write a simple Android app, just to see what it's like. And what do you know - it's a lot of fun!

I was tired of the timer app that I've been using for a long time - it takes too much work to start it up, and then it has annoying ads that keep the screen turned on. So, I decided to write my own. Also, a timer app would be one of the simplest I could write.

I used the MIT App Inventor program. It uses a click-and-drag programming model which is easy to get into, but has all the disadvantage of graphical programming frameworks - great for small apps, but burdensome when it gets more complicated.

I specifically designed this timer app to be very simple, and called it Timer - Super Simple. It has just 2 buttons - Add 5 minutes, and Clear. And there's a display that shows the amount of time left. This is probably not an app many people would want to use, because the time can only be increased in 5 minute increments. But it's what I wanted.

Here's a link to download the app: https://sites.google.com/site/timersupersimple/

And also, here's a link to my very first video screencast that I uploaded to YouTube. I was having a hard time figuring out how to center things in App Inventor, so once I figured it out, I decided to make a video for it, using the free version of Screencast-o-matic. But it turns out there's a much easier way of doing it, so I made the video unlisted.

Post mortem—teaching an online database programming class

I was very excited when I got an email from the University of Washington Professional and Continuing Education program last year, asking me to be one of the instructors in the SQL Server Developer's Certificate program for Spring 2015. I jumped on the opportunity.

I've finished the class now - it's a big relief to finish it.  It was a great experience in terms of doing something very different from what I normally do, and also, teaching a class was on my lifetime bucket list. But there was also a lot of stress involved, and far, far more work than I anticipated originally.

Any experienced teacher will laugh at this, but I calculated the hourly pay based on the number of hours I needed to be online, doing the class session, which was three hours a week. I completely ignored the amount of prep work that I'd need to do, the interactions with the students, and the grading. All of these took a tremendous amount of time.

The prep work in particular usually took more than a full day, for every single session. I had a set of PowerPoint slides from the previous instructor, and the assignments. But, I thought the flow was not as good as it could have been, and there was too much emphasis on things that weren't very important, and not enough on things that were. So I decided to redo it. I still think that was a good decision, but it made preparation far more time consuming - I was almost starting from a blank slate.

For the textbook, I ended up choosing the most highly ranked book on SQL Server database programming from Amazon. It would have been great as a reference book, but it didn't work very well as the basis for my class, because it didn't proceed in a logical sequence for a class, and also grouped a lot of topics together that would have been better split up into separate chapters. So, that was my first mistake, and it continued to cause problems throughout the class.

I thought I would have more flexibility in the class structure than I ended up with. I ended up having to be online, presenting the lesson, for about 3 hours, every session. I tried to make it as interactive as possible, with labs, quizzes, etc. However, 3 hours is a very long time to talk, even with breaks! I had to prepare a tremendous amount of material, and be prepared with cough drops and water bottles for my throat. I, as the instructor, was talking most of the time, and the students were interacting via the chat window, or the quizzes. I felt bad for the students, because I know it must have been boring for them to listen to me for that long, no matter how I tried to jazz it up.

Education is changing quickly, that's for sure. In the short lifetime of this certificate program, about 7 years, it started out as an in-person class, then went to a combination in-person/online class, and now it's entirely online. I'm sure that transition to online was driven by student demand to not need to commute to congested locations, with scarce parking, during rush hour. It makes total sense.

However, they've preserved some aspects of the in-person class that don't make sense anymore - namely the 3 hour segments where the students are required to be online, listening to me give my presentation. It's boring for them, difficult for me, and there are much better ways to absorb information. I think a better route would be to have a very well thought out syllabus, textbook, selection of online videos and reading, and most importantly, labs and assignments that have deadlines, and are carefully graded. Then, have 1 or 2 optional online office hours a week, where you cover questions and review assignments.

Another downside of online learning in general is that it's hard to develop a community feeling, to connect with people. For me, that was one of the main drawbacks. I felt like I was talking to an empty room much of the time, and it was hard to feel connected with the students when I only got emails from them, and saw comments and questions written by them in the chat window of the class session. I imagine the students felt the same. It's not like you're going to make friends easily in an online class, unlike in a regular class. I'm not sure that I have any bright ideas to fix that problem, though.

The actual course management was done in a software program called Moodle. It was a real pain to use, the user interface was very clunky and non-intuitive, with a steep learning curve. I did eventually get used to it, and figured out the best way of setting up the assignments, and the interface for the class. It felt like the software hadn't been updated in about 10 years, though.

The online class sessions (as opposed to the classroom management website) were hosted in Adobe Connect. That was another software package that I'd prefer to never use again. I had some major connectivity issues - almost every time I did screen sharing, I had a random connection problem, and the screen share on the students side turned black. After the first few times, I realized that exiting and re-entering the online session seemed to fix the problem  But it was very disruptive for the students, and completely stressed me out. I had some discussions with the previous instructors before I started teaching, and the clunky software used for the class was one of the things they warned me about.

The courses weren't cheap for the students. There were three courses in the certificate program, at about $1000 each, so a total of $3000. What were the students buying? The information that I was giving them is out there online—more current and complete than what I gave them. Also, they could get a book for less than $50 that covered the same information.

Here's what I figured. The students are buying the following:

• The prestige of being able to say that they completed a certificate in SQL Server Development at the University of Washington Professional and Continuing Education
• Having somebody curate and manage what they need to learn - although everything they need to learn is available for much less elsewhere, having someone knowledgeable slice through the jungle of information and pick what's critical is very helpful.
• Here's the critical one - the pressure to perform. I set homework assignments every week, and graded them stringently. If those assignments weren't completed by a particular time, the students would lose points.  

There were a couple things that surprised me. One was that there was a very broad range of skills in the class. On the high end, I had people with 15 years experience with relational databases, just not Microsoft SQL Server. On the low end, I had complete novices who didn't understand many of the basics, even though they'd been through the introductory course already. So even though most of them got certificates, they have very different skill levels.

Also, trying to make the course useful to students of these varying skills levels - i.e differentiation - is very difficult. In reading about education in general, I had come across this concept of differentiation, and how that's something that teachers should be doing more of. But it's really difficult. What I ended up doing is creating the regular assignments for each lesson, and then also creating what I called bonus assignments, which were optional, but had much more challenging materials. The problem was that since they were optional and didn't affect the grade, only the absolute most motivated students completed them, which ended up being very few - from about 1 to 4 students a week, out of a total of 29 students. The lesson being - if you don't HAVE to do it, most students will NOT do it, even though it would be very useful to them. 

My stress level when teaching the class was sky-high. I was way out of my comfort zone - which is good, occasionally - but I was quite anxious. That faded towards, the latter half of the class, thank goodness. 

One last note - I had 6 women out of 29 students total. However, they were my most diligent students by far, in terms of completing or attempting the bonus assignments. I don't know if that's anything that could be generalized, but that's how it was for this class.