Final Writeup

Summary:
The question my report set out to answer was "can some of the core concepts of parallelism be taught to people with no programming experience?"  I tested two populations, one consisting of CMU students and one consisting of people from the internet via Mechanical Turk, and found moderately positive results.

Approach:

Overall, I tried to teach the following topics, using a very analogy-heavy lesson to make it easy to understand from a non-programming perspective:

Speedup
Challenges of Parallelism:
   Communication Overhead
   Workload Imbalance (*)
   Inherently Sequential Portions of a Task
Pipelining
Deadlock/Livelock(*)/Starvation

(* = added in Phase Two)

Phase One:

In Phase One, I experimented with CMU students.  I had two populations, one who had programming experience but no systems background, and one who had little or no programming experience.  My methodology was to give them a 5 minute quiz, then teach a 5 minute lesson on parallelism, and then give the same quiz again.

Slides for my lesson: https://docs.google.com/presentation/d/1s54jTbYw8fX2KDjsAzvpwQ2ukfGhxhMLBpuGXC0a0mA/edit#slide=id.p  (these were not the focus of the lesson, just a visual aide, so they were pretty hastily done and mostly taken from 418 slides when those demonstrated a helpful analogy)

My quiz (this stayed the same for Phase Two): https://drive.google.com/file/d/0B_D9EQm0S0FPRjJKMUN2eFVhWEU/edit?usp=sharing

My script (this was mostly a guideline, I played it by ear so if people were getting a topic, I wouldn't go through every single detail of my script): https://drive.google.com/file/d/0B_D9EQm0S0FPanZZMTFuZDFhZ0k/edit?usp=sharing

Phase One was pretty helpful in developing my lesson, but it was inherently limited in the number of people I could test the lesson on, since they had to know me well enough that they'd be willing to take 15 minutes of their time on a busy week in exchange for just candy to help me.

I also observed that even among the non-programmers, my lesson was feeling a little light on content (which is why I added additional topics going to Phase 2 and fleshed out the existing ones a little more).  Another note was that all the people with programming experience successfully got the pipelining question right on their first try, compared to half of the non-programmers, and generally did much better.  My quiz tests parallel thinking more so than explicit memorization of concepts or anything programming-intensive, so it makes sense that even without systems background, people more used to problem solving would do better.

The sample was also a little un-diverse in the sense that even non-programmers at CMU pick up a lot of that way of thought from the prevalent culture and often the programming gen-ed that even humanities majors take.  To really test non-programmer performance, I would need to seek (*ominous drumroll*)...the internet.

Phase Two:

In Phase Two, I used Mechanical Turk to experiment.  I had participants take the quiz, then watch a Youtube video, then take the quiz again (they got paid 25 cents each).  There were multiple safeguards to make sure they at least attempted to learn - I said three letters at different points in the Youtube video that they needed to copy down to get paid for the task, and the place to enter that was before the second take of the quiz, so they at least had to listen to the video, if not understand it, to get credit for the task.

Youtube link: https://www.youtube.com/watch?v=_xqHr7ZxOuk

Results:

Overall, both groups seemed to improve.  There were a lot of 'eccentric' answers to the Mechanical Turk quiz, although only accepting correct confirmation codes filtered the most insane gibberish out.  I had 8 answers to the in-person version, and 32 at least somewhat reasonable answers to the Mechanical Turk version.

Some simple measurements:

6/8 before lesson, 8/8 after lesson of CMU sample got a basic pipelining question right (part c of the Cake question), versus 9/32 before lesson vs. 11/32 after of the MTurk sample.

7/8 both before and after lesson of CMU sample got a basic question testing how completely sequential portions of the work affect its parallelism right (part a of the Cake question).  MTurk results for that: 16/32 correct before lesson, 21/32 correct after lesson.

For the most thorough measurement, we need to compare the quiz averages (out of 10) before and after:

CMU pre-average (programmer): 8
CMU post-average (programmer): 8.8
CMU pre-average (non-programmer): 3.67
CMU post-average (non-programmer): 7.67
MTurk pre-average: 3.13
MTurk post-average: 3.26

Statistics aside, it definitely seemed like both populations learned at least a little from the lesson.  Many of the MTurk second-attempt quizzes referenced things like the cost of coordination and terms like deadlock, though not always correctly, and on the whole there was an improvement.

There are many, many future directions such a project could be taken.  Expanding the project to try to test the lesson on specific groups like younger kids, or the elderly, could be interesting.  Making a much more animated video could also be interesting since my current one is basically all spoken with a few visual aids.  The challenge in getting results is finding people who don't have much experience with these topics/ways of thinking (problem with the CMU sample) but are motivated to try their hardest to learn (problem with the MTurk sample).

Presentation: https://docs.google.com/presentation/d/1Or9wFOS6b4QOBKRL381jhkeFpaZpfGWa26eaY89idUE/edit?usp=sharing

Fun class, fun project!

Initial Results

In the end, I ended up running my in-person lesson on about 10 people, with the quiz before and after to test their progress.

I also got a video version of my lesson up and running, and then got people on Mechanical Turk to do the process of "take quiz, watch lesson, take quiz again".  (Using a confirmation code to make sure they actually watched the video)  Currently I have about 50 results from this.

I haven't fully analyzed the results from the quizzes yet, but it seems like overall, people improved in their understanding of the concepts and scored better the second time.

Progress Update

I've run my study on 6 people so far.  So far, most of my sample has been people who know programming but not parallelism concepts, so I'm going to look at the results and think about how to improve my lesson plan for that audience.  I'm also going to look for more people from the non-programmer camp, since I'm currently having trouble finding them compared to the first camp.

The other major direction I'm going to experiment with is trying to make a video with my lecture, post it online, and use Mechanical Turk to get a much larger audience than the convenience sample my real-life results are.

Project Proposal: Bringing Parallel Education to the Masses

The fundamental question my project seeks to answer is "Can many of the core concepts of parallel programming be taught to people without much background in systems programming, or even without any background in programming at all?"

While a lot of the material covered in 418 involves intensive programming, specific hardware details, and advanced reasoning about program performance, many of the core concepts and themes of the course are simple and elegant enough that I believe they can be taught to a non-technical audience.  There's still the question of "what will that audience get out of this lesson, if they never plan to write a program, let alone a parallel program?" but I think parallelism permeates enough of everyday life that even people who never write a program in their life can learn something useful.

The structure of my lessons will be:

A 5 minute quiz

A 10 minute lesson

The same quiz, again

I plan to run these lessons on both technical majors with no knowledge of parallel concepts and non-technical majors.  My sample will likely consist solely of CMU students for practical reasons, but a reach goal is to be able to run these experiments on high school students or an even wider demographic.  Since they'll take the same quiz before and after the lesson, I should be able to use the changes in performance to measure the effectiveness of my lesson.  To avoid the issue of performance improving because of "teaching to the test," I'll make sure that nothing included in my lesson directly relates to the quiz (The examples I use in instruction will be more than superficially different than the examples I ask as quiz questions)

I plan to cover these topics:

• Why parallelism is helpful/the concept of "speedup"
• Challenges posed in parallelism:
• Communication overhead
• Pieces of a problem that are inherently sequential
• Deadlock
• Starvation

My deliverables will be:

• The results of my tests with analysis of why I think they happened
• The lesson(s) I gave (if I go through multiple iterations of the lesson/quiz, I'll include this for each iteration):
• The quiz I used to measure performance
• The script I used for the lesson itself
• Any visual aids I use in the lesson

And my firm, will-definitely-be-completed goals are:

I will definitely run at least one iteration of my lesson on at least 5 technical majors and at least 5 non-technical majors and hand in all deliverables of that lesson as written above.

Stretch goals:

I'll try to run this on even more students than my firm goal of 10 people. 
I'll also try to contact the local high school and see if I can try this there.
If I get unsatisfactory results from my initial lesson plan after a few trials, I'll try to figure out its flaws, modify my lesson plan and try out the new plan on the remaining trials.

Day-by-day Timeline:

April 26: Develop lesson plan
April 27: Study for Parallel Exam :O
April 28-29: Develop/finalize lesson plan and find people to test it on
April 30-May 2: Test lesson plan
May 3: Modify lesson plan if necessary, otherwise begin analysis
May 3-5: Continue testing lesson plan
May 6: Study for and take a final (in another class)
May 7: Continue testing lesson plan and begin analysis
May 8: Begin writing up analysis, prepare presentation if chosen for the competition, study for a final in another class
May 9: THE MOMENT OF TRUTH