This is what the doc looks like after a week of students working on it. There has been a flurry of activity in the last two or three days. I have already edited the first two categories (Mathematical Challenge and Solution). The discussion pointed to the students desire to have Solution and Annotation condensed into a single category so that's what I did. The really interesting stuff is at the bottom of the doc. A student added a "Comments" section and there's a well thought out discussion about how the categories of "Presentation" and "Creativity" should be incorporated into the final document.

I'll also be publishing (soon) the assignment as it was published on the student's blogs prior to our collaboration (over 60 people including the three mentors) on this rubric. We used Google Docs to create this assessment tool.

It may be difficult to read some of the text against the background of the blog. (Use your mouse to "select" the text; that'll probably make it easier to read ... that's what I do. ;-) ) Different authours identified themselves by using different coloured text and/or differently highlighted text. As with the flickr rubric the document is not linear; different people added different content at different times and places. Also, Lani was using the comment feature of Google Docs to add her thoughts and these comments are automatically removed when Google Docs publishes directly to a blog, which is what was done here. (These notes were added afterwards by editing the blog post.)

---

Developing Expert Voices Rubric...

To All My Students
I've started putting together a rubric to grade your Developing Expert Voices assignments. I'm opening this document to all of you to help build the rubric. So far I've included 4 categories (see below).


Some Questions: What other categories should we have?
Should it be marked on a 5 point scale (0 to 4) or just "done" or "not done?"
I've included some suggested weightings for each category. Are they appropriate? Should one (or more) of them be increased/decreased?
Should there be marks assigned for creativity? Should "creativity" be another category? How would you describe the various "levels" of creativity? Maybe creativity should just be for bonus marks. What do you think?

Feel free to add to this document and make whatever changes you like. Together we'll build a better assignment. ;-)

To Do List
• We need to have "descriptions" for each "level" within each of the classifications below. Each description should be brief but include enough detail so that we all know what "excellence" does (and doesn't) look like.
• You can change anything and everything: Rename the levels, add/delete or change the classifications, change the grade weightings, etc.
• I've included some of my initial thoughts about what kind of things we should incorporate into each description. These are suggestions only. Feel free to expand on any or all of it. These need to be worked into descriptions of the various levels of achievement outlined below.

When we are done we should have something that looks like this:
http://adifference.blogspot.com/2006/12/flickr-assignment-rubric-v10-were-out.html

Levels
Instead of levels 1-4 (lowest to highest) we will use these descriptors. They better describe what this project is all about.
Novice: a person who is new to the circumstances, work, etc., in which he or she is placed
Apprentice: to bind to or place with a master craftsman, or the like, for instruction in a trade.
Journeyperson: any experienced, competent but routine worker or performer.
Expert: possessing special skill or knowledge; trained by practice; skillful or skilled



RUBRIC CLASSIFICATIONS (percentages indicate weighting of total grade)

The teaching of mathematical concepts is the main reason for this project; so we can teach other people and at the same time learn.

Mathematical Challenge (25%)__________________________
- breadth of topics
- appropriate level of difficulty
- problems are "doable"
Novice: Problems illustrate only an introductory knowledge of subject. They may be unsolvable. The solutions to the problems are obvious or easy to find. They do not demonstrate mastery of the subject matter.
Apprentice: Problems are routine, requiring only modest effort/knowledge. The scope of the problems does not demonstrate the breadth of knowledge the student should have acquired at this stage of their learning.

Journeyperson: Not all the problems are "routine" in nature. They span an appropriate breadth of material. At least one problem requires careful thought such as consideration of a special case or combines concepts from more than one unit. Showcases the writer's skill in solving routine mathematical problems.
Expert: Problems span more than one unit worth of material. All problems are non-routine. Every problem includes content from at least two different units. Problems created demonstrate mastery of the subject matter. Showcases the writer's skill in solving challenging mathematical problems.

Solutions (55%)__________________________

- notation
- all steps clearly delineated
- easy to follow
- mathematical work is correct
Novice: One or more solutions contain several errors with not enough detail to understand what's going on. Explanation does not "flow," may not be in sequential order and does not sufficiently explain the problem. May have improper mathematical notation.
Apprentice: One or more solutions have a few errors but are understandable. Explanation may "flow" well but only vaguely explains one or more problems. Some parts of one or more solutions are difficult to follow. May include improper use of mathematical notation.
Journeyperson: All solutions are correct and easy to understand. Very few or no minor errors. Explanation "flows" well and explains the problems step by step. Solution is broken down well and explained in a way that makes it easy to follow. May have minor use of improper mathematical notation. May point out other ways of solving one or more problems as well.
Expert: All solutions correct, understandable and highly detailed. No errors. Explanation "flows" well, explains the problem thoroughly and points out other ways of solving at least two problems as well.


Presentation (20%)

__________________________
_I think this should be worth only 10% because the presentation does not have to do with a person's grasp of the concept and they may already be receiving bonus marks for there creativity. I agree with Kasia on that. (Aichelle)

I second the motion. (John D.)

I think that presentation should be worth 40%. If your work doesn't captivate the reader then it will not be read. What will mathematical challenge and solution matter if they are not even looked at? (Grey-M)

Hmm.. putting some thought into it, the presentation should be worth 20%. I kinda agree with Grey-M's point because if the presentation is dull, it won't grab much attention. It shouldn't really be worth a lot since we're looking at how much the person understands the concept. (John D.)

It would still be 60 percent for the math and the solution. I think that is plenty. (Grey-M)
I don't think that 60% is quite plenty. But then again, we would put a lot more effort on our projects, making our projects look presentable and professional. I'm still sticking to 20%. Or maybe 20%-30%. And we're already getting bonus marks for creativity. (John D.)

I don't know about you, but if I was looking at a couple projects I would read the one that looks more professional over one that looks amateurish. Then the content, though superior, of the more drab one is lost to the limitless void of the internet or as Mr.K would say the project would die and I believe the whole point of this being on the web is so that our projects won't die. (Grey-M)

Ricky: Actually I agree with Kasia and John with this because for me presentation is just the final touches to make the project look right. Even if the presentation is superior, the internet community, depends on who are you actually looking at it, say a grade 8 person has no idea what there looking at and they look at the presentation aspect and say it's awsome but the mathamatics are not correct. A university student would be looking more at the mathamatical aspect and would be judging the math, not the presentaion of it. Putting Presentation at such a high mark is not a real good option beacuse this is a mathamatical project not an art project and should have a higher concentration on mathamatics in general instead on how it looks. I'd rather have it at a 20% mark. And we also have bonus marks for Creativity so it works for me.

Ten say:Well i think Presentation 's mark is good , because if you can presentting your project is a really good way to learn

Great point Graeme, I completely agree. A project that is going to be viewed on the internet must have some visual appeal to it, otherwise it will be overlooked. Not that it is the main focus of the project, but if it is worth more, it will ensure that we will put a lot more effort into it. But John is correct the project focuses on how well we can teach what we know, not how pretty it is. My number is: 20%. («Craig»)

- doesn't detract from content
- holds learner's interest
- aids in understanding content
Novice:The project is presented without the help of any media tools and barely stands out from the majority due to low level of creativity. (Aichelle) the project is given some effort. John D. - The project doesn't quite hold the learner's interest, dull, and not much of an effort was given. The project is presented with very little, if any, effort and the reader either is not attracted to it initially or becomes uninterested easily. "Took the easy way out" in terms of online presentation (media tools, etc.). («Craig»)Ricky: Project is presented in blogger with help of minor visuals using tools like paint for example. Also, work seems repetive and boring to readers.
Apprentice:The project is presented through a single media tool and only entices a small group of people (minor captivation). Creativity is sufficient but not overwhelming.(Aichelle) the project is given effort. Ricky: Project is presented in blogger with help from online tools like fooplot and slideshare, showing some knowledge of online presentation. Presentation is also somewhat intresting and holds details that keeps readers intrested
Journeyperson:The project is presented through different types of media tools and captivate the majority of the viewers due to a high level of creativity. There is XFACTOR but it's not enough to completely deliver the message.(Aichelle) the project is given good effort. Ricky: Project is presented in blogger but uses a wide variety of tools to create a unique presentation access. People are intrested in what's going on in the problems and creates an atmosphere that wants readers to come back and read again.
Expert:The project displays an immense use of creativity and is presented through several types of media tools that entice the people to view the project. The level of presentation and creativity further aid in the understanding of the desired solution to the problem. It makes you say, "WOOOOWW!" (Aichelle) project is given excellent effort. John D. - A lot of media/learning tools were given, learners are definitely interested. The learners fully understand the content given and are captivated. Ricky: Project is presented in such a way that wows the audience to come back for more, it also is represented in tools that work with the project in general, and is organized in blogger from top to bottom. It also helps the other aspects of the project and makes the project look extrodinary well.

COMMENTS__________________________

Vincent says: I suppose we can associate CREATIVITY with PRESENTATION due to the fact that if something is creative, it is very presentable. What attracts attention is CREATIVITY and this will determine the level of presentation required. On the other hand, if something is plain and bland, it is not considered PRESENTABLE thus, it is not creative at all. If this doesn't make sense, then I'll just say that CREATIVITY exposes the level of PRESENTATION of the project. Therefore, they should be combined and cateogrized as one. I also suggest that MATHEMATICAL CHALLENGE should be just 20% due to the fact not everyone knows their MATH level yet. Besides, if a quarter of the mark is based on the level of the challenge, this will require further explanation of WHICH is considered a challenging problem. Also, i think that SOLUTION AND ANNOTATION should both equal 50% because they are almost synonymous. Therefore, there is no reason for us to separate the two categories. 55% is a bit too much and is an odd number. It is not a very nice whole number. 50 is a good number. This leaves us with PRESENTATION and CREATIVITY. If we have decided that they should be two different categories, PRESENTATION should be worth 15% and CREATIVITY should be worth 15% or if we have decided they should be combined, then simply say, XFACTOR PRESENTATION = 30%. Therefore, MATH CHALLENGE (20%) + SOLUTIONS and ANNOTATION (50%) + CREATIVITY and PRESENTATION (30%) = 100%. Bonus marks are in Mr. Kuropatwa's discretion.

Mark: I agree with what Vincent said. I think that is a good idea to put most of the percentage into Solution and annotation because it is what shows our own mastery about that topic.

Ricky: For presentation... What if we are colaborating with the other group?? How will that effect the project in general?
I also think that solution and annotation should be combined b/c you should explain the steps involved with solving while you solve the problemMr. K. says: I think this is a good idea. We just have to include level descriptors that include this. Also, how shall we weight the mark? If "solution" and "annotation" are combined should it be worth 55% of the mark? How would you redistribute that?I agree with this. If they are combined it would be good to make it a slight majority of the marks. 55% is my magic number, under the category SOLUTION!. For the rubric of it, combine the Novice, Apprentice, Journeyperson, and Expert from each of the two categories to make one long explanation for each level or work. («Craig»)

If we do collaborate with another class then we can update the rubric to incorporate that, meybe even invite the students in the other class to help us figure out the rubric together.

MArk Says: I think that we should add a creativity category so that people will add more effort on their projects. I believe that the creativity category will bring our creative side out and even make this project interesting.
Aichelle says: I agree wth Mark I think there should be a creativity category or maybe it could go towards bonus marks.

What might the descriptors look like for a creativity category? What words might be appropriate? Laninhall 3/21/07 9:09 PM

Jennifer says: I also agree with mark about the creativity category. But Mr. K, you mentioned something about if people are afraid to experiment with the "presentation" part of the assignment, there will be these grade 9 students who would help us do the presentation part if we are interested. Would that boost up our mark for collaborating with these students, or would it lower our mark because we got help from these students?

Mr. K says: If we end up outsourcing your work to New York we can look at these descriptors again but my first reactions is this: You won't be penalized for outsourcing the presentation but it still must meet our class' standard for excellence in the way it is presented. If you are outsourcing your presentation then you have to oversee how it is done to ensure that the standard of excellence is met. Just because you are not putting together the presenation yourself doesn't mean you are not responsible for how your work gets presented.

Kasia says:In my opinion the creativity should be for bonus marks. It isn't the creativity we are being marked on, it is our understanding of the concepts and ability to explain them to others through our problems.

I don't think the category of creativity should be mandatory. I think it should just go towards bonus marks. In our scribe posts, some people are more creative than others... but it doesn't mean that they don't understand what's going on. Some people like to keep things simple, and others like to be creative with their scribe posts... so I think creativity should be bonus. If we make a category of creativity, then everyone will be required to be creative I think that being creative should be of your own will.. because in order to be creative, you have to do something without being told, and being creative is something you do unexpectedly.

John D. says: Creativity should just be for bonus marks. We are being marked based on how much we understand the concept and our ability to explain the problem to others, not on how pretty our project is... could be up to 10%.

OK, after all of the above how does this sound: I pasted the "creativity" description from the flickr rubric where it's worth bonus marks to a maximum of 5%. That means that the highest possible mark on the assignment is 105%. Should just paste this into this rubric as well? Lani asked about what language should we use to describe what creative work looks like. How does this sound? It was written for the flickr assignment, it needs some editing to reflect the work we are doing in this assignment:

Creativity (up to 5% bonus)

The maximum possible mark for this assignment is 105%. You can earn up to 5% bonus marks for being creative in the way you approach this assignment. This is not a rigidly defined category and is open to interpretation. You can earn this bonus if your work can be described in one or more of these ways:

• unique and creative way of looking at the world, not something you'd usually think of;
• if the problem does have roots (corny math pun, couldn't resist) in the real world, then does it have any applications in real world problems? Would it have uses in industry or the sciences? This is important because what good is math if it does not have use in a real world setting in some way?
• original and expressive;
  
• imaginative;
• fresh and unusual;
• a truly original approach. (method is unique, presented in a way no one would expect, eg. song, movie, etc.) («Craig»)
  

Ten say: I think this is fair tho, its just 5 percent creativity ,because everyone is different and everyone does the project in a different way, so this mark should be hard to get