Does Microsoft Learning listen?

I often tell people who are worried about taking Microsoft exams for fear of failing that I have failed more exams than most people have ever taken – to date over twenty failures, including one miserable exam that I only passed on my fourth attempt.

Don’t worry, I do pass more than I fail.  Something else that I often tell people is that there is a lot that you can learn from failing a certification exam – if you are only willing to learn from it.  Every time I fail an exam I try to remember the questions that have me stymied, and as soon as I walk out of the exam room I write down whatever I can remember so that I can look them up later on.  That is how I learned the ocsetup command.  In fact, that is how I learned a lot of the PowerShell cmdlets and command-line switches that I use.

Every exam – in fact, every exam question – gives you the opportunity to comment.  Unfortunately I am always reticent; I know that I don’t know everything, and frankly I used to be worried that if I commented that I thought something was wrong, I would be showing my ignorance.  This phobia doubled as a ridiculous assumption that someone at Microsoft Learning cares how much Mitch Garvis knows or doesn’t know… beyond the obvious pass/fail requirement of the exam.

I have heard people tell me that they don’t leave comments because they don’t think that people actually read those comments.  I can tell you with absolute certainty that someone does… most of the time that someone is a wonderful woman named Liberty Munson.

I first met Liberty in 2006, and have often referred to her as a witch – not because she is anything other than kind and warm, but because I am reasonably sure she can sit an exam on any subject matter in the world with complete ignorance on the subject… and pass.  You see, Liberty is a Psychometrician – an expert in exams.  Although I still don’t like commenting on exams, occasionally I have a question or comment about something on an exam, and I ask Liberty.

I took issue with a particular question on exam 74-409 (Server Virtualization with Windows Server Hyper-V and System Center).  I walked out of the exam with a sigh of relief at 10:45am… and at 4:30 that issue was still nagging at me.  I sent an e-mail about it and made my point.

Within an hour I got a reply from Liberty – I had not actually taken my issue to her, but rather to a friend at Microsoft Learning, who passed it on for me.  Nonetheless Liberty replied directly.  In the hour since I had sent my ‘complaint’, she had tracked down the exact question I was referring to, figured out who had written it, and gotten a response from the item writer.  While I completely disagreed with the response (it was not a right/wrong argument, rather a in-scope/out-of-scope argument), I was thrilled that I got the response.

Now here’s the thing… we can’t all e-mail Liberty every time we disagree with a question on an exam.  That is simply not how it works.  The best way to contact her if you don’t agree with a question is to click that Comment button on your exams.  You may not (read: will not) get a direct reply, but is the best way for you to help the Learning Experiences team to maintain the highest level of quality in their exams.

By the way, while I do not know this for sure from personal experience, my best advice is to never play cards with a psychometrician for money… it will never end well Smile


2 thoughts on “Does Microsoft Learning listen?

  1. A little insight from the other side of the fence.
    I have taken Microsoft exams and I have created Microsoft exams and a little insight in to how an exam is created would be useful. When Microsoft decides to create an exam they get a group of SME’s together and they block out what knowledge is needed to pass the exam and how much weight each part gets. They may go so far as to create a few questions, but their job is to basicly outline the exam and give weights to each area of the exam so the final product uniformly covers the subject area. Another group of SMEs will try to refine that list. Another would create sample questions. Another would rate those questions and suggest changes or additions. Another group would make sure that the questions covered the blocks accurately. So you could create an exam by randomly selecting questions from each block where each block got the same weight as was originally decided. Finally there will be several rounds of trial tests and the feedback is used to improve the actual exam.
    Many SMEs go in to creating an exam and it takes many days for each get together to do their part. These get togethers also include experts in creating exams to help the SMEs.
    One rule from the exam experts to the SMEs was that they were not to create trick questions, difficult yes, but not trick. The valid answers had to be correct and the invalid answers had to be plausible, but wrong. No wishy-washy answers. (Working on creating the invalid answers was hard.) A second rule was that if you knew the subject matter you should be able to get 60%.
    So the chances are low the question is wrong, but SMEs are not perfect and the question could be better written. It is also possible that you did find something the SMEs overlooked. Remember you are taking the exam because you consider yourself an SME or a soon to be SME.
    SME – Subject Matter Expert

    • John I agree with you… This question was not WRONG, it was just out of scope 🙂 I’ve been involved in exam creation before, and I was surprised that this question was not batted down. -MDG

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s