Home » Microsoft

Category Archives: Microsoft

A Big, HUGE Microsoft Security FAIL.

(NOTE: This article was written December 7, 2016. Not one word has been changed since that date.  To understand why it can only now be published, read the article on this site called 107 Days: A Microsoft Security Nightmare. -MDG)

For reasons that will become obvious, I am going to delay posting this article until the issue has been resolved.

A few days ago a colleague of mine discovered the password to my Microsoft Account.  I won’t go into the how and why… I knew that my password had been compromised and I took the immediate steps to change it.

image

Ok, I understand that things break… I tried a few times, and then I decided to follow the advice and try later.  I trust my colleague not to actually use my password, so even though I felt uncomfortable with it being compromised, I knew I could wait a couple of hours.

Throughout the evening I tried (unsuccessfully) to change my password.  As I was sitting with my father having dinner, as I had drinks and cigars with my friends… no joy, I still got the same message.  ‘There’s a temporary problem with the service. Please try again. If you continue to get this message, try again later.

I want to be clear… if my network had an error that was preventing users from changing their passwords I would consider it reasonably important, and I would take immediate steps to fix it.  But having trusted Microsoft for so many years, I assumed this would be fixed eventually.

Four Days Passed.

Yes, it was literally four days before I decided that my passivity would not eventually lead to a solution.  I sat down and figured out how to request support. I was hoping to be able to speak with a human being.  Before I could, however, the Virtual Support Assistant got me to try this link and that link.  It then made me go through seventeen steps to finally confirm that the account in question was mine… and once it confirmed that I really am me, it tried to reset my password… and I ended up with the same error message that ‘There’s a temporary problem with the service. Please try again. If you continue to get this message, try again later.

Okay, it’s been nearly an hour… and I am chatting with someone who is quite obviously not their first round draft pick.  After all, I asked for help with Outlook.com, not with something that people actually pay for.  I spent twenty minutes explaining to him the situation, and the added (and I assume rare) complication that I have two accounts with the same address… my Office 365 account and my Microsoft Account are both the same address that are completely different.  ‘Please don’t touch my Office 365 Account, I only want to change my Microsoft Account.’  This led to another five minute discussion on the meaning of the word change.

He had me fill out another form on-line.  I did.  At the end of that form I got a message that said that the product team would contact me within 24-48 hours to help me.  I told the Support Agent that I had filled out the form.  He told me that now I had to wait until they contacted me.

All in all, my Microsoft Account (which is the account I use for my MCT & MCP Benefits, Skype, and myriad other features) will have been compromised for the better part of a week… and there was nothing I could do about it.  Yes, I could have contacted Answer Desk a few days earlier, so it would have been compromised for only three days.  I want to know in what world is that considered an acceptable delay to be able to change a compromised password?

Some time ago I started using Multi-Factor Authentication (MFA) for many of my most important systems, which is why I am never concerned that my blog or my password vault could be compromised.  For various systems I have a hard key (Yubikey) and soft keys (Google Authenticator and Microsoft Authenticator) which keep most of what I do safe.  But most of the Microsoft systems do not support MFA and I am stuck with only a password.  I use reasonably complex passwords so I usually am not concerned, but in a case where my password is compromised and I am not able to change it, I wonder how it is that a company as advanced as Microsoft (in this case) does not allow me to use MFA.  I would love to be able to require my Yubikey in order to log in to Windows and many of the on-line systems I use, but it is simply not an option.

I am disappointed by Microsoft this week… and I hope that they take the lessons learned from this experience to improve.  However I sit here today, thinking of the myriad occasions I stood on stage in over a dozen countries on five continents and defended Microsoft’s security systems as among the best in the world; I was always sure in my knowledge that I spoke the truth.  Today I would not feel comfortable making that claim… and my faith in their systems, like shattered glass, will not be easily fixed.

107 Days: A Microsoft Security Nightmare

I have held off talking about something for quite some time.  I do not mess around when it comes to security, especially for my critical accounts. When the actual security of an account has been compromised, as was the case with my Microsoft Account, I do not advertise it. 

On December 7th I sat in the Second Cup cafe on Bank Street in Ottawa and wrote an article called A Big, HUGE Microsoft Security FAIL.  I wrote about how I had been unable to change my password and that their engine to do so was broken, but that it turned out it was not everyone, it was just me.

There’s a temporary problem with the service. Please try again. If you continue to get this message, try again later.

It took several days for anyone at Microsoft to take me seriously, but my issue was finally escalated to a 2nd Level Support Tech named Gary (who, I want to be clear, was a nice guy, and as helpful as he could be under the circumstances).  Gary and I got to know each other sometime in mid-December.  Remember, the issue started happening the last days of November, I reported it on December 5th, and the case was escalated (grudgingly) around December 9th.

Gary spent a couple of hours trying to help, and then in discussions with the Product Engineering Team trying to get it fixed.  By the end of the day he said something to the effect of: “Yeah, neither I not our engineers have ever seen a problem like this.  It might take some time, but if you are willing we will work with you to get it fixed.”

Some time… It might take some time… that was on or about December 9th.

I am happy to say that the problem has now been resolved… As I sit and write this, with the resolution less than an hour old, it is 1:15pm, Wednesday March 22.

107I spent nearly a decade touting the virtues of Microsoft’s security… and then from the day I informed them that my password had been compromised, and that a glitch in their system was preventing me from changing it, it took 107 days to resolve the issue.

So let’s take a quick rundown of some of the sites and services that are accessed with my Microsoft Account:

  • Skype (One of the ways I communicate with hundreds of people)
  • OneDrive (All of my files!)
  • Microsoft Certified Trainer (MCT) account (including my MCT renewal, courseware downloads)
  • Microsoft Certified Professional (including my MCP Transcripts)
  • MSDN Subscription (including all my software licenses and keys)
  • Windows Store (including credit card information)
  • Microsoft Volume License Center (VLC)
  • Microsoft Store (including credit card information)
  • Bing
  • Microsoft Partner Portal
  • MSN
  • Outlook/Hotmail
  • MY WINDOWS COMPUTERS

And so, you can see, this is not like having my Words With Friends account compromised… This is extremely serious and far-reaching.  This was… everything.

Once a week I would get an e-mail from Gary telling me that they had not yet resolved the issue… but they were still working on it, and he would continue to keep me informed.

Proof Of Hack 2On March 6th a hacker compromised my Skype account, and sent a link to dozens of my contacts with malicious content.  Naturally those contacts let me know, and I reached out to Gary and told him that now that hackers had indeed compromised my account, they needed to resolve the issue and pronto.  Gary replied with: “I have taken a look into your account, to look for any evidence of unauthorized access, and I did not see any. Was any account info changed? Can you still login?”

a few days later that he had not been able to open the embedded picture, and asked that I resend it as an attachment.  Thank Heavens for that, because had he taken the next step immediately I would not have been able to renew my Microsoft Certified Trainer (MCT) credential in time.

So when Gary did finally get the picture (as seen above) he wrote (on March 18th):

If you received that message, then it could be that someone attempted to access the account.

To prevent that, I have placed a suspension on the account that will prevent any login activity. While my engineering team investigates this issue, no one will be able to break into the account. I have also left a note on the account so that the attacker will not be able to attempt to remove it.

Wonderful.  You are suspending my account now, probably after the damage has been done, but all this is doing is punishing me.  FIX THE DAMNED PROBLEM!

On the same day as I received this e-mail I wrote the following one line response:

Gary this is no longer acceptable. I am calling a lawyer.

On Tuesday (March 21) I received Gary’s reply:

In light of this recent reply, I have escalated this issue to a second team within Microsoft, and am awaiting to hear their response.

I understand the frustration, but please know that I cannot do anything to speed up the engineers and Ops teams working on this issue.

Wouldn’t you know it… The following day (that’s today, Wednesday March 22, 2017 – 107 days after I first reported the issue) I received a call from Gary that started with:

Well Mitch, it seems that when you threaten to call a lawyer things get done faster.  I think we have solved your problem.

Indeed, before the phone call ended I had successfully changed my password.

One hundred and seven days after I first reported the problem.

One hundred and seven days since I told Microsoft there was a problem with their security.

One hundred and seven days since I told Microsoft that my account had been compromised, that someone had my password, and that I needed their help to secure my data and reputation.

One hundred and seven days.  Actually it was only 105 days since I wrote the original article (which will be published shortly after this one, untouched since the original writing).

So why didn’t I publish sooner?

There are a handful… maybe four or five people who know the story and who understand some of my frustrations with this case.  These are also people who know I have a great bully pulpit in the form of this blog.  They have all asked me ‘Why didn’t you publish sooner?’  Two of them asked why I did not go to the mainstream technology media to let them know about this.

Simple… I have an account that is easy enough to guess, to which I could not change the password.  If the wrong people knew about that they would have focused on getting that password and, once they had it, they knew I couldn’t change it.  They would have literally owned me. 

And so I sat quietly, seemingly patiently, waiting for Microsoft to fix the problem.  I waited those 107 days knowing that when it was finally resolved I would a) breathe a big, huge sigh of relief, and b) sit down and write this piece, venting my facts and frustrations.

MICROSOFT! HOW DARE YOU? How can you let ANY problem, let alone one as serious as this, fester for so long unresolved?  Do you think you owe me nothing?  At this point I am still considering a lawsuit, and if you don’t think damaging my reputation and peace of mind is worth damages in a court of law then you are seriously misreading the system.  You should be ashamed of yourselves, and you should be tracking down who is responsible for this travesty, this shame, and firing them.

I got that off my chest.  I have, over the past two weeks, asked friends and colleagues for recommendations on lawyers.  I might just reach out to one this afternoon.  We’ll see.

Welcome to What’s Next…

There is irony in the title of this post… What’s next.

I posted on Friday that it was my last day working full time at Yakidoo.  I really enjoyed my time there, and am glad that my next venture will allow me to stay on there on a limited basis.

This afternoon I am meeting a colleague at the airport in Seattle, and that will begin my first day at my new gig.  I will talk more about it in a few weeks, even though today will be my first billable day.  That is what’s Next.

However the reason he and I will be in Seattle – Bellevue/Redmond actually – is the Airlift for Windows Server, System Center (WSSC), and Windows Azure vNext… the next generation of datacenter and cloud technologies that Microsoft is ‘showing off’ to select Enterprise customers several months prior to launching them.  It will be a week of deep-dive learning, combined with the usual Microsoft Marketing machine.  How do I know?  It’s not my first kick at the can Winking smile

It is, of course, not my first such Airlift.  The first one I attended was for System Center Configuration Manager (SCCM) 2007, back in November of that year. It was a consulting firm that had sent me, in advance of my heading off to Asia to teach it.  I have since been to a couple of others, each either as a consultant, a Microsoft MVP, or as a Virtual Technology Evangelist for Microsoft.  I have not given this a lot of thought, but this will be my first Airlift / pre-Launch event that I am attending as a customer.  It will be interesting to see if and how they treat me differently.

I suspect that the versions of WSSC that I will learn about this week will be the first that I will not be involved in presenting or evangelizing in any way dating back to Windows Server 2003.  I will not be creating content, I will not be working the Launch Events, and I will not be touring across Canada presenting the dog and pony show for Microsoft.  I will not be invited by the MVP Program to tour the user groups presenting Hyper-V, System Center, or Small or Essential Business Servers.  I will not be fronting for Microsoft showing off what is new, or glossing over what is wrong, or explaining business reasons behind technology decisions.  It is, in its way, a liberating feeling.  It is also a bit sad.

Don’t get me wrong… I will still be blogging about it.  Just because Microsoft does not want me in their MVP program does not mean that I will be betraying my readers, or the communities that I have helped to support over the years.  I will be writing about the technologies I learn about over the next week (I do not yet know if there will be an NDA or publication embargo) but at some point you will read about it here.  I will also, if invited, be glad to present to user groups and other community organizations… even if it will not be on behalf of (or sponsored by) Microsoft.  I was awarded the MVP because I was passionate about those things and helping communities… it was not the other way around.

What else can I say?  I am at the airport in Toronto, and my next article will be from one of my favourite cities in North America… see you in Seattle!

Skype Collaboration Project, and quick tips to detect Phishing scams

skypeA few days ago I posted a quick post called Free Skype Premium for a year! and I got a few interesting questions about the voracity of the offer.  Some of you were worried that it was a scam, and believe me I am the first person to say you should be skeptical.  However before I posted about it I checked it out.

As one reader noted, the first thing I look at is the domain name.  Behind all of the mess of incomprehensible much, every URL you browse to on the Internet will begin http:// or https:// and will be followed by child domains, sub-domains, and eventually the parent domain – for example I might, if we were larger, have a page with the URL http://we.all.love.mitch.garvis.ca/And?Wish/himAndHisFamily/?Well.  The actual domain is directly to the left of the first (really the third, but I don’t count the two in http://) slash.  So while a phishing scam might use http://24.200.34.12/we.all.love.mitch.garvis.ca/And?Wish/himAndHisFamily/?Well the first slash is after the numbers, which means that it is a scam.

Unless my domain name itself has been compromised – if someone has actually hijacked the DNS (Domain Naming Service) of garvis.ca they cannot create a child domain to it… so they could no easier use http://scam.garvis.ca than I could http://garvis.skype.com.

The link to the Skype offer was https://collaboration.skype.com/promotion/?cm_mmc=AFCJ%7C1250_B1-_-11129583-1225267.  To the left of the first slash is collaboration.skype.com.  This means it came from Skype.

imageNow let’s look at the next objection I got.  ‘This page does not have a Skype logo on it, it doesn’t have Skype’s (or Microsoft’s) branding or look and feel.  That is a big red flag.’

That is something that you would definitely take pause at… the only thing on this page that looks remotely like Skype is the colour they used for the top line.  While this is a good observation, it is something that I throw away when determining the legitimacy of the page.  I have examined hundreds or more phishing schemes and hijacked sites, ranging from banks to credit cards to e-commerce sites to the White House and United States Department of Justice.  No matter how good the look and feel might be, they cannot get beyond the technical – if the domain name is right, then either the domain itself has been compromised… or it’s legitimate.  In truth, most phishing scammers spend more time on look and feel because they understand that most of us would look for that first… if they wanted to hijack your Skype password, they would spend the time to make the site look legitimate, including the colour scheme and logo.

The next concern was that we entered our e-mail address, ticked the appropriate box, and clicked on SEND… and nothing happened.  We didn’t get an e-mail right away (or even ten minutes later) as we were promised.  Actually we were not promised that… we were told to ‘Look out for the voucher codes coming to your inbox in the next 48 hours…’  Well most of us are not that patient, and we assume that while some sites do claim it will take that long, we should really be getting something in the next fifteen minutes.

We shouldn’t be so impatient… especially when out of the blue someone offers us something for free.  In fact, I posted the article just after 1:00am on December 18th (EDT) and received the e-mail at 10:27am on December 20th (EDT)… so in fact it was closer to 58 hours than 48.  Okay, no problem… It was later than they promised, but it came.  Thank you Skype for giving me a free service that I will absolutely use, and I forgive your slight tardiness Smile.

image

imageAs I had been promised in the original invite, I saw this screen – my vouched was successfully redeemed.  However it did say that it would take no more than 15 minutes, so it could still be a scam, right?  Well a few seconds later the bottom-right corner of my screen popped up with this Notification… and I knew I had gotten what I asked for.  Thank you Skype, I will use the Premium services well!

Free Skype Premium for a year!

It looks like this is legitimate, although I will keep an eye out and let you know immediately if it isn’t.  Microsoft is giving away Skype Premium for a year.  Simply submit your e-mail address, and you will receive a code.  Click on the link for more info!

http://microsoft-news.com/get-premium-skype-features-free-for-one-year/

Thanks for your support!

Image representing Microsoft as depicted in Cr...

Image via CrunchBase

Over the past few days I have received an incredible number of you asking what happened, if I am okay, and if I will be alright.  I can assure you I am.  Let me explain.

A great many of you have known me as a Microsoft contractor.  I have been for quite some time, first as a Virtual Partner Technology Advisor, then as a Virtual Technical Evangelist, and most recently as a member of the Server and Tools Business.  So when e-mails to my @microsoft.com account started to bounce (Tuesday this week) a lot of people expressed their concern.  I am quite touched by the outpouring of support!

I have always contracted to Microsoft through its Canadian subsidiary, Microsoft Canada.  In September of this year I accepted a contract with Rakuten, Inc – a Japanese company – that would see me spending most of my time in Tokyo.  Although we tried, there was no good way for Microsoft Canada to keep me on.  It was not done maliciously – in fact, my skip-level (my manager’s manager) did everything he could to a) keep me on, b) communicate the issues with me, and then c) accommodate my request for a timeline extension.

So let me answer some of the ‘Best Of’ questions… the ones that seem to be coing up most often.

1. Did your decision to leave Microsoft have to do with being turned down for a particular position?

No. Although over the past year I have indeed been turned down for a position, it has worked out very well for me in almost every way imaginable.  While taking that role would have been good for me, I have been able to grow in the direction I have wanted to grow.  Because of my independence I have been able to accept the consulting project I am currently working on, which is one of the mot exciting projects I have worked on in years.

2. Did you leave Microsoft because of a disagreement?

No… and yes.  I suppose in the end we disagreed on geography – my consulting role needed me to be in Japan, and Microsoft Canada would have needed me to be in Canada.  Other than that there was no disagreement whatsoever.

3. Did you leave because you did not like the direction in which the company was heading?

Not at all.  In the army I topped out at Staff Sergeant, and as such I learned quickly that some things were above my pay grade.  At Microsoft that was the case as well – I know that a lot of things are out of my control, but I also knew that whatever direction the company would take, my position (should I have elected to keep it) was safe.  Whatever decisions the company made, as a VMware Compete expert I was reasonably safe 🙂

4. Do you feel any disdain toward Microsoft, Microsoft Canada, or anyone you worked for or with?

ABSOLUTELY NOT.  I loved working there, and while I may have had the occasional issue with someone they were always resolved.

5. Did you leave Microsoft to work with competing technologies?

NO.  Although over the past couple of weeks I have made a habit to wear my non-Microsoft branded shirts more than usual, I have not ‘gone over’ to any other competing technology.  With that being said, I am carrying an iPhone now not because I left Microsoft… because Windows Phone 8 is not available in Japan, and this is what the company I am working for gave me.

6. Will you be going back to Microsoft?

That is a very good question. What I once thought of as my dream job no longer holds the same appeal to me.  With that being said, there are a lot of jobs at Microsoft, and should the right opportunity present itself I would be glad to go back, either for the right contract or for the right full time position.  However one thing is for certain: I no longer view Microsoft as the Holy Grail of companies.  I think they are a great company to work for, but there are a lot of other great companies out there.

7. What will you miss most about it?

I had to give this question a little thought.  My first knee-jerk reaction was the people, but then I realized that the people I got to know are still there, and are still available to me.  I am still a Microsoft MVP, a Microsoft Certified Trainer, and an influencer.  My friends are still my friends.  When it comes down to it, I suppose what I will miss most is having Lync… having the ability to call my family from Japan was a great tool!

8. Any regrets?

None at all… for the remainder of my time in Japan I will continue to work closely with Microsoft, but not with the Canadian team.  It is a really exciting project, and I would not trade it for anything.

I want to thank you all again for your concern and support, and hope to be able to continue working with you in the future!

Exchange Issues with iOS 6.1

There has been a lot of chatter over the last few days about the most recent update to the iPhone’s iOS 6.1.

I have been saying for years that Patch Management is one of the most critical steps to protecting your infrastructure, both on the server-side and on the client-side.  However I have also stated that before implementing any patch the IT department should be testing it to make sure that it does not do more harm than good.  Of course, vendors do not release patches that they are not confident with, but they do not always test them in every scenario.

One of the common scenarios we see with the iPhone is with it being connected to an Exchange Server for its mail and such, whether that Exchange Server belongs to the organization or a public cloud solution such as Office 365.  While it works and it fully supported by Microsoft, it is not a scenario that Apple seems to test extensively for.  And so, with this most recent patch, there are issues (excessive logging causing enterprise-wide issues for all users).

It is the very reason that I have always advocated maintaining a lab environment that mirrors your production environment, and testing patches in that setting before approving them for your organization.  However with the iPhone being an unmanaged device end-users are prompted to apply their own patches without waiting for approval from the IT department.

And so this past weekend following this patch release mail server administrators around the world were scrambling to find a solution to the problem.  Unfortunately for many the immediate solution was to block iPhones from syncing to the mail servers until Apple releases a new patch.  I expect this will not make a lot of people happy, but in this case iPhones really are bringing down entire mail server farms.

With Windows Intune and System Center 2012 Configuration Manager there should be a solution to this, although I have not had the opportunity to test it yet.  The latest version of Intune (commonly known as Wave D) allows the management of iPhone and Android devices, and just may allow the IT department to regain control of patch management, preventing such issues going forward.

My friend and fellow Microsoft MVP from Israel.  He is a Microsoft Infrastructure Practice Manager at Ankor Computing Infrastructures, a leading Integration company in Israel.  Although his award category is the same as mine (Windows Expert-IT Pro) he is an expert in several technologies, including Exchange Server.  In 2011 he wrote an excellent paper on P2V Migration for Microsoft Exchange Servers that I published on this blog.  He has written a very interesting white paper about this recent issue, including solutions and workarounds.  You can download his paper by clicking here.

%d bloggers like this: