Fifteen Tabs: Shut em all down!

helpThose of us who try to keep  the banging of our heads against a wall to a minimum will have long since learned that we don’t know everything, and more often than not there is someone out there who knows more than we do.

“But Mitch, how can you say that?  I read your blog religiously, and when I don’t know something about computers I can usually find the answer on your blog!”  I know, but even I have to go looking for the answers sometimes… in fact, more often than not.

The problem is, very often the answer is not so easy to find…

This week I found myself trying to solve a problem that concerned a High Availability SQL Cluster.  It was maddening… I spent three days trying to find the answer, and from very early on I knew that I was going to have to look to others for help. 

As a blogger, the first place I start looking is… other blogs of course!

FrustrationNow here’s the problem with that… when the answer is not as simple as a yes/no to track down, you may have to look through myriad blogs and articles and documents and forums before you actually find the answer… and often enough one blog or article or document or forum may not give you the solution, but it will point you in the right direction – either down a rabbit hole or sometimes into a snake pit.  Either way, I keep each page open in a new tab in my web browser, because often enough I will want to document how I got from zero to hero – or hero to bum, as the case may be.

So three days into trying to solve this problem, which took me to a dozen different articles, forums, and conversations on Facebook, Twitter, and LinkedIn (which indeed led me to three other articles), I found in four of those articles the pieces to the puzzle that eventually helped me to solve my issue.  It was done, I solved it… not alone mind you, but I solved it.

I looked at my desktop and sighed… I hadn’t rebooted in three days for fear of losing track of my path toward the solution… which of course I had to document for my client.  I could now close them all down.  Each browser tab (and yes, there were actually fifteen of them, excluding Facebook, LinkedIn, and Twitter) was now beckoning to me to shut them down.  One by one I clicked on the X in the top right-hand corner… and I was quite satisfied as they all disappeared into the ether.

As frustrating as technical problems may be – and I always tell people that knowing more will only lead to more technical and even more frustrating problems – when you solve them the feeling is truly euphoric.  That stress-relieving satisfaction that you fought AND WON.

Now how is that for a way to end a day? Happy Friday everyone, and have a great weekend!

Will you pay?

CPUsAn article showed up in my Inbox today: Intel Core i9: It’s not whether you need 12 cores, but whether you’ll pay for them.  It is an interesting read, and a very good question.

I have always liked ZDNet.  Their people do a good job of keeping a pulse on the industry.  Their question is a valid one… will people be willing to pay for 12 CPU cores (presumably on the desktop… people will definitely pay for them in servers).

I used to be very good friends with a man named Willem.  Willem is brilliant, and was (almost) always a positive influence on me.  He is an IT Professional who moved to Virginia some time ago, but until then he owned and operated a company in Montreal called Saturnus True Data Services.  They were not the first computer company I ever worked for… but they were certainly one of them.

One day in late 2005 I was talking to Willem about the new laptop I was buying, an Acer Ferrari 4000.  It was sleek, it was gorgeous.  It was the first computer I ever owned that had a 64-bit CPU.  When I told Willem about it he asked why I would ever need or even want a 64-bit CPU?  I admit I did not really have a good answer for him then.  Later on I would… starting with the 3.2GB limitation on 32-bit CPUs.  However, when he asked me in November of 2005 I couldn’t tell him why I would even need more than 3GB of RAM, because back then nobody really did.

Fast-forward nearly twelve years, and 64-bit CPUs are ubiquitous.  I haven’t tried in a while, but I doubt you could even buy a laptop today with a 32-bit CPU.

The hybrid laptop I am writing this article on – my Surface Pro 4 with an i7 CPU – cost quite a bit more than my Ferrari laptop did, and it has a 4-core CPU with 16GB of RAM.  Had Willem asked me 12 years ago why I would ever want 4 CPU cores on a hybrid laptop I would have answered honestly with a question: What’s a CPU core?  And yet, here I am and it is my go-to machine.

The way our world works is simple: Something is invented and it is expensive… at first.  As time goes on prices go down… usually as newer versions are invented.  Eventually they become obsolete, and if you are lucky they become collector’s items… usually they become junk.  The Ferrari that I paid over $1500 for in 2005 is now selling (used) on eBay for $200… and that is probably because of the Ferrari logo because equivalent laptops from other manufacturers are selling for much less than that.  One day my Surface Pro 4 will be nearly worthless too.

CPUSo Intel invented a desktop CPU (the Intel i9) with 12 cores.  Today nobody needs it.  In 20 years nobody will understand how we got by with such primitive technology.  The founder of that company, Gordon Moore, predicted in 1965 that “…the number of transistors on integrated circuits doubles about every two years.”  So 12 cores is simply what was next.  Our computers get faster and as they get faster they get more expensive.  Then something even faster comes out, and that other one becomes less expensive… until they become obsolete.

So who will pay for the 12-core CPU on a desktop?  Probably very few people… now.  But give them time; prices will come down, and we will see them out there.  Slowly at first, but eventually 12-core CPUs will probably become the standard desktop processors.

…Now who among us feels really old, and nostalgic about our 4.77MHz CPUs?

A New Perspective…

This blog is older than I ever thought it would be.  So every once in a while I like to give it a facelift.  This morning you should notice a big difference.

I picked a new template last week.  I have modified it though… the pictures in the cover are shots I took in Cuba this year.  I hope you like it!  Let me know if you don’t!

Mitch

Windows Server 2016: A pet peeve

Windows Server 2016Over the next few weeks, as I do my first production infrastructure implementation based on Windows Server 2016 and System Center 2016, I am sure this list will grow longer.  In the meantime, I have uncovered my first pet peeve in the new version.

Don’t get me wrong, overall I like Server 2016… but to find out that it is no longer possible to install Windows Server with a GUI (Graphical User Interface) and then later to uninstall the GUI (see article for Windows Server 2012) is fairly annoying.

Throughout the launch of Windows Server 2012 I was with the Evangelism Team at Microsoft Canada and I traveled the country – first for the launch events, and then evangelizing and teaching that platform.  I spent a lot of time talking about Server Core because of the benefits for security, as well as for the reduced resource requirements (which, in a virtualized infrastructure, can be staggering).

Of course, Server Core looks a lot like where we started out… if you were a server administrator back in the 1980s and most of the 1990s, you were using command line tools to do your job.  However it had been too long ago, and the vast majority of admins today were not admins back then.  So I was able to discuss a compromise… Install Windows Server with the GUI, and when you were done doing whatever it was you needed the GUI for (or thought you did), you could uninstall it… or at the very least, switch to MinShell.

I showed up at my client site this week and was handed a series of brand new servers on which to work.  They all had the GUI installed.  So I went to work, and typed in that familiar PowerShell cmdlet to remove the GUI.  I was greeted by that too-familiar red text which meant I had done something wrong.  I will spare you the boring details, and after several minutes of research I discovered that Microsoft had removed the ability to remove the GUI in Windows Server 2016.

I understand that the product team has to make difficult decisions when developing the server, but this was one that I wish they had not made.  However confirmation comes directly from the product group in this article, in which they write:

Unlike some previous releases of Windows Server, you cannot convert between Server Core and Server with Desktop Experience after installation. If you install Server Core and later decide to use Server with Desktop Experience, you should do a fresh installation.

I wish it weren’t so, but it is.  Once you install the GUI you are now stuck with it… likewise, if you opted for Server Core when you first installed, you are committed as well.

Sigh.

Firewalls: Trust me!

I have several clients who have multiple sites, as well as multiple Active Directory (AD) forests.  As security is so important they want to lock things down the best they can, but they also need to open up the necessary ports to allow the domain trusts to work.  The ports required for this are:

Port Number Protocol Traffic Type
53 TCP/UDP Domain Naming Service (DNS)
88 TCP/UDP Kerberos
389 TCP/UDP LDAP
445 TCP Server Message Block (SMB)
636 TCP LDAP (SSL)

These ports should work for every version of Active Directory dating back to Server 2000, but I have not tried anything earlier than 2012.

Windows To Go Gotcha in Windows 10

So here’s an interesting fact about Windows To Go.  When Windows 10 first came out I was still running Windows 8.1 on my corporate desktop, and when I went to create my WTG image I couldn’t because the Windows 8.1 WTG engine did not support building Windows 10 WTG keys.  Ok, that is understandable.

Windows 10: The last operating system Microsoft will release, right?  Well my corporate laptop is on Build 1607, and when I downloaded the latest build (1703) it would not recognize it.  So my two options are:

  1. Download the earlier build and make my key based on that build; or
  2. Take the time to upgrade my laptop.

With all due respect Microsoft, if you are going to tell us that Windows 10 is the last desktop OS, don’t pull these games.  As a tech guru I understood right away what the problem was… How much time do you think the regular Joe trying to use your products would have spent on this?

Scheduling Server Restarts

If you manage servers you have likely come to a point where you finished doing work and got a prompt ‘Your server needs to reboot.  Reboot now?’  Well you can’t reboot now… not during business hours.  I guess you’ll have to come back tonight… or this weekend, right?

Wrong.  Scheduling a reboot is actually pretty easy in Windows.  Try this:

  1. Open Task Scheduler (taskschd.msc).
  2. In the Actions pane click Create Basic Task…
  3. Name the task accordingly… Reboot System for example.
  4. On the Task Trigger page click the radio button One Time
  5. On the One Time page enter the date and time when you want the server to reboot.
  6. image
  7. On the Action page select Start a program.
  8. On the Start a Program page enter the name of the program shutdown.exe.  In the Add arguments box enter /f /r /t 0.  This will force the programs to close, restart the server (instead of just turning it off), and set the delay time to 0 seconds.
  9. image
  10. Once you have done this your server will reboot at the precise time you want it to, and will come back up.

**NOTE: Don’t forget to check.  it is not unheard of in this world for servers to go down and not come back up as they are supposed to!

Do it in PowerShell!

Using PowerShell to script this will allow you to not only save the script, but also run it on remote servers.  From Justin Rich’s blog article I found this script:

register-ScheduledJob -Name systemReboot -ScriptBlock {

Restart-Computer -ComputerName $server -Force -wait

Send-MailMessage -From mitch@email.com -To mitch@email.com -Subject "Rebooted" -SmtpServer smtp.mail.com

} -Trigger (New-JobTrigger -At "04/14/2017 8:45pm" -Once) -ScheduledJobOption (New-ScheduledJobOption -RunElevated) -Credential (Get-Credential)

 

Have fun!

A Big, HUGE Microsoft Security FAIL.

(NOTE: This article was written December 7, 2016. Not one word has been changed since that date.  To understand why it can only now be published, read the article on this site called 107 Days: A Microsoft Security Nightmare. -MDG)

For reasons that will become obvious, I am going to delay posting this article until the issue has been resolved.

A few days ago a colleague of mine discovered the password to my Microsoft Account.  I won’t go into the how and why… I knew that my password had been compromised and I took the immediate steps to change it.

image

Ok, I understand that things break… I tried a few times, and then I decided to follow the advice and try later.  I trust my colleague not to actually use my password, so even though I felt uncomfortable with it being compromised, I knew I could wait a couple of hours.

Throughout the evening I tried (unsuccessfully) to change my password.  As I was sitting with my father having dinner, as I had drinks and cigars with my friends… no joy, I still got the same message.  ‘There’s a temporary problem with the service. Please try again. If you continue to get this message, try again later.

I want to be clear… if my network had an error that was preventing users from changing their passwords I would consider it reasonably important, and I would take immediate steps to fix it.  But having trusted Microsoft for so many years, I assumed this would be fixed eventually.

Four Days Passed.

Yes, it was literally four days before I decided that my passivity would not eventually lead to a solution.  I sat down and figured out how to request support. I was hoping to be able to speak with a human being.  Before I could, however, the Virtual Support Assistant got me to try this link and that link.  It then made me go through seventeen steps to finally confirm that the account in question was mine… and once it confirmed that I really am me, it tried to reset my password… and I ended up with the same error message that ‘There’s a temporary problem with the service. Please try again. If you continue to get this message, try again later.

Okay, it’s been nearly an hour… and I am chatting with someone who is quite obviously not their first round draft pick.  After all, I asked for help with Outlook.com, not with something that people actually pay for.  I spent twenty minutes explaining to him the situation, and the added (and I assume rare) complication that I have two accounts with the same address… my Office 365 account and my Microsoft Account are both the same address that are completely different.  ‘Please don’t touch my Office 365 Account, I only want to change my Microsoft Account.’  This led to another five minute discussion on the meaning of the word change.

He had me fill out another form on-line.  I did.  At the end of that form I got a message that said that the product team would contact me within 24-48 hours to help me.  I told the Support Agent that I had filled out the form.  He told me that now I had to wait until they contacted me.

All in all, my Microsoft Account (which is the account I use for my MCT & MCP Benefits, Skype, and myriad other features) will have been compromised for the better part of a week… and there was nothing I could do about it.  Yes, I could have contacted Answer Desk a few days earlier, so it would have been compromised for only three days.  I want to know in what world is that considered an acceptable delay to be able to change a compromised password?

Some time ago I started using Multi-Factor Authentication (MFA) for many of my most important systems, which is why I am never concerned that my blog or my password vault could be compromised.  For various systems I have a hard key (Yubikey) and soft keys (Google Authenticator and Microsoft Authenticator) which keep most of what I do safe.  But most of the Microsoft systems do not support MFA and I am stuck with only a password.  I use reasonably complex passwords so I usually am not concerned, but in a case where my password is compromised and I am not able to change it, I wonder how it is that a company as advanced as Microsoft (in this case) does not allow me to use MFA.  I would love to be able to require my Yubikey in order to log in to Windows and many of the on-line systems I use, but it is simply not an option.

I am disappointed by Microsoft this week… and I hope that they take the lessons learned from this experience to improve.  However I sit here today, thinking of the myriad occasions I stood on stage in over a dozen countries on five continents and defended Microsoft’s security systems as among the best in the world; I was always sure in my knowledge that I spoke the truth.  Today I would not feel comfortable making that claim… and my faith in their systems, like shattered glass, will not be easily fixed.

107 Days: A Microsoft Security Nightmare

I have held off talking about something for quite some time.  I do not mess around when it comes to security, especially for my critical accounts. When the actual security of an account has been compromised, as was the case with my Microsoft Account, I do not advertise it. 

On December 7th I sat in the Second Cup cafe on Bank Street in Ottawa and wrote an article called A Big, HUGE Microsoft Security FAIL.  I wrote about how I had been unable to change my password and that their engine to do so was broken, but that it turned out it was not everyone, it was just me.

There’s a temporary problem with the service. Please try again. If you continue to get this message, try again later.

It took several days for anyone at Microsoft to take me seriously, but my issue was finally escalated to a 2nd Level Support Tech named Gary (who, I want to be clear, was a nice guy, and as helpful as he could be under the circumstances).  Gary and I got to know each other sometime in mid-December.  Remember, the issue started happening the last days of November, I reported it on December 5th, and the case was escalated (grudgingly) around December 9th.

Gary spent a couple of hours trying to help, and then in discussions with the Product Engineering Team trying to get it fixed.  By the end of the day he said something to the effect of: “Yeah, neither I not our engineers have ever seen a problem like this.  It might take some time, but if you are willing we will work with you to get it fixed.”

Some time… It might take some time… that was on or about December 9th.

I am happy to say that the problem has now been resolved… As I sit and write this, with the resolution less than an hour old, it is 1:15pm, Wednesday March 22.

107I spent nearly a decade touting the virtues of Microsoft’s security… and then from the day I informed them that my password had been compromised, and that a glitch in their system was preventing me from changing it, it took 107 days to resolve the issue.

So let’s take a quick rundown of some of the sites and services that are accessed with my Microsoft Account:

  • Skype (One of the ways I communicate with hundreds of people)
  • OneDrive (All of my files!)
  • Microsoft Certified Trainer (MCT) account (including my MCT renewal, courseware downloads)
  • Microsoft Certified Professional (including my MCP Transcripts)
  • MSDN Subscription (including all my software licenses and keys)
  • Windows Store (including credit card information)
  • Microsoft Volume License Center (VLC)
  • Microsoft Store (including credit card information)
  • Bing
  • Microsoft Partner Portal
  • MSN
  • Outlook/Hotmail
  • MY WINDOWS COMPUTERS

And so, you can see, this is not like having my Words With Friends account compromised… This is extremely serious and far-reaching.  This was… everything.

Once a week I would get an e-mail from Gary telling me that they had not yet resolved the issue… but they were still working on it, and he would continue to keep me informed.

Proof Of Hack 2On March 6th a hacker compromised my Skype account, and sent a link to dozens of my contacts with malicious content.  Naturally those contacts let me know, and I reached out to Gary and told him that now that hackers had indeed compromised my account, they needed to resolve the issue and pronto.  Gary replied with: “I have taken a look into your account, to look for any evidence of unauthorized access, and I did not see any. Was any account info changed? Can you still login?”

a few days later that he had not been able to open the embedded picture, and asked that I resend it as an attachment.  Thank Heavens for that, because had he taken the next step immediately I would not have been able to renew my Microsoft Certified Trainer (MCT) credential in time.

So when Gary did finally get the picture (as seen above) he wrote (on March 18th):

If you received that message, then it could be that someone attempted to access the account.

To prevent that, I have placed a suspension on the account that will prevent any login activity. While my engineering team investigates this issue, no one will be able to break into the account. I have also left a note on the account so that the attacker will not be able to attempt to remove it.

Wonderful.  You are suspending my account now, probably after the damage has been done, but all this is doing is punishing me.  FIX THE DAMNED PROBLEM!

On the same day as I received this e-mail I wrote the following one line response:

Gary this is no longer acceptable. I am calling a lawyer.

On Tuesday (March 21) I received Gary’s reply:

In light of this recent reply, I have escalated this issue to a second team within Microsoft, and am awaiting to hear their response.

I understand the frustration, but please know that I cannot do anything to speed up the engineers and Ops teams working on this issue.

Wouldn’t you know it… The following day (that’s today, Wednesday March 22, 2017 – 107 days after I first reported the issue) I received a call from Gary that started with:

Well Mitch, it seems that when you threaten to call a lawyer things get done faster.  I think we have solved your problem.

Indeed, before the phone call ended I had successfully changed my password.

One hundred and seven days after I first reported the problem.

One hundred and seven days since I told Microsoft there was a problem with their security.

One hundred and seven days since I told Microsoft that my account had been compromised, that someone had my password, and that I needed their help to secure my data and reputation.

One hundred and seven days.  Actually it was only 105 days since I wrote the original article (which will be published shortly after this one, untouched since the original writing).

So why didn’t I publish sooner?

There are a handful… maybe four or five people who know the story and who understand some of my frustrations with this case.  These are also people who know I have a great bully pulpit in the form of this blog.  They have all asked me ‘Why didn’t you publish sooner?’  Two of them asked why I did not go to the mainstream technology media to let them know about this.

Simple… I have an account that is easy enough to guess, to which I could not change the password.  If the wrong people knew about that they would have focused on getting that password and, once they had it, they knew I couldn’t change it.  They would have literally owned me. 

And so I sat quietly, seemingly patiently, waiting for Microsoft to fix the problem.  I waited those 107 days knowing that when it was finally resolved I would a) breathe a big, huge sigh of relief, and b) sit down and write this piece, venting my facts and frustrations.

MICROSOFT! HOW DARE YOU? How can you let ANY problem, let alone one as serious as this, fester for so long unresolved?  Do you think you owe me nothing?  At this point I am still considering a lawsuit, and if you don’t think damaging my reputation and peace of mind is worth damages in a court of law then you are seriously misreading the system.  You should be ashamed of yourselves, and you should be tracking down who is responsible for this travesty, this shame, and firing them.

I got that off my chest.  I have, over the past two weeks, asked friends and colleagues for recommendations on lawyers.  I might just reach out to one this afternoon.  We’ll see.

Outlook / Hotmail Issues Acknowledged

When I posted my last couple of posts Microsoft was reporting that ‘All is Well.’  Dashboards were green, nothing to see here.  That has now changed:

image

So we know now that the issues are pretty serious… at least, serious enough that they are now acknowledging them.

As for me, I have been having an issue with my Microsoft account that has been ongoing since December 5th.  I have held off talking about it for security reasons, but with all that has been going on today, and the fact that yesterday my account might have been hacked, I am planning on writing about it tonight.  And boy are you guys going to be shocked.  All I will say for now is this: DO NOT TRUST MICROSOFT SECURITY. 

Yes, I said it.  For a decade I have been espousing the virtues and benefits of Microsoft’s security.  Unfortunately I have had to change my position on this, and in a very big way.  DO NOT TRUST THEM.  It has cost me terribly, and I will tell you about it tomorrow.

M

Outlook / Hotmail Down: Update

So the worst fear of hundreds of millions of computer users has been realized today.

image

As I reported a little while ago, Outlook.com and Hotmail.com are down.  But it goes much deeper than that.  If login.live.com is down (see screen capture) that also means that OneDrive, Skype, and even XBox Live are out as well.  If you are a Microsoft Certified Trainer and you were planning to download courseware today, that’s not available either.  In fact, any service that requires authentication with a Microsoft Account is down right now.  We are still awaiting word from Microsoft as to when these services might be restored.  But for now, you (and I) will simply have to wait… in the figurative dark.

Outlook / Hotmail Down?

After receiving notification on my iPhone this morning that there was a problem with one of my Outlook.com accounts I went onto my computer to try it out.  It did not work there either, so I did some investigating.  It looks like this system is suffering a major outage today.  I am not sure if this is strictly the e-mail, or if other services relying on the extremely popular Microsoft Account (formerly Microsoft Passport, Live ID) are out as well.  One this is for sure… there are a lot of unhappy users on-line this morning who are not receiving their e-mail.  More to come!

Resize Live Virtual Hard Drives

 

I have used Hyper-V for as long as there has been Hyper-V.  Today I use it much less than I once did, but it is still handy for running VMs on my laptop.  I run a particular VM called ‘Sandbox’ in which I do all sorts of things that I would not want to do on my live system… things that I can simply try and then wipe.

When I built the Sandbox VM I was spending a lot of time at home, and portability was not a huge issue.  I ran it on one of my external drives, and it worked fine.  I allocated 100GB and was good to go.

When I realized I was going to be on the road again I could just as easily take my external hard drive with me, but the shortage of USB ports on my Surface Pro meant making a decision… I was going to shrink the VHDX file and put it on my internal hard drive.

76GB free space.  That’s going to be a problem.

Step 1: Shrink your partitions

My 100GB virtual hard drive (.vhdx file) meant that somewhere within the VM I had a 100GB partition (or at least a few partitions that added up to that).  I had to shrink as much as I could.

  1. If you have a Pagefile.sys, Swapfile.sys, and Hiberfil.sys you should eliminate them now.  Remember that even if you turn off Memory Paging the files don’t disappear until you reboot.
  2. Defragment the disk.  We may not talk much about it anymore, but the old faithful defrag.exe C: still works. 
  3. Use the Disk Manager console to shrink your C: as much as you can… but not too much.  When I tried it I had the option to shrink it down to 11.5 GB… I’m pretty sure that would render my VM pretty useless.  Pick a number that works for you.  I chose 60GB.
  4. Using the diskpart tool delete any partitions at the end of your drive.  I had a 450MB Recovery partition on mine.

image

Because it was a Recovery (read: SYSTEM) partition I needed to do the following:

Select Partition 5

Delete Partition OVERRIDE

image

Good… Now we can shrink the VHDX file.

In the older versions of Hyper-V this would have meant shutting down the VM.  You don’t have to do that anymore… but you do have to run PowerShell as an Admin.  Once you do:

Step 2: Shrink your VHDX file

The cmdlet is easy…

  1. Navigate to the directory where you keep your VHD file;
  2. Resize-VHD -Path .\Sandbox-PC.vhdx -SizeBytes 60GB

It will only take a minute and you will be done.  Simple as pie!

Free E-Books for your Technical Library

Every so often Microsoft releases a bunch of new books that they give away for free.  Most of the time they are for the latest and greatest software and systems, so they are worth checking out.  Currently they have books for:

Microsoft SQL Server 2016

Windows Server 2016

Microsoft Azure

Microsoft Power BI

Windows 10

Microsoft SharePoint Hybrid Capabilities

Mobile App Development

…and much more.  Check out the list here.

While it is nice to be able to download these as PDF files, I am really happy that I can also download all of these files as proper e-Books from www.kobobooks.com for my Kobo, and I assume from www.Amazon.ca for my Kindle (I don’t have a Kindle… but you know what I am saying).

Here is what I like to do to make sure I don’t miss a thing:

  1. I log on to my Kobo account.
  2. I enter the name of the product I am looking for in the Search bar (say, Windows 10).
  3. When the list comes up I change the Sort parameters to Sort By: Price: Low to High.  This ensures that the FREE books come up first.

image

Let me be clear though, there are some excellent books on all of these subjects that are not free, and are probably worth buying.  That is not the point of the article though Smile

Thanks Microsoft Virtual Academy!