The View from the Geekside

As IT Professionals a core element of our industry is the fact that technology is constantly evolving, advancing, and changing into newer ways of doing things.

This movement is coupled with, and magnified by, the fact that technology companies cancel support for the older products and technology.

A natural byproduct of being an IT Pro is that we develop the opinion that it’s our job to ‘champion’, or evangelize, our individual stance on what direction we feel these changes should be made with those we support.

Always pushing our employers and our customers into implementing the newest technology and replacing the systems that are no longer supported.

“The solution to your issue is  to implement BYOD now!”

“XP is no longer supported, you HAVE to upgrade immediately!”

“You’re still on OSX 10.5, are you crazy?!”

“That system is still running Ubuntu 10.04. Are you an idiot?”

“OMG! My Android phone is so much better than your stupid iPhone!”

“Why in the world did you reformat that old computer? It’s slower than my phone. Throw it away!”

The Internet is full of articles on “how” to upgrade or “why” to upgrade, all written by IT Professionals that are true experts in their fields.

I want to approach this topic from a different angle. To perhaps offer another perspective and a potential reminder to others in this industry.

1. First off, no it’s actually not our jobs to push our employers and customers into replacing their systems.

As IT Professionals we stand between an Industry that moves at unrelenting speed and the Consumer/Customer/User/Employer that implements new things at their own pace, if they do it at all.

It’s our job to provide a cushion between these two equal and opposite realities. We examine both the needs of the end user and the available new products and technology in order to be a trusted voice of reason for both sides.

We review the requirements of the user so that we can advise on the best solutions for their needs.

We also provide feedback into the industry as to those end user requirements so that we can help shape the future technology movement.

We are the only ones who have view of both sides so we are the experts at making the best fit between the two.

2. Don’t be a dick.

Seriously, the IT Professionals industry is full of Champions, Evangelists, Experts, Gurus, Consultants, etc.

What it doesn’t need are Fanboy/Fangirls.

Support your opinions and preferred solutions, sure, but don’t be a mindless zealot.

No solution to any problem was ever properly solved by someone with an agenda, on some kind of campaign.

Just look at our Governments as an example.

When you’re surrounded by round holes and all you have are square pegs, and a hammer, it’s not your job, or your right to make those square pegs fit anyway.

Shoving your views and opinions on what the end user should do down their throats, purely based on your own opinions, is an abusive use of our position.

3. I know it’s a shock but sometimes “just upgrade” is actually not the right answer.

Heresy! I’m going to be strung up on my server rack and get my coffee taken away by the flaming keyboard wielding mob of IT Professionals!

It’s a scary idea, but it’s true! There might actually be reasons that the end user is using their outdated technology.

Hardware compatibility with certain peripherals such as specialized controllers.

Software compatibility with expensive or irreplaceable software suites.

The system might still be perfectly capable of doing what it’s supposed to do.

Hell, maybe they just like it.

Or, yeah, they might not know what their options are.

Whatever the situation, there IS ALWAYS a reason someone isn’t implementing changes on their own.

Frankly you should be happy that they aren’t making their own changes because if they were, we’d be unemployed.

Use your knowledge and view of the situation to carefully review the users needs, and wants, and see if there is a way for them to get the same experience with more modern technology.

If there isn’t a way, then support their decisions, offer advice and use your skills to make their current systems work the best way possible.

If they don’t want to, or simply can’t, upgrade their XP machine, let them know all the reasons it’s important to replace it if/when they can but also go over all the options for making it as secure as possible so that they can continue using it. Then, revisit the situation at a later date to see if anything has changed.

Don’t belittle them for their decisions because when it comes down to it, you work for them. Not the other way around.

Or, what I like to call “Hah!, neener neener, No, you can’t go behind my back and install iTunes anymore!

(Disclaimer: There’s undoubted way more knowledgeable readers than me, especially in more Security related skills, that will cringe and tear apart what I’m about to write. But, this worked for my specific issue and it severely limits the security repercussions of using Local Admin. So, use these tips at your own discretion)

You know them, you hate them.

The poorly coded Windows software that simply will not run without Local Administrator privileges.

So, you’re forced to give your Users way more privileges than they need, simply to support some critical software that was designed wrong.

Well, I’ve found ways to fix this by changing only privileges on a granular, more focused level instead of the global use of Administrator accounts.

First some real world background on what caused me to need to learn this stuff.

We’re, (as of this writing but upgrading very soon), a Windows 2003R2 Domain. (don’t judge)

Plus we have some VERY expensive, and now irreplaceable, critical Enterprise line of business software from the XP era that was written to require Local Admin rights of the user or it simply didn’t run.

The software in question, since I’m going to reference it later, is called Agile Advantage. It’s some niche software for Product Lifecycle Management used in the Manufacturing industry.

Agile was an awesome company, but back in 2006 Oracle bought them and shut them down. Well, the software is essentially the spinal cord of our company, so here I am trying to keep it working as the Windows world advances along.

Ok, so Security Nightmare Number One. I am (now WAS, but we’re getting to that) required to give everyone who uses this software Local Admin privileges. Which, was essentially the whole company.

Now, leading into Security  Nightmare Number Two, as I mentioned above, we’re a 2003R2 Domain so we use a KIX script at log-in to map network drives. I know there are much better ways to do it, even under 2003R2, like Powershell, but it has been in place for a long time, and it works. Well, here’s the thing.

Security Nightmare Number Two The User Account Control (UAC), in Windows 7 forward, blocks the KIX script when ran under Local Administrator level accounts.

(Something I didn’t know before this, when ran under Local User UAC runs with the same privileges as Windows Explorer so it doesn’t cause this problem)

You can probably see where this is going.

Combine these two and now I’m giving Local Administrator to all my Users on Windows 7 PCs, AND turning off UAC. Both highly not recommended but I had no choice.

Moving on to the reason you’re reading this. I finally had enough and undertook the massive project of trying to turn this train wreck around.

(Tool #1 for you TL;DR visitors)
Being a huge fan of the Microsoft SysInternals suite of tools, I finally realized I could use Process Monitor to get a view of exactly what was going on behind the scenes in Windows when Agile Advantage was running.

Now, I at least have not been able to get Process Monitor to run except under Local Admin so this was a brief bump in my plan as the whole point was to test this application under Local User.

The solution in my case was to log into Windows with Local Admin privileges and then do a Right-Click Run As on the Agile Advantage exe and select a Local User level account.

This essentially tricks the application into behaving as if I was logged in as a local user but lets Process Monitor run as intended.

Process Monitor, if you happen to not know already, scans every single active system call that is going on in Windows. So, the critical trick to using it is Filters because it goes nuts when you first run it because it’s really a busy place behind the scenes of Windows.

By adding a filter for Process Name is, in this case AgileAdvantage.exe, to Include and then a filter for Result is Denied I was able to focus the background noise down to just the traffic I was looking for.

Keep in mind Process Monitor is highly customizable so I recommend playing around with the filters so you can get it to do just what you want for your specific situation.

This lead me to seeing Denied attempts against two local files in the C:\Windows and C:\Program Files(x86)\Agile Advantage 2006\ directories and also Denied against several Registry keys in the HKCR and HKLM hives.

By targeting those specific entries and changing the Advanced Security Properties for of each one to give LOCALPCNAME\Users Admin level privileges to just those files I then got Agile to run. There was a couple instances in the Registry keys where the LOCALPCNAME\Users account wasn’t even on the Security tab, so I just added it as needed.

Now, I used the Local\Users group because my intention is to create a generic image for future PC roll-outs and I don’t want to re-do this every time for each specific Active Directory account that might use the PC.

Ok, so I’ve gotten the application to run, but it was still acting squirrely. Some functions that normally worked immediately were now taking a couple minutes, etc. Details are going to be different for each person and each application.

For my specific case, this lead me to poke around forums for other possible tools to use to get a deeper view of what was going on. This lead me to

(Tool #2 for you TL;DR visitors)
The Microsoft Application Compatibility Toolkit. Now, this software does way more than I even know, or needed in this project so I’d recommend playing around with it more. The part I specifically used is called the Standard User Analyzer (SUA) under Developer and Tester Tools. When I directed SUA to my target application and launched it, I got some more in-depth views of the system calls on the various tabs.

On the Registry tab there was a couple more HKLM and HKU keys getting denied and on the Other Objects tab there was a sub-directory in the hidden C:\ProgramData directory that was trying to get written to and getting denied.

So, essentially what I’m saying is to use both of the tools listed above for your troubleshooting investigation.

Another similar tool that was brought to my attention by Chris Jackson, an Architect at Microsoft (@appcompatguy on Twitter) is called LUA Buglight.

For my specific instance I didn’t need this tool but it looks to be very useful for this and generates a concise text file report of the issues it runs into.

These steps fixed my specific problem that I’ve been fighting for years. It was a giant pain in the butt and I repeated these steps for a couple other key applications but combining all this work with a reference PC that I plan to use for imaging future systems, all the effort now will make future issues non-existent.

Now, some further adjustments I made because I’ve just found a lot of Windows Weirdness in the Event log over the years on systems running as Local User.

In Computer Management, Local Users and Groups, I added the following accounts to the following Groups.

NT AUTHORITY\Authenticated Users and DOMAIN\Domain Users added to the Distributed COM Users group. Again, I did Domain\Users versus a specific person because I’m creating a generic image system.

I also added them to a few others, like Event Log Readers, Performance Log Users, Performance Monitor Users, etc. just to make any future work I might need to do on the individuals computer under their account later easier. This is just something you can decide for your own individual instances.

Happy Hunting and I hope my project steps here help others in similar situations.

To reference back to my initial Disclaimer above. I’m sure there are reasons why what I did are probably not recommended by some professionals but I weighed the potential issues against allowing global Administrator privileges and decided these options were far preferable.

For the sake of full disclosure, this blog piece is just a ‘whiny, negative, rant’. As such, you can feel free to not continue on. I’m not fishing for sympathy, or arguments, or reassurance, or looking for people to tell me to stop being so negative. This is simply, ‘text therapy’. It’s dumping what’s in my head out into words.

I have often heard people refer to their jobs with the analogy of “running in a Hamster Wheel” because they feel like they’re not actually getting anywhere.

For me, my career in IT is more like swimming in an Endless Pool. For those who might not know what it is, an Endless Pool (also often called a Swimming Machine) is a small swimming pool, usually 12-16 ft. long by 7 ft. wide. They use an underwater pump system to create a seriously strong current for swimmers to swim against for exercise. This allows for the swimmer to use all their effort in swimming without actually ‘going anywhere’. So they don’t need huge Olympic sized pools to exercise in.

This is a far better analogy for my view of working in IT because, in a Hamster Wheel, while you still ‘get nowhere’ from the effort, if you stop, you simply stop.

In an Endless Pool, if you stop swimming against the current, you get pushed back to the beginning and can potentially drown.

Swimming in an Endless Pool will make you a stronger swimmer, sure, but you’re only ever in your own little pool. The view never changes, the challenges never change, all the effort you put out is simply to maintain your exact same position and not get shoved backwards.

Meanwhile, I see all the other ‘swimmers’ out there in my IT environment. Swimming in rivers. Moving forward. Expanding their skills by experiencing different environments. All of their effort is the same effort I put forth but they’re actually improving, moving forward.

Every single one is stronger, faster, and better at ‘swimming’ than I am. Comparing myself to other ‘swimmers’ only brings frustration.

Information Technology is a really tough industry to be in if you do any sort of introspection comparisons of your own skills compared to everyone else in the industry. Like comparing my furiously dog paddling in my little pool to Michael Phelps.

Don’t get me wrong. I’m pretty good at what I do. The problem is what I’m doing is simply trying to maintain the exact same position in my little pool, day after day. While there is no active discouragement to improve, and it really is a pretty nice pool there’s no reason to ‘swim any harder’ because there’s simply nowhere to go.

Some days it’s difficult not to consider letting the current win.

I am the Network Administrator for a high-tech circuit board manufacturing company. A ‘struggling to survive’ manufacturing company that is currently running in ‘survival mode.’ I have been here for 12 years and this is nothing new to me.

When a corporation is struggling to survive it is like a wounded animal and some really difficult decisions are often made.

As the Network Administrator I’m often called upon to clean-up the fallout after these decisions.

The inevitable decisions are made and I cringe as I’m handed ‘the list’.

“The company has to restructure to help bridge the gap this year. A head count reduction is regrettable but necessary.” “Don’t worry Mike, you are not affected by the head count reduction.”

“You are not affected.”

<sigh> I cringe and glance down ‘the list’.

I see names on the list I don’t even recognize, which makes me feel a little sad. “You are not affected.”

I see names on the list I actually am kind of glad to see, which makes me feel a little guilty. “You are not affected.”

As I glance farther down the list I see names I wasn’t expecting which knocks the wind out of me. “You are not affected.”

I see names of friends, I see names of people I felt were indispensable to the success of the company. “You are not affected.”

I see names of co-workers I’ve worked side by side with for over a decade. I see people who were more family than co-worker. “You are not affected.”

I feel the weight of the world come crashing down upon my shoulders. “You are not affected.”

I feel the pride I once had for the skills I possess, for the company I once supported without question, for the company I once looked at more as a second home than a place of employment, vanish.

“Don’t worry Mike, you are not affected by the head count reduction.”

In my mind I see dozens of laptops, PCs, and cellphones which, until today, were the tools of the co-workers on ‘the list.’ “You are not affected.”

In my mind I see the gigabytes of Personal Drives, My Documents, Outlook Archives, etc. that I now have to clean up and archive. The data that, until ‘the list’ was the products of someone’s career. “You are not affected.”

In my mind I see dozens of accounts that will soon be disabled. The electronic remains of people who are my friends. Remains that I now have to clean up. “You are not affected.”

“Restructure”, “Bridge the gap”, “Head count reduction”

Corporate Human Resources rhetoric that sounds better than “The loss of people’s support structure for their lives.”

“You are not affected.”

I sigh, my shoulders sink, my soul aches, and I move on. I begin the task picking the remains of people’s professional lives apart. “You are not affected.”

As I begin to gather the remaining equipment, the ghosts of friends who once used them, someone walks by and says “Hey, at least you weren’t affected. You’ve got job security.”

In the back of my mind I scream “Fuck you! Take your ‘job security’ and shove it!” and I silently move on with my morbid responsibilities.

“Don’t worry Mike, you are not affected by the head count reduction.”