Today’s topic, “What were you thinking?” is being hosted by Wayne Sheffield. Thanks Wayne! And this is the 10 year anniversary of T-SQL Tuesday. Time flies!
The Peter Principle
Have you ever heard of the Peter Principle? In essence one is promoted up either in a company or in their career to their level of incompetence.
Several years ago, I had to work with a very difficult person at a client. Let’s call her Susan. During the interview with the IT department she stated that everything was fine and they really didn’t need outside dba help. Well, the client was experiencing major stability issues with their website and they needed to get off of SQL2005 (upgraded from SQL2000) and onto SQL2012. There was also a long list of issues to be addressed and things came up too during this whole client engagement.
When I started, I was pulled aside by the IT Manager and told that Susan had several HR violations against her, mainly for shouting and arguing with people. So heads-up and tread lightly around her. She was older (maybe ~60+?) and I think that the company and people were either waiting for HR to fire her or she would get laid off. Either way, she was the only database administrator on the team. Other dba(s) had quit over the years because of her and the financial health of the company. I got to know the Director of Software Engineering pretty well and he flat out told me that his developers hated asking her for anything. In fact, many had stopped and would rather try to figure things out for themselves.
A Corner of the Institution
At one point I was flown out on a week long trip in the spring of 2012 to the client’s office in Santa Monica and had a chance to see Susan’s desk firsthand.
I tried to control my confusion and facial expressions but honestly I had a hard time trying to keep it together.
1. She was running Windows XP when everyone else in the company was running Windows 7. She said she couldn’t take the downtime to upgrade.
2. SSMS was connected to all of their instances using the IP address and sa login. When asked why, she said, “Per Microsoft best practice” <- this was a common refrain from her. When asked to provide a link, she never would.
3. She did everything via the GUI- barely typed any T-SQL code. It was like watching paint dry watching her work so slow.
4. The few technical books she did have were either SQL2000 or SQL2005. Nothing new, nothing about SQL2008 or SQL2012. I’m not judging but that doesn’t seem to me to be the bookshelf of a senior dba. Maybe she had a SQL library at home? My gut tells me no.
Other Bad Signs
Short (!) list. No SQL Agent jobs to run CHECKDB (S: it blocks users). Tempdb had 8 data and 8 log files (S: Per Microsoft best practice). All tables were heaps (S: it is not worth the time to create clustered indexes on ~350 tables. I won that battle but it sucked). Didn’t want to attend any of the Project Manager stand-up meetings regarding the migration to SQL2012 (S: I’m just too busy. But you are going to be managing this when we are gone. S: I’ll figure it out then).
All Good Things Must Come to an End
I was put into a bad situation. From me, we expanded the consulting team during the engagement to do a lot of work that needed to be done. Was the project successful? On all technical accounts, yes. Stability and performance were AWESOME when we were done. Upper management was very pleased with our efforts. The public end-users could see major speed improvements on the website too.
But the raw nerves and yelling… Yes, I initially got into a shouting match with her about not running CHECKDB (S: because it blocks users so we don’t run it) when they had many entries in msdb.dbo.suspect_pages. I got so fed up with her I said, “You know more than Paul Randal about CHECKDB?” (S: Who is Paul Randal?) I had to go above her head and won that battle too but it sucked.
Sometimes in life you have to work with difficult people and I’ve had my fair share. Be civil and professional when you can. Stick to facts. But when people don’t want to believe facts and it is harming the business, don’t be afraid to explain it to C-level executives on why something needs to be done.