jump to navigation

Friday Philosophy – The Worst IT Person I Have Met October 15, 2010

Posted by mwidlake in Friday Philosophy.
Tags: , , ,
9 comments

A couple of weeks ago I extolled the virtues of someone I felt was a great person to work with. This week I’m going to do the opposite (and it will be interesting to see which posting gets more hits).

The worst person I have worked with in IT is Mick. I’ve only known a couple of Micks {and if you are one of them, but you don’t know Barry, you are not the Mick}. In an ironic twist of fate I met Mick at the same time I met the best person I have worked with, Barry. We were all in the same team you see, a UNIX sys admin team I got parachuted into. Maybe the vast difference between the two help make them so distinct in my mind.

Mick was very knowledgable and technically very capable. No, that is not fair, he was extremely good. He actually knew all this system admin stuff and several variations of shell programming, perl, C and a few other two-steps-from-assembler type languages. And he was an absolute and utter pain in the behind.

Barry and I did not know much (or in some cases, any) of this sys admin stuff. If we needed to do something and did not know how, Mick was supposed to show us. It worked something like this:

“Mick, I need to copy all the files that were changed last week from this directory on box X to box Y, keeping the directory structure – Can you help?”. Mick would not hear. He suffered from “intermittent deafness” – though he never missed any announcements about free food. You had to go and stand by Mick and wait for him to deem to notice you. If you actually interrupted him he would swear at you and utterly refuse to help, you had to wait quietly. If it was a good day he would deem this acceptable after a minute or two, but he would do his utmost to convey the impression he despised your lack of knowledge and your concerns were beneath his talents… but he would stoop to help.

You would repeat the task you were trying to do and, pausing only briefly to pour scorn on such a trivial thing, he would turn his back and start typing. He’d write a script to do it. “no, no, don’t write it, just tell me the basic commands and I’ll work it out!” No, he insisted on writing the script.

The script would be a thing to behold. Mick would write it in as few lines as possible and the least number of letters. For ages. Oh, he would have a working version in about the time it took Barry or I to explain the task, but he would not give you that version, oh no. He would ignore you until he had made all variables 1 character, took out all whitespace, replaced anything obvious with something obtuse, replaced a small chain of simple commands with one or two arcane commands. Every script was an attempt to win an “obfuscated code” competition. If we waited for the end result, it was impossible for Barry or I to decipher. The only benefit to the process was you would see the commands he was using and you could wander off and start with the unix Manuals yourself and get the job done.

He had other methods with which to demonstrate his greater worth.
Mick would agree to help (under duress of the boss telling him to do so) with an urgent task, but keep asking you to wait all day – then go home without doing his bit.
He seemed to love to intercept anyone coming to you for help, tell them he would sort out the problem for them – only to not. And then tell the user the next day that it was Barry or My problem to sort out. Correct, Mick would not have mentioned this to us.

Mick was fair though, he would treat everyone the same. With scorn. Any expertise in a field he did not know was unimportant and anyone with skills in his field was just competition to be shown who was best. Sadly, he usually was best, if best means biggest smartass.

Over time, as Barry and I learnt stuff (almost never from him), Mick became redundant. Not because we caught him up, not by a long way, but because no one else in the department would ask him anything. They would come to Barry and I. We might be slow and we sometimes screwed it up but we did not sneer and we fixed the problem in a way they could understand.

The reason Mick is the worst person I ever worked with is, unlike people who simply break stuff or lie about their skills or are stupid, he was actually very talented and capable – and yet took a perverse pleasure in not doing so. Mick would put effort into the art of maximizing his unhelpfulness. It was the difference between his potential to help and his drive to not do so that made it so hard for me to deal with him. I’d rather work with a talentless, idiot liar because at least you don’t need or expect much from them.

*sigh*

Friday Philosophy – The Best IT Person I Have Met September 24, 2010

Posted by mwidlake in Friday Philosophy, Perceptions.
Tags: , ,
3 comments

I’ve had the pleasure of working with and meeting a lot of talented and capable people in IT – some of them have even been nice people too :-) {In fact, most people I have met do not match that annoying myth about IT’s reputation for social awkwardness). However, for me one person sticks out in my mind as the best person I have worked with in IT.

It’s Barry. I’m pretty sure none of you have met Barry, and in fact as I knew Barry back in 1996 I’m not so sure I would recognise him (and I have no hope of remembering his last name) if I met him now.

Barry and I met when I got press-ganged into a Unix system administration team. I was just getting started at being an Oracle Performance person and knew very little about Unix Sys admin. But, for reasons I won’t go into now, I went home on the Friday as an “Oracle expert” and came in on the Monday to find my desk had been physically lifted and moved into the Unix sys admin corral and I was now a “Sys Admin not-expert”. My protestations were listened to – and then ignored, with the information that if I did “not knuckle down and get on with it”, the money would stop flowing. So, rather dazed and just a tad unhappy with the situation, I sat – and sulked – at my desk. And there, sat next to me, was Barry.

You are probably expecting me to now tell you that Barry knew Unix sys admin inside out and how he took me under his wing and showed me the ropes. Well, he didn’t. I have no idea where they got Barry from, I think he was a pro-C developer, but he had just been similarly abused by management and deposited into a team he had not signed up for. And he knew even less about being a sys admin than I did. I at least knew my way around a few monitoring commands like top, w, ps, “glance” etc.

Barry was also not very quick with IT. Don’t get me wrong, he was not stupid, but he was not one of these people who just had an affinity for technology and spent all his spare time building their own media server when CD-ROMS for PCs were still quite new. In fact, he seemed to find the whole of IT to be something of a challenge.

What Barry had though was enthusiasm, commitment and curiosity. Not in an annoying, bouncing all over the place crying “this is great” way, but more a case of “OK, server Falcon has run out of disk space. What can I do about it? How do I find out where the storage has all gone, who is using it and can I get it back off them?”. And he would set to. He’d start with what he knew (which was little more than the “Man” {Online Manual} command in the first week) and the bits he could suck out of my head and work through it. Every few minutes he’d be tapping me on the shoulder and saying things like “Look, you can get information about disk usage here, and map it to the real physical disks by greping for this”.

It was Barry’s attitude that made him stand out, and also his ability to infect you with the same attitude. I started off in that team furious and demoralized, determined to find a new position and resign ASAP. But Barry got over his annoyance and started working. He asked me for advice and discussed the issues over with me, even though I was as clueless as him. When he found something he showed me it. When I found something, he was keen to learn it.

Between us, we got by. We knew very little and it was hard work, but because Barry was not daunted and would keep working on the problem until he had it sorted, he dragged me along with him. Often I would still be there with him into the evening, sorting something out when everyone else had gone home. He did not just take on every problem people came to us with though, he would stick with what he felt was the biggest issue until it was sorted, and he would keep with it, and ask for help, and try what you suggested.

{oddly enough , the worst person I ever worked with was already in this team. Maybe that is why the others left and Barry and I were pulled in!}

It only lasted a few months as we both escaped to jobs more suited to our skills, but I learnt a few things. One was that a crummy job could be made a lot better just by your attitude and another was that some people (Barry, not me) had a real talent for enthusing people and thus getting things done. And also, that you did not have to be highly intelligent or knowledgeable to do a very good job. That’s lucky for me, then :-)

Friday Philosophy – The power of cooperation June 27, 2010

Posted by mwidlake in Friday Philosophy, Perceptions, performance.
Tags: , ,
3 comments

Being the person responsible for the performance of an Oracle-based system can be an oddly lonely working experience. It can also be a very gregarious position as you get to meet a lot of people and discuss lots of different aspects of many systems – all systems seem to have performance issues and so people come along to you, hoping you can identify a “work faster” database setting for them.

But you are often the only person who specialises in Oracle performance. You generally need to be in an organisation that is very large or where performance is key to success for there to be justification for dedicating a person to the topic. To have more than one person dedicated to performance your organisations has to have a very strong focus on getting the best performance out of the Oracle and related systems {or really, really atrocious performance issues :-) }. So usually there is no one else around who is as experienced (or more so) as yourself to discuss such things over with or ask for a second opinion.

Which is why I am very lucky at the moment. I’m working in a team of oracle performance people. There are 2.5 of us (one is a manager with other responsibilities, so he only counts as half). Being able to say “Hey, Dave, What do you think of the wait times on scattered reads?” or “how in heck do I force this nested subquery on a view to use a hash join?” and get some other ideas is very valuable.

What is also interesting is how opinions and preferred techniques on tuning can be different and just as valid. As an example, last week I was working on a poorly performing statement. I was at home and it was the evening, so I was not communicating with the rest of the team. I managed to get the code down from 40 minutes to just under 20 by using a combination of a LEADING and USE_HASH hint. I sent the code back to the user. Only to see that within thirty seconds of each other my colleague Graeme had also sent the user a response, again getting the code down to around 20 minutes. Graeme had pulled a chunk of the code into a subquery factoring “WITH” clause and added cardinality hints. Totally different changes.

So Graeme and I then had a “philosophical” discussion about the different changes {“Mine is best” – “No! Mine is, yours is all bloated and complex”- “Your hint is less future-flexible!!!”}. Only joking, we actually discussed the changes and why we each chose what we did. Graeme commented that is showed that tuning was an art and not a science and I countered that it was a science, as we had both identified where we felt the execution plan could be improved but used different techniques to get there. The thing is, Oracle is so complex and has so many options to influence the optimiser that you have flexibility to chose different tools and techniques.

We had both identified the same main improvement but had each come up with different tweaks for later in the plan.

The end result was that we went with Graeme’s main plan {he is bigger than me} but we pulled in my tweak. That bought the execution time down to around 10 minutes, so about four times faster over all and twice as fast of either of us alone. That is one of the advantages of not working alone.

We also then discussed how we could get this code down to seconds with the use of either Materialized views or changing the process that generated the report to do so incrementally and store the daily results. Until one of us realised we had reached the boundary of compulsive tuning disorder. The report running in 10 minutes was enough improvement to satisfy the business, the report was only going to be run over the next couple of months, so spending a day or two re-working it further was going to be fun – but of no advantage to the business. We told each other to finish for the day. So another advantage of not working alone is that not only do you get more technical input but your help prevent each other losing sight of the overall aim.

It really does help to have two people working on the same area.

{There is a sneaky way of getting beyond being a lone performance specialist. If you are in an organisation long enough you can usually find some other idiot who is silly enough to want to learn more about performance and you can train them up. It does not take long before they know enough to start coming up with things you never thought of. Oracle is, after all, full of many ways to do the same thing and you can’t know it all}.

Friday Philosophy – CABs {an expensive way to get nowhere?} March 11, 2010

Posted by mwidlake in biology, development, Friday Philosophy, Management.
Tags: , ,
3 comments

A few years ago, my wife and I went to New York for a holiday. We got a cab from the airport into Manhattan. It was an expensive way to see, at great length, some of the more uninteresting automobile transit routes through New York. We arrived at our hotel a great deal later than we anticipated. And with a lot of our paper dollars no longer in our possession.

I’ve also taken cabs through London, usually at the weekend to get back to Liverpool Street Station. The trip is generally quick, painless and not too expensive, no matter what bit of London is currently being dug up. Those black-cab drivers know their stuff.

Of course, the CABs I refer to in the title of this Friday Philosophy are not private cars for hire. In this context CAB is Change Advisory Board. A term that can make grown developers weep. If you do not know, the Change Advisory Board is a group of people who look at the changes that are planed for a computer system and decide if they are fit for release. My personal experience of them has been similar to my experience of the taxi variety, though sadly more of the New York than London experience.

You might expect me to now sink into a diatribe {ie extended rant} about how I hate CABs. Well, I don’t. CABs can be a part of a valuable and highly worthwhile process control mechanism. Just as proper QA is core to any mature software development process, so CABs are important in getting informed, talented stakeholders to review proposed changes. They check for overall system impact, clashes with other proposed changes that individual development streams may be unaware of, use their own {hopefully deep and wide} experience to consider the changes and to verify Due Diligence has been invoked {that last one is a bit of a minefield and where, I believe, many CABs fail}.

Sadly, though this is often the aim, the end result is too often a bunch of uninformed and technically naive politicos trying to wield power, using the CAB meeting as an extended game of management chess.

I’ve seen CABs trade changes. “I’ll let you have X if I can have Y and Z”. I’ve seen CABs turn down changes because the form had spelling mistakes in it. I’ve seen CABs object to a change that will save the company 5 million pounds a day because it lacked one signature.

That last one just stopped me in my tracks {I’m not exaggerating either, if anything I am underplaying the cost impact of that decision. I saw the figures and I wasted a couple of days of my life checking, 5 million pounds a day was the least I felt I could prove.} We are talking about enough money every day to pay the salary of everyone on the CAB for several years. And they blocked it because the DBA team had not signed off the change. No effort was made to address the lack of the signature in any way, the change was just refused.

The DBA Team had not signed off the change because the one and only DBA Team Leader who was allowed to sign off was on holiday for two weeks. They needed that holiday too, for other but I suspect linked reasons.

Now, I knew the DBA Team Lead and he was a good bloke, he knew his stuff and he was not paid 5 million pounds a day. His deputy was paid even less but was no less talented but she was not allowed to sign off the change as she was not the DBA Team Lead.

That was a CAB gone very wrong. The process of the CAB had been allowed to over-rule good business sense. It was also overruling general and technical sense, but that really is secondary to what keeps the business making a profit.

I’ve seen the opposite of course, technical teams that just apply whatever changes they feel are fit, with no oversight or CAB. To be honest, this less controlled process seem to mess up less often than a poor CAB process as the technicians know they are the ones who will spend the weekend fixing a mess if one occurs. But that mess up will occur eventually, if control is lacking, and the bigger and more complex the IT environment, the greater the chance of the mess up.

So, I feel CABs are good, no make that Great, if you have the right people on them and you have a sensible cascade of authority so one person being away does not block the system. That is quite a bit harder to put in place than a simple “Dave A, John, Andrea, Alex, Raj, Dave P, Mal, Malcolm and Sarah have final signoff” which most CABs effecively become.

But there is one last fault of CABs I want to highlight. They tend to treat all changes in the same way and all changes are not the same. Upgrading the underlying OS is not the same as adding a cardinality hint to one Business Objects report.

If your CAB or change process treat the two above examples the same, then your CAB or change process is broken. Now, in all IT “rules of thumb” there is an exception. In this case, I am truly struggling to think of one. My feeling is that if your change process treats an OS upgrade the same as adding a hint to a report, it is not fit for purpose.

Those are my main issue with CABs. They should be of significant business importance, but nearly always they are implemented with one process to deal with all situations and then get taken over by people with an “Office Politics” agenda as opposed to a “Getting the best job we can reasonably expect done” agenda.

I’m very passionate about this and I have a way I hope can throw this issue into context, an analogy.

Ask yourself this senario.
You go to your doctor with a niggly cough you have had for a week OR you go to your doctor because you almost passed out each day you got out of bed for the last three days.
If your doctor treated you the same for both sets of symptoms, would you be happy with that doctor?

Why are all IT changes handled by most CABs in exactly the same way?

(BTW if you ever almost collapse when you get out of the bed in the morning, do NOT go to work, go instead to your doctor and ask them for a full medical and if he/she does not take blood pressure readings and order a full blood chemisty test, go find a new doctor.)

Command Line or GUI – Which is Best? February 18, 2010

Posted by mwidlake in performance.
Tags: , ,
15 comments

At present I am suffering ever so slightly from “split personality disorder”* in respect of my liking for Command Line and GUI interfaces.

On the one hand, much to my colleagues mild reproach, I use SQL*PLus and not PL/SQL Developer for my day-to-day work. Even worse, I got sick of using notepad to hack around scripts {I am working in a windows client environment and you simply can’t use MS Word with SQL files!} so I have retrograded recently and downloaded a windows-complient ‘vi’ interface and it is soooooo nice to be able to use powerful ‘vi’ commands on my files once more. “:.,$s/^ /,/”. Ahhh, it is so much easier. I can do stuff in 3 seconds in ‘vi’ that would take me 10 minutes in Notepad in a large, complex file. That and, I’m sorry, but notepad seems to be unable to manage a 100MB file, despite me having 2GB of real memory and a decent machine, but ‘vi’ has no problem with it at all.
Even more retrograde, I have direct telnet access to my linux servers and I am getting back to that as it makes me so much more productive. “ls -alrt ^d” for all directories anyone? “df -k .” to see how many data files I can add? Yep, it’s all arcane and means so little to many modern IT “Java/Struts/CDE” people but boy it is direct and fast. I might even dig out that book on SED and AWK.

On the other hand, I have finally (after much very painful discussions back and forth) got agreement that my site probably has access to AWR, ASH and all that good performance repository stuff. So I am hacking around with the OEM screens that focus on performance and snapshots and stuff. Now, I am traditionally not a big fan of GUI DBA tools. Partly it is because I am a bit old and stuck in my ways and partly it is because GUIs are really just “menus of options”. You are limited to what options are available in your DBA GUI tool and you have a harder time learning all the options available or what is really going on “under the covers”.

But with AWR and all those graphs, links and utilities, you can drill down into real problems real time or in the past so effectively that, well, once they start using this tool properly they will not need me at all. It is a fantastic boon to performance management and problem resolution, as well as proactive performance management.

So there you are, I am with Doug Burns on this one, in that I have Learned to Love Pictures. When the Pictures are well thought out and well connected and simple enough to help make sense of a complex system {and Oh Boy Oracle performance has become sooo Complex!!!!}

So right now, I spend half my day in vi/linux/command line world and half of it in pretty picture/GUI world. I think what really makes me happy is to leave behind the half-way-house of text-like Windows World {Windows SQL*Plus, Notepad}.

Just to finish, you can’t mention AWR without someone raising the ugly issue of licence cost and how Evil Oracle Corp were to charge for it. Well, I think it has been established that the guys and gals who developed AWR/ASH did not expect it to become a cost option but it did. And I suspect that what kept it a cost option was the community’s OutRage at it being a cost option. Anything that popular, hey, a commercial company is going to charge for. I still reckon Oracle Corp ballsed up as making it free and helping people use it a bit would have made 90% of customers’ lives easier and would have translated into user happiness and a certain % of sales for training courses to learn more, but heck my day job is to make things work, not maintain sales percentages, so my opinion counts for nowt. *sigh*

(*apologies to real sufferers of Dissociative Identity Disorder, I am using the term in the loose, non-scientific, “common usage” term of “not sure of my opinion” rather than having truly disparate personalities and memories.** And note, I certainly do not mean schizophrenia which, despite the on-going public-opinion misunderstanding, is rarely anything to do with multiple personality disorders or “spit minds” AT ALL, and is more to do with a difficulty in determining between reality and hallucination. ).

Making Things Better Makes Things Worse February 11, 2010

Posted by mwidlake in development, Management, Perceptions.
Tags: , ,
12 comments

This could be a Friday Philosophy but I’ve got a couple of those lined up already. Anyway, I am suffering at work at the moment. I’m encountering a phenomenon that I have talked about with Dennis Adams a couple of times. It probably has a proper term, but basically it is the odd situation that when you make things better, you get more complaints. {Dennis, do you know the proper term?}

{Update. Dennis was good enough to link to this paper he wrote on customer feedback}

Anyway, Let me explain. You have an application sitting on a database. The screens are slow, the reports take an age to come out, your might even have considerable system instability and unplanned outages. The users are not unhappy. They were unhappy last year. Now they are just cynical and they just expect the system to be slow, unresponsive, flaky. So they do not report any problems.

Then things change. The system gets some much-needed care and attention and now the slowest reports get tuned up, the screens come back faster and less spontaneous department-wide coffee breaks are caused by the system crashing. Everything gets better. But not for the help desk, now they start getting calls. “This report is too slow”. “Why can’t I jump straight from the customer screen to the pending orders screen?”. This happens because the users now realise that something can be done. There is a point in complaining as there is a chance their piece of misery could be made better. I certainly went through this many years ago when I inherited a system that crashed every week. No one mentioned it, they just went to tea and complained about it. The first time it crashed after I arrived I could not believe that no one had called before I had realised it had died. Once it had been up solidly for a couple of months, then when it crashed boy did we hear about it!

Also, when you improve a system and things generally get better, occasionally something will improve and then fall back a bit. Hardly anyone says “thanks” for the initial improvement but they will say something if it improves and then drops back.

That is what is happening for my main client at the moment. The system was not that bad, but it needed some help. Why else would I be there? I’ve been beavering away with the rest of the team and we are making things better, so far mostly at an underlying “getting the overall system straight” level. A new chap has joined to help concentrate on performance and he is really making inroads into specific things, individual reports and processes that need a good sorting out.

So now that things are getting better and performance is generally improving, anything that is still slow is being brought up by the development and support teams. Also, we’ve made a few things slower (I’m sorry, it just happens like that) and they certainly get mentioned.

So, I’m busy. And I could get annoyed at people asking why X is slower when Y and Z are faster. But I don’t, because Dennis explained this counter intuitive theory to me.

I know things are getting better as people are annoyed as opposed to apathetic :-)

The Evenings are Drawing Out December 14, 2009

Posted by mwidlake in Perceptions.
Tags: ,
1 comment so far

Today, December 14th, sunset was later than yesterday. In London it was 15:51 and 50ish seconds. Tomorrow, the sun will resolutely stay in the sky until 15:52 {and a few seconds}. The days are drawing out at last.

English Sunset by Angie Tianshi

But it is not the shortest day of the year {I should say daytime really, all days are the same length give or take odd leap-seconds)

What is the shortest day, I hear you all cry?

This year, 2009,

The shortest day is December 21st

The date with the shortest period of daylight {in the Northern hemisphere} is the the 21st/22nd December, depending on how long ago the last leap-year was. And everyone knows that the the shortest day will also be the day where the sun sets earliest, it makes sense.

Except it does not quite work like that.

We probably all remember from our school days that the earth goes around the sun at an angle from the “vertical”, if vertical is taken as at 90 degrees to the circle the planet takes as it spins around the sun. Think of it like an old man sitting in a rocking chair. He is rocked back in his chair, head pointing back away from the sun in the Northern Hemisphere’s winter and feet pointing slightly towards the sun. Come Midsummers day, he has rocked forward, head pointing towards the sun for the Northern Hemisphere summer. One rocking motion takes a year.

Well, old earth is also slumped slightly to one side in his chair. This results in a slight skew on when sunset starts drawing out and when sunrise starts drawing in. Sunrise will continue to get later until we hit the New Year (Western New Year, not Chinese :-) ). It just so happens that sunset gets later at a slower rate than sunrise gets later until the 21st, when we hit the Shortest Day.

To be fair, I missed the boat slightly, the point at which the evenings started to stretch out was actually the 13th or 14th December but I did not have time to blog until today.

The below tables help make this situation clearer, I include one for the UK and one for Australia. The jolly nice site it links to allows you to change the location to wherever you are in the world (well, the nearest Capital).

Table of sunrise/sunset times for London

Table of surise/sunset for Sydney, Australia

If you want to know a little bit more about the relative position of the Sun and Earth over the year, you could check out my little rambling comments on when we are closest to the sun. It comes as a surprise to most people on the “top half” of the planet.

What has this to do with Oracle, Performance and VLDBs? Nothing much, except to highlight that the obvious is not always correct, just as it is with Databases, IT and in fact science in general.

I’ll finish with a sunset picture from Auz. Ahhhh.

Outback sunset from ospoz.wordpress.com

Friday Philosophy – What Was My Job Again? December 11, 2009

Posted by mwidlake in Management.
Tags: , ,
2 comments

How much control do we have over what job we are in?

I know I have mentioned this before, but I did not choose to work with Oracle – I fell into it by accident. I found myself in a job which I was no longer enjoying, for a boss who had issues with me and I had issues with a salary (mine, of course). So I decided to apply for a job with a company a friend of mine worked for:

For those of you who are not UK-born or have seen less than three decades go by, in 1990 “Oracle” was a teletext company that provided information on your TV screen for the TV company Channel 4. Basic news, entertainment, sports etc via chunky text and chunkier graphics thrown up in glorious low-res.

I honestly thought I was going for an interview with that company, as opposed to the other “Oracle”, some database company with the same name which went on to pretty much conquer the corporate database and applications world. That was a pretty lucky break for me and I made a conscious decision to stick with this database stuff.

Many years later I was on a contract doing database performance tuning. Someone, a manager, came and asked me about how they could save space, not in the database world but in the Unix filesystem world. So I made the suggestion that they check out the man page on compress. We compressed some big files and he went away happy, leaving me to get on with my database stuff. My big mistake was, when he came back a week later and asked how he could get the compressed data out, I promptly showed him. I revealed too much knowledge.

I came in to work the next Monday morning and my desk had gone. There was an oblong square of dust and hula-hoop crumbs, nothing else. Even my pile of “documents to get back to” from under the desk was gone. Had I been sacked? No, I had been put in the Unix system administration team. My desk had been picked up and physically moved across the room to the Unix Corral, along with everthing on, under or next to it.

No one has asked me, it had not been mentioned to me at all, the managers had just realised on Friday that half the existing team (contractors) had left at the end of the week and no replacements had been found. That devious manager I had helped had told the others I was a whizz at Unix and so my fate was sealed. I was not a whizz at Unix, I was barely competent at basic shell programming. But I learnt a bit before deciding I wanted to stick at the Database stuff and went off to a contract doing that again. I still kind of wish I’d done the Unix a little longer though.

The final shift I’ll mention, and is probably more commonly echoed in other people’s experience, is coming in one day to find you are a manager. This had happened to me small-scale a couple of times, taking on a contract where I ended up in charge of a team, but in this particular case I was a permanent employee managing a team of 4 DBAs and my boss left. Within the week I found that I was being treated as the manager of 5 or 6 teams, totalling about 30 people. More by them than by upper management, but upper management cottoned on and asked me to do the job. Long story cut short, I resisted the move upwards but it happened anyway. Not, at that time, what I wanted at all.

I’ve told the above story a few times when doing presentations on management-related topics and many people, a surprising number to me,  have said to me afterwards that the same sort of thing happened to them. I am also now chair of the Management and Infrastructure Special Interest Group of the UK Oracle user group. That SIG is full of people with a similar story.

What is the point of this particular Friday Philosophy? Well, these experiences have made me realise that a lot of people are probably doing jobs they just found themselves in, or in the case of managers, just got pushed into.

If you did not chose your job, you are unlikely to be a good fit, especially to start with.

I’m sure most of us have experienced this and, looking back, can see that initially we lacked the skills, the background, even the inclination for the role. But we either got on, moved on, or become morose and bitter.

This also means that we are all probably encountering a lot of people in that exact situation, all the time – People doing a job they just found themselves in. So, if someone seems to not be doing a job as well as they could, check how long they have been doing it. If it is a recent change, remember your own experience and cut them some slack. You would have appreciated it when you were them.

It also explains Morose and Bitter Geoff who manages Accounts too, doesn’t it?

The Frustrated User’s perspective. November 28, 2009

Posted by mwidlake in Perceptions.
Tags: ,
1 comment so far

I got the below email from a friend this evening. Said friend does not work in IT. He works in a large organisation with a large and active IT department that might just be forgetting they provide a service as opposed to laying down the law…

****************************************************************
Hi Martin

For the last few weeks since {an edited out software virus disaster} we have been bombarded with unsolicited security policies from I.T. They pop up during the 10-15 minutes it takes to logon to our computers. You then have to download the policy and sign at the bottom to say whether you accept or decline the policy. When I scanned through the 10th policy I was struck by the fact that none of it applied to my area of responsibility except for one small part that had been covered in excruciating detail in one of their previous pathetic attempts at communicating what is expected of us. And all said missives using what looks like a variation of the english language. Having skipped the policy during a number of recent logons I was now being informed that it is “mandatory” to accept the policy or decline it giving a reason. I declined giving the above observation on the lack of relevance to my role as a reason.

I have now been informed that it is not possible to issue only the relevant policies to individuals (and presumably having identified this is not possible, have not bothered trying in the first place?) and in any case there might come a time when I “might” be given a task where the latest I.T policy applies and therefore I have to be aware of the existance of the policy. I think this latest one was something to do with purchasing software packages from suppliers -although this isn’t entirely clear. There is no way that I would be allowed to purchase software packages, which is a shame as there are off the shelf products that do what we require, whereas the in-house system foisted upon us simply does not provide any reliable or useful information what-so-ever.

The following senario occurs to me. I write a policy on controlling legionella – not unreasonable given that we have swimming pools, showers, air con etc. in our premises. I then send a copy to every employee requiring them to open it — expect them to read it —- understand it —- and accept it, “just-in-case” they get asked to go and run a sports centre. What response do think I would get?

Although the risk of catching legionella is low, people have died as a result, but we do not require everyone to sign a policy for this or any of the other more serious hazards they face at work. I am not aware of any software-purchasing-related deaths of late. For dangerous stuff employees sign one policy when they join the organisation. If they have to deal with a hazard we make them aware by warning them about it and if necessary give them additional training, guidance and support so that they can manage the risk in accordance with the overall policy.

Perhaps we have got this wrong. Maybe we should require all computer users (just for example) to complete a workstation assessment online every day when they start work – and if they don’t their computer should blow up in their face and a guilotine then drop from the ceiling removing their hands so they can’t sue for RSI or eyestrain.

That’ll teach them
************************************************************

I hope I have never been responsible for inflicting enough inconvenienve on my users to make them as aggrieved and angry as my friend.. Thing is, I now worry that I might have…

Friday Philosophy – How many Consistent Gets are Too Much? October 30, 2009

Posted by mwidlake in Perceptions, performance.
Tags: , ,
2 comments

One of my good friends, Piet de Visser commented on a recent post that “380 {consistent gets} is too much” per row returned. He is right. He is wrong. Which?

Piet is actually referring to a given scenario I had just described, so he is probably {heck, he is spot on} correct as his comment was made in context – but it set me to thinking about the number of times I have been asked “is the number of consistent gets good or bad” without any real consideration to the context. The person asking the question usually just wanted a black/white good/bad ratio, which is what Piet also mentioned in his comment, a need to discuss such a ratio. I am on holiday in New England with my second bottle of wine, memories of having spent the day eating Lobster, kicking though Fall leaves and sitting by a warm fire reading a book, so I am mellow enough to oblige.

Sadly, out of context, no such ratio probably exists. *sigh*. There evaporates the warm glow of the day :-).

The question of “is the number of consistent gets per row good or bad?” is a bit like the question “is the pay rate good enough?”. It really depends on the whole context, but there is probably an upper limit. If I am helping my brother fit {yet another} kitchen then the pay rate is low. He has helped me fit a few, I have helped him fit a few, a couple of pints down the pub is enough and that equates to about 30p an hour. Bog standard production DBA work? 30-60 pounds an hour seems to be the going rate. Project Managing a system move that has gone wrong 3 times already? I need to factor in a long holiday on top of my normal day rate, so probably high hundreds a day. £10,000 a day? I don’t care what it is, I ain’t doing it as it is either illegal, highly unpleasant, both or involves chucking/kicking/hitting a ball around a field and slagging off the ref, and I ain’t good at ball games.

I have a rule of thumb, and I think a rule of thumb is as good as you can manage with such a question as “is {some sort of work activity on the database} per row too much?”. With consistent gets, if the query has less than 5 tables, no group functions and is asking a sensible question {like details of an order, where this lab sample is, who owes me money} then:

  • below 10 is good
  • 10-100 I can live with but may be room for improvement
  • above 100 per record, let’s have a look.

Scary “page-of-A4″ SQL statement with no group functions?

  • 100-1000 consistent gets is per row is fine unless you have a business reason to ask for better performance.

Query contains GROUP BY or analytical functions, all bets are pretty much off unless you are looking at

  • a million consistent gets or 100,000 buffer gets, in which case it is once again time to ask “is this fast enough for the business”.

The million consistent gets or 100,000 buffer gets is currently my break-even “it is probably too much”, equivalent to I won’t do anything for £10 grand. 5 years ago I would have looked quizzically at anything over 200,000 consistent gets or 10,000 buffer gets but systems get bigger and faster {and I worry I am getting old enough to start becoming unable to ever look a million buffer gets in the eye and not flinch}. Buffer gets at 10% of the consistent gets, I look at. It might be doing a massive full table scan in which case fair enough, it might be satisfying a simple OLTP query in which case, what the Hell is broken?

The over-riding factor to all the above ratios though is “is the business suffering an impact as performance of the database is not enough to cope”? If there is a business impact, even if the ratio is 10 consistent gets per row, you have a look.

Something I have learnt to look out for though is DISTINCT. I look at DISTINCT in the same way a medic looks at a patient holding a sheaf of website printouts – with severe apprehension. I had an interesting problem a few years back. “Last week” a query took 5 minutes to come back and did so with 3 rows. The query was tweaked and now it comes back with 4 rows and takes 40 minutes. Why?

I rolled up my mental sleeves and dug in. Consistent gets before the tweak? A couple of million. After the tweak? About a hundred and 30 million or something. The SQL had DISTINCT clause. Right, let’s remove the DISTINCT. First version came back with 30 or 40 thousand records, the second with a cool couple of million. The code itself was efficient, except it was traversing a classic Chasm Trap in the database design {and if you don’t know what a Chasm Trap is, well that is because Database Design is not taught anymore, HA!}. Enough to say, the code was first finding many thousands of duplicates and now many millions of duplicates.
So, if there is a DISTINCT in the sql statement, I don’t care how many consistent gets are involved, of buffer gets or elapsed time. I take out that DISTINCT and see what the actual number of records returned is.

Which is a long-winded way of me saying that some factors over-ride even “rule of thumb” rules. so, as a rule of thumb, if a DISTINCT is involved I ignore my other Rules of Thumb. If not, I have a set of Rules of Thumb to guide my level of anxiety over a SQL statement, but all Rules of Thumb are over-ridden by a real business need.

Right, bottle 2 of wine empty, Wife has spotted the nature of my quiet repose, time to log off.

Follow

Get every new post delivered to your Inbox.

Join 171 other followers