jump to navigation

Friday Philosophy – What Makes a Community? November 10, 2017

Posted by mwidlake in conference, Friday Philosophy, Perceptions, Presenting, UKOUG, User Groups.
Tags: , , ,
1 comment so far

Earlier this week Daniel Westermann asked a question on twitter: “What is community?”

What is a community?

Daniel was not specific about if this was a work/user group community or a wider consideration of society, but my first thoughts were about the Oracle community (or communities) that I have been involved in. By community I mean anything from a national or regional user group; a regular, geographically agnostic conference; a special interest group; even just a bunch of people who meet socially who share a common interest (such as London Oracle Beers which I help run). You could actually think of it as the world-wide collective of all such smaller Oracle communities.

I’ve thought about this a lot over the years and you can see my answer in the right. Quite obviously an Oracle community needs a shared interest in Oracle, in some aspect of it or a broader view. All tech communities focus on a brand of tech, I don’t think you get a “computers” community as it is just too broad. But the parts that make up the community are, I think, alwyas the same.

1) A large group of people willing to take part
+
2) A medium group of people willing to share
+
3) A small group of people willing to drive the community

Taking a regular conference as an example, the first group are the delegates. If not enough people are willing to pay for it and turn up then your conference will fail. The second group are the speakers and people who will help with organising. The third group are the ones who get the second group involved, manage the effort and sell the idea of the conference.

Community at UKOUG

That third, small group is the key. If you lack that, you have no community. Sometimes, especially for smaller groups, that third group could be very small, even just one person. Delegates and speakers can come and go but it’s not so easy with the drivers of a community.

For several years we had a small but active Oracle user group in the centre of the UK, in Birmingham. It was run by one person, Mike Mckay-dirden. He almost single handedly started it up, organised the venue and corralled some of us speakers into coming over to talk. It ran successfully for several years but then Mike decided he could not keep doing it. He stopped, no one took it over – and the community died.

With larger communities such as UKOUG or DOAG there will be several people driving it all forward and usually, if one drops out you can keep going until another driven person turns up to help. But it is always a very small group of people doing a hell of a lot of work.

Over the years I’ve watched some communities get stronger or weaker and even die off as those key, driving people change. You can tell who they are, they are the ones who look knackered all the time :-). The LOB is in danger of dying as a couple of the driving people are no longer around and I can’t get to London very often now.

The chances are that as you are reading this blog you are part of an Oracle community. If so, I’d encourage you to support the drivers in those communities. If you lose them, it could really badly impact your community. Would I encourage you to become one of those drivers? Well, I would. But you have to want to do it yourself – it’s a lot of hard work and sometimes it feels like none of the first and second group really appreciate what you are doing, which can be very demoralising. And it eats up a lot of time.

I went from being one of the large group willing to take part to a member of the medium group willing to share pretty quickly. After a few years I stepped up to being in the smaller group, for a couple of groups actually. I took those steps up because I wanted to, not with any real expectation of gain (I could see it was going to be me working “for free”!). But I am absolutely sure that I would not be where I am now if I had not. I would not be an OakTable member, I would not be an Oracle ACE Director, and I would not be a known international speaker if I had not at least joined the middle group. Joining the band of drivers introduced me to a lot of really nice, really helpful people too.

This blog has been all about Oracle communities but I think the three-group-theory might apply to all communities. I’ll have to think on that a little longer before I voice an opinion. One thing I do know – It’s really nice being part of communities.

Advertisements

Friday Philosophy – Smart or Smart-Arse? October 20, 2017

Posted by mwidlake in Friday Philosophy, humour, Perceptions, rant.
Tags: , , ,
1 comment so far

Many of you know what a “Smart-Arse” is. For those who do not…

A “Smart-arse” a person who is irritating because they behave as if they know everything or try to catch you out by misleading you.

A smart person will look at your problem and say something like “have you tried checking the array size?” and, 8 times out of 10, their input will help you solve your problem. It may not be THE answer but it makes you think about root causes.

A Smart-arse will say something more like “well, I would never have your problem as I would not have joined a company full of Java Nerds!!!”. Yeah, maybe that would have avoided my specific problem #1, but it is of no practical worth right now. .

You can usually pick out a smart-arse just by the tone of their voice. Think about the two situations above. The Smart person probably gave their advice in a quieter voice and with a passive or supporting manner. The Smart Arse person will usually have a higher, louder voice with a slightly sarcastic tone. Basically, in a superior or attention-seeking manner.

Another “Smart-arse” thing to do is to try to catch you out, in the misguided belief it makes them look cleverer than you.

In these situations always ask – “what is a Smart-arse hoping you won’t spot”

I’ll put my hand up right now. Sometimes, especially looking back on my past, I have been the smart-arse. (And, as humans, we hate the fault in others that we see in ourselves). And I bloody hate smart-arses. And I keep seeing smart-arse crap all over the internet. Let me give you an example. Look at the figure on the right.

This is the perfect example of the “Smart-Arse” question. You are faced with what looks like a simple logic puzzle and normally the tag line is something like “93% of people will get this WRONG!!! – Only Geniuses can solve it!!!!!!!”. They never cite a basis for the “93%” as it is as utterly made up and is as asinine and bloody annoying as whatever trick is in the post. What they are doing is giving you what looks like a genuine puzzle that needs a bit of thinking about but most of us can solve (though not you Dave, you really are an idiot). BUT! But they have hidden a detail. The are purposefully leading you astray so they can go “Aaa-Haaaa!!! Noooo! You forgot to check for the closed pipes!” (check tank 5 to 2). Or whatever the trick is.

This is “Smart-Arse”. It is not checking if you can solve a problem, if you are smart. It is checking if they can con you. Checking if they can give you a load of information and then go “Ahh HA!! Got ya!!! You did not check for the tiny bit of info we hid from you!!! O-hohohohho we are so clever!!!!”

Well, I have news for you, Smart-arse. You are a bloody idiot. Your answer is wrong, as any fool can see. (Bear with me on this…)

More boxes, same smart-arse shit

You may have seen other “tests” like this such as the one to the left – a bit more complex but the question is the same, which one fills up first.
In my head my response is always “which one fills up… *first*????”.

First! That is my response. Not which one but the fact that the question itself is wrong. It should be “which one fills up” full stop, as “any fool can see”. Not which one fills up first.

I better justify that claim.

Look at this second example, clearly labelled with the icon of utmost smarts Mr Einstein (who, I bet, could not plumb a toilet let alone all these pipes as, back in his time, there were no push-fittings – just copper and solder. I think he once said he was rubbish at practical tasks). They think the key “got ya” is that the pipe from C to D is blocked so water goes from C to J. And then from J to L, as the pipe to I exits higher than that to L. One sneaky trick and then a bit of good physics – it is not the input but the output that counts. So water pours into L and then to F – but not H as, again, a sneaky block has been inserted. So F fills up.

And only F fills up!!!

As as soon as it is full it overflows. It overflows below the height of any other buckets the fluid had flown through – and so no other bucket will fill. Their initial question is flawed. “Which will fill first” indicates one will fill second. No other bucket will fill second. The question is not logical! Bloody idiots.

I can’t say why I initially was struck by the fact that only one bucket would fill when I saw the first example of this, even before I spotted the blocked pipes, but we all think in different ways. It does not make me smarter, just different. I’m interested to see if any of you can point out a flaw in my logic above though as I have a sneaking suspicion I could still be wrong.

However, this is not the major flaw… (I told you that I was also an utterly insufferable smart-arse).

Figure 2 has a drip filling the A tank, figure 1 has a gushing tap filling tank 1. Now ask a simple question. No matter if the tap is gushing or dripping, can the pipe out of tank A (or 1) empty the water faster than the tap supplies it? Well, if the tap is dripping you would say “yes” – but if these tanks are 5mm cubed and the pipe out is less than 1mm thick then no! No scale is given. And in fig 1 the tap is gushing. Have any of you had a shower where the plug hole drains slower than the shower produces water? After 2 or 3 minutes your feet are in a shallow bath and if you keep the shower running it overflows into the rest of the bathroom.

With figure 1, the one with the gushing tap, my brain says that tank 1 will fill as the tap supplies water faster than it will exit through the pipe to tank 5. Tank 1 will fill and piss water all over the shop and whatever goes down the pipe to 5 will eventually fill that tank. Which of tanks 1 and 5 fills first is “it depends” (the classic answer to most I.T performance questions). The question is how much slower is the flow out of the pipe from tank 1 – if it is, on average, above half the rate of the tap flow then tank 5 will actually fill first. In any case, you have soaked the bathroom floor and the people in the flat below are banging on the front door…

With that new idea in your head, if you turn up the tap in figure 2 you can now see that which tank fills first is probably A or.. C – depending on the max flow out of the pipes (all pipes are the same bore so flow rate is the same, increasing header pressure in each tank as they fill allowing…) I think it might be C as it’s outflow is higher in relation to the tank top than B or C…

So depending on the tap flow rate, the drain pipe flow rate and the relative height of the clear output pipe it could be…. absolutely NOT the answer of the Smart-Arse original poster. That is the problem with smart-arses! They are so fixed on their clever “gotcha” answers that they stop thinking of the real world.

And don’t get me started on those images where bananas are added to cans of beer and divided by a plate of cakes, designed to look like some sort of Algebra test. Always they are being smart-arse. They try and hide the introduction of multipliers where all the first examples are addition, or you need to count the number of items, or yellow is 3 and green is 6, or it is in base 23. I was going to include an example (again, a really wrong one) but I’ll save that for another week when I am also in a bad mood.

And, of course, I am a “smart-arse” for pointing this all out. Did I say how much I dislike smart-arses?

I promise you, when you start looking for the smart-arse aspect to all those “are you smart enough” bollocks things on social media it just turns into so much blargh and you can either answer them easily or just decide you can’t be bothered being misdirected. And you can use that saved time for looking at funny kitten videos or, I don’t know, doing some productive work?

Is there any other relevance to your working life? Maybe. Next time your management structure asks you a seemingly benign question about what you are doing this weekend or when you think you are on leave (hang on, “think”?), or how minor will be the impact of a small change to how the business functions to the application you are developing – just switch on the bulb with “smart-arse” painted on it. They are asking you a question where they are expecting you to think in the clear, simplistic way most of us would. Now ask what the bloody hell they are up to.

Overloaded Indexes (for ODC Appreciation Day) October 10, 2017

Posted by mwidlake in database design, development, performance, SQL.
Tags: , ,
add a comment

ODC Appreciation Day is an idea that Tim Hall (aka Oracle-Base) came up with, to show out appreciation for the Oracle Technology Network (OTN)/Oracle Developer Community.

Fig 1 This is an efficient range scan

I want to show my support but rather than mention an Oracle “feature” I particularly like I’d like to mention a “trick” that I (and many others) often employ to help performance. That trick is Overloaded Indexes.

We can all appreciate that accessing data in a table via an index is a very good thing to do when we want just the few records for a given index key from the thousands or millions of rows in the table. As an example, we are looking up all the records for someone and we know their first name, last name and date of birth – a very common occurrence in things like hospital or billing systems. So our PERSON table has an index on those three columns. Oracle will read the root block of the index, read the one relevant block in each of the branch levels for the key, find all of the leaf block entries for that key – and then collect the relevant rows from the table. Fig 1 shows how we think of this working. i.e. most of the records we want to find will be close together in the table.

Actually, a lot of people who just “use” the Oracle database as a store of information sort-of think this is how an index always works. It efficiently identifies the rows that you want and that is the end of it. If the index is on the value(s) you are looking up rows for (say LAST_NAME, FIRST_NAME, DATE_OF_BIRTH) the index is ideal and that is as good as it gets.

Fig 2 – a more usual, less efficient index range scan

But in reality, the index lookup is often far less efficient than this and is more like fig 2. Working down the index structure to get all of the required index entries is exactly the same, but the rows you want are scattered all over the table. Oracle has to fetch many table blocks to get your data, maybe as many blocks as there records to be found. This is far from efficient.

So what can you do about this? You already have the “perfect” index, on LAST_NAME, FIRST_NAME, DATE_OF_BIRTH, the values you are looking up. Maybe you could add another column to the index to avoid those situations where there are many people with the same name and date of birth. But you may not have that extra information or it is simply not possible to identify the values in the table any more accurately, you really do need all the rows scattered though that table for the given search key.

There are “architectural” things you can do such as create the table as an Index Organised Table (see my little set of blogs about them starting here). You can also use various methods to group the relevant rows together in the table. But all of those methods are Big Impact. You need to recreate the table or plan for this eventuality up-front when you design the system.

But there is a very specific, easy thing you can do to address this particular problem, for the SQL statement you need to speed up. You can add all the columns your query needs into the index. This is an Overloaded Index.

An Overloaded Index holds not only the table columns in the WHERE clause but all the columns needed from that table for the query.

Why does this work? Because when Oracle identifies the range of keys for the key (LAST_NAME, FIRST_NAME, DATE_OF_BIRTH) all the other columns it needs are also in those index leaf entries and there is no need to get the rows from the table. All those lookups to the table are avoided. Fig 3 at the end of this article demonstrates this.

However, I’ll give you a real world example I had recently. A client had a SELECT statement something like the below, with the execution plan shown, and it was running too slowly for the business requirements:

SELECT (SUM(NVL(T.TRAN_VALUE_CR,0))-SUM(NVL(T.TRAN_VALUE_DB,0))) ,
        COUNT(*)
FROM    ACCOUNTS     A ,
        TRANSACTIONS T
WHERE   A.ACC_XYZ_IND      =:3
AND     A.ACC_ACCOUNT_NO   =:1             
AND     A.ACC_SUBACC_NO    =:2
AND     T.TRAN_XYZ_IND     =A.ACC_XYZ_IND
AND     T.TRAN_ACCOUNT_NO  =A.ACC_ACCOUNT_NO
AND     T.TRAN_SUBACC_NO   =A.ACC_SUBACC_NO
AND     T.TRAN_P_IND       =:4 
AND     T.TRAN_DLM_DATE    >=TO_DATE(:5,'YYYYMMDD')

------------------------------------------------------------
| Operation                                 | Name         | 
------------------------------------------------------------
|   0 | SELECT STATEMENT                    |              |
|   1 |  SORT AGGREGATE                     |              |
|   2 |   NESTED LOOPS                      |              |
|   3 |    NESTED LOOPS                     |              |
|*  4 |     INDEX RANGE SCAN                | ACC_PRIME    |
|*  5 |      INDEX RANGE SCAN               | TRAN2_3      |
|*  6 |    TABLE ACCESS BY LOCAL INDEX ROWID| TRANSACTIONS |
------------------------------------------------------------
Statistics
----------------------------------------------------------
       4740  consistent gets
       3317  physical reads

The index used on TRANSACTIONS is:

INDEX_NAME                   TABLE_NAME       PSN COL_NAME
---------------------------- ---------------- --- --------------
TRAN2_3                      TRANSACTIONS     1   TRAN_ACCOUNT_NO
TRAN2_3                      TRANSACTIONS     2   TRAN_SUBACC_NO
TRAN2_3                      TRANSACTIONS     3   TRAN_DLM_DATE

The index TRAN2_3 on the TRANSACTION table that you can see being used in the plan was for all the columns being used in the WHERE clause that actually helped identify the TRANSACTION records required – TRAN_ACCOUNT_NO, TRAN_SUBACC_NO and TRAN_DLM_DATE (the TRAN_XYZ_IND and TRAN_P_IND were always the same so “pointless” to index).

I added a new index to the TRANSACTION table. I added a new index rather than change the existing index as we did not want to impact other code and we wanted to be able to drop this new index if there were any unexpected problems. I added all the columns on the TRANSACTION table that were in the SELECT list, were in the the WHERE clauses even though they did not help better identify the rows needed. If there had been TRANSACTION columns in an ORDER BY or windowing clause, I would have added them too. So my index looked like this:

create index TRAN2_FQ on TRANSACTIONS
(TRAN_ACCOUNT_NO   ,TRAN_SUBACC_NO   ,TRAN_DLM_DATE
,TRAN_P_IND        ,TRAN_XYZ_IND
,TRAN_VALUE_CR     ,TRAN_VALUE_DB)

It is very, very important that the new index holds every column from the TRANSACTION table that the query needs.To prevent accessing the table, all the data the query needs for that table must be in the index.

The query could now satisfy the query by just using the new index, as the below explain plan shows.

----------------------------------------------------
| Id  | Operation                      | Name      |
----------------------------------------------------
|   0 | SELECT STATEMENT               |           |
|   1 |  SORT AGGREGATE                |           |
|   2 |   NESTED LOOPS                 |           |
|*  3 |    INDEX RANGE SCAN            | ACC_PRIME |
|*  4 |     INDEX RANGE SCAN           | TRAN2_FQ  |
----------------------------------------------------
Statistics
----------------------------------------------------
         56  consistent gets
         52  physical reads

There is now no line in the plan for visiting the table TRANSACTIONS and we are using the new TRAN2_FQ index. The consistent gets and physical reads to satisfy the query have gone down from 4740 and 3317 respectively to 56 and 52. I think that is good enough.

Fig 3 shows what is happening. The new index is effectively a “mini IOT” designed to support the given query.

Fig 3 – Tables? Who needs tables!

There are of course a few caveats. The new index needs to be maintained, which is an overhead on all INSERT/UPDATE/MERGE/DELETE activity on the table. The index will only remove the need to visit the table for queries that are very, very similar to the one it is designed for – ones that use the same rows from the TRANSACTIONS table or a subset of them. If you alter the query you, e.g. select another column from the TRANSACTION table you would need to revisit this overloaded index.

Finally, be careful of modifying existing indexes to overload them to support specific queries. If the index is there to support referential integrity you need to think carefully about this and the modified index may be less efficient for other queries that used the original index (as adding columns to an index make it “wider”).

BTW if you think you recognise this from a recent Oracle Scene article then you would be right. I was pushed for time, I used something I had written before 🙂

Free Webinar – How Oracle Works! September 15, 2017

Posted by mwidlake in Architecture, internals, Knowledge, Presenting.
Tags: , , ,
3 comments

Next Tuesday (19th September) I am doing a free webinar for ProHuddle. It lasts under an hour and is an introduction to how some of the core parts of the Oracle RDBMS work, I call it “The Heart of Oracle: How the Core RDBMS Works”. Yes, I try and explain all of the core Oracle RDBMS in under an hour! I’m told I just about manage it. You can see details of the event and register for it here. I’ve done this talk a few times at conferences now and I really like doing it, partly as it seems to go down so well and people give me good feedback about it (and occasionally bad feedback, but I’ll get on to that).

The idea behind the presentation is not to do the usual “Intro” and list what the main Oracle operating systems processes – SMON, PMON, RECO etc – are or what the various components of the shared memory do. I always found those talks a little boring and they do not really help you understand why Oracle works the way it does when you use it. I aim to explain what redo is, why it is so important, what actually happens when you commit, how data is written to and read from storage to the cache – and what is actually put in the buffer cache. I explain the concept of point-in-time view, how Oracle does it and why it is so fantastic. And a few other bits and pieces.

I’m not trying to explain to people the absolute correct details of what goes on with all these activities that the database does for you. I’m attempting to give people an understanding of the principles so that more advanced topics make more sense and fit together. The talk is, of course, aimed at people who are relatively new to Oracle – students, new DBAS or developers who have never had explained to them why Oracle works the way it does. But I have found that even some very experienced DBA-types have learnt the odd little nugget of information from the talk.

Of course, in an hour there is only so much detail I can go into when covering what is a pretty broad set of topics. And I lie about things. I say things that are not strictly true, that do not apply if more advanced features of Oracle are used, or that ignore a whole bucket full of exceptions. But it’s like teaching astrophysics at school. You first learn about how the Sun is at the centre of the solar system, all the planets & moons revolve around each other due to gravity and the sun is hot due to nuclear fusion. No one mentions how the earth’s orbit varies over thousands and millions of years until you have the basics. Or that GPS satellites have to take into account the theory of relativity to be as accurate as they are. Those finer details are great to learn but they do not change the fundamental principles of planets going around suns and rocks falling out of the sky – and you need to know the simpler overall “story” to slot in the more complex information.

I talk about this picture.

I start off the talk explaining this simplification and I do try to indicate where people will need to dig deeper if they, for example, have Exadata – but with a webinar I am sure people will join late, drop in and out and might miss that. I must remember to keep reminding people I’m ignoring details. And amongst the audience will be people who know enough to spot some of these “simplifications” and I think the occasional person might get upset. Remember I mentioned the bad feedback? I got accosted at a conference once after I had done this talk by a couple of experts, who were really angry with me that I had said something that was not accurate. But they had missed the start of the talk and my warnings of simplification and did not seem to be able to understand that I would have needed half an hour to explain the details of that on thing that they knew – but I had only 50 minutes in total for everything!

As I said, this is the first Webinar I will have done. I am sure it will be strange for me to present with “no audience” and I’m sure I’ll trip up with the pointer and the slides at some point. I usually have some humour in my presentations but that might not work with no crowd feedback and a worldwide audience. We will see. But I am excited about doing it and, if it works, I may well offer to do more.

As a taster, I explain the above diagram. A lot. I mostly just talk about pictures, there will be very few “wordy” slides.

I invite you all to register for the talk – as I said, it is free – and please do spread the word.

click here to register for the Webinar

Friday Philosophy – Are Leaving Presentations A Quaint British Tradition? August 11, 2017

Posted by mwidlake in Friday Philosophy, off-topic, working.
Tags: ,
9 comments

A few days ago a friend, Robert Lockard, started a discussion on Facebook about bad bosses and the strange things they did. I mentioned how one of my first bosses had refused to do my leaving presentation, arguing with his boss (very loudly so all could hear, despite it being in “an office” – a plastic box in the corner) that he did not want to be nice to me and do the presentation. Neither did his boss!

Another friend of mine, Jeff Smith (oh he of SQL*Developer fame) replied to my comment saying “what is a ‘leaving presentation’ – they let you get up in front of everyone and invite them to kiss your a$$ goodbye? Because, that sounds pretty amazing”.
That took me by surprise, it had never occurred to me that “leaving presentations” were not universal. That’s one of the great things about the global reach of social media, it helps you realise that so many things you thing are normal are, in fact, local to your region and are seen as bizarre by others in other cultures.

So that made me wonder how common “leaving presentations” are? I could have asked on Twitter or mailed a few friends, but I have this blog thing I can use…

I suppose I should describe what a “leaving presentation” is, in case other cultures do the same but call it something else (or just do it without a title). So:-

For most of my working life in the UK, if you are a permanent member of staff and it is known you are leaving (so it’s not a case of you being sacked) and it is only you (so it’s not a case of the company slashing the workforce) then “Shirley from Accounts” will take charge and will buy a card. I don’t know why, but it is nearly always a woman who gets the card and the same woman organises most people’s leaving card. The card is sent around the office in an envelope and people sign it (maybe adding some words like “begone foul demon”) and, depending on the organisations, there may be a collection made at the same time. It is beholden on you, the leaver, to pretend to never notice the card going around (or checking how the collection is going. I did know one guy who put money into his own collection to make himself seem more popular!).

Back in the 1980’s/90’s, the places I worked at did not have email – not even internal systems on the mainframe, so lots of envelopes would be going around with company memos or things you had to read and sign you had read. So the odd card going around was easy to ignore. These days of course everything is email so the last few times I’ve noticed a card going around, it stuck out like a sore thumb and you knew it was a leaving (or “congratulations” or “get well soon”) card.

Then on your last day your boss gets the team around, (s)he says you did not steal much and you did not piss off all the users, they give you the card and pretend to care what happens to you in the future. People then clap politely. This not the US, there is no whooping or saying it is the saddest day of their lives. If a collection had been made they will have bought you something with the collection. It is always almost, but not quite totally, useless. You might have to do a small speech and then, the best bit, you take them down the pub and buy everyone a drink (it used to happen at lunch time but now it tends to be more at the end of the day). The round generally costs you more than the collection they gathered for you. Sadly the last bit seems to be dying out.

As a contractor/external consultant you tend to avoid the mild discomfort of it all as you are not around long enough to become part of the team and, well, it’s just not done for over-paid contractors.

Personally, I have always found the whole thing a bit weird and, if I am the leaver, mildly uncomfortable. I try to avoid the whole thing by keeping my exit quiet or stealing enough stationary so that management do not feel I deserve a leaving presentation. Of course, in the case I cite above, I nearly avoided it just by making my bosses hate me. ho Hum.

But I do still try and do one bit, the “taking people to the pub” at the end of the day, even when I am a contractor and we are not supposed to get leaving presentations.

So what if anything do they do where you are? Is the leaver expected to do something (bring in cake, kiss everyone, do a dance)? Do you have a tradition that is eminently sensible and common in your country but, not you you come to think of it, maybe it’s a touch strange? Or do people just leave quietly and no one notices much – except for the scramble for the chair or your higher-res screen?

Friday Philosophy – Sometime The Solution Has To Not Only Match The Problem But Also… August 4, 2017

Posted by mwidlake in Architecture, development, Friday Philosophy, Perceptions, Programming.
Tags: , , ,
3 comments

…The People!

When you design a system for end users, a good designer/developer considers the “UX” – User eXperience. The system has to be acceptable to the end user. This is often expressed as “easy to use” or “fun” or “Quick”. But in reality, the system can fail in all sort of ways but still be a success if the end user gets something out of using it. I’ve said it before and I’ll say it again and again until I give up on this career. In my opinion:

User Acceptance is the number one aim of any I.T. system.

OK, you all know about UX probably. But what about solutions that have no End Users? I’m thinking about when you create a technical solution or fix for an internal system, to be used by fellow I.T. professionals. How many have you considered the skills and temperament of the people who are going to house-keep the solution you create? I suppose I have had opportunity to think about this more than some of you due to how I work:- I’m a consultant who gets called in to fix things and then leave. At times I have chosen a solution that has been influenced by the people who will be looking after it.

I’ll give you an example. At one site that I worked at for about 9 months, I did a lot of work for one system. The developer/systems administrator who looked after the system was…stupid. I don’t really like saying that, we all vary in our skill set, experience, intelligence, *type* of intelligence (I know some people who can speak 3 languages or know a lot about history but could not wire a plug). But this guy really seemed to struggle with logic, cause-and-effect or learning anything new. And I had to help him look after this database application with one main, huge, hulking table. It had to be partitioned, those partitions maintained and the data archived. I implemented the partitioning, I explained partitions to him several times, what was needed to maintain them, where to look in the data dictionary for information. It was like talking to my mum about it. He just seemed not to understand and his efforts to code something to do what needed to be done were woeful.

I knew it was not me, I’ve run enough training sessions and presented so often that I know I can explain myself (Well, I hope so! Maybe I am deluded). He just was not getting it. Maybe he was in the wrong job. So I wrote him a set of SQL-generating scripts to get him going. He kept messing up running them. In the end, I knew I was about to leave and when I did within 3 months the real customer would have a broken system. So I wrote a mini-application in PL/SQL for him to do what needed to be done. And set it to email a central team if it failed. The team he would call when he broke it all again. I also simplified the solution. My original system had some bells and whistles to help with future changes, such as over-riding where new partitions went or how old ones were compressed. I stripped it out to keep it as simple as possible. I altered the solution to suit the person that would run it.

I’ve done something like this a few times over the years. Usually it is more to do with the skill set of the team as opposed to actual ability. I’ve on occasion worked with people who are new to Oracle and my time is limited so, rather than give them a solution written in PL/SQL that none of them know, I have done so with SQL and cookery instructions/shell scripts. It’s not the best solution but it is something they can live with.

More recently I had to look at fixing the performance of some SQL statements. Baselines would have done the job perfectly. However, the team were all Java experts and had no desire at all to learn about database administration. (To be frank, they had no time to learn either, it was the usual situation of them having 75 hours of work each every week as management thought just shouting would get things fixed, not hiring enough people). I strongly suspected that they would forget about the baselines and if they had a problem they would be confused as to what was going on. So I fixed the key SQL statements with a set of hints to force both the overall structure of the execution plans as well as which indexes to use etc – and said over and over and over and over that if they ever changed indexes or migrated to a later version of Oracle, those hints would need reviewing. They were, in effect, part of their code base. A big advantage of the hints was that they would see them in their code and it would remind them what had been done. They seemed happy with that.

My point is, sometimes the “best” solution is not the correct one, even when you are keeping within the walls of the computing department(s). Sometimes you need to think about who you are giving the solution to and change the solution accordingly.

Friday Philosophy – Improving Your Working Life July 14, 2017

Posted by mwidlake in Friday Philosophy, humour, Perceptions.
Tags: , , ,
1 comment so far

If I got you all to write down the top 5 things that make working bearable, and then got you to make a list of the top 5 things that make working enjoyable, I have a suspicion there will be one thing high on the “Enjoyable” list that may not even be on the “Bearable” list:

Being in a good team.

This one thing can make a real difference to your working life. I know this is true for me and it’s something I’ve heard other people say a lot. The team you are in can make up for a lot of negative things about any given job. I’ve found myself in roles where I am unsuitable for the task, or under a ridiculously high workload, even being paid much less than I know I am worth. But if I have been in a good team, working with people I like (well, at least some of them!) it makes it all a lot better. A lot, lot better. Think about the jobs or roles you have most enjoyed in your life. In any of them did you not like the team you were in?

Unfortunately in most teams there are people like Miserable Kevin, who does nothing but complain and slag everyone off. Or Oddly Quiet Katrina who is about as much fun to be with as eating a jar of pickled frogs.

You might expect me to now suggest you leave any team you do not like being in – but that is impractical advice for most people and impossible for many. And in fact I think there is a much, much better option:

Don’t change which team you are in – Change the team you are in.

You may think that it’s the responsibility of the team leader or maybe “management” to create a good team, perhaps by punishing or getting rid of Kevin and Katrina. And to a certain extent you are right. But most team leaders got the job for reasons other than their soft skills (the ones that allow them to understand and work with people – Heck, most of us in I.T. are there at least partially as we do not like other bloody people!). But actually, anyone in the team can change the team into somewhere more pleasant to be. I’m willing to bet that if you have worked in teams that are fun and satisfying to be in, the person or people who made it that way were not the team leader. Or at least not limited to the team leader.

Be warned, I am not claiming it is easy to change a team and it can take a while. But I think anyone can improve their team, if they put in a little effort. And you can do it in small, easy steps.

It helps if you know or learn a little about how different people think, a little bit of pop-psychology can go a long way (all those management training courses I once went on helped me a lot in this) but in essence you just need to help people to talk, relax, interact and get to know each other a little better. Try to see things from their side, consider why they are being objectionable or difficult. And be nice to people. Not in a creepy way, but just try to not lose your temper at someone who is being annoying, do not join in with the weekly team moan about Kevin behind his back. I’m not saying you should not disapprove if someone is being an arse, but you can make it clear you are not happy with them without being antagonistic or retaliating. Just one person doing this can make a big difference. I know, I’ve done it.

Another thing to do is try to include people more. If you and another team member are discussing an issue, maybe ask Oddly Quiet Katrina what they think. If you can find a common interest with someone you don’t know well in the team, try to talk to them about it. I don’t mean do what one bloke I know did – come in the office, ask how your weekend went and then look stunned with boredom when you told him – He had no real interest, he was playing a role, and doing it badly. If you are going to try and draw someone out a little, it really helps to be interested in what they say.

Once thing I have found helps significantly in creating a better team is suggesting the occasional coffee, go to lunch together (especially if it means going out the office to get something, even if only a sandwich) or have an after work pint. {I’ve previously mentioned this as Team Ice-Cream}. It really helps if you know at least 1 other person who will join in; and you are not trying to get everyone there. You just let people know, go and have the coffee or drink and let whoever wishes to join in. Sometimes there will be someone who will refuse to join in but, heck, the last thing you want to do is try and make it “enforced team fun” – as that never works well. I’ve done this in 3 or 4 roles now and after a few weeks (and it can take weeks and months) most of the team was coming along most of the time. And the intra-team bitching had plummeted. I even saw people help each other without being told to!

One thing to mention – don’t be too enthusiastic and gung-ho about it. Don’t come in first thing and cry “Hi team! How’s it all going! Hi-Fives! Who’s for a pint after work!?”. This will make you into Psychotic Barry. No one trusts Psychotic Barry.

The key principle is to be a bit more friendly and inclusive without people really noticing you are doing it. Keep it all low key.

Why should *you* be the person to put in the effort? Well, think of it selfishly. It will be nicer for you if being at work is less bloody awful. Also, it’s a skill you can use everywhere! Each time you change team, you can see if you can improve the new team a bit and see if you can do it more effectively. You can use it in social situations too. Maybe even the wider family, but if you try that and it all goes horribly wrong, don’t sue me.

Another reason to do it? Many people who know me in the flesh may be surprised to know that I’m not naturally very good in a crowd. I got a lot better at it, and more comfortable with people in general, by trying to improve the team.

So go change your team. It’s actually easier than you think and, heck, what have you to lose but some wasted effort and a couple of quiet evenings in a pub on your own (or maybe worst case scenario, just you and Psychotic Barry)? At least you will know that you gave it a go, it is now definitely the team leader’s problem.

One final word of warning. That bloke over there who is a militant vegan, self-appointed know-er of all and despises all contractors on philosophical principles? Don’t expect to get anywhere with him and don’t try too hard. Some people need professional help!

Friday Philosophy – Robots Rising & Tech Taking Over? July 7, 2017

Posted by mwidlake in ethics, Friday Philosophy, future, Perceptions.
Tags: , , ,
7 comments

Today I saw some cracking photographs of a lighthouse. How many of us have at some point wondered if it might be nice to be a lighthouse keeper? The solitude, the scenery, the stoic fortitude in the face of the storm – quite literally. (Thank you Brendan Tierney for the photo I’ve stolen from him).

No one lives here anymore

It’s an odd job lighthouse keeper, it holds a special place in Western culture and literature. A job to be held by those a little apart from society and yet with a dedication to the betterment of mankind. I suspect a lot of people in I.T. (and the wider community) find a resonance in that, as so many of us are a little bit apart and yet intelligent & care.

Well, you can’t be a lighthouse keeper anymore. At least, in the UK you can’t. Check out This web site about UK lighthouses and lighthouse keeping. That job, that vocation, was handed over to automated I.T. systems a few years ago, effectively handed to robots & technology. You might think you know where I am going with this, and initially you will be right, but bear with me.

I’ve been thinking a lot over the last 2 or 3 years about the increasing use of technology and robotics to do tasks that we humans have been doing. An obvious one is autonomous driving vehicles, where The I.T. smarts and sensors are leaping along incredibly fast. I am in a long-running “argument” with a friend about when fully autonomous vehicles will be a reality on public roads. He says under 5 years, I think it is more (I started saying more than 5 years to him in 2016, so, giving some leeway, I say not before December 2021 will we see fully autonomous vehicles driving from a town centre to another town centre, sharing lanes with human drivers – specific enough Neil?). But self-driving vehicles will be safer than humans “soon”, and cheaper than employing humans, so companies will swap to it. That will end a lot of employment.

I know others have pointed this out and it is not as if history isn’t almost a continuous tale of technology assisting or replacing human effort. Tolpuddle martyrs, dark satanic mills and all that. Industrialisation of farming has put a lot of farm labours out of work but we could not feed the current mass of humanity without it. People move on to new tasks.

But the difference now is not that we are handing jobs to a slightly better automated system where we still need some human control, we are removing the human element. And we are doing this in a lot of areas in a very short space of time. Factories are becoming far more automated, we order our goods online and huge conveyor robotic systems are being built to do the packing, with fewer people involved and lower long-term costs.

But it’s not just physical tasks that are being automated. Genetic algorithms, neural nets, deep data and machine learning is starting to replace tasks that have needed human interaction. Chatbots are getting smarter, to the point where they are used by companies as first-line support {often laughably poorly at present, but it is getting better – and Oracle do have an interest as was covered in Oracle Scene recently {sorry, that link might not work for long}. Expert systems have been developed that can judge simple court cases such as parking fines and beat humans at spotting pre-cancerous cells in tissue samples.

Oracle and the Bots

We now see expert computer systems breaking a lot of barriers and doing things that until now have been deemed uniquely human cerebral tasks. Computers won at playing chess 10+ years ago, they triumphed in “Go” last year and now they can win at versions of Poker where you are not sure of the data and have to read the play of your opponent – in effect second guess a human. Currently all these systems are very expensive, highly focused and specific to a task, built on huge data sets and using fine-tuned sets of algorithms, to do one task. We have nothing as generally capable as a 5 year old child or even a dog.

Only, we keep building systems that are better and better at specific tasks.

So why do I say this bothers me but not in the way you would expect? It’s because I keep seeing “thought leaders” present the same denial of these impacts on us in I.T. of the systems we as an industry are developing, platitudes that we are a special case and will be OK. Several times over the last couple of years I see some utter pillock in a suit from upper management telling a crowd of I.T. experts that we will be just fine as we are smart and we can stop doing the easy tasks and concentrate on the harder ones, use our brains more.

This is balls for two reasons. Firstly:

What about everyone who is below smart?

Most of us in I.T. are not only above average intelligence (probably IQs of 125 and upwards), we are surrounded by similar smart people. Our life partners are generally above normal intelligence, we work in teams who are above-average smart, we probably mostly socialise with generally intelligent people (as a raft of psychological studies show, we gravitate to those at a similar IQ to ourselves, irrespective of where we are on the scale). Even the end users we abuse tend to be above average intelligence. I suspect that most of us somehow don’t “get” that well over 60% of people are not only less intelligent than we are but they have few options if our society passes the jobs they can do to computers and robots. And they are not that likely to be philosophical about having no point to their lives and being poorer. They’re probably going to be very angry, very poor and pretty pissed off with smart-arses who say that “we are OK” – and there are a lot, lot more of them than us.

And that leads to the second reason it is balls.

The smart work will also be doable by Tech

As I’ve said already, we can already create technological systems that can beat us at specific cerebral tasks and there is going to be a small and smaller pool of work for highly-intelligent workers. Let’s face it, a lot of what we do now in I.T. is drudge and boring, there is not really that much smart work needed doing, even in this industry stacked by us smart people. And doing work that really needs you to be smart is tiring (well, I find it tiring!). And our work in I.T. tends to be logic-based and what are computers good at? We will just have a breathing space before our work is also more cheaply done by computers.

I’m annoyed as I think those of use who are involved in this revolution are being told a deluded lie that we will be OK if it pans out like I have just said. Those extra 25+ IQ points are not going to keep us special for very long.

So if computers can drive the taxis & lorries, manage the steel works and build the cars, derive the best drug treatment and give the perfect injection (yep, theoretically a robot already wins on that) what do we as humans do?

Only a few people can be “utterly human” – artists, poets and philosophers. And we do not need 7 billion of them anyway.

We could try and legislate against it, tax robots hard. But those who make a lot of money already run the “free market economy” and will continue to do so. If Robots and computer programs do tasks more cheaply, companies that uses robots will rise to the top of any monetary-based society, i.e. a capitalist society. What will change what has been in place for 100+ years? I can’t see the currently rich and powerful objecting to working methods that increase their wealth. Even if it means more and more poorer people.

Some argue for a basic living wage to keep us all alive – fed, warm and basic healthcare whilst machines do the work. That would give us that often cited nirvana of being free to do “what we want”. But if you have no job, what do you do? Again, for those of us with high IQ we can maybe come up with things to do. Maybe. I seem to be relatively happy being semi-retired, but I’ve done a lot of stuff and had my time of striving to achieve. And still do. But how about those who are IQ 100 and below? I suspect entertaining yourself is not as easy. I think anger, resentment and the feeling of being disenfranchised is just going to continue increasing. I think it’s why the UK is leaving Europe and why the US has an egotistical man-child as president. More and more normal people are unhappy with their lot and see no good future – so they vote for a change. ANY change. Even if it is crazy change.

I know, not a very happy Friday Philosophy. Will someone please tell me it will all be OK? And I mean OK for everyone, not just us “smart” people.

Friday Philosophy – “Technical Debt” is a Poor Term. Try “Technical Burden”? June 30, 2017

Posted by mwidlake in database design, development, Friday Philosophy, Management.
Tags: , ,
5 comments

Recently my friend Sabine Heimsath asked a few of us native English speakers what the opposite of “technical debt” was. My immediate reaction was to say:

I’d say (sarcastically) “proper development” or “decent designer” or even “what we did 25 bloody years ago when we were allowed to take pride in the software we created!”

But my next comment was less reactive and more considered. And that was to say that I did not like the phrase “Technical Debt”:

A debt is when you owe something to someone, to be paid back. You do not owe anything to someone when you build poor systems, you are actually creating a “technical burden” – something those in the future will need to live with and may eventually have to sort out. Those who created the bad app or design will probably not be the ones fixing it – as in paying the debt.

That is of course not always true. Some of us have had to end up fixing a poor solution that we implemented – usually implemented despite our protestations that it was a daft thing to do. But the usual case is that a badly thought-out solution is implemented in a rush, with little design, or with inadequate testing, because of a management pressure to be “agile” or “fast moving”. And it is done with cheap or over-stretched resource.

Also, “technical debt” to me sounds too organised and too easy to fix. If you have a financial debt, you simply pay it back with some interest. In almost all situations I have seen where there is a “technical debt”, the interest to pay – the extra effort and time – is considerably more than was saved in the first place. Sometimes it is more than the original cost of the whole project! Loan Shark territory.

When the poorly designed/implemented system falls over in a heap sometimes the hard-pressed local staff lack the skills or bandwidth to fix it and “Experts” are called in to sort it out. And part of the time taken to fix it is the expert going “why in f**k did you ever think this was a good idea?” (Maybe using better terminology, but that is what they mean!). Being more serious, sometimes the largest slice of time is when as an “Expert” you have to persuade the people who own this mess that it really does need sorting out properly, not just another quick hack – and it really will take much , much more effort than what they originally saved by deciding to implement this fast & dirty. Sorry, I mean “lean & mean”.

This situation often has a secondary impact – it makes the people who originally implemented the solution look poor. And that may or may not be fair. I’ve seen many cases where the original staff (including myself) were forced to do things they did no like by timing constraints, lack of budget or simply the ridiculous demands by someone higher up the organisation who thought simply shouting and demanding would make good things happen. They don’t, you end up creating a burden. Though I have also seen poor solutions because the original team were poor.

I think at the moment a lot of what is called “systems development” is more like a desperate drive to constantly cut corners and do things quicker. If it goes wrong, it’s just a debt, we pay it back. No, no it is not. It’s often a bloody mess and a Burden for years. I keep hoping that, like many things in I.T. this will be a phase we can cycle out of and back into doing proper planning and implementation again. Yes, anything that speeds things up without losing planing/design is great. And if you have the skills, you can do proper Agile, designing the detail as you go – IF you have the general over-arching design already in place. But I wish there was more consideration of the later cost of quick & dirty.

So what was the term Sabine wanted? Well, I can’t speak for her, I am not 100% sure what she was looking for. But from my perspective, we should not say “Technical Debt” but “Technical Burden”. And the opposite might be “technical Investment”. You spend a bit of time and effort now in considering how you can create a solution that can expand or is flexible. I know from my own personal experience that it is when you are given the chance to do those things that you provide a solution that last and lasts and lasts. Only when I have been allowed to properly consider the business need do I create something still used in 10 years. Or 15. Or maybe even 20. That might need some checking!

So, if you really want to build systems to support a successful business, and not a short-lived flash, perhaps you should be saying something like:

You are asking me to create a Technical Burden. Would you rather not help me create a Technical Investment?

If anything else, you might at least be adding a new entry to “Buzzword Bingo”.

Friday Philosophy – New Conference, Same Sad Old Faces Up Front June 2, 2017

Posted by mwidlake in conference, Friday Philosophy, Presenting.
Tags: , ,
5 comments

I’ve been on the Oracle conference presenting circuit for well over a decade now and I must confess I enjoy it. Part of this is that I see lots of friends at nearly every conference – even in countries I have never been to before. This is because many of those friends are fellow presenters, who have been presenting for well over a decade now…

There he goes again, banging on and on and on…

This is not totally true of course, there are some relatively new presenters, even a couple I can think of that have been presenting for only a year or two (Pieter, Frank…). I’m proud to say that there are some less-experienced presenters I actually helped get going at this lark and even a couple who are better at this than I am.

But the truth of it is, if you were to go to 5 conferences in one year across Europe (or maybe even 5 across the USA, let me know) you will keep seeing the same bunch of mostly older faces up there, sometimes even doing the same talk (or talks) – That is maybe not such a bad thing as the real audience, the local Oracle community members, are mostly from that region, won’t have been to the other conferences and get to see current talks that have been trialled and tested and tweaked elsewhere.

However, if you go to the same conference 5 years in a row – you will STILL see the same bunch of mostly older faces up there (all getting slowly older, greyer, wrinklier – except for those who hit Mid-Life-Crisis and suddenly develop gym-bodies and oddly darker hair…. 🙂 ). Again, maybe not a bad thing as these are the people who like presenting, get selected again based on the fact the audience liked what they said, they did not lie too much and did not get too many things wrong. And most of us try to not do the same presentation 2 years in a row, so the material moves on {I do repeat presentations after a year or two’s break, usually updated and aimed at the newbie audience, but that’s just me}.

So is this “same old faces” a problem? Most of us conference organisers agree that it is as people drop off the presenting circuit or seem to run out of material. So you need a new influx. And you need younger presenters to keep the older ones on their toes (or just help them on and off the platform). And younger or just new people to give another perspective or add their considerable experience to the mix. One of the newer presenters I can think of is actually retirement age and a great addition to the circuit.

But the problem is, how do you encourage new presenters? You lot reading this who do not present are a damned hard bunch to motivate to give it a go! Now, I know that presenting is not for everyone and that some of you would rather stick your arm in a wasps’ nest than present, but some of you can bang on for ages in a social situation and actually know your stuff. So how do we get you to present?

The same ideas come up. One is to say you only need to present for 5 or 10 minutes. Sometimes we will even organise a full session made up of such short session to let people give it a go. It does not seem to work to me, you get one or maybe two new people and then fill the other mini-sessions with experienced people – who then complain about how hard it is to do a decent talk in such short time!

Another is to specifically ask at SIGs and smaller meetings if anyone fancies trying out presenting, in the small and friendly arena that they have just experienced. You know, the one where after presenting the presenter cannot really hide in the crowd and everyone there knows if you did a good or bad job… We do get the occasional new presenter but not really. And I suspect most of them would have submitted papers eventually (and I’m ignoring the issue of new presenters getting papers selected, I’d need a whole post on that).

Another route it to co-present and this is the one I have used a small number of times. You get someone you know, who understands the material, to share a presentation with you. If they stumble or forget what they were saying, you can just nudge things along, and hopefully cope with any tricky questions that might worry the new person. It worked once (and you now see his sad, old face ALL the time), partially worked the second (though I’ve not seen her present for a while) and utterly failed the third.

The UKOUG is trying this at the next UKOUG Tech conference, but in a more formal way. They are getting some of us more experienced presenters to offer ourselves to co-present with new people. I’m not sure how well this will work if we experienced presenters are not finding the inexperienced presenters ourselves. Can you imagine someone who has never presented before wanting to step up to the podium with one of the Oracle Names, unless they also know them? If you said something wrong, would they correct you in front of everyone (no, probably not, we are generally nice people). Anyway, it’s something to try and I am happy to help. The UKOUG have started promoting this a little, but I don’t think everyone is going to find it appealing. In fact, my friend Dawn saw this and thought it sounded…:

Creepy! That made me laugh.

Nevertheless, if you are a potential new presenter or just inexperienced and you want to present on something I also know about, I’d be happy to consider co-presenting with you. Just let me know. And generally speaking, if co-presenting appeals to you but not with me {I would not blame you}, get in touch with the UKOUG.

About the only way I know of really getting new presenters is… to get people drunk and make them agree to it. Then remind them about it endlessly until they feel obliged to do it. It does work, but it ends up being a self-selecting set of new presenters, i.e. people I drink with, which rather annoyingly tends to be sad, old men. I’ve tried drinking with young, vivacious women but I usually get asked to leave the club, as I am coming over as creepy.

So, if you are someone who has considered presenting or would consider it – what would help you give it a go? Tell me, I’ll see if I can arrange it.