jump to navigation

The Knowledge Curtain. June 8, 2009

Posted by mwidlake in development, Management, Perceptions.
Tags: , , ,

I came across the concept of “The Knowledge Curtain” in the late 90’s, from a man called Lee Young, who I worked with for 2 or 3 years, along with a friend Mike Cox, who was key to showing me how to peek through the curtain.

The below is taken from a powerpoint presentation I sometimes give on project disasters (how to avoid, not how to encounter!):

When systems designers talk to end users, both parties usually end up not really understanding each other

When systems designers talk to end users, both parties usually end up not really understanding each other

The basic principle is that, the knowledge of your users/customers is partly hidden from you. When you ask people for whom you are designing a computer system about their job, they don’t tell you lots of stuff.

  • Things that they assume you know.
  • Things that it does not occur to them to mention.
  • Things that they do not want to admit to doing as it sounds silly or badly managed.
  • I’ve even had people not mention parts of their job as they don’t want their manager knowing they do it.

But that is OK, when you talk about information technology and computer systems to them, they have exactly the same problems with what you say :-).

Lee’s presentation, with N. Mehandjiev, predated the all-encompasing rise of the internet and this is one of the few references to it I can find. {Among the relatively few other hits on the topic, amongst the ones about knowing how to make curtains, are references to the “Knowledge Curtain” as the concept that the Web is not as available in other areas of the world. A related but very different issue}.

So, how do you peak beyond the knowledge curtain? Systems designers and developers with experience learn how to ask the questions “what are the exceptions to what  you have just told me” and “what else do you do” in many, many ways and without giving offence. After all, you have to keep asking these two questions and people naturally get irritated with that and some feel you are suggesting they are either stupid or lying. It can be tricky. 

I think that unless you have someone who is fully IT literate and has done the job the computer system is to assist with, you will inevitably only catch a part of the requirements.

For massive projects over many months or years, I think this lack of a clear understanding of what people do is a major factor to their failures. This is made even worse when the analysis is done by one group of people and the specifications are then shipped out to an external party for the application to be developed. With all that missing knowledge, it is no wonder some systems are initially so poor.

I only know of one method that reliably allows you really see beyond the knowledge curtain. That is prototyping and user feedback. Only when you show the people who are going to use the system what you have put together to help them will they know if it works. These sessions are incredibly valuable and only in development projects where they have been key to process have I seen the project deliver something truely useful.

I now have a general way of developing anything.

  • I ask the users for 3 or 4 major things that they want the system to do.
  • I develop those 3 or 4 features as quickly as possible and show them to the users.
    • One will be almost exactly what they need.
    • One or two will be close to what they need.
    • One will be utterly useless.
    • During the above, one or two critical new needs will come up.
  • Fix the close ones, dump or redo the useless one and add the new needs to the list.

Simply cycle around the above, asking for new features when you have got less than 4 features you  are actively working on. And only polish features (add all the nice screen touches and widgets) once is is exactly what they need or pretty damned close. {You hardly ever run out of features before you run out of time and money!} You end up with an excellent system that evolves to better help the customer as time goes on.

There are lots of development methodologies that have the above principle (or a variation of it) as their core, so I am certainly not the first person to find that this method works. Which is why I find it difficult to understand why so many projects don’t use it?

BTW, I thought I would just add that one of the factors in my starting a blog was a comment by Andrew Clarke on his, several years ago before Blogging really took off. It was for a presentation I did which included the Knoweldge Curtain concept. He was very nice about my presentation and I still appreciate it. This is the link, but as it is an archive you will have to search for my name.



1. APC - June 17, 2009


Here is the exact URL for the post which discusses your presentation: http://radiofreetooting.blogspot.com/2005/11/ukoug-annual-conference-retrospective.html

Cheers, APC

2. joel garry - June 22, 2009

Heny Ford said it best: “If I had asked people what they wanted, they would have said faster horses.” I guess it has come full circle with HD branded F-series pickups.

Prototyping methodologies only work for certain size and type of problems. And for those, you have a real problem of how to stop the cycle and worse, how to maintain it, when the users cycle and when the base system cycles. If the base system can even cycle after such work.

Prototyping large systems with 3 or 4 features? Good luck. Making your developers “stretch?” A recipe for burnout after 3 or 4 cycles.

3. mwidlake - June 23, 2009

Thanks for your thoughts Joel.

I’m not sure what you are refering to by ‘making your developers “stretch” ‘? In my experience, being asked to develop a complex system based only on endless specifications which you then have to redevelop many times when it becomes obvious they are not fit for purpose causes burnout. Especially as the pressure to fix everything when you are 6 months plus behind already is enormous 😦

I’ve worked on many large, complex projects and you can use prototyping on them. You black-box the massive system into chunks. I was lucky enough to be shown this when working on the most complex system I have seen; a hospital system intended to cover a huge amount of functionality. It was far more complex than any billing or purchasing system I have worked on.

The project was broken down into major areas and then the major areas into sections worked on by small teams of 2,3 or 4 people. Each chunk was prototyped, rapidly developed with user feedback, rounded off and then plugged together and integration tested. It worked brilliantly.

There are still major challenges of course – getting the whole data model good enough to start and defining the feeds in and out of the black boxes, but over all it worked. {The project as a whole did not work, but that came down to not managing the politics and personalities, which was a crying shame}.

I had to write one module that went over the whole system, recording a patient death. This involved cancelling some activities, prompting others, notifying departments, making sure letters did and did not go out as appropriate. You would expect it to be a nightmare to do this for a system written in little bits, but because each bit was a box and we had documented the feeds in and out, it was simply “challengingly difficult” 🙂

4. joel garry - June 24, 2009

The “stretch” was in the link provided by APC.

Thanks for your thoughts on my thoughts. 🙂

I can see where you can chunk a massive project. Most projects I’ve seen aren’t quite massive enough to do that.

Failures are more interesting, anyways.

5. Peaking behind the knowledge curtain | Oracle - August 11, 2009

[…] threatening for years to start a blog Martin Widlake has finally put fingers to keyboard. Some of you may recall that I am a fan of his UKOUG presentations. His writing is entertaining and […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: