The Nightmare of Knowledge

In the dream I am late, I can't move, I am overpowered by forces beyond my control, unable to prevent disaster, or stop myself from performing acts I know to be wrong. It is a humbling experience, even a nightmare.

It fascinates me that in my dreams I have a clear sense of right and wrong, but that I am not always able to exercise it. It feels natural that diverse elements of my life are connected into a web of intrigue and nonsense as prosaic as a trip to the post office: why is my colleague from work living in my house, in cahoots with my mother, who is guarding the kitchen with a knife? Why is my mother blaming me for something that he did, and in an uncharacteristically threatening tone? I find myself using uncharacteristically harsh language back to her, even though I am aware that it is wrong to do so -- and I reflect on this wrongness in the dream itself, but I cannot stop it.

Why did I leave my room naked, even though I know I should dress first? Mid-region nudity seems to be a recurring theme in many people's dreams. I wonder if primitive tribes attach the same importance to nudity in their dreams.

Meaning to be found?

I dream most vividly and bizarrely when I am hot. There is clearly some connection between temperature and dreaming. This is just one of many things that I would like to understand better, but perhaps there is some simple sense to be found.

A lot of speculation has been directed at the subject, of course, much of which I consider to be New Age nonsense: for instance that we dream in black and white. I confirm this to be false from personal experience. That dreams can be interpreted is, I believe, greatly exaggerated. I find dreams to be largely ad hoc journeys through events that we have accumulated as memorabilia of our lives.

People look for meaning in their dreams, but meaning is an odd thing, and I have written about this before from an engineering point of view. Meaning seems to be (in a sense) the opposite of information (read more). It seems to be related to what stands out from its background most clearly. As humans, it sometimes seems that the more implausible a story is, the more likely we are to scour it for meaning and pass it on.

Sleeping dreams are not magical. They seem to place us in mundane situations, where in waking life we would typically avoid incident, threat or danger. But in dreams our body and mind often fail us at a crucial moment. We are unable to press the brake pedal, or turn the steering wheel and so we crash. We need to run to catch the train or plane, and our legs give way and we collapse in paralysis, powerless to do anything about it. We meet a stranger and kill attack them, or faced with a steep fall, we throw ourselves from the edge.

Loss or abdication of control is common in sleeping dreams and paradoxically we lose control over the taken-for-granted essentials, but the loss of strict control over a story-line allows us to explore quite bizarre imagery: to see the familiar reinvented. Until we experience something so terrible at the boundary been waking and sleep that primitive reflexes intervene with their hard-wired programming and we jerk back from the beyond.

In daydreams, by contrast, we are in control. We rehearse heroic interventions, or become architects of our own destiny. We use these reveries like thought-experiments. In daylight our thinking seems to be directed by us, decisions premeditated and guided by a catalogue of importances of the moment. We do not blunder into catastrophe, we do not choose to dwell of humiliation or failure, features and uncertainties in daydreams. Perhaps we do so at night precisely because we actively suppress them during the day (hence emphasizing their importance).

Freudian psychology has long maintained that sleep-dreams reflect our inner state of mind. I don't buy that explanation for a moment. What I see in my dreams is an arbitrary placement of familiar things into a completely improvised scenario (this penchant for creativity is interesting in itself, making me wonder if especially creative people have different kinds of dreams to more logically driven people). However, the cast and subject matter of a dream is sometimes related to things we dwell upon.

Things of greater importance to us (either because they have etched a permanent place in our thoughts or because they happened more recently) are usually prominent in sleeping dreams, as though these ideas have been ranked into some kind of importance scale by Sandman Promotions, or some kind of Google-Sleep engine. What it might say about the brain's powers of improvisation when we forget to dress, or fail to deal with mundane boundaries is harder to understand. Why does sleep make me do things I know to be morally wrong or even physically inadvisable? What part of inhibition is switched off during sleep to make these impending catastrophes that often wake us in the end?

Years ago I read the hypothesis that the brain is trying to erase short term memories during sleep. This makes no sense to me either. I do not dream of only recent events or past events, unless they seem particularly important to me. Importance rather than novelty seems to be the key. And importance ranking is a characteristic of networks, of which the brain is an undeniable example (as search engine's know).

I am sure that many have entertained the idea that dreams might be like weather systems, storms of neuronic activity guided by random firings, but this begs two questions:

  • A random firing would not have enough structure to generate a flow or story-line. Where does the story-thread come from? Perhaps our brains are hard-wired to perceive story-lines in events, just as we see shapes in clouds?
  • We know that there are brain-waves or patterns of activity in the brain from EEG traces, which are far from random. Indeed they are quite organized, perhaps more like traffic patterns guided by daily rhythms on major and minor roads. How do these waves affect the thoughts we have? I have yet to see a clear account of what brain waves tell us.

I would like to understand brain activity better to understand the role of patterns. Intuition would lead me to expect dreams to be brought on my the same kinds of processes that govern ordinary thinking. The lack of control suggests that some inhibitors that provide self-protection are inactive during sleep. I am waiting for V.S. Ramachandran to come up with one of his astonishing experiments to answer these questions.

Perhaps what dreams really illustrate is how our brains, loosed of external pinnings or stimuli, traverse the network of knowledge and experience of a lifetime -- in which case there might be valuable lessons to be learned. There is, of course, a random element: where does one start? From such a starting point, the dream draws a time-line, a clock or arrow of time that drives our thinking in a single direction from the starting place. We make up stories.

Tall stories in the rough

This propensity for us to rank the importance of thoughts, and connect the dots into stories led me to think last year of whether storytelling can help us in learning. Visual pattern recognition does not require a special appreciation of time because patterns come with a ready coordinatized sequence or timeline already. Even spatial patterns are laid out in a sequence or position. Time and space are somewhat equivalent, as Einstein or even Parsifal realized.

What stories do is to connect points that are not a priori arranged into a pattern or sequence. There is no unique way to do it, so there are many possible answers, none of which is necessarily the right one. I have often used story telling as a mechanism is teaching, but I was interested in whether one might use it to advantage in non-interactive, distributed learning.

Alva Couch and I spent some time talking about this on his sabbatical visit to Oslo last year. We came up with some interesting models for knowledge representation based on forming networks of ideas (using Topic Maps) and then loosening of the meanings underpinning associative connections in the network. I wanted to see if we could use this as a way of finding `inspiration' in organizational Knowledge Management. I told Alva of the idea, and he made a number of models. His eventual conclusion was that more useful stories come about by relaxing the constraints more fully. The ultimate approach is to remove all meaning from connections and simply allow any idea to follow from any other without caring about the kind of relationship, provided there was some meaningful relationship to start with. Use the sense and then throw it away. A little dream madness.

Meaning becomes an ordering principle, but microscopic details are exchanged for a whole-greater-than-the-sum-of-parts effect.

Why does dreaming improvise linear narratives from complex networks of things we know? Is there a relationship with the need we have to put things in order, see patterns? Freud saw a catharsis in rehearsing these combinations, but I suspect that our brains are simply evolved to see patterns. We cannot help it. We have evolved with senses that work in a linear fashion. Vision, sound and touch require us to parse from beginning to end to identify a pattern. It is perhaps not surprising that, faced with a much more complicated structure, we try to trace it in the same way.

It is not unlike a chemistry of ideas: from a single idea, we form a bond to the next one, and from there to the next and so on. If we try to fit things together in a rigid pattern that seems safe, we end up with a brittle crystal, pretty but no more than we expected. If, on the other hand, we don't mind that the kinds of bonds are different or unexpected, we don't question the pattern, we just go with it and see what we end up with. The result? A new plastic? A new protein? Something more messy, but the result could be something useful, even though we would never have seen it from the bottom up. This is the way that evolution took us from primordial soup to primate. It is how knowledge accumulates and builds on past experience. It is rather like dreaming, simply because both are network phenomena.

If we see possible directions of travel from one idea, we will `brain storm' our way forward to places we never imagined we could reach. In my career, I have often had success in marrying unlikely bed-fellows into a new hybrid idea. This is somewhat the kind of experiment that dreams perform.


Knowledge management in organizations is a serious and pressing issue today. As the pace of our world increases and people move more quickly from task to task, or even job to job, knowledge that is locked inside people's heads tends to remain there. Organizations spend a lot of money and resources on retraining new employees because their core processes were not well documented and could not easily be transferred to newbies.

Knowledge networks are not just inside people's heads but in between them too. We combine co-workers to form new networks, combining knowledge across the organization -- a meta-mind. Yes, you've no doubt heard it before.

But we should not get too excited. Others have made the same observation before. What I see is that we are actually not that smart, and the world does need that much intelligence to get by. We all spend our most of lives repeating simple patterns, not inventing new ones. Stability, after all, is a key force in our ability to survive, and we create organizations that. DNA is almost by definition `that which survives when you repeat the the same banal process of biological copying for ever'. Repetition leads to the exploration of possibilities -- a giant search engine.

We learn effectively by rote (though this has become somewhat politically incorrect in schools) because we mostly need the same basic skills time and time again. It is minor variations that lead us to new insights, so it makes sense to rank the importance of ideas and stay close to the tallest ideas in the rough.

Knowledge management is hard not because knowledge is hard to find but because people set up barriers that they won't cross. `I don't want to know all that stuff'... If they relaxed their inhibitions, the fragility of the knowledge might go from that fragile crystal to a more resilient plastic. It would work like a dream!

Have Topic Map -- will travel

I always felt that artificial intelligence was barking up the wrong tree when it turned to logic for answers, as logic is a constraining, brittle force whereas what thinking requires is the insertion of freedom to travel around the cognitive network. So now as I work on knowledge management for IT system, building these ideas into Cfengine, I find that there is value in a more amorphous exploration of knowledge. What might the implications be for knowledge management? If dreams show us what happens when we associate more freely, without inhibitions to narrow our thinking, then we should dream more in organizations. It is a creative force.

People mostly want to repeat things they know. Making a change is risky. We are stressed when too many things happen outside a familiar framework. We can write them off as unimportant to try to integrate them into our mind map. Repetition provides the inertia of stability. We rehearse, revise, repeat. Perhaps dreams are doing something similar. Dreams are based around familiar things because these are the persistently stable fixtures of our lives. The cortex interprets and organizes information from the environment during consciousness. It may be that random but importance-weighted fluctuations from the brain stem during REM sleep get filtered through the cortex, and it tries to interpret these signals as well, creating a "story" out of fragmented brain activity, when the barriers are relaxed.

The interesting question about dreaming of course is what is he survival value of a sleeping brain? Dreams themselves are probably just a side effect of a brain that can organize knowledge into story-line.

In terms of evolution, the function of sleep is not so mysterious; sleep would represent a perfect opportunity for bodily maintenance. It is known that most mammals die without sleep after a matter of days. Some few individuals who have been in accidents and have suffered brain damage find themselves unable to sleep without ill effects(see example). Others have a genetic disorder that can prevent sleep and die eventually of sleep deprivation.

These are fascinating mysteries that I hope to read answers to one day. For now, I believe there is something here for the matter at hand: knowledge in organizations. And there is one more thing we learn from sleep, apart from all the networking: sometimes you just have to switch off and let your hair down to make room for exploration.

Isn't that actually what research is supposed to be about?