Tuesday, 5 January 2021

Towards a Theory of Everything

This post may seem way off-topic, but for writing science fiction it helps to stay informed on the latest developments, as a sprinboard for our imagination. And a new approach to a "theory of everything" from Stephen Wolfram certainly provides fuel for the imagination. ("It's hypergraphs all the way down".)

I follow Joscha Bach on Twitter, and spent an hour reading a link from a tweet from him about an interesting blog post by Stephen Wolfram on Mathematics, Combinators and the Story of Computation. I was intrigued by its opening paragraph about a mathematical tool called 'combinators' (maybe discovered by a mathematician in 1920, Moses Schonfinkel). I think these combinators underlie what Wolfram believes may be a new framework he's created for thinking about physics, that will probably lead to a Theory of Everything.

Back in high school I read some excerpts of Newton's Principia Mathematica, in which he described some of the laws he'd discovered (like the inverse square law of gravity), in English.  It was a struggle to understand what he meant: expressed in English, the statements were really hard to understand, but easy to understand when expressed in modern mathematical notation.

Reading of the argument between Newton and Leibnitz regarding their (simultaneous and independent) discovery of calculus, I was struck similarly that Leibnitz's notation seemed more elegant and easier to work with than Newton's (which is presumably why Leibnitz's notation is the one we adopted).

And probably the capstone was reading the SF book Babel-17 by Samuel Delaney.

Those things made it clear to me that introducing excellent symbolism (notation) for the right concepts, coupled with rules for how to manipulate those symbols to make true statement, can produce really powerful tools for reasoning.  (That and asking the right question, or framing a problem the right way.)


But Wolfram's (long but fascinating) blog post offered me two more big ideas:

1) That there were and are people who think about symbolism and notation, and invent new ways to think about stuff. And that Moses Schonfinkel was one of those people, who also tried to distil mathematics down to a minimal framework - and through his invention of combinators reduced all maths down to just three combinators.

(A note for any computer scientists out there: combinators are equivalent to Turing machines which are equivalent to the Lambda calculus which is equivalent to cellular automata.)

2) Wolfram's claim that probably the single most important idea of this past century is that of universal computation: that in an absolutely real sense, the universe is an engine like a cellular automata, that's in operation.


His blog post also mentions his new physics project (Finally We May Have a Path to the Fundamental Theory of Physics…,and It’s Beautiful), announced and released several months ago, and today I've listened to the 4+ hours of his fascinating interview by Lex Fridman about it (Stephen Wolfram: Fundamental Theory of Physics, Life, and the Universe | Lex Fridman Podcast #124).  I highly recommend it.  Fridman conveniently breaks down the video with time coded links in the description (or you click on the "Chapters" link near the bottom of the video to call up a more graphical view of all those topic areas), if you want to just dip in.

(His explanation of the fundamental idea of his new framework, the hypergraph, is clearly described in that link above, in the section "How It Works".)  Wolfram is hoping to find "the right rule" (or small set of rules?) that would produce our observed universe and physical laws.

He says in the article:

"But in the early 1980s, when I started studying the computational universe of simple programs I made what was for me a very surprising and important discovery: that even when the underlying rules for a system are extremely simple, the behavior of the system as a whole can be essentially arbitrarily rich and complex.

"And this got me thinking: Could the universe work this way?"


To summarise some parts of the interview, I think he and his team have come up with a new mathematical symbolism for working with physics, and I think it's probably a major breakthrough.  I think it's important because it offers deep insights into quantum mechanics, as well as special and general relativity.

A few points that stood out for me:

- Space is quantised (I think he said thinks, at about the scale of 10^-100); and that there may be 10^400 or more points

- Everything is just space

- The key part of the symbolism is the idea of what he calls a hypergraph that captures the relations between points in space

- You could represent it as a graph (pairs of nodes connected by edges), but it's better to connect a node to multiple other nodes by a hyper edge (a surface?)

- Time is the sequence of applying cellular automata style rules. There may have been about 10^500 moments of time so far

- You can estimate how many physical spatial dimensions there are by how many dimensions you need to represent any specific hypergraph to avoid lots of crossings. For some hypergraphs that comes out as three.

- You can make statements about the curvature of space, and the expansion of the universe, in a hypergraph.

- Quantum mechanics, as formulated in the 20th century, falls out naturally from the representation.

- Ditto for general relativity, and also special relativity

- The new formalisation, the new mathematics, is relatively easy to learn, and there's plenty of low-hanging fruit (insights) from applying it.

"It’s always a test for scientific models to compare how much you put in with how much you get out. And I’ve never seen anything that comes close. What we put in is about as tiny as it could be. But what we’re getting out are huge chunks of the most sophisticated things that are known about physics. And what’s most amazing to me is that at least so far we’ve not run across a single thing where we’ve had to say “oh, to explain that we have to add something to our model”. Sometimes it’s not easy to see how things work, but so far it’s always just been a question of understanding what the model already says, not adding something new."

- One example is that fermions and bosons are fundamentally different because in his formulation the fermions are the particles that like to bifurcate in the hypergraph and the bosons like to join branches.

- I gather integer spin and half-integer spin particles have interesting explanations in the theory

- He has an estimate that in the hypergraph that represents our universe, there's 10^200 times more "activity" going on to "maintain the structure of space" itself, than into maintaining all the matter we know exists in the universe.

Wolfram Physics Project: https://www.wolframphysics.org/
Stephen Wolfram's Twitter: stephen_wolfram
Stephen's Blog: https://writings.stephenwolfram.com
His Books:
- A New Kind of Science
- A Project to Find the Fundamental Theory of Physics

Wolfram writes:


"Will we be able to bring together physics, computation and human understanding to deliver what we can reasonably consider to be a final, fundamental theory of physics? It is difficult to know how hard this will be. But I am extremely optimistic that we are finally on the right track, and may even have effectively already solved the fascinating problem of language design that this entails."

and

"For me, one of the most satisfying aspects of our discoveries over the past couple of months has been the extent to which they end up resonating with a huge range of existing—sometimes so far seemingly “just mathematical”—directions that have been taken in physics in recent years. It almost seems like everyone has been right all along, and it just takes adding a new substrate to see how it all fits together. There are hints of string theory, holographic principles, causal set theory, loop quantum gravity, twistor theory, and much more. And not only that, there are also modern mathematical ideas—geometric group theory, higher-order category theory, non-commutative geometry, geometric complexity theory, etc.—that seem so well aligned that one might almost think they must have been built to inform the analysis of our models.

"I have to say I didn’t expect this. The ideas and methods on which our models are based are very different from what’s ever been seriously pursued in physics, or really even in mathematics. But somehow—and I think it’s a good sign all around—what’s emerged is something that aligns wonderfully with lots of recent work in physics and mathematics."

He's also doing this all out in the open (publishing the software and papers), and inviting collaboration.


Exciting days (in a good way!) may lie ahead.

Friday, 31 July 2020

Unconscious Thought Theory as a Creativity Tool

(This is a near transcript of the workshop I ran at the 2020 World Science Fiction Convention, held virtually in Auckland, New Zealand.)

Hi, my name’s Luke Kendall. I’m writing a series of sci-fi/fantasy books about a lovable young female assassin. Thanks for choosing to attend this session, “Unconscious Thought Theory as a Creativity Tool”.

I’ve never run this workshop before: so, welcome to a world first!

I’ve laboured on this and I’m enthusiastic about the subject both because I know it works, and I know why – and also because it’s not just true, it’s based on a deep truth.

Because there’s a lot to get through I’ve practiced and rehearsed this, and ended up with a script to keep myself on-topic. So you won’t need to take notes because I’ll be posting the transcript on my website, A Toe in the Ocean of Books. It’s also being recorded, and I’ll share the presentation slides too.

Why Listen To Me?

Although I’m no expert on the Unconscious Thought Theory (UTT), I may be one of the few people with experience in consciously applying it to the writing process – until after today?

From 1991-2015 I worked for Canon’s Australian research centre. Around 2014, they paid for a full day workshop for thirty or so people on what UTT was and how it could be used in innovation.

In 2014 my wife of 30 years died. A few months later I was retrenched.

But thanks to a life-changing suggestion from an insightful career counsellor (Karen Tisdell), I decided to pursue my lifelong dream of becoming an author.

I had a 140k word MS I’d first drafted in 1993 and reworked off and on up until around 2005 when I just set it aside.

Mid 2015, following advice from my editor in Ireland (ThEditors.com) that I had 2-3 books crammed into this single MS, I had a big problem – after splitting it in two, I needed a new plot and climax for what had been the midpoint,

I found the solution by taking what I’d learned about UTT for innovation, and applying it to my creative writing.

How can Unconscious Thought Theory (UTT) help in writing?

BTW: make sure you have a piece of paper and a pencil or pen to hand for a little later on…

In this workshop I’ll share what I know about using your unconscious mind for creative work – specifically, by applying the UnconsciousThought Theory. We’ll also try it out with a practical exercise.

So here I’ve listed some things I’ve found it useful for:

  1. Plotting

  2. Inventing stuff

  3. Restructuring a story

  4. Coming up with twists

  5. Solving continuity problems

  6. Fitting new elements into an existing story

  7. Overcoming some kinds of Writer’s Block

  8. Escaping corners you may have painted yourself into

What is UTT, and the science behind it

It’s a relatively new, and mildly controversial theory about unconscious thought processes, that’s grounded in science.

The handout has lots of links to papers and studies and talks – they’re not to look at now, they’re for the convenience of anyone who wants to dig a little deeper, later.

I summarise the theory this way: we have a rational, conscious part of the mind, and an unconscious part of the mind – and the two work very differently.

  1. The rational part is more logical, analytical, and sequential.

  2. The unconscious part is highly parallel, and great at finding patterns and making connections.

  3. We can consciously prod our unconscious into tackling problems for us, provided we distract our conscious mind in the right way at the right time.

Here’s a quote from the German philosopher Arthur Schopenhauer who wrote:

“One might almost believe that half of our thinking takes place unconsciously… I have familiarized myself with the factual data of a theoretical and practical problem; I do not think about it again, yet often a few days later the answer to the problem will come into my mind entirely from its own accord; the operation which has produced it, however, remains as much a mystery to me as that of an adding-machine: what has occurred is, again, unconscious rumination.”

A few references:

One of the first psychological experiments leading to UTT involved asking people to solve some real life problems like rating a set of apartments.

The experiment presented four apartments – one desirable, one undesirable, two neutral – and provided information like how big the apartment was, how old, whether it was close to public transport, hospitals, and so on… twelve aspects across four apartments, giving a total of 48 pieces of information to absorb.

Then they broke up the people in the study into three groups.

  1. One group was asked to work out the best apartment rationally, in three minutes;

  2. a second, to decide immediately.

  3. But the third group was distracted for three minutes then asked to decide.

They found that only the 3rd group performed significantly better at choosing the desirable apartment.

Functional MRI studies have shown that when your unconscious mind is at work on a problem, up to fifty areas of the brain go active. In contrast if you’re doing a logical task, there will be just one or two parts of the brain active.

The unconscious works in parallel, the conscious mind is serial, working on things one step at a time.

The unconscious can process, store, and link large amounts of information; the conscious comfortably remembers about seven things at a time. But the unconscious is very poor at working out logical stuff: it can’t do arithmetic, it doesn’t do if-then-else chains of reasoning. But it is great at finding patterns and weighing up fuzzy pros and cons.

Just don’t ask your unconscious mind to do your tax return for you.

The key idea of UTT is just this:

“Simply stated, people can either think about things or not.  UTT adds the idea that we [can] think unconsciously.”

Practical exercise (part 1 of 2)

Okay, with that basic information, and given this is a workshop, let’s try the technique ourselves right now. Grab that paper and pencil, and what I want you to do now is think about some problem that needs some creativity to solve.

  • Step 1: Write down, in bullet points, what you know about the problem
    • That consciously freshens up what you know about the problem
    • That information feeds into your unconscious and helps signal that it’s something requiring attention
  • Step 2: read over those notes
    • Add more points if they occur to you while doing this
  • Step 3: think about the problem any way you like (logically, idly, …) for just a little while
    • But if you do start thinking logically/analytically, be very wary of that; this is the phase we’re just collecting all the pieces.
  • Step 4: distract your conscious mind.
I’m going to keep waffling for a little, while write down the problem and its parts – not solve it. Just ignore me – turn me down somehow or take off your headphones for three minutes. I’ll wave when the time’s up.

When you jot down whatever you can about the problem, that includes things like all the conflicting goals you’re trying to achieve. I had a tricky plot point where I ended up listing six or more things I wanted to achieve, all at the same time. Write down the things you must have, and the things you must avoid. Or that you’d like to have, or avoid. Stuff that could be involved; including people.

What about the tone of the solution? Write down some possibilities. Dramatic? Touching? Sad? Happy? How much time might pass – minutes, days, years? What resources do the characters have available? Basically, write down anything you feel could be relevant. If you were alone and working on your own time, you might also consider whether you’ve already made notes of ideas to consider later, fragments of things that could be useful to you one day – in which case you could dig out those old notes and read them. Or do some searches for helpful information that might factor in.

Once you’ve written them all down, read them. You might think of extra details you want to factor in. You might have bits of inspiration already, ideas that feel like they might be pieces of the puzzle. Write them down.

Once you feel you’ve got everything kind of piled up and buzzing around in the back of your mind, you can think about them in any way you like – rationally or not – for a little while. But if you find you’ve started to think logically about the problem, be very wary of that. Unless your gut instincts are telling you you’re finally on the right track: if so, charge on ahead.

[Waves.]

At some point though, if you haven’t solved the problem, especially if you feel you’re going in circles, that’s the point to stop. So at that point we want to stop the logical part of your mind from trying to calculate the answer, and let your unconscious mind get to work on it. By now it knows you want a solution to this problem, but it doesn’t hurt to tell yourself that explicitly as well. Even write that down if you want. And then you distract your logical mind for a while so it can’t get in the way.

Right. So what we’ve just done is the first half of the practical part of this workshop. I’m not claiming you’ll definitely have an answer by the end of it – but you might! This will be our little experiment. I think this is being recorded, so if at any point you want to mute me or take off your headphones because you’ve just had an inspiration, then go ahead. You can replay the part you miss later, or skim the handout to get the gist of it and catch up.

So now I’ll distract your conscious mind by telling you…

How UTT worked out for my writing

So for my 1st book, the major feedback from my editor handed me a BIG plotting problem to solve. After going round and round in circles and getting nowhere, it occurred to me to take the theory I’d learned about for business innovation, and try it out for writing.

I did, and a few days later came up with a solution. Afterwards I realised, “Holy smoke, that worked!”

Later in that 1st book I had another smaller plot problem to solve, so I applied UTT again, and soon tweeted that I’d solved it with what I called a ‘horrifying teacher’ idea.

So the next time I needed a creative solution, I tried it again. Lo and behold, it worked again. I kept using it, and it kept working.

What I did discover was that depending on how tough the problem, the more time it took.

So, not knowing if the solution was there yet, I’d ‘check in’ by popping back to the problem and sit with pencil and paper and write down any fresh thoughts I had – more aspects of the problem to consider, or more ideas of things that might help, that might partly solve it. If my unconscious had the solution ready, I found that when I started writing, the solution would emerge, a sentence or a thought at a time, and I’d write that down, and then the next, and the next, until I had it all – or mostly – solved. At other times, nothing would come, and I’d realise it wasn’t ready yet.

Now, I think this approach keeps your unconscious working on the problem.

For me, using UTT often feels like working on a jigsaw puzzle – sometimes finding pieces, sometimes fitting a few pieces together.

Another of the nice things about UTT is it takes some of the self-critical pressure off you. Instead of thinking “I don’t have a solution; I’m stuck”, you can honestly tell yourself “I’m still working on it: don’t rush me! Good things take time.”

Suppose you’re writing about a character and they’re currently stranded in a desert. If you have negative thoughts, like “There’s no way she can survive in the desert without water!” either add that point to the list of things to factor in: “My character needs to find a way to survive in the desert” – or if that point’s already on your list, just say “Yep, so that’ll make a great surprise for the reader when she overcomes that!”

Do others use their unconscious for creativity?

You bet! I was listening to a Ted talk by Elizabeth Gilbert speaking about the pressures that creative careers place upon you: Your Elusive, Creative Genius. She said the notion that all creativity came from within the individual – and so the artist alone deserved both all the joy of success, or all the pain of failure – only arose in the last 500 years.

Before that, people felt the burden of both success and failure was a shared one. E.g. the ancient Romans believed in a ‘genius loci’, a spirit that lived literally in the walls of their home, and when propitiated helped the creator invent, among other things.

She interviewed the great American poet Ruth Stone in her 90s who spoke of being able to sense when a poem was on its way: to her it was like a thunderous wind barrelling toward her across the plains, a great storm. And unless she could get inside and get to a pencil and paper, by the time this mighty storm train of a poem crashed through her she would lose it. So she’d run for the house: if she got there in time, she could capture the poem and copy it down; but if she was too late, it would surge through her and past and be lost. But if she could grab a pencil before it had passed completely through her, she could kind of reach out spear it with her pencil tip, and drag it back inside her body, gripping it until she could find a piece of paper, in which case she would then haul it back inside her, word by word, transcribing it onto the paper. And most fascinating of all, it would be in reverse: word for word, backwards.

[As further evidence of UTT, thanks to a dream this morning, I worked out why it came out in reverse!]

Elizabeth also interviewed the musician Tom Waits, who had similar experiences, until in middle life he did something novel one day which changed his whole process. She tells how he was driving on the freeway in Los Angeles, unable to stop, and suddenly heard in his head a most delightful melody, and knew he had to capture it at once or it would be lost. So all the usual stress and tension started building up. But instead of panicking he looked up into the sky and said out loud “Excuse me, do you not see I’m driving? Does it look like I can do anything for you right now? So, either you come back later, when I can give you the care and attention you need, or go and bother someone else; go and bother Leonard Cohen.”

She told the story of one of the depressing times while working on Eat, Pray, Love, when she became convinced that what she was writing was not just going to be the worst book she’d ever written, but the worst book ever written. But remembering what Tom Waits had told her, she leaned back in her chair and said to an empty corner of the room: “Look, you and I both know this isn’t all just on me: I’m doing the best I can; I’m putting everything I have into this; but it needs the both of us. So if you’re not going to do your part… well, I’m going to sit here and keep working. But I’d like the record to show that I showed up for my part of the job.”

Elizabeth’s thought was that by putting all the credit for success and failure on the shoulders of a single person, we’re putting incredible pressure on each creative individual, and it’s this awful pressure that leads to so many well known stories of writers and other artists literally dying: if not through conscious suicide, often through disabling addictions or disastrous life choices. And that by acknowledging a spiritual dimension to the process, that huge burden can be shared.

So just knowing your unconscious is able to help solve your problems, eases that burden.

It’s hard to avoid stumbling over evidence the unconscious mind is routinely used to solve creative problems.

  • James Watson dreamed of a spiral staircase as part of his and Francis Crick’s discovery of the double helix of DNA.

  • Neil Gaiman talks of his ‘compost heap’, which he fills with random facts and ideas, from which ideas flower.

  • Even our expression “I’ll sleep on it,” is an explicit recognition of the creative problem-solving abilities of our unconscious minds.

If you search the internet for how to come up with ideas for stories, you’ll often find suggestions like using the I-Ching, or tarot cards, which may be ways of tapping into a spiritual well of creativity. I basically agree, except I see that well as being inside us, and the springs that feed it are everything we experience.

I sometimes use a stirring piece of music to get into the right frame of mind to write a dramatic scene. Music is known to activate almost every part of the brain – quite like how the unconscious does. Interesting, eh?

UTT for Writer’s Block?

Writer’s block is a fuzzy sort of term, possibly unhelpful or damaging. Some say it’s not a thing in itself (a symptom not a cause), others disagree. But let’s set the term aside. If you suspect something is blocking your writing, put some conscious thought into that. There’s no reason you can’t tackle that itself with UTT!

Of course there are many, many things that can interfere with or stop you from writing, so how to get past the block probably depends on the cause. But I’ll mention a few I overcame with UTT.

For me, remembering the elements for a scene or story arc – characters, plots, resources, setting, tensions, and so on – takes mental effort. I found by writing that stuff down it removed that burden. On more than one occasion, I couldn’t progress because I was mentally juggling all these things and had no spare capacity to invent what I needed to move forward. So that’s one thing that can block me.

Writing it down also primes the unconscious, so it can start looking for patterns, making connections, and extrapolating from what’s there.

On other occasions I had my plans for what was going to come next, but somehow kept finding other things to do first. Sometimes they were real life issues I had to deal with as a priority – but not always.

But then, more than once, when I stopped and had time to let my mind wander, a great new idea emerged very quickly. So quickly, in fact, I suspect I’d felt distracted or demotivated because my unconscious had the idea waiting but I hadn’t given it the stillness it needed for the idea to surface consciously. On a couple of occasions, the new idea saved me from heading down a plot path that would have dead ended and needed to be thrown away.

(Sometimes of course it was a real life problem that needed to be dealt with.)

Practical Exercise (part 2 of 2)

I think it’s time to return to our workshop exercise. Go back to the problem you posed yourself. Now skim to the end, and over the next thirty seconds, start writing something relevant: ideally the solution. It might emerge as you start writing. But don’t start trying to reason out the answer, just start writing. I’d like you all to silence me for 40 or 50 seconds so I don’t distract you as I keep talking. Let’s see how we all do. I’ll keep talking to avoid dead air… (but it’s captured on the next slide, so you won’t really miss it).

I think the unconscious is often the source of lots of our ideas. The trick to using it is threefold:

  1. First, just knowing it exists and that you can use it to solve some kinds of problems;

  2. Second, feeding it enough raw materials, so there are bits and pieces for it to connect to solve the problem;

  3. Third piece, leaving a silent space for it to present its answer to you.

I’ve found the more I use it, the more I come to know the feeling when my unconscious has something ready for me. It’s by no means obvious. At least for me, it’s not a voice in my head that tells me the answer.

[Waves to try to attract attention.]

How did we all do, I wonder? I’d like to hope that many people cracked a knotty problem, or at least now see a path to the solution. If not, just keep checking back in every day or few, to see if your unconscious has cobbled together the patchwork genius idea you need. Feel free to leave a comment in the Chat, either way.

Luke’s Tips

So here’s my list of other things to do, to help you use UTT in your creative endeavours. They’re in no particular order.

Write stuff down. That’s a key point: don’t rely on your memory. If you don’t record the ideas, you’ll waste mental energy consciously remembering them or worrying about them. Also, once they’re written you can re-read them if you find yourself in a hole.

Clear the decks of clutter. For the intense parts of the creative thinking process – the parts where you make the act of creation your top priority and give yourself time exclusively for that – you need clear mental space. If you have other tricky problems you need to solve, either deal with them first, set them aside, or accept that you’ll take longer because your unconscious will be working on multiple problems at the same time.

Sometimes you can clear mental space for a problem, by writing it down just as a “to do” thing.

Don’t allow yourself to get daunted. The funny thing is, that although the number of pieces to a problem might look scary, each issue is just another piece of the puzzle. In a way, it adds to the richness of the solution; it adds to the stuff your unconscious can use to create an arrangement that makes them all fit, or at least blend.

Allow yourself time. Set aside distractions as far as you can and give yourself time. Do this as often as you need to, in each attempt to come up with the needed idea(s).

Try and try again. You don’t need to solve the problem in a single session. Sometimes the solution will turn out to be a jigsaw that needs to be assembled. Sooner or later though you’ll probably start sensing the big picture: and then it will start filling in quickly.

Trust your instincts. Your gut will often tell you “This idea feels right” – trust it. Think about the idea, and around it. ‘Your gut’ is your Unconscious talking to you. Don’t ignore it!

Missing Facts? It’s possible you’re blocked because you’re missing some pieces of the puzzle. E.g. in one instance I simply hadn’t imagined the physical layout fully, so I stalled when trying to write the action parts of the scene.

I’ve blogged about UTT and creativity several times, probably in too much detail. But that may be helpful for anyone who’d like to read more about applying the technique, and I captured just such an example here, which happened to me while writing book four in my series.



For authors and artists, everything you do, every movie you watch, every poem or piece of music that resonates with you, every person you meet, every pain you suffer or joy you feel, it all goes into the Aladdin’s treasure cave of your unconscious, where the alchemy of the most complex thing in the universe – the human brain – goes to work to transmute it into thoughts and ideas and works of creation.

We each carry around inside us a fount of creativity that we can tap into quite easily. It’s sensitive to our moods, it’s sensitive to and absorbs facts and impressions, sights and sounds and smells, from the world around us; it’s synthesising ideas and joining facts together, juggling things and looking for patterns, and best of all, if something is important to us and we consciously pose a problem, that mighty unconscious array of mental powers goes to work trying to solve it.

Humans are imagination engines, naturally generating ideas like the sun generates light.

It’s what we do.

Thanks for listening to me, and I hope this little workshop will prove useful for you in the future.


Some further references

  • The article that kicked it all off (about 45 very readable pages): A Theory of Unconscious Thought

  • An overview of incubation (sleep, shower…) and its underlying causes: Creativity—the unconscious foundations of the incubation period
  • The nature of consciousness itself: Passive Frame Theory – suggests it’s a kind of spontaneous symphony bubbling up:
      “the information we perceive in our consciousness is not created by conscious processes, nor is it reacted to by conscious processes. Consciousness is the middle-man, and it doesn’t do as much work as you think.”
  • This study shows complex ideas can enter consciousness automatically.
  • Exploring the elusive gap between anecdotal and empirical support for UTT in creative problem solving: The Merits of Unconscious Thought in Creativity:

      “unconscious thought cannot ‘create’ knowledge; conscious learning and thought process are needed to establish a knowledge base.”
  • In case you’re wondering: the company that provided the innovation training was called Inventium. They have a few blog posts about UTT.

Post-workshop chat

I think dreams in which ghosts tell us things or do things that predict real-life experiences are your unconscious communicating to you that it’s been acting like a detective, and worked something out from subtle clues you had not consciously noticed.

I realised I conflated two separate Kate Bush songs when I was reaching for the Gurdjieff connection - it was “Rolling the Ball” where she mentions Gurdjieff as one of the “wonderful people [ready?] to teach me”.  The link from Cloudbusting to the strangely subversive Wilhelm Reich and his Orgone Energy was a complete furphy, sorry!

I think the purest experience of our unconscious mind is when we’re dreaming.  It’s why dreams can be so illogical: because the conscious, rational part of our mind is not participating (or minimally participating), because it’s asleep.

And the explanation for why Ruth Stone transcribed these poems backwards?

I think it’s simply because that’s how we can remember dreams. I think dreams are our unconscious telling us stories: which are largely the things it creates.

Usually the last part of a dream is the thing you remember - maybe the only thing.  But if you try to remember what led to that, how you got there, you can remember the previous scene in the story.  And from that one you can step back to the one before that, and so on.  So you recover the memory by working backwards, chasing the links from one piece of story to the one before, and that’s why her poems had to be remembered backwards.

Copyright © L. J. Kendall 2020. Permission granted to reproduce for free, provided this notice is included.

Sunday, 5 April 2020

A Novel Ending

A Novel Ending

A funny thing happened on the way to the ending…

I’m loving my new being-an-author career, despite my inability to earn a living from it. (Yet?)

One of the things I enjoy is that each book feels like it comes with its own unique challenges.  I hope that means I’m learning as I go along.

For the current book in my Leeth Dossier series, Lost Girl (#5), there were two challenges.  The first was my feeling that on the subject of plot, I fly too much by the seat of my pants, so the whole journey from start to end is like a billy-cart racing downhill out of control.  Exhilarating, but also nerve-wracking.

Looking back through my revision control files, I deduce I started writing the Lost Girl segment of the story back in Apr/May 2009.  I wrote a chunk more in Jan 2015 before posting the chapters for review on the Online Writing Workshop for Science Fiction, Fantasy and Horror over a few months and receiving valuable feedback.  By Oct 2015 it had grown to about 25,000 words.  I think that was when I set it aside to devote all my energy to completing the earlier books in the series.

That took five years.

So, for book #5 I was starting with a broad idea of the story, 25,000 words of draft, and thanks to a discussion with Jon Marshall one night, a good feeling for the theme of the novel.  I also wanted to try a hybrid approach to plotting.

Pantser or Plotter?

I love the organic surprises and creativity that comes from flying by the seat of my pants, but at the same time, having watched his video Misconceptions About Rewriting, and from my own experience in algorithmic development, I deeply understood the value in at least high-level planning.  So I hoped that creating the bones of a plot might work well, since it would only be a sketchy outline and thus pretty painless to change it if the story started going in a different direction.

I started reading John Truby’s “The Anatomy of Story” in early 2019, reaching a quarter of the way through before getting distracted. (I see there’s also a video interview here.)  The book and advice is fine, but it wasn’t sparking any flames for me and I set it aside.  (I do intend to return to it and finish reading it though.)

I’d downloaded a free writing book from Daniel David Wallace titled “How to Write Better Sentences”, and watched some of his videos, and liked them, so when he offered a course on his character-driven plotting course in December 2019 I decided to sign up for it.  I did that diligently from mid Dec 2019 to end of Jan 2020, and was very happy with the outcome.  I had lots of ideas for how the plot would unfold, all of course driven by the characters, which is how I have to write.

The problem that remained was that I only had a satisfying end.  I think you also need the ending to be unpredictable.  I (still) don’t know how you plan that.  For me it’s just happened by luck or inspiration, and I was still in the same position, lacking that final piece of the puzzle.

As I’ve written on several occasions now, I’ve really come to trust Unconscious Thought Theory.  It’s an excellent way to use the full power of your mind and surprise yourself.  So I carried on writing, always with the lightly-plotted ending in mind, and things flowed along pretty well.

Now, because the World Science Fiction Convention was to be held in New Zealand in July/Aug 2020, a place I’d wanted to visit and a convention that seems exciting, I planned to attend.  I booked my travel and accommodation, with the goal of publishing the book late July and maybe even launching it or promoting it in some way at the WorldCon.  To do that I needed to send the MS to Dave at ThEditors.com by E/April.

So I was trying hard to meet a deadline of E/March to have a good 1st draft of the MS.  I was extremely lucky to be little affected by the unprecedented Australian bushfires of 2019, but no one could remain unmoved by those events - from the heroism of our fireys and communities ( (even this Aussie speculative fiction anthology put together for fire charities: Stories of Hope), to the jaw-dropping failures of our so-called political leader to actually lead.

There was also the ongoing drama of the Democratic run-up toward the 2020 US presidential election, which I had grown addicted to (since the US has an inordinate impact on world affairs, and I could see what a turning point the 2020 election could be, with luck).  Then of course, the global pandemic of Covid-19 arrived, making the insignificance of my own efforts even clearer.  Now, I may not be as stubborn as Leeth, but I am stubborn, so I carried on.

Like the period after the death of my wife Stella in Dec 2014, my writing sustained me and kept me positive. So for that reason as much as any I stuck to my plans.

Which brings me to the part of the story which most surprised me — and in the end delighted me.

How did it go in the end?

Approaching my deadline, I had set myself a (120,000) word count limit to try to hold myself to, to keep the book a manageable size for all concerned.  And as the days fell away and March 31 approached, it looked like I might just do it.  But when the days dropped from three to two to one and I arrived at the cusp of the ending, I’d run out of time and still didn’t know fully what the ending would be.

Partly this was just having confidence in my unconscious to pull a rabbit out of the hat; partly it was the belief that if I didn’t know what the ending would be, it might mean that when (if?) it did arrive it would also surprise a reader; and partly it was having faith in my characters.  I also had an interesting mix of actors there at the ending.  There was the potential to have… (counting…) seven major players in the scene at the end.  Unfortunately that meant there was a huge range of ways the ending could play out.

So I think there was a certain amount of choice-paralysis at work, although it’s also possible my unconscious was simply beavering away at the complex problem I’d set it.  Of course I was also worried I might not be able to pull it off at all.  Would the ending be good enough?  I didn’t want just an exciting one, I wanted it to have emotional depth too.  So the stakes felt high to me.

And still I couldn’t seem to make much progress.  Here’s how the last several days went:

March 29: Had reached the climactic scene — I mean, all the players were on stage together.
March 30: Wrote 3700 words of the scene.
March 31: Day off, visiting Mum.
April 1: (astute readers will see this is after E/Mar) Strangely blocked, until I forced myself to design the physical location. Then: +600 words.
April 2: Wrote 3,600 words and halfway through the “action” part of the ending.
April 3: Finished the “action” part (wrote 1000 words).
April 4: Still didn’t know how it would end. I simply considered each actor and let them do their thing. Various agendas in play, conflicting, and stage fully set and conducive to just letting the movie roll. Each one being clever and true to themselves, a dramatic race to the finish with real challenges and emotional drama. At the very end of the day (11:50pm), I felt delighted. My characters had thrilled me and moved me. I’d been biting my nails more than once, and in tears more than once. 5,100 words and the MS was basically complete.

I’ve never experienced anything like that last day.  My feeling was one of excitement and satisfaction.  I made some small additions while cooking dinner (9pm), and when I sat down with it I realised I didn’t want to watch any TV as I ate, I instead wanted to dwell on those final scenes, to savour and digest it all.  I suppose it felt a bit like I’d just stepped off an emotional roller-coaster!

I still have thirty one small gaps to fill or issues to check, and then lots of work to polish it up enough to be ready to send to my editor, so I can’t really relax much yet.  But the weight on my shoulders of finishing that critical 1st draft, and perhaps as importantly, writing a ‘good’ ending, is over.  So a lot of unacknowledged stress has fallen away, and I must say I’m looking forward to the next few weeks.

I’ve written this because the experience felt really special, and I wanted to capture it. It was nothing like I’d had for the previous books.  Once again, I feel I learned a lot.  My fingers are crossed in the hope that readers will have the same opinion!


Sunday, 8 December 2019

Overcoming Obstructions to Writing

(Photo by Travis Saylor from Pexels)

Just a quick post to share something small I learned about how my mind works when it comes to writing.  For the last week I’ve made no progress on Lost Girl (Leeth Dossier #5).  There were everyday tasks which I felt needed to be done (like coffee: harvest time won’t wait), but that wasn’t enough to explain it all.  Those tasks only took up the time and energy for a portion of each day.

I’ve written about the Unconscious Thought Theory several times now, and I know for certain it’s heavily involved in my own creative process.  (I strongly suspect it’s true for everyone, but I don’t know that.)  So I know a lot of the work happens in the unconscious.  I also noticed previously that I was blocked when my conscious plan for how a story would unfold didn’t agree with what was developing in my unconscious.  In hindsight, I felt my unconscious threw up obstacles to stop me heading down that wrong path I had consciously planned.

I also know the neuroscience says the unconscious parts of the brain have something like fifty ‘processing units’ running in parallel. That sounds like a lot, but it’s a finite number.

Anyway, what happened was that tonight I tackled a task that had been weighing on my mind for a couple of months now.  (Selecting and booking a hotel for the world SF convention next year in Wellington, New Zealand, to be precise.)  I felt some relief as soon as I’d done so.

What I realised later however was that when I started to think about the next scene in Lost Girl, my mind didn’t instantly shy away and offer up a distraction.  Then I recognised that mental shying away was something I’d learned to do a long time ago to protect myself from worry about a situation I had no control over.  At that point I finally put two and two together and realised my unconscious was trying to solve the hotel booking problem and didn’t want to spare time for book creation.  Each time I tried it distracted me in a non-creative direction.  Sneaky, wise unconscious!

So, if you’re finding yourself blocked, go looking for a cause and then tackle that.  There’s probably a lot of different things that could be the issue.  Two I’ve discovered are 1) your plan is wrong, and 2) you have something else that needs your unconscious attention first.  The unconscious is great at making patterns and weighing up combinations of possibilities, but it has limits!

Friday, 11 October 2019

Leeth’s Journey So Far: “Four Books!”

Chatting to an author friend today (Barbara Strickland), about what lay ahead for Leeth, she asked whether my idea for the series in the longer term was to pursue the action path or the emotional side of things?

I’m four books in now, and so far I believe I’ve delved deeply into both sides.  I had to stop and think though, to consider Barbara’s question.  Thirty years ago, I thought the series would be a sci-fi/fantasy action series, but when it came down to it, I couldn’t tell the story without honoring the depths of the characters.

(Because I love Mirella’s cover designs, let me just brighten this short post by including them here:)

Anyway, I realised my answer to Barbara’s question was “both”, even though it means I’ll need to continue a tricky balancing act.  I mustn’t let the excitement of the action elements of Leeth’s dramatic life distract me from the compelling human and emotional side of her story as it unfolds.  There’s no chance of losing sight of that in Book 5, Lost Girl, but in later volumes it may be something I’ll need to keep in the forefront of my mind.

Meanwhile, back at the ranch…

I’ll finish with a quick status update.  In August, Galaxy Bookshop hosted the launch (see the video) for Violent Causes (The Leeth Dossier Vol. 4): all my books are available on the shelves there!  As well, I’ve been learning bits and pieces about book marketing.  I’d much rather be writing, so I plan to go back to devoting myself to that — soon.  Maybe I’ll spend one day a week on the marketing side.

Things I’ve been learning about are Facebook ads from Mark Dawson and Nick Stephenson, Amazon ads from Bryan Cohen (I even created some ads), drawing up a far too exhaustive marketing plan, and learning bits about getting reviews and starting up a mailing list.  I also set up my Facebook Author Page (which I gather if you Like it, you’ll be notified when I post something there).  I also 99% converted my Blogger blog site into a kind of web site, only to stumble at the final hurdle.  Two days before I discovered how to overcome that last problem, I gave up and recreated my website in Wordpress.org.  It’s in pretty good shape, though I still have a small list of things to do.  Via my hairdresser Sebastian I discovered the talented Kellie M Davies and through her, the Aussie Speculative Fiction (ASF) FB group.

I also sent some books down to Bendi-con as part of an ASF contingent, and some are on their way soon to Supanova Adelaide and Supanova Brisbane with the same group of Aussie authors.  (We also had a small get-together in Sydney CBD, which was a great evening.)

Roger Patulny kindly agreed to shift the meeting date for our local writers’ group Authora Australis to a Wednesday that didn’t clash with the Sydney Mediaeval and Renaissance Group’s monthly meetings (thanks, Roger et al!), so I can attend regularly.  And this Sunday (Oct 13) Kristyn M Levis will be talking to our Book Marketing discussion group about Instagram for Authors.

I also created my first NielsenBook2Look” book widget (“biblet”) for Wild Thing.  Biblets let you look inside a book, share it on social media, find out where to buy it, and more.  Oh, and I had a flash fiction sci-fi story accepted by Antipodean SF magazine, called Dangerous January (which will be available there for a limited time).  Which also reminds me: I completed the Neil Gaiman Master Class too, and enjoyed it!

I guess I have been working hard after all (but not with blogging — shame, Luke!), even though there’s been far less writing than I like.

Thursday, 4 April 2019

Sleeping Giants

I don't think I've written a social blog post before, here, but I read the article Why the Anglo World is Collapsing by Umair Haque today (Thursday, 2019/4/4) and felt it was important and valuable, but also not quite on the mark in every respect. So I wrote this response. (This is a slightly expanded version of a comment I left on Umair's article.) I encourage people to read the original article.


I share Umair's deep concerns, but I think assigning the cause to the ‘Anglo idea of superiority’ is off the mark. As others have commented on the article, the US and Britain are not the only countries facing very similar problems, and I’d note that Australia, which is another ‘Anglo country’, though more multi-cultural, is not in such dire straits. The imminent Federal election here will likely determine whether we follow the US down its current path, or stay true to our own ideals of mateship and a fair go for all.

I read a fascinating essay by the biologist Stephen Jay Gould, titled “Kropotkin was no Crackpot”, in which he noted that for many people, only the “Nature red in tooth in claw” message in Darwin’s Theory of Evolution was absorbed, elevating the concept of competition to stand alone on the peak. Western societies largely overlooked the part of the theory that noted that co-operation frequently occurs in nature, too. The Russians, perhaps having more experience of living things surviving in harsh conditions, were much more attuned to the idea that co-operation is often necessary for survival.

Around 2010, I saw a study which compared Japanese, Australian, and US business cultures, along multiple dimensions. One stand-out fact was that in the areas of the individual vs the group, and co-operation vs competition, Japan lay near one end of the spectrum, the US at the opposite end, and Australia was about halfway between. I think a reasonable mix of the two is far healthier than valuing one and disregarding the other.

I have heard it argued that toxic capitalism is the cause of the problem (not Superiority), and that capitalism only emerged as a result of the religious idea of Predestination — that God knew each individual’s destiny, and what happened in their life was not something that could be changed. So success and failure was not something that could be altered by helping (or hurting) another, and that in turn led to the concept that fair compensation for work done was unnecessary. That divorced labour from a fair share in the profits of that labour. The extreme end result of this is the situation we find ourselves in globally, where a tiny percentage of the population owns the vast majority of the world’s wealth, despite the fact that that wealth was created by the efforts of billions of people over centuries of effort. It can only be rationally accepted if the wealthy believe, however falsely, in their innate superiority over others.

Anglo societies also place the rule of law above all, education for all, and equality. So do many other societies (all other healthy societies, I would argue). That these values are being eroded nowadays is not due to an Anglo belief in superiority. They’re being eroded by greed; by powerful elites who have been co-operating among themselves to “Manufacture Consent” among the majority (thanks, Noam Chomsky), and to influence policy and law makers to tilt the playing field in their favour (read “Winner Take All Politics” — it’s a heartbreaking eye-opener).

Personally, I think the problem Umair has written about has several causes, and he has attributed far too much to a single one of those.

I also think that Bernie Sanders and Alexandria Ocasio-Cortez are right in saying that courage can fix this, in our lifetimes (within ten years?) — though it will need “Moon Shot” levels of courage, determination, and effort. I also agree 100% with Umair (and AOC!) that the zero sum thinking is a big part of the problem. How anyone can believe that zero sum conditions apply to social systems, or the world’s wealth, boggles my mind. Billions of people around the world are building and creating and inventing and helping one another. Humanity is fantastically wealthier than it was a thousand years ago, a hundred years ago, twenty years ago. It sometimes seems not so, because the 1% have taken so much of that for themselves. So, new thinking, like the New Green Deal, or Kate Raworth’s economic Donut Theory, Modern Monetary Theory, is needed to solve the problem.

At heart it’s so simple, too. Co-operation and respect is all that’s needed: for each other, and for this precious planet. Do Unto Others As You Would Have Them Do Unto You. Basic goodness. How can that message still not have sunk in, after two thousand years? It’s because of the power of stories.

If you’re in one lifeboat, where everyone is paddling in their own direction, where are you going to be compared to a second lifeboat where everyone has agreed to paddle in the same direction?

Another thing, too: one of humanity’s greatest strengths—the desire to socialise and belong to a group—is also our biggest weakness, when we exclude others because they’re not enough like us to be included in our group. And stories are the basis of the group identity, determining who is inside, and who is outside and alien.

The Pen really is mightier than the Sword. The stories we tell ourselves control our fate: and if we believe lies that destroy our societies rather than truths that lift us all collectively, then we have only ourselves to blame. (I remember an advanced driver training course instructor saying: “If you’re ever sliding, out of control, and see a tree ahead, don’t look at it. Look at the gap you need to get through. If you look at the tree, that’s where you’ll end up.”)

So, the stories we tell ourselves, and the fact that a powerful few have been actively co-operating to push down the majority to better themselves. In my experience, 5% of people are bad. They will take what they can without respect for others, and we let them creep into power and poison our institutions at our own risk. That’s why the US founding fathers said “The price of Democracy is eternal vigilance.” Because there really are some bad people out there, and some of them are smart, and charismatic. If we don’t oppose them when we find them, they institutionalise their mindset and corrupt all those around them.

It’s time for the sleeping giant of the vast majority to wake up and step onto the playing field.