Showing posts with label kindle. Show all posts
Showing posts with label kindle. Show all posts

Thursday, 6 December 2018

Withdrawing My Books from Kindle Select

This is just a quick post to announce I will be withdrawing my books from KU/KOLL, in response to Amazon's algorithm changes that mean the KU money is largely now being shared by just two classes of people:
  1. unscrupulous people who game the system
  2. best-selling authors who get millions of pages read per day
My books will still be available for purchase from Amazon, and in print as both A-format and B-format paperbacks. They simply won't be available for free once the Kindle Select renewal periods ends.

For me personally this will drop the royalty payments for the ebooks from 70% to 35%, so I will also be increasing the prices in the next week or so. (I've also been thinking for a long time that I may have been pricing them a little too low.)  This should have a negligible effect on sales, because I am not competing with the scammers to pay for ads, so my books are not being bought.  Very few are even found by people borrowing them (perhaps one or two books per month).

So the ability to borrow my books will end, for each, on the following dates:

Wild Thing - Feb 26, 2019 (The Leeth Dossier #1) (Wild Thing paperback)
Harsh Lessons - Dec 21, 2018 (The Leeth Dossier #2) (Harsh Lessons paperback)
Shadow Hunt - Jan 19 2019 (The Leeth Dossier #3) (Shadow Hunt paperback)

Once my ebooks are no longer exclusive to Kindle Select I'll be able to enrol them in all the other ebook distribution platforms, which do not appear to suffer from the same problems Amazon has engineered for itself.

I doubt I will be returning the books to Kindle Select because I anticipate Amazon will fail to take effective action until the average quality of books available there has fallen noticeably.

For any other authors who wish to withdraw their books, the process is quite simple, as described in the relevant KU help page:
  1. In your KDP Bookshelf page, next to the book you wish to withdraw, click on the "..." (ellipsis) button beside the "Promote and Advertise" button.
  2. In the menu that pops up, click on the "KDP Select Info" item.
  3. On the page that leads you to, clear the tick from the "Automatically renew this book's enrollment in KDP Select for another 90 days" in the panel that pops up. (You might need to click on the Manage KDP Select Enrollment button to get that panel with its check box.)

In happier news, I hope to put the finishing touches to Violent Causes (The Leeth Dossier #4) by mid December (about a week), and launch it a couple of months after that.  I'm pleased with how it has turned out -- with Dave's help (and Jon's) as usual!

Saturday, 3 November 2018

Amazon is failing its authors, its readers, and itself

There’s a growing awareness that using computer algorithms to handle decisions — from debt assessment or even being fired to self-driving cars — can have unexpected and even disastrous effects. Around October 2017, Amazon joined those unfortunate ranks.
Sounds scary? It is, when you dig below the surface a little.
Over a million new books are now published each year. Over half of those are in English.
I started publishing in December 2015, using Amazon exclusively for my eBook editions. Sales and reviews were low but showed a steady growth. My approach was to concentrate on writing each new book and do the bare minimum on the marketing side.
For almost two years that seemed to be working — until October 2017.
At that point, sales plummeted. Had I done something, or said something to upset people? Highly unlikely, since I’d been silently working on the current book. I assumed my silence was the cause. But after discussing sales with a few other authors who used Amazon ads and active marketing, I changed my mind. They told me that from around October 2017, they found if they pay $X to Amazon for ads, they came out ahead by about 10%.
What happened? What follows is my best guess. There are three parts to it: book discovery, Kindle Unlimited, and unscrupulous people.

Book Discovery

Readers can discover books on Amazon through various searches, but in addition there are three important ways Amazon promotes books:
Recommendations
  1. Emails from Amazon to you suggesting books you might like.
  2. Suggestions for other books based on your searches, on Amazon web pages.
Note: included in these recommendations are books that have paid to be promoted.
Best-seller lists. In each category, a list of the books that have earned the most revenue in the last 48 hours.
Popularity lists. Like the best-sellers, but across the last 30 days.

Kindle Unlimited

Readers who pay the monthly subscription fee can borrow books from Kindle Unlimited (KU) at no extra cost. Some of the subscription fees go into a pot of money paid to the authors.
This ‘Cockygate’ article by the Verge notes: “There are over 5 million books available via Kindle, with over a million books available on the Kindle Unlimited system. Amazon is currently paying out approximately 20 million dollars every month to authors on Kindle Unlimited.”
U$20M lying on the table: does that sound like a tempting target for unscrupulous people?
At first, Amazon divided up the pot based on number of books sold. Absurdly short ‘books’ started appearing. Amazon changed the algorithm to count ‘pages read’ instead of ‘books read’. Scammers started stuffing books full of ‘bonus material’ to dramatically boost their number of pages.
It’s been an arms race between scammers trying to grab money unfairly from the pot, and Amazon trying to counter them. This article by Derek Haines argues Amazon’s algorithms can’t count pages or words, but he goes on to add:
KU scammers and ebook stuffers are not the real issues here.
[…] Until Amazon can find a better system to track and record KU ebook reading correctly, the scammers will keep finding ways to make money, and honest authors will lose out.
Or be terminated.
But wait, there’s more…!

How recommendations used to work

Before Oct 2017, Amazon’s algorithms recommended books based on some score related to average review rating, the sales rate, and purchase by other people of similar books. (“People who bought TitleX — as you did — also often bought TitleY.”)
I think this system is still used for the 30 days after a book is released, perhaps extended if the book hits a best-seller list or while it is earning high revenue.

How they work, post Oct 2017

The only books being promoted were those from paid ads. At first, I guessed that Amazon had changed its algorithms to only promote advertised books. There are so many authors publishing, Amazon can profit from authors buying ads.
Amazon don’t disclose how their algorithms work, and rightly so. They’re in a war with those who try to game the system to make money. I have a lot of sympathy with Amazon and what they face.
But since October 2017, the KU pool of money has been paying large sums of money (perhaps most of it) to some unscrupulous ‘authors’. I say ‘authors’ because some are not authors: they are business people who pay ghost writers for content.
The result is, only books with paid ads are being promoted.

Unscrupulous People

David Gaughran’s Kindle Unlimited -- A Cheater Magnet and Kayleigh Donaldson’s Book Stuffing, Bribery and Bullying articles were eye-openers for me — I recommend them. The problem is that payments to unscrupulous people has a terrible effect on honest authors, harms readers, and has started damaging Amazon’s brand (and, I assume, the revenue from self-published books).
The effect on the readers is the smallest: the unscrupulous people can’t afford to annoy them too much, since readers determine the KU payouts. But even for readers, it’s making it harder to find and read books they’ll enjoy.
The unscrupulous people spend big, buying Amazon ads to turn a good profit, paying for those ads from the profits. If collectively they spent 18 million dollars for a 19 million dollar payout from KU, they’d be happy — and hey, that even leaves a million dollars to be divided amongst the remaining 500,000 authors! <sarcasm off/>
The greatest harm is being done to the honest self-published and indie authors. Like most skills, writing follows a Bell curve. A minority are really good; a minority are really bad; and there’s a big spread in the middle between those extremes. (The true picture is more complex, because everyone’s taste differs. But that’s not important for this argument.)
But most good and great writers have no interest or skills in marketing, nor in spending time and effort on it. So most authors can’t compete when it comes to marketing.
If you’re an author selling via Amazon and your readers have declined and your income has plummeted, take heart at least in knowing it’s likely not any indicator of how good your book is. Your small voice is being drowned out by the people gaming the Amazon system. The unscrupulous are producing ten or fifty books to your one, and paying for vast numbers of Amazon recommendations of their books, rather than a meagre few algorithmically-generated recommendations that yours are getting.
Writing a book is hard work. A lot of writers manage one. Producing a book proves to an author they can do it. But without a lot of luck, or a big and skilful marketing effort, few readers will discover the book, so the return is low in terms of money and in appreciation. And with Amazon’s ineffectual response to the cheaters, a lot of new authors will try, but won’t find out if they are great or even good. Unless new authors stumble over this article or one like it, they won’t know the failure may well have been Amazon’s, not theirs.
On top of that, the Amazon algorithms only seem to promote books for the first 30 days. This of course encourages writers to produce shorter books and pressures them to cut corners. For the unscrupulous, it actually plays into their hands since producing content quickly is something they can pay or cheat to produce — they just need to keep one step ahead of Amazon’s lacklustre attempts to rein them in.
By inducing many of the good and some of the best new authors to give up, while still ‘feeding the scammers’, over time the overall quality of Amazon’s book marketplace will decline. At first, not massively. Just a little. But the trend will continue, and in the long term will make Amazon the place for second-rate books which are clearly lower quality than those from other publishers.
The Amazon book marketplace will swell with the people who pay for the ads, and the people with good marketing skills. There’ll be some very good books in there, but on average, the rewards will go to the best marketers and to the scammers, not to the best writers.

How Amazon Could Fix This

It’s in Amazon’s interest to fix this, despite what some cynical people argue. Like other observers, I believe the problem lies as much with Amazon as with the scammers. Amazon seem determined to address the problems with technology (algorithms), not people, and I can see the sense of that: the solution must scale to handle millions of books, hundreds of thousands of authors, as well as defend against sets of active and inventive scammers trying to snatch an unfair share of the money available.
Taken together, this also means that until Amazon develop a solution to the problem, the company appears disinterested and unsupportive, damaging their brand even as the quality of their content declines.
So here’s my set of suggestions for Amazon (the first two of these I owe to a friend, Geoff Dash).
Follow the money. The problem of the scammers is proportional to the amount of money they’re taking. So audit the highest-earning authors in order of the revenue they earn. Examine their books to ensure they’re meeting Amazon’s Terms of Service rather than gaming the algorithms. Add legitimate authors to a register of Verified authors (and even mark their books as such). Add scammers to a list of banned authors and remove them from Amazon.
Suspect the absurdly productive. Amazon could reduce the number of authors they need to Verify by examining them in order of productivity: the authors producing the most books per year (or, per month).
How to Verify. It doesn’t have to be done manually, although the above two principles make that feasible. Instead, deep learning algorithms could be trained with examples of scam content and genuine content. Amazon could use the Mechanical Turk to help examine the books selected for audit. It could build a set of trusted Verifiers. It could use some of its own people to examine the books and explain to the algorithm developers what patterns and qualities to look for.
Only Count Content Once. Use its internal plagiarism detecting algorithms to identify content so that when a reader ‘reads’ a book, that content is only counted once. I imagine something like MD5 markers for a book, stored in a reader’s account. That information could even improve the user experience, by offering the reader a chance to skip the previously-read content.
Be more Transparent. It’s fine to have algorithms making decisions for you, and you need to keep their workings secret to make things harder for the scammers. But have a good and fair system for resolving disputes. Amazon needs to do this because its record of hurting good guys with bad algorithms has damaged its reputation in this respect.
Algorithmically select for quality. By this I mean, engineer algorithms that over time will encourage high quality work by offering reasonable revenue to the author. At the least, stop using algorithms that encourage low quality. The current system that only seems to recommend books published in the last 30 days encourages ‘publish fast’ over ‘publish high quality’. Blocking reviews from professional bloggers and reviewers who are gifted with so many books they don’t spend over $50, is another algorithm that does not select for quality.

What Will I Do?

At present, I see no value in being in Kindle Unlimited, and little value in selling your books through Amazon at all. Being exclusive with Amazon is worthless after the first three months. I recommend anyone who has currently chosen exclusivity with Amazon, to terminate that exclusivity. You may need to wait three months for your current exclusivity period to end, but when your sales are zero there’s no difference in revenue between 35% or 70%. Start publishing your books elsewhere, where you won’t be drowned out by ads and scammers.
If enough authors take this step, it will also send a signal to Amazon. Hopefully, such a market-savvy company will recognise and react to that. Certainly, from reports I’ve read, Amazon doesn’t appear to react to any other signals.
I'm quoting from the Verge’s ‘Cockygate’ article, in which Zoe York (romance author) notes:
“All of these people, without exception, all of them, publish exclusively on Amazon,” she said of the book-stuffers. “This is not a problem on iBooks, on Kobo, on Barnes and Noble, or on Google Play, which are the other major ebook retailers.”


Amazon did a wonderful thing in basically creating and nurturing the ebook marketplace, and through that, stimulating the self-published marketplace. It would be sad if it misstepped now and its know-how and support became irrelevant.
Amazon now rewards good marketers (honest ones, but also the scammers), not good authors. That needs to be fixed, or it will likely lead to the collapse of Amazon as a reputable self-publisher.

Sunday, 27 May 2018

Amazon book tweaks — Categories, Keywords, reviews, and links

Following some discussions with Barbara Strickland (Amorina Rose's blog) about Amazon Categories and Keywords, and remembering some of what Lama Jabr explained in one Amazon publishing seminar, and after reading this article Six Things You Should Be Including in Your EBook (and Probably Aren't) by David Kudler, I decided to revise all three of my books for Amazon.

David Kudler Thanks

What I wanted to do:

  • Make sure I had links to each book in the series so far.
  • Follow the suggestion to add a review link right at the end of the book.
  • Make sure my categories and keywords were good.

It took me about a day to update the first book, four hours for the second, and an hour for the third. Hopefully this article will help you tweak your first book in less time than I took for mine.

The tweaking is in two parts: tweaking the copy of your MS (probably in .docx or .odt format), and editing your ‘book details' on Amazon.

Links

It also occurred to me (largely because Barbara Strickland had recently commented on how attractive the covers were), that a thumbnail of each cover would make a more attractive link than just the plain text title. Like so:

Novels by L. J. Kendall

The Leeth Dossier:

Wild Thing

Harsh Lessons

Shadow Hunt

Violent Causes


(By the third book, I even realised I should include a link to my Amazon author page!)

While doing this, I found Calibre doesn't preserve hyperlinks on images when you convert to .mobi format, which is what I had been using for upload to Amazon. But the developer, Kovid Goyal, said that format was pretty obsolete, and recommended a different one. I tried epub, and that was just fine.

I wanted to put country-independent links to my other books, in each book. Links to the appropriate Amazon store for the reader's country.  I didn't want to do it through Amazon's OneLink system:
https://affiliate-program.amazon.com/help/node/topic/202165020?ac-ms-src=faqonelinklp&cid=faqonelinklp

merely because I myself am leery of systems that earn money from people clicking on links.  Granted, I think it's only a few cents, and only if they actually buy the product, but still...  Maybe that's just me being weird, but maybe other people have a distaste for that, too.

Image composition on Linux

I spent some time grabbing images for those buttons to make my own.



I then spent 90 mins finding image editing software that would let me combine them together, for Linux. Lots were recommended, but apparently the idea of a canvas on which you simply drop some images and move them around is a bizarre mental model. Well, apart from the Gimp, but I thought if I could find any other program to do that, I'd probably save time compared to working out how to perform even that simple feat using the Gimp's diabolical UI.

In the end, I found the KDE program KolourPaint, which was very simple and intuitive — it was easy to open up images, and copy them onto a single canvas and then move them around.

Reviews

Before I did any of this, I first contacted Amazon directly to ask if it was okay to use generalised links that pointed to the reader's local Amazon store, instead of to the US one only (say). I specifically asked if it was okay to use bookshow.me. The representative responded and said that was fine.

E.g. the country-independent links for my books in Amazon look like this:

Wild Thing: http://bookshow.me/B019DQVR76

Harsh Lessons: http://bookShow.me/B01HA4FALC

Shadow Hunt: http://bookShow.me/B06XVH355Q

Previously my Afterword asked people to review the book, but didn't offer them any help to do so. In contrast, the Six Things article described how to construct a link directly to a "rate/review" page for your book so the reader just needs one click, with no typing or searching to arrive there.

I also added a note that reviewers will need to login (to Amazon or Goodreads respectively), to be able to write a review.

So the idea is to add a note at the end, saying something like:

"I hope you enjoyed reading this. If you did, consider rating and/or reviewing it. Thank you!"

(You'll need to sign in to your account(s).)

The Six Things article used an image to click on for the reviews, which I think is more appealing and ‘button-like' to click on than just text. So I grabbed some images from Goodreads and Amazon to make the buttons. I feel that's safe, as Amazon owns Goodreads, and I'm only directing readers to Amazon sites. If you're not selling through Amazon, of course you'd need to pick review sites related to your retailer.

Goodreads review

But the link structures for reviews described by the 2013 Six Things article have changed since it was written. E.g. for Goodreads they'd said:

… um, I can't find it, actually. Maybe David didn't spell it out? He does describe how to find the book ID, though. (You can find it out just by clicking on a Goodreads review link and inspecting the URL.)

Anyway, the review link needs to look like:
https://www.goodreads.com/review/edit/<goodreads-Book ID number>
Just search for your book in Goodreads to determine your book's ID and examine the link. E.g. for Shadow Hunt the book link is https://www.goodreads.com/book/show/34823396-shadow-hunt, so the book ID is 34823396, and the review link becomes https://www.goodreads.com/review/edit/34823396

Amazon review

For Amazon, you determine your Amazon book ID (ASIN) the same way: find your book and examine the URL.

The example book link http://www.amazon.com/Your-Book-Title-ebook/dp/B00EXAMPLE/ is pretty correct, except the http has changed to the more secure https. E.g. Shadow Hunt is https://www.amazon.com/Shadow-Hunt-Leeth-Dossier-Book-ebook/dp/B06XVH355Q/

But the link the article describes for reviews, "http://www.amazon.com/Your-Title-ebook/product-reviews/", is now quite different, and the Amazon review link is now coded like this:
https://www.amazon.com/review/create-review/ref=cm_cr_dp_d_wr_but_top?ie=UTF8&channel=glance-detail&asin=B00EXAMPLE

E.g. since Shadow Hunt's Amazon ASIN is B06XVH355Q, the URL for its review is: https://www.amazon.com/review/create-review/ref=cm_cr_dp_d_wr_but_top?ie=UTF8&channel=glance-detail&asin=B06XVH355Q

I also thought it made sense to repeat the links to all the books in the series at the end, since at that point the reader is more likely to know if they'd like to read more of your work.

Amazon Categories and Keywords

The categories & keywords topic is where things get interesting...

Amazon have a good help page about setting them up:
https://kdp.amazon.com/en_US/help/topic/G200652170

It turns out that everything on that page is essential reading. I had ignored it.  So my keywords were not doing what they should.  E.g. I didn't know you had to also apply one (or two) of your categories as keywords, for certain Amazon searches (category searches?) to work for your books.
Watching the video that's also on that page suddenly made a whole lot of things really, really clear.  E.g. the stuff at the 3min40s mark where she says something like "Words in the title, categories, and description, will already be part of the search criteria: try to say something new".  Also, she says you should play around with the keywords and see which work best — I've never done that!

It's worth watching — maybe several times.

Setting Amazon categories

From https://kdp.amazon.com/en_US/help/topic/G200652170:

"Browse categories are the sections of the Amazon site where customers can find your book. Think of browse categories like the sections of a physical bookstore (fiction, history, and so on).

"During title setup, you'll select a BISAC (Book Industry Standards and Communications) code. The codes you choose, along with your selected keywords, are used to place your book into certain categories, or browse paths, on Amazon. After your book has an Amazon listing, your book's category will appear under the Product Details section of your listing. This will be the path a customer can follow to find your book. [...]

"You can choose up to two BISAC codes for your book. We may also assign additional browse categories depending on your BISAC selections and assigned keywords."

The article also includes tips on choosing categories, and you should definitely read it.

Setting Amazon Keywords

Also note the topic "Categories with Keyword Requirements." For sci-fi and fantasy that links to this page: https://kdp.amazon.com/en_US/help/topic/G201216150


Now, for each of my books I'd just made up keywords that I felt best reflected it, and entered those. I suspect I'm not following best practice even now, but at least I'm choosing keywords from the above list. I assume these are words which are most used by readers in Amazon searches.

You might be able to discover them by typing a search term and seeing what words auto-complete in the Kindle store?

Here's my before and after list of keywords for each book:


Keywords — Wild Thing, before
Science fiction Urban fantasy
Strong female lead Magic
Near future Abusive relationship
Adult situations

Keywords — Wild Thing, after
cyberpunk Urban
Strong female lead paranormal
horror Coming of age
Adult situations

Keywords — Harsh Lessons, before
strong female lead some adult situations
some humor some violence
assassin covert agency

I also changed the category from Fiction > Science Fiction > Action & Adventure, to Fiction > Science Fiction > Cyberpunk

Keywords — Harsh Lessons, after
cyberpunk urban, fantasy
strong female lead paranormal
spy, spies, assassin, covert, agency bully, bullies, bullying
thriller

(I also made the same category change for Shadow Hunt.)

Keywords — Shadow Hunt, before
thriller cyberpunk
mutants wizard
magic coming of age
vampire

Keywords — Shadow Hunt, after
thriller cyberpunk
strong female lead magic, wizard, paranormal
spy, spies, assassin, covert, agency coming of age
vampire

So, that's it: a bit fiddly, bit not too bad. I don't expect dramatic changes, but hopefully the rate of reviews will climb from nothing (for almost a year) to a handful of new reviews!

To Do

What I've failed to do (yet) is add the attractive-clicky-button links to make social media sharing of your book super easy. I think that might even be the most important thing to do!

The Six Things article describes how to set up sharing links for Facebook, Twitter, and Pinterest.

Saturday, 29 April 2017

Stuffing things up

Well, this is a pretty embarrassing blog article to be writing, but it's an important public service announcement. I stuffed up.

Here's the executive summary of how it affects people who want to read Shadow Hunt (vol 3 in The Leeth Dossier): please wait an extra day before you start reading it. If you pre-ordered it, please wait for an email from Amazon telling you that a new version is available.

Here's why….

I made Shadow Hunt available for pre-order on Amazon (April 30th, 2017), uploading the draft I had a month earlier (March 26th), while I was working through my editor's 2nd and final critique. (Or 3rd, or 4th, if you include the two earlier book splits!) I thought five weeks would give me a comfortable period — comfortable enough to even allow me to attend the sci-fi/fantasy convention SwanCon, in Perth, and spend a couple of days extra seeing the sights.

Which would have been all well and good if I hadn't made a couple of other errors. The first was not checking the details of when I needed to upload the final version of the book: I assumed the day before. In hindsight this was remarkably stupid, because naturally Amazon needs extra time to check the book carefully before launch. (In fact, they need 72 hours.) My second big mistake was a really stupid failure of basic arithmetic: 30 minus 3 = 27, not 28, right? Yeah, well my brain did the sum this way instead "Let's see: 30, 29, 28: that's three days. So, April 28 is the deadline."

Don't ask. Even I don't see how I could have made that error. Partly it was wishful thinking that the time counted from the end of April 30th, not the start of the day! D'oh!

Maybe by this time I was getting tired, too, because my editing of vol 3 was taking longer than I had expected. Anyway, I completed all my edits and did my 1st polish of the new parts, and then the 2nd polish (a bit rushed, I admit), and sat down one hour before the deadline (I thought) to convert the book to Amazon's format and upload it. I finished that with ten minutes to spare (again: I thought) and then tried to upload. Bzzt: "locked for pre-order release checks."

Uh oh.

It was only then that I realised my stuff-up. I contacted Amazon, but there was nothing they could do: my choices were either to cancel the pre-order at the last minute (and thus prove I couldn't be trusted with the ability to manage a pre-order for a period of 12 months), or upload the revised version with the usual detailed description of changes, and request Amazon to push out an offer to all existing purchasers that a new version is available.

So I chose that 2nd option. This means that the draft copy will only be there for a short period after the pre-order date (a day, I hope), and that people who pre-ordered will soon get the updated version.

The draft version that ‘escaped' can be distinguished from the final version in many ways:

  • At the end of the table of contents, the release version is noted as "pre-4" — not "1".
  • The Prologue is inside Part I — instead of coming just before it.
  • The book is 141,854 words long — not 149,450. (Yes, this is despite cutting about 10k words, too.)
  • The draft version has 79 chapters (+epilogue) — not 82 (+epilogue).
  • Something like 2,000 other changes, big and small. I won't enumerate them here, but one small example in the prologue is the changing of the sentence that begins "Yet thanks to it, no one would expect to find human remains" to "Yet thanks to it, with no bodies to be found".
  • I'm not going to mention any of the bigger changes, because they'd be spoilers, but for the 30 days or so following my upload of my late draft version to Amazon, I think I averaged about 9-10 hours per day working on improving the book. The changes were based on Dave's critique and some extra Q&A with him, an early beta read by a friend as those changes neared completion (thanks, Jon!), and feedback from two beta readers (Sandra, AndyK) to try to resolve a key point. So parts of the plot are strengthened, there are fewer errors and questionable parts (it all makes more sense), and is just generally vastly improved, thanks to all the feedback.

    At this point I should also add that on one point, Dave and I couldn't agree (but neither of us were sure any more who was right). I also couldn't reach consensus by the beta readers. I vacillated for a while, but in the end, trusted my gut. So don't blame Dave if there's some thing (or things) about the book which you feel could have been improved: you've no doubt found a place where I didn't follow his advice!

    Tuesday, 18 April 2017

    Publishing: Where to Begin

    For my first large sci-fi/fantasy convention, I travelled across the country to Perth — another first, for me. SwanCon is Australia's longest running speculative fiction convention. In this, its 42nd year, the background theme was honouring the work of Douglas Adams. They had wonderful guests of honour (Michael Troughton, Joyce Chng, Traci Harding, Sean Williams, Alan Baxter, Davina Watson, and Wesley Chu via Skype). The SwanCon committee had organised a huge range of panels and activities — generally about four talks on at the same time across each day from 10am to 11pm or later; along with four or more activities running in parallel — from board games, to live action role-playing, to console gaming to children's and family activities.

    (Michael Troughton and Sean Williams holding up the SwanCon costume party sign)

    SwanCon 42 was held at the Metro Hotel, Perth, which did a heroic job to support the convention (and feed a large crowd several times a day). The hotel strained at the seams, but in my view can be proud of the job they did.

    I was also impressed by the dedication of the organising committee, and their ability to fix things and cope when things went unexpectedly wrong.

    This is just a short piece to record and share some notes I made as homework for one of the two planned panels I was on. It's a companion piece to a related panel focussed on writing your story (and hence, is over on All About Leeth). (I just intended to type up my notes, but thought they'd be a bit too cryptic if I had done literally just that.)

    The topic of the panel discussion (organised by Michael Cogan, I think), was "Publishing: Where to Begin".

    "Do you have a really awesome story and need a way to get it out to the masses? Come and hear from some who have been there and done it before."

    My most excellent fellow panellists were Amanda Bridgeman, Satima Flavell, and Glenda Larke.

    I think we collectively provided good information, and there were excellent questions and comments from the very engaged audience. Please understand this is not a record of what we all said, but merely some notes I made beforehand as a memory jogger. Some of these points were made by other panellists independently, in their own words. Because we had only an hour and there was a lot of ground to cover, only some of my notes were covered in the talk.

    The following applies after you've written your story.

    Before publishing, you need to have put in the effort to make it the best you can. This is critical for your first published work, since the biggest problem for new authors is being discovered and tried by readers. If you make a bad first impression, there are so many new stories coming out all the time that readers may not come back and try you again.

    So it pays to invest in your first story. By all means get friends or family to read your story. Note though that you're putting them in a difficult position, especially if you take criticism personally: criticism of what you've written is not a criticism of you. And if your story is not the sort of story your reader would normally read, don't ask expect anything other than finding typos and bad grammar.

    You're much more likely to get insightful criticism from other writers, and from avid readers in your genre. Having portions reviewed by other writers will be especially valuable to you, as will feedback from beta readers.

    Neil Gaiman said "When more than one person tells you something you've written is wrong or doesn't work for them, they are almost always right. When they tell you exactly what they think is wrong and how to fix it, they are almost always wrong. You are the creator: only you can know how to fix it."

    Most valuable of all will be a paid-for critique by a good professional editor who understands the story you are trying to write. And if you only spend money on one thing to help publish and sell your book, spend it on that, even if you have to save up for it.

    You learn writing by doing. When you think your work is good enough, invest some money in getting a good professional editor to critique it. It's an investment in yourself. And unlike other such professional development courses, the "practical exercise" from this "training course" directly contributes to making your story the best it can be.

    But you need to do the homework to get to a stage where the editor will be helping you make it the best it can be, rather than spending your money having them teach you how to write! (A good editor would not agree to take on work that's not up to that standard, IMO.)

    And you need to find an editor who ‘gets' your work and is willing to explain why they've advised what they have, and will listen to you when you explain why you did what you did. You need to be able to discuss and things when your opinion differs. Usually, then, you'll reach some agreement — often, some blending of both points of view, or some other wording or approach or scene that you can both agree on. I highly recommend ThEditors.com. They even offer a free assessment of the 1st 3k words of your work.

    Regarding editing, I've blogged about it here. And this article I stumbled over looking for that Neil Gaiman quote is pithy and wise.

    So, let's go:

    Traditional Publishing (including Indie-publishing) and self-publishing have a lot in common. If you are being published by a company, you'll also need to write a "pitch letter" and/or an outline and/or a synopsis. These are all different kinds of writing, and require time and practice to learn how to do well. They each serve a different purpose, related to making the publisher's (or agent's) job easier. Those people have hard jobs, so these communications have evolved to tell them exactly what they need to know, as clearly and succinctly as possible. There are books on the topic and web-sites, including examples of Things To Do, and Things Which Will Instantly Get Your Story Rejected If You Do Them. A key thing to know is that the people who will be reading your words have very little time and are under a lot of pressure. Try to put yourself in their shoes, and think about things from their point of view, so you can tell them what they need to know.

    But the other panellists (and some audience members — Alan, Sean, Bec…) know much more about that than me, since all my attempts to follow that path went nowhere!

    For self-publishing, you don't need to write those things, though you will need to write an excellent blurb, and ideally a tag-line for your book (the snappy/intriguing teaser line that may appear on the cover), and also "the elevator pitch".

    Be aware that these things, though very short, are very difficult to do and you're likely to go through hundreds of iterations before you get each of them "right". It's because they're like poetry in a way: you're trying to pack the maximum impact into few words. You have to distil everything down to its essence. However, for these far shorter pieces of writing, you can pester anyone you know for their opinion!

    The text of your book must be proofread before publication. This is the easiest and most straightforward kind of editing there is. Unfortunately, it's also hard to do yourself since your mind has stored the words you intended to write, and you'll tend to read those rather than the words you actually wrote. This includes punctuation, too. But few errors will throw a reader out of a story like typographical or grammatical errors. (Unless it's a glaring continuity or plausibility error: they're probably worse!)

    Book cover. Unless you're a graphic designer, preferably with some background in book cover design for the genre of your story, pay for a book cover design. You can get super-cheap cover designs via web sites like fiverr, or others that will have people bid for your job. There are sites that offer pre-made book covers across wide ranges of genres, requiring just the text to be supplied (Title, author, etc.), and who will sell you exclusive use of a design for a relatively low cost ($100 — $200). You can also look for good book cover designs in your Twitter feed, and contact the author to ask for their cover designer's details. (That's how I found my cover designer, the wonderful Mirella de Santana)

    An audience member asked if we each would recommend trying to go via the Traditional or Indie route for their first book, or just self-publish? The other panellists made excellent points, and observed very correctly that you'll get a lot of support if you're accepted for publication (especially with a small, independent publisher): you'll get an advance, they'll provide the editor, whose services you won't have to pay for, they'll write or help you write the blurb, they'll design the cover, and best of all they'll handle the marketing for the book's launch period. They all, I think, said "On balance, yes, it's probably a good idea."

    I however think the decision is not so clear-cut. If you self-publish, you are indeed taking on all that extra work: that of a whole publishing company — and you're probably unskilled in most of it! Admittedly, all that work is just for your story alone.

    But if what you're writing is the first volume in a series, the balance shifts so that it's probably better in the long run for you to self-publish, in my opinion. That's because the first book in a series is a really powerful tool for marketing. Having complete control over the cover, the blurb, how and where it's sold, and most especially, the ebook price (so you can greatly reduce it, or even make it free for short periods), is enormously helpful in getting your work found by readers, and generating follow-on reviews (typically, only something like 1% of readers will also review your book).

    Another reason to opt for self-publishing is my belief that we're currently only about halfway through a tectonic shift in the book publishing environment, which in future will be more like self-publishing. With self-publishing, the authors and the readers are in pretty direct contact, via social media, blogs, review sites, Facebook groups, and mailing lists. This interaction will be mediated by giants like Amazon (who are in business to make it easier for authors to sell and publish their books, and readers to discover and buy books they'll enjoy) and other ebook publishers and distributors, as well as the social media companies and purpose-built web sites that help solve the "discovery" problem.

    Even traditionally-published authors need a social media presence these days, since companies can only afford to market their work for roughly the "two week launch period", unless you're one of the super-star authors.

    Of course, you should produce an ebook version of your story! It will be your best marketing tool, and probably also generate you the most income. And producing and publishing it will cost you nothing except a day or so of time (including time spent learning). If you choose to distribute exclusively with Amazon, you will earn 70% royalties provided you stay within their recommended (and sensible) price band. If you choose Amazon exclusively, also sign up for Kindle Unlimited and allow borrowing. I earn more from my share of Amazon's subscription-model payments than I do from individual ebook sales. (I earn least of all from my print book sales.)

    Produce a print edition, too. If you produce a PDF file, that will be exactly what you'll get when you print. Though for colour parts (the cover and back), you'll need to be aware of CMYK colour and how to produce a PDF/X file with the right profile. I've blogged about that in too much detail here.

    Decide how you'll publish. If you're accepted by an Indie or large publisher, they'll tell you. Otherwise, you get to choose.

    If you're self-publishing, I think it's wise to buy some ISBNs. In Australia, you buy ISBNs from Thorpe-Bowker. One ISBN costs $125; if you buy ten, that costs $250. One hundred costs $575. Book distributors' databases tend to see contiguous ISBN numbers as belonging to a specific publisher (in this case: you).

    It costs nothing (or next to nothing) to apply for an ABN (Australian Business Number), too. You just fill out some forms on a government web site.

    Smashwords will make your ebook available to Amazon, Barnes and Noble, iBooks, and I think all or most of the major players. This will save you the effort of producing an ebook in each of the formats they all require. (Though with the exception of Amazon, my guess is that they'd all accept epub format, or else Word's .docx format. Amazon prefers Kindle/.mobi format.)

    Producing the ebook is very easy if you do it the right way. There's some wonderful free software called Calibre for managing your ebooks, including producing your own. It's amazing. The first time you use it, it'll probably take you some hours to learn it and set up the couple of options you'll need to alter.

    Each time you add a new book you've written to Calibre, it'll take you about an hour to enter in all the book's metadata (Title, Author, series info, ISBN, keywords, blurb, cover). You produce the ebook by choosing "convert to ebook" and selecting the appropriate input — typically Word format — and output: e.g. .mobi. Click Cobnvert: it will take about a minute. Replacing the book with an updated/improved version will take a few seconds; generating the new ebook version will take a minute, again. It's really that easy, thanks to all the hard work by Calibre's creator, Kovid Goyal, who has also produced heaps of good tutorials for using it (also free). I've blogged about that, here.

    Then, basically, you'll upload that output ebook file, and a separate cover file, to your publisher's web site.

    If you've chosen Amazon, or are using Amazon at all, you should register with them as an author and create your Amazon Central Author Page. Provide bio details and so on in the online forms they lead you through. You need to "Add" and link your books explicitly, manually, to your Author Page.

    For each edition of each book that you add and link to your Author Page, you inform Amazon of its ISBN. If you don't have one, they'll provide you with one. Though if you choose to publish them elsewhere, you'll need to use a fresh ISBN, not Amazon's one.

    Oh, and Amazon also make it super-easy to register that you are overseas and only 5% tax should be subtracted and handed to the US government (as long as you live in a country like Australia, that has a reciprocal tax arrangement with the US). You just fill out a form, and provide your Australian taxation number, and which bank account they should pay the royalties into.

    You can choose to create a print edition via Amazon, through their CreateSpace subsidiary.

    For my print editions, I chose a larger publisher/printer than Amazon, that also uses Print On Demand, called IngramSpark. I think it costs about $50 to sign up with them (once-off), and then it costs US$50 each time you upload a new edition/version of your book. ($25 for the cover, $25 for the contents.) So if you can get it perfect 1st time, and don't need to upload a corrected edition, you'll minimise your cost there. There's also a one-off cost ($80, IIRC) per book that you can opt-in to, to "advertise" your book. I think this just pushes your book's metadata out to all the book distributors' databases, which therefore makes your printed book available across about 29,000 stores, worldwide. Not bad, eh?

    One trick/note: don't tick the box that says book stores can return unsold copies. The downside is that almost no book store will stock your book on its shelves, for fear of not selling them. The up-side is that you won't lose potentially large sums of money paying for the shipping back of unsold copies if a store over-estimates how many they'll sell. They'll still order your book in, though, if asked. And of course, online booksellers in each country will happily order and ship your books to readers. You will need to set your discount to around 40% so the book store can make a profit. IS provide tables so you can work out costs and thus set your price so you make at least a small profit on each copy.

    I chose IS to reduce delivery costs, since the books will be printed in or near the country in which they're bought. This is in contrast to Amazon, which prints only in the US (and maybe UK)? Also, IS supports a much wider range of print edition sizes and bindings.

    You should inform the National Library of Australia of each edition of each of your books. It is I think worthwhile to also apply form and get a Cataloguing in Publication (CIP) entry for your book. And remember to send off a print copy (and the ebook) to the NLA after publication.

    I think it's also worthwhile joining the Australian Society of Authors (ASA). They've recently upgraded their infrastructure. (Remember, Luke, to re-upload all your author details into the new system.) And the Writers' Association for your state, too.

    Social media. The big problem for new authors is being discovered. You need to use some social media to make your presence known. Be authentic, is my advice. Be yourself, and offer useful and interesting content. Don't just repeat, in various forms, "Check out my book". But don't let social media dominate your time or energy. Limit yourself to at most an hour a day, is my own rule of thumb.

    Use free give-aways to attract new readers to try your work. The first book in a series is ideal for this. Never try to trick/cheat/deceive/coerce a reader. How would you feel if that were done to you? Good reviews will hugely increase your visibility, so write the best book you can. And make sure the book description sets the reader's expectations correctly: most reviewers who don't enjoy a story will give it a low rating even if they think it was really well-written. You want the people who will enjoy your book to be the ones reading it.

    The best thing you can do to increase your sales is to write more good books.

    Go to it!

    Monday, 6 February 2017

    Book 3 – Shadow Hunt – is almost ready

    Still alive!

    I've been silent so long (October last year!), because I've been working very hard on book three, so I thought I'd make the time to offer a short update.

    When will it be published?

    I wrote in the Afterword of Book 2 that I was confident I'd have Book 3 ready early in 2017. I've been trying very hard to publish the ebook by mid-March. Right now I'm getting some much-delayed other book-related work done while I wait to receive the detailed critique of the 2nd half of Shadow Hunt from Dave at TheEditors.com — I should get that today or tomorrow. If I can manage to average 15pp/day then I'll have the updated draft ready about Feb 28. I'll then spend a week to 10 days checking and re-checking that, then upload it to Amazon. If I can manage 20pp/day, then it will be ready earlier, obviously.

    I'm doing my best, and working very hard. I finished writing and polishing the draft and sent it to Dave at the end of 2016 (in two parts: 1st half on Nov 28th, and the 2nd half on Dec 3rd).

    Shadow Hunt cover

    Mirella, too, has been hard at work on the cover design, working from a brief I prepared last year. This image, below, is not final, and maybe I shouldn't include it... but I just can't resist giving you a sneak peek, even though it's not quite finished. (It will change just a little, but I love it already!)

    (You can see the final cover over on my other blog.)

    Some nitty-gritty

    Dave's critique has been invaluable, as ever. We spent some time discussing his key recommendations at a high level, and I sketched out a plan, and then sat down and worked my way through chapter by chapter. I also received very helpful feedback from Jon Marshall (again!) as a beta reader of the same MS, and he found several important problems and oversights (fortunately, none hard to correct).

    Incidentally, the reason for breaking the critique in two was purely a time-saving method: I could get started on the first half while Dave continued on the second half. This was only possible because he'd sort of already critiqued the whole thing twice before, because the MS has been split in two twice from Dave's feedback (that's not counting once, before, based on my own assessment). A side benefit of splitting it in two is also purely mechanical: the powerful, free word processor I use, LibreOffice, becomes unusable when the number of comments in a file exceeds about 1,500. (It must have what's called an order N2 algorithm in its handling of comments, which is a fancy way of saying the time it takes to do anything rises as sharply as the up-sweeping arm of a parabola!) As it stands, before I start deleting comments, LibreOffice takes about 15 — 20 seconds just to respond to a menu click. I've added to the problem report (bug 60418), and it's on their list of things to consider. But it's not considered a big problem.

    ISBNs & CiP

    Today I logged in to Thorpe-Bowker to assign ISBNs to the ebook and the print edition. Armed with the correct ISBNs, I then visited the National Library of Australia website to request a Cataloguing in Publication entry for three editions of Shadow Hunt (the ebook, the 5"x8" edition, and the 4"x7"). But again, all three 4"x7" editions are on hold until I can find a workaround for another bug in LibreOffice, that messes up the page headers. (Bug 103078)

    About Vol. 3

    Shadow Hunt is currently a little longer than the 1st, Wild Thing, though Dave is recommending I cut 30 pages from one 100-page stretch. If I manage to do that, it's shaping up to be the same length as Wild Thing. I'm very happy with how the book turned out (is turning out?).

    April plans

    I'm also planning to attend the SwanCon this year (I've never visited Perth)! It's billed as "Australia’s longest running Sci-Fi, Fantasy and Speculative Fiction Convention".

    My next blog posts here will probably cover these topics:

    • Break up that indigestible previous blog that covered too much in too much detail
    • a quick intro to Scribus for preparing your book covers, given a brilliant cover design as your starting point
    • setting up your Youtube author channel (mine is here)
    • a rant about the new publishing world.

    Over on my other blog, All About Leeth, I plan to post a Q&A from a fan (yes — I have a fan!) about the series, and especially about Harmon and Leeth's relationship.

    I think that covers thing. Now to press on with some other work — like finalising the blurb, and more polishing of the new parts I've written over the last few weeks. I'm doing my best!

    Tuesday, 12 January 2016

    Adventure #1 in publishing: Creating the ebook!

    Ebook conversion.

    Hoo, boy! This should be the easy step, but I think this is where most of my effort went, in the ten days before my publication deadline. You need to convert your book to the ebook format required for the publishing platform you've chosen. This should be straightforward and painless, but I'm told it's anything but. My early experiments bore that out: but read on, I found a good and simple approach using Calibre.

    Lest this sound a bit scary, I should cut to the chase and say that once you've cleaned up your source document, and entered the metadata into Calibre for your book (by typing in the blurb, ISBN, title, author, etc.), the conversion to epub or Amazon's .mobi using Calibre requires only a few minutes of effort, and produces a good and reliable result.

    What you want is the ebook format to be as nicely laid out and as well-formatted as what you have in your word processor. But for a various reasons, the conversion can have major problems.

    I was using LibreOffice 5.0.3 for my word processor, incidentally, at the time I wrote this blog post. Most people probably use Microsoft Word. A few people (especially if they're producing something like a photo book or a children's illustrated story) might use InDesign. I think InDesign is poorly suited to producing a reflowable ebook: InDesign is, after all, designed to do the opposite, to place each piece of text and artwork exactly where you put it, and not to be movable or resizeable. (That said, it would be great for non-reflowable ebooks, like children's picture books or coffee-table books with lots of images.)

    In contrast, one of the biggest strengths of an ebook is that the reader can enlarge or reduce the font (or possibly change the font completely), to suit their eyes and/or the lighting conditions. So this means that you have to be careful about preparing your MS so this reader-directed reformatting can work well: you need to use true page breaks when you want a new page, and an indent or centring style when you want to indent or centre something. An awful mistake is to use spaces/tabs and blank lines to manually position some text so it looks right for you in your MS. Likewise, it's a bad mistake to make a piece of text look like a chapter title but not mark it as such. In short, you should be using paragraph styles exclusively. Doing so also means that if you decide you need to make a change to how you've laid out every instance of something, you need only change the paragraph style, not each paragraph!

    Here are some problems I've encountered, using LibreOffice:

    The HTML it produces is far more complex than it needs to be. I have basically two kinds of paragraphs (body text and chapter titles), and a few pieces of text that must be centred. I use italics for emphasis. But LO (currently) splits the runs of text into shorter spans of differently-named but identical styles (usually, showing where I went in and edited the text), and for my MS produced literally thousands of paragraph styles.

    Worst problem for me was when I copied the whole MS into the new template I wanted to use: after resetting all the paragraphs to the correct body text style, and setting all the chapter titles to the chapter title style, it lost (randomly as far as I could see) about half the places where I'd applied the italic style. I delved into the XML and after a days work of writing some programs, was able to recover about 90% of the italics. The rest I just had to notice and re-do manually during one of my many read-throughs.

    Conversion to Microsoft docx and back seemed to help somewhat: but then other aspects of the page formatting got messed up (left and right margins, footers, etc.). So that was a dead-end.

    The table of contents generated for the ebook format was just bizarre. You don't want page numbers matched to chapters in the TOC, since page numbers basically don't exist in the ebook format (because the ereaders don't want to calculate this up-front and store and display the information: the technology may change in future). Anyway, not only did my chapters have page numbers, but after conversion, each chapter in the TOC occupied a separate page. Ouch!

    Loading up the LibreOffice .odt file into Calibre for conversion also produced poorish results when I then used Calibre (v2.45) to convert it to epub and Kindle (.mobi) format. Calibre seemed to work better when I saved as a .docx and then loaded that up. But the formatting wasn't what I wanted, so more investigation was needed. I made this note to myself when I started this step:

    "I have some notes on a page at Goodreads about tips for conversion to ebook formats, and a few web pages to read, as well as my own experiments to perform. Worst case, I may have to unzip the epub format and have a look at the raw HTML, and simplify it down to the bare minimum. This is not meant to be rocket science, after all."
    (Which is what happened, incidentally.)

    Oh, I also had a plug-in for LO that converted straight to ebook formats: but that seemed to have its own problems, so I uninstalled it.

    Here's a very good explanation of self-publishing on Kindle.

    Now, at this point I could have delved into the HTML side of things, since Kindle uses a subset of HTML as one of the possible input file formats for producing a book. (The others are PDF and mobi.) If you want to delve in and get your hands dirty, and take control of lots of aspects of the layout, then learning about the HTML may be the best approach. I didn't do that in the end, but rather than ignore all the tips I've gathered, I'll stick them in an "appendix" after this main article ("Kindle's HTML").###Fix this link

    The approach I took in the end was much simpler, and I believe generally more useful, and should only continue to improve with time, so I'll focus mainly on that.

    My approach was to rely on Calibre, since one of Calibre's important main functions is to perform ebook conversions.

    I thought the best starting point was probably to see what it produced, and to look into any problems I found. In that way, if I found something that looked like a genuine problem in Calibre, I could inform its creator, Kovid Goyal. Or, it might point to problems in what I'd done.

    Importing my .odt file and converting to epub was painless, as was reading the resulting epub version within Calibre – but I noticed quite a few issues. Here's a screen shot of Calibre in action, with Wild Thing imported into it:

    Calibre is powerful, flexible, and has a well thought out user interface that makes it pleasant to use. But the flexibility (necessary to deal with all the different tasks it can perform), meant that I found myself at a bit of a loss as to work out what to do. Calibre also comes with lots of good documentation, including video demos.

    Another thing I found was that Calibre itself has what looks like an excellent editor built in, with a well-written user manual available for free here. (I did what I normally do: "printed" the web page to a PDF file, then opened it in Adobe's "acroread" program, selected "booklet" printing, and sent that to the printer, stapling the result with my long-arm stapler to make a convenient A5 reference booklet for easy reference.) Obviously, editing the MS in Calibre is a last resort: if I make changes to the original MS and re-convert in Calibre, I'd need to make any edit in Calibre again – or else make the Calibre version the main source file for the book, which wouldn't be a good idea since then I'd have different "source file" for the print and the epub editions.

    The trouble for me was that my deadline was now so close I simply didn't have the time to study and learn Calibre thoroughly. I needed some shortcuts.

    Because I'm technical, I decided to poke about under the bonnet and see if I could work out what was happening. Now, most of this poking about was unnecessary and is merely a distraction: so I'm again going to put most of it into yet another "appendix" to this article ("Poking about in Calibre").###Fix this link, too

    But one aspect of it proved so useful that I'll shortly describe some of my poking about, within this article proper.

    The ".epub" file is actually a ZIP file with specific files and file formats inside it (that's good design!). One of the intentional consequences of this is that you can poke about inside. If you know what you're doing, and don't break the rules, you can even change things. (You just zip it all back up into an .epub file again.)

    So I poked about and spent a while developing some scripts to try to fix what I saw as problems, and continued delving into and learning the technical details of how the .epub format worked, and what Calibre produced. I made some progress, but then reached a point where things looked sufficiently wrong that I filled in a bug report to describe what I was experiencing.

    The three big errors I had were:

    1. After importing my LibreOffice MS into Calibre and converting to epub, the Table of Contents (of about 70 chapters), appeared as one separate line on each page: you have to page forward through 70 pages to get to the actual 1st page. Or, if you wanted to "jump" to chapter N, then you have to page forward N-1 to get to the ToC entry for Chapter N, and then click on the link.
    2. I was getting white square rectangles appearing for some of the non-breaking space characters when I uploaded the .mobi file to KDP.
    3. Lots more fonts than I expected in the output file, leading to strange changes in the paragraphs.

    The developer of Calibre, Kovid Goyal, was very helpful. And, the executive summary of the long and detailed description of my trials and tribulations are that all the problems were caused by either a wrong setting I'd chosen in Calibre, or a genuine bit of messiness in my MS.

    To fix problem (1), Kovid pointed out two simple corrections to what I was choosing for the conversion operation:

    1. He recommended converting from .odt to .docx format , and give Calibre the .docx file, since a lot more work had gone into the conversion from .docx. So I chose "Save a copy" as .docx and tried that.
    2. In the Structure Detection section of the conversion dialogue, either change the "chapter mark" to none, or change the chapter detection expression to "/". So I set the "Chapter mark" to "none", and also changed both the "Detect Chapters at" and the "Insert Page Breaks before" to just a "/" (this means, "disable this function").

    Together, those two pieces of advice fixed pretty much all the errors I'd found in the conversion process – apart from errors obviously coming from my MS itself. Here's another screen shot, showing all three (changed) settings in Calibre:

    For problem (2), it turned out to be a character I was occasionally (accidentally) somehow typing instead of a non-breaking space, in LibreOffice. The character appeared to be a non-breaking space in LO but was illegal for Kindle (specifically, it was the Unicode character U+FFF9). I just had to find and delete all those characters from my MS.

    Problem (3) – fun with fonts! When I paged through the ebook in Calibre, doing a visual check, I noticed there were some sections in the wrong font. Checking back into the MS I found that, sure enough, there was a genuine font change in the MS – with a visual difference so small that it was almost undetectable in LibreOffice. But then I realised that finding all these by visual inspection (i.e., manually) was an unnecessarily laborious way of finding and fixing them. By manual inspection, I had learned that the font errors so far had been because some of the text was still in Garamond 9pt. Because Garamond is not installed on my system and the license costs for me to acquire it and pay the annual fees were prohibitive, I had changed the body-text paragraph style to Georgia 10.5pt. Anyway, the odd slippage back into Garamond 9pt would have happened because I must have done some manual formatting, so that when I changed the font style within the paragraph style, those manually-applied formatting changes persisted.

    So I called up the Find&Replace dialogue, clicked on Format, and then on the Fonts tab. Now, since Garamond isn't installed, I couldn't pick it from the list. However (and this is really praiseworthy on the LO developers' parts), I was able to type Garamond (with leading cap) into the font text field, choose "Regular expressions, and then enter ".*" (no apostrophes, naturally) into the Search field.

    Now, I could fix them one by one, by clearing direct formatting, after noting whether the text should be italic or not, and then setting it italic as needed, after clearing direct formatting. But that, too, was tedious. Useful to do a little, though, to get a feel for the errors and start correcting them.

    I could see exactly how it had happened, and it had made sense at the time, but it did have this side-effect that I'd been unaware of. Anyway, understanding the types of errors, I then did a Find All (such a cool LO feature: you end up with multiple separate selections of the matching pattern throughout the document, so you can make changes to those pieces of text in one operation). LO told me it had 299 words and 1,457 characters selected, and I noted that the font size was 9pt in every case (by simply observing that "9" showed in the font size box on the main menu-bar), and then changed the font to Georgia and the point size to 10.5, and that was that!

    (Except for LO bug 62603 – aarghh!)

    The joy of the "stylesheet.css" file. I thought I'd do a search for other typefaces that might have crept in (Arial, Calibri, Times Roman). I found that there were a few places I'd unintentionally slipped into Times New Roman. Then I realised I could see every single font I had actually used just by looking inside the "stylesheet.css" file inside the .epub file.

    And, yes, there were a whole bunch more there. Here's the list, for maximum self-embarrassment:

    $ grep "font-family" WildThing-exp5.unz/stylesheet.css | sort -u
        font-family: "Liberation Sans", Arial;
        font-family: "Liberation Serif", serif;
        font-family: "Times New Roman";
        font-family: Calibri;
        font-family: Garamond, sans-serif;
        font-family: Georgia1, serif;
        font-family: Georgia;
        font-family: Times New Roman1, sans-serif;
        font-family: Ubuntu, sans-serif;
    

    Actually, I couldn't make much sense of what was going on when I searched for Liberation Serif: it seemed like a high percentage of paragraphs in the document were in this font: I could clear the formatting so they weren't found, but doing so made no visible difference. And these weren't paragraphs that had proven to be a problem in the conversion. I wondered if some weird font substitution was still going on? The particularly weird thing was that these searches would also find the chapter and part headings – but when they were selected, the font showed as Times New Roman, not Liberation Serif. Yet when I'd searched for Times Roman, they hadn't shown up. (The only pieces of text in the whole MS in Times New Roman were the title and "by L. J. Kendall" on the title page. Because of a bug (feature?) in LO's ToC generation, I had had to add a space character to the end of each chapter title (to make the chapter names actually appear in the ToC). Because of that work around, it seemed like these were all appearing in Liberation Serif. Finding and fixing these was tedious. I couldn't do a Search& Replace because the word "Chapter" and the chapter number were supplied from the paragraph style (they weren't "in" the document body). It turned out that the text is provided via the Bullets and Numbering dialogue's Options tab; the font is kind of defined by the "Character Style" you choose; but I couldn't at first see where these character styles were defined (or modifiable). It was set to "None"; when I changed it to "Header Char", it changed to Liberation Serif, even though it appeared as Time New Roman because that was the font style defined for the Chapter Title style defined in the "Styles and Formatting" dialogue. A bit confusing, eh?

    So, in the "Styles and Formatting" dialogue I needed to select the "Character Styles" section, and then change that to Liberation Serif. While there, I set the font size and colour, too. That changed all the auto-formatted chapter titles to be the same: I changed the Character Style for the Prolog to match, but chose not to do so for the Parts nor the Acknowledgement, nor Afterword etc.

    So, I checked that after making that change, I checked to see if the "added space" to the title was still being detected as in Liberation Serif when I did my Find&Replace: it was, but I figured that since this was a non-printing character, letting it be changed to Georgia shouldn't make a visible difference. So I went ahead and did a Find All for the Liberation Serif font: this time, it worked harder, finding I think 66,000 words in that font. So I went ahead and changed the font to Georgia, and the font size to 10.5 for all of them. That seemed to make no visible difference, but now when I searched for Liberation Serif, there were no matches.

    Whew!

    Okay, next on the list of probably-spurious fonts was Liberation Sans: that found only the Acknowledgement and Table of Contents headings – good!

    I then checked the others – none of them were used, except for Ubuntu in four places. So, after that, it seemed that I had (finally!) cleaned up my font mess. Oh: and in the Find&Replace dialogue I clicked on the "No format" button to clear the search for font, and also unchecked the regular-expression, so as not to accidentally confuse myself the next time I used the dialogue.

    In Calibre, you update the book by choosing "Add files to selected book records" from the Add books menu, and then selecting the (preferably .docx file) for your MS.

    I noticed some odd justification; checking back, I discovered that my two-space pedantry had come undone: all the occurrences of a non-breaking space followed by a normal space, between sentences, had been lost somehow. A quick regular expression Find& Replace fixed that. Oh, and for some reason, every paragraph was no longer fully-justified. Re-converting fixed not just the weird spacing problem but also the justification problem. Now all that remained was trying to work out how to centre the chapter headings for the auto-titled chapters (and ideally, add a little space below).

    Ha – in checking, I found that the back cover image I'd inserted on the last page, and anchored to the page, had stayed on that page even when the font change had increased the number of pages. It now appeared about 50 pages before the end. So I fixed that, too.

    After that, "all" that was required was about half a day of intense and error-prone work to find the single-character "smearing" of regular-into-italics and italics-into-regular caused throughout the MS in every place where there had been a style-change at the edge of where a Find&Replace had operated.

    Sigh. But at least that was the last of my problems, and things proceeded smoothly from there.

    Okay – once again, this blog post has turned out to be much longer than I expected, so I'll need yet another separate and new post to cover the topic of the nitty gritty of uploading the file to Kindle and making your Amazon Author page and making updates. So I'll leave it there, for today. Stay tuned!

    A Nice Table of Contents

    Oh, one thing I did to make the Table of Contents nicer was to delete the leader-characters and page numbers from the auto-generated table of contents, leaving just the (useful) hyperlinks for the ebook edition. I've also found a problem, due to LibreOffice's bad habit of breaking runs of characters into separate logical spans of text. This means that if you edit any text in your ToC, and new text you type in will be in a separate text span. This was very visible because I'd chosen a colour for my ebook chapter titles (not plain black), so edits became very visible, as thed edit was in black and the auto-generated text was blue.

    Editing the ToC manually to delete the leader characters (the row of dots) and the page number reference (mostly useless in an ebook), after the 1st time, instead of doing it manually, I used the Find&Replace dialogue: I turned on the Other Options, selected Regular Expressions, put nothing in the Replace With field and this text (without the quotes) in the Search For field: "\[0-9][0-9]*", and then quickly replaced each one.

    Poking about in Calibre and .epub

    So I copied the .epub file Calibre had produced, into a temporary directory to play with. The 1st step was to unzip the .epub file. Now, I noticed there were far more "index_split" files than there were chapters, and so I opened the first few of these files, and the content.opf, page_styles.css, titlepage.xhtml, and toc.ncx files. I immediately noticed a few unexpected things: text was still being broken up into some very short runs of characters, with different paragraph styles for each run. When I got to the stylesheet.css file, things started becoming clearer. That file defines all the paragraph styles in a very human-readable format, and it became obvious that I apparently had sections of text that looked the same, but which weren't. Since the paragraph style names created by Calibre are pretty distinct (like p-p1, p-p2 and so forth), it meant it would be easy to search the html files to see where they were used. In that way, I could find where I'd used these odd styles, and change the original MS to remove the weird styling.

    Now, the 1st problem in my epub version is that Table of Contents (TOC) puts each line of the TOC on a separate page! Looking in the toc.ncx file, I see it looks like this:

    <navPoint id="ujwaAQQK4di35KOrpmPFchB" playOrder="1">
          <navLabel>
            <text>Prolog 1</text>
          </navLabel>
          <content src="index_split_004.xhtml"/>
        </navPoint>
        <navPoint id="utE7axnMMjYH5AzmPYfyOp2" playOrder="2">
          <navLabel>
            <text>Part I 14</text>
          </navLabel>
          <content src="index_split_005.xhtml"/>
        </navPoint>
        <navPoint id="uyHLH5SBCmIBt42iiswSe24" playOrder="3">
          <navLabel>
            <text>Chapter 1 15</text>
          </navLabel>
          <content src="index_split_006.xhtml"/>
        </navPoint>
    

    So it looks like the page-breaks must be inside the actual content files like "index_split_004.xhtml". Looking inside that shows that the text is "Prolog 1" as you'd expect, but that instead of being a normal paragraph, it's defined to be a header element (H1). That seems odd. Look:

    <h1 class="p-p18" id="calibre_toc_2">
       <a href="index_split_075.xhtml#anchor8" class="s-t">
           <span class="s-t4">Prolog 1</span>
      </a>
      <a id="anchor9" class="s-t"></a>
      <a class="s-t"></a>
    </h1>
    

    Another odd thing there is that there are three anchor points: the 1st with the text "Prolog 1" and ID "anchor8", a 2nd with no text/content but an ID of "anchor9", and a 3rd with, again, no text/content, all the empty anchors with class "s-t", the "Prolog 1" text with styles-t4. I don't understand that. Looking again at the stylesheet.css, I see that the "s-t" class defines a style with 0 margin, 0 padding, and a 1.2 line-height, and s-t4 is the same but with font-height 0.77419em.

    Ahhh...! Perhaps because I manually reformatted the main block of the TOC to be 9pt, and the PARTs to be 10pt? And the 1st anchor is for the left text, the middle for the leader/spacing dots (omitted) and the 3rd for the page number? Sounds plausible, except the page number is in the 1st anchor, not the 3rd. Hmm. It kind of matches the definition at http://www.w3.org/TR/html5/text-level-semantics.html#the-a-element which says that if you leave out the actual link, "then the element represents a placeholder for where a link might otherwise have been placed, if it had been relevant, consisting of just the element's contents." So: why is the page number inside the 1st anchor, not the 3rd? The anchors are often used to indicate clickable content.

    What is a class, anyway? Sitepoint says that an HTML element's name (like "h1" or "div" or "a" or "p" specifies headier, division, anchor and paragraph), the "class" attribute let's you specify one or more subtypes. These subtypes are used to label semantically-similar things for identification, so that CSS or JavaScript code can then do clever stuff like define some properties or do something to all elements in the whole document structure (DOM) that have that subtype.

    It also means that if I want to auto-remove the page numbers from the TOC entries, I may have to remove it from both the index_split_004.xhtml file and the toc.ncx file – something to keep in mind. So: does this mean I should manually delete the TOC from the MS and regenerate one with page numbers turned off? Or write a script to do the edits, so I can keep a single MS file? Or maybe insert two TOCs, one designed for epub and one for print, and write a script to delete one or the other depending on whether I'm generating an ebook or a pbook? Or, is there a tweak in Calibre itself I could do? If you click on "Convert books" and then on the "Structure Detection" button, it shows you the code it uses to "Detect chapters" (which looks perfect to me), and also to"Insert page breaks", which again looks good, and explains where the 1-page-per-TOC-line problem is coming from: each line is of style "H1", so it's of course getting a page break.

    So it sounds like a 1st step might be just to regenerate the TOC in the MS, and not fiddle at all with the formatting, and then see what Calibre produces.

    Ah, and I see in Calibre's "Heuristic Processing" conversion option, one that says "Renumber sequences of <h1> or <h2> tags to prevent splitting" – that sounds like what I want, so the problem may become very simple to solve. Hmm: nope. It now has two lines of the TOC per page! An improvement but not a solution.

    Oh, and while fiddling with fonts, noticed the paragraph indent was an absolute unit of measurement (cm) instead of relative (points). So:

    To change the measurement units used in this dialogue, choose Tools - Options - LibreOffice Writer - General, and then select a new measurement unit in the Settings area.

    Hmm, tried again after fixing up far more font-change issues than I expected: I did have several messes. Now, it's all in Georgia, 10.5pt. TOC still wrong. Changed an option to force it to use the auto-generated TOC – I assume that means the one in the .odt file…

    "Appendix – Kindle's HTML"

    Most of the text below (apart from the occasional editorial remark in italics) is directly quoted from the Support Indie Authors forum on Goodreads.

    Never specify fonts or font sizes.

    Use percents for indents and em's for margins.

    Limit use of symbol characters (try to use HTML symbol codes) and in-line formatting. (We only use bold and italics -- not even super and subscripts.)

    But Morris noted (good tips here http://www.morrisegraham.com/):

    You can, and should specify font sizes for chapter and title headings. This is done by specifying size in the style part of your header by using a CSS style.

    Like....

    p.h1
    {
    font-size: 1.5em;
    text-indent: 0em;
    text-align: center;
    font-weight: bold
    }
    

    note that when I use an h1 header in my body, it causes the text to be 1.5 em, which is 1.5 times the width of the letter "m." So when the customer on the end adjust font size in their ereader device, the titles and chapter headings stay relative in ratio and proportion to the deafult font size. You can only get this kind of control by doing your book in notepad and then converting it to eBook with a converter later.

    But Owen then noted:

    If H1, H2, H3 etc tags are used, each device will display them in their own different default size. You can of course define your own styles and adjust the sizes if you want to, but we've always been happy with the defaults.

    Micah noted:

    One thing that always bugged me about the KDP defaults is that they automatically stick a huge extra line of space between paragraphs. This makes intentional section breaks very difficult for the reader to interpret. To avoid Kindle sticking in extra space, you do this (presumably in the CSS section):

    I just define the normal paragraph properties:

    p
    {
    text-indent: 1.3em;
    margin-bottom: 0.2em;
    }
    

    I found that setting the bottom margin to 0 seemed a bit too tight, and also thought the default text indent was too large for most eReaders. But you can adjust them as you like.

    And he added:

    Oh, and another thing which I find a bit odd about Amazon's eBook formatting: if you do not hard code in justified text, Kindle eReaders will justify the text by default.

    That's not an issue, however what may be an issue is that this default justification does NOT show up in their Look Inside feature.

    I have not hard coded justified text, so in the Look Inside preview, it appears as if my books have left justified text. But in an actual eReader they show up with fully justified text.

    Owen shared lots of good info:

    In our case, we specify a "normal" style in our style sheet for all normal text that is:

    p.Normal, li.Normal
    {margin-top:.60em;
    margin-right:0;
    margin-bottom:0;
    margin-left:0; 
    text-indent:5%;
    }
    

    Our paragraph style (which is hardly ever used) is:

    p
    {margin-top:0;
    margin-right:0;
    margin-bottom:0;
    margin-left:0; 
    }
    

    Our heading styles follow this format:

    h1
    {margin-top:2em;
    margin-right:0;
    margin-bottom:.30em;
    margin-left:0;
    border:none;
    padding:0; 
    font-weight:bold;
    }
    h2
    {margin-top:0.6em;
    margin-right:0;
    margin-bottom:0;
    margin-left:0;
    border:none;
    padding:0;
    text-transform:uppercase;
    font-weight:bold;
    }
    
    h3
    {margin-top:0.6em;
    margin-right:0;
    margin-bottom:0;
    margin-left:0;
    border:none;
    padding:0;
    font-weight:bold;
    }
    
    h4, h5, h6
    {margin-top:0.6em;
    margin-right:0;
    margin-bottom:0.6em;
    margin-left:0; 
    text-align:right; 
    font-weight:normal;
    font-style:italic;
    }
    

    We've never set the text justification manually. I was unaware until Micah mentioned it that anyone would think this "unprofessional", since I would have thought that readers of eBooks would be aware that the Amazon "Look inside" feature formats things differently than eReaders. I don't think I want such people reading our books anyway (our characters often speak and act "unprofessionally" and that might annoy them?)

    I have read that some readers find left justified text easier to reader. I suppose there might be some worth in not hard-coding the justification in case their device allows them to select that option?

    And also:

    Igzy wrote: "If it's not too much trouble could one of you offer an example of two properly formatted paragraphs laid out together? I'd be much obliged if I could see the tags in action, as I'm not familiar with HTML formatting. "

    No problem. Of course GR interprets HTML code, so this example used parentheses in place of pointy brackets, thus ( = < and ) = > . This is how we begin a chapter, with notes after the ||:

    (br clear=all style='page-break-before:always') || This tells the Kindle converter to break the page. Below are the headings: heading 3 for the main, and heading 4 for the sub, which is italic and aligned right.

    (h3)(a name="_Toc410049128")Prologue: Zero Day(/a)(/h3) || name ID's the target for the TOC link.

    (h4)Janin Station;(br) || br is a linefeed (line break)

    Tau Verde, Vulpecula Region(/h4)

    (p class=NormalBlock)It was make-and-mend day for the Halith Imperial Navy’s Kerberos Fleet ... ruled the lives of Halith mariners—especially when the fleet was lying up at a comfortable port like Janin. (/p) || This a paragraph container. HTML designates the beginning of a container with a code, just p here -- h3 above -- and ends it with a / in front of the code: /p. "class" defines the type of paragraph as defined in the style sheet. The is flush-left paragraph with extra space at the top. The CSS entry for it is below.

    (p class=Normal)Watchstanding and sensor sweeps ... guarded by a ring of monitors. (/p) || This is a standard text paragraph. 
    
    
    p.NormalBlock
    margin-top:2.5em; || creates the extra space. Note there is no text indent. 
    margin-right:0;
    margin-bottom:0;
    margin-left:0; 
    }
    

    That is 90% of an HTML text doc right there (with the CSS examples above, in the previous posts). Yes, you see a lot of godawful gibberish in the code here and there, like style="much gibberish" or (span style="much gibberish") (/span). Almost all the time, that is unwelcome. Word will put it in to try to mimic the exact look of a doc in IE -- not what you want.

    And Amazon have their own guide which comes recommended:

    "Building Your Book for Kindle", and the ebook is free.