Is My Writing Improving? Observed Vs. Expected

A couple of months ago I got a little frustrated with my progress in my writing, having what appeared to me to be a lengthy lull in publications.  This got me thinking about probabilities and how many submissions I should have to make for each acceptance, etc., which is something I've written about at length before.

But one thing that occurred to me was that it would be nice to have some kind of metric on an ongoing basis so I could see how well I'm performing compared to the expected number of submissions required for each acceptance.  So I think this is pretty cool: as I've noted before, The Grinder posts acceptance rates, which can loosely be used to approximate a submission's probability of acceptance for a given market.  Summing up these probabilities over time gives a rough indicator of how many acceptances should be expected during that time, which can then be compared to the actual number of publications that occurred.

Here's my "over-under" for the past two years since I started submitting to science fiction markets:

Year	Quarter	Subs	Exp	Obs	Exp (Yr)	Obs (Yr)	Obs-Exp
2012	Q4	9	0.6359	0
2013	Q1	9	0.6796	0
2013	Q2	8	0.9949	1
2013	Q3	4	0.1591	0	2.4695	1	-1.4695
2013	Q4	1	0.025	0
2014	Q1	9	1.108	1
2014	Q2	3	0.3819	0
2014	Q3	10	0.9089	1	2.4238	2	-0.4238
Total			4.8933	3

The "Subs" column is the number of submissions I made to science fiction markets during each quarter.  The "Exp" column is the sum of the acceptance rates for all the markets I submitted to during each quarter.  The "Obs" column is the actual number of publications I had during each quarter.  So based on the submissions I've made so far, I should have had four stories accepted for publication.  I've had three, which puts me below the expected value for the average writer who logs submissions on The Grinder.  However, subsetting it by year, that one missing publication can (in my mind) be attributed to the learning curve that I went through that first year of submissions.  The "Obs-Exp" column captures the difference between the observed and expected values for each year, which I'm calling the over-under.  My over-under for the first year was -1.4695, which put me a whole publication behind where I should have been.  But in my second year, my over-under was only -0.4238, meaning I actually hit my target of two publications based on the markets I submitted to.  Which means I've improved as a writer (either in the writing itself or in selection of better fitting markets for the stories submitted).

I think this is a really good way to measure improvement in writing performance over time.

Now I just have to exceed the expectations long enough to make up for that missing publication....

Am I Competing Against Other Writers?

I keep reading postings on authors’ websites (for example, here) about how writers aren’t competing with each other, if someone else sells a book, it doesn’t mean I won’t sell mine, we’re all in this together, blah blah blah…. 

 

It’s not that it’s annoying me (per se), it’s just that the idea seems so prevalent (a quick Google search yielded here, here, and here, for example) and yet is so mathematically unfounded…at least in my (I believe justified) opinion.

So some quick math (yay):

A writer’s probability Pof getting published comes down to two variables (ignoring the option of self-publication for the moment): the number of opportunities for publication, X, and the number of writers trying to snag one of those opportunities, Y.  The probability is a fraction, P = X/Y.  Pretty straightforward.

Introductory calculus tells us that if we hold X constant (meaning we have a fixed number of opportunities for publication), then as Y gets larger (meaning there are more and more writers trying to get published), the limit as Y goes to infinity is zero.  In other words, the more writers trying to nab one of those publication slots, the less likely it is for any of them to get one.  Conversely, in this scenario, fewer writers trying to get published would increase the probability of publication for any given writer.

Similarly, if we hold Yconstant (a fixed number of writers), then as X gets smaller (meaning more and more publication opportunities are being taken by other writers), the limit as Xgoes to zero is zero.  In other words, the more writers get published, the less likely it is for any of the other writers to get published.  Conversely, fewer writers taking up publication slots increases (temporarily) the likelihood that any given writer will get published.

Now this probably isn’t applicable for authors who are already published (like in Scalzi’s case, where he’s talking about book sales, since he and Dan Brown are already published), where there are other market forces at play, but for the majority of aspiring writers, there are only so many publishing opportunities to go around.

Using the SFWA list of professional publications, I figure there are about 1,000 opportunities to have a short story published per year, accounting for professional magazines that publish a set number of stories daily, weekly, quarterly, etc.  Whatever the actual number is, it’s a fixed number.  The number of writers trying for those slots is roughly fixed as well, though I have no idea what it is.  I do know that one magazine I’ve submitted to a few times always has about 60 stories in the queue, so figure about 60 submissions per day for every magazine, which gives very roughly about 500,000 submissions to pro magazines per year.  Again, whatever the actual number is, it’s fixed.

The more writers there are “competing,” the lower the probability of success.  The more writers achieve success, the lower the probability that anyone else will have it.  At least in this context -- though I would expect it to extend beyond professional short story publications to other mediums as well.

Assuming for a minute that these numbers are correct, an author trying to get published this year could expect a probability of about 0.002, or 1 in 500.  Not terrible, but it means the average writer could expect to have about 500 rejections before getting published.  Naturally, there’s a lot of variation between authors, and many (most likely the successful ones who write about how we’re not competing with each other…) will publish much sooner than that.  And they probably have strong anecdotal evidence that says it doesn't matter, that they're not competing with anyone else.  After all, they got published, and no one else's success impeded them one bit.  Right?  But then there are the many writers on the other side of the curve, who will take much longer than 500 submissions -- or at least they would if they didn’t give up long before getting that far, which is what I imagine happens to most.  There are obviously major differences in writing ability between authors, and I like to think that it’s the better ones who get those limited few publication slots.  But what if, hypothetically speaking, every writer was at the exact same skill level?  There’d still be only 1,000 slots available. 

It’s a numbers game.

Nothing against self-publication, and certainly nothing against celebrating other writers’ achievements and trying not to have a cutthroat capitalistic perspective on it all, but don’t lie to yourself!  I can see why it’s not necessarily intuitive.  It doesn’t seem fair.  But the numbers is what they is.