Solving Experience Survey Results

Results of the Survey on Solving Times, Use of Aids, & Ratings

+ – + – + – + – + – + – + – +

Mr Kitty here.  A big thank you to the 352 solvers who filled out my survey on their experience with last Tuesday’s back-page puzzle, DT 28566.  Extra thanks to those who left insightful comments under Question 6.  I’ve read every comment and shared all of them with BD.  I’m including some of them below, along with responses to those that posed questions.

Some key findings from the survey are that:

  • Only 10% of solvers reported both filling the grid and parsing all the answers without using any aids.
  • 70% of solvers reported spending more than 40 minutes on the crossword.
  • The average suggested difficulty rating was 3.3 stars.

I’m going to take a short break from surveying, but suggestions for questions that might be asked in a future survey are welcome now, either as a comment below or via BD using the contact link at the top of the page.  Already on my list of possible questions are number of crosswords solved per week, other crosswords solved regularly, and solving method: online, with an app, in the newspaper, or on a printout.  What would you like to know about your fellow solvers?

Click on each spoiler box to see detailed summaries of the responses to each question.  Clicking on a chart will open a larger version in a new browser window.

Question 1: What aids, if any, did you use when filling the grid on this puzzle?

[Filling the grid here means finding the answers, which could be accomplished unaided via understanding the wordplay, spotting the definition, guessing based on the checking letters already in place, etc., or by using solving aids.  Please select all applicable answers.]

Only 34 solvers of the 352 completing the survey answered “None” to both Q1 and Q2.  Everyone else used some form of assistance for finding the answers or parsing the clues or both.

[collapse]
Question 2: What aids, if any, did you use when parsing the answers in this puzzle?

[Parsing in this context means understanding how each answer is obtained from the wordplay in the clue.  Please select all applicable answers.]

[collapse]

Question 3: How long did you spend solving this puzzle?

[Ignore any breaks, and include only the time spent thinking about the puzzle before you stopped working on it.  An estimate is fine if, like most of us, you didn’t have a timer running.]

The line shows the percentage of solvers spending less that the corresponding time on the puzzle (i.e. it is the cumulative distribution of times).  This data shows that 95% of solvers spent more than 20 minutes on DT 28566, 70% spent more than 40 minutes, 40% spent more than 60 minutes, and 20% spent more than 90 minutes.  The median time is 58 minutes.  In addition:

  • The three fastest solvers all reported spending 12 minutes on the puzzle, and all completed the solve unaided (I hope BD has them signed up to blog).
  • Several of the sub-20-minute solvers reported using aids, suggesting that some readers chose to do whatever it took to get a fast time.
  • Several of the unaided solvers reported spending hours on the puzzle, suggesting that other readers chose to do whatever it took to solve the puzzle without using any aids.
    [collapse]
Question 4: What difficulty rating do you feel would be appropriate for this puzzle?

[The BD scale runs from 0 (trivial) to 5 (fiendish).  Half stars (e.g. 3.5) are allowed.]

The average of this distribution is 3.3 stars.  That’s higher than 11 of the 14 difficulty ratings given in comments posted on my DT 28566 blog.

[collapse]
Question 5: What enjoyment rating do you feel would be appropriate for this puzzle?

[The BD scale runs from 0 (ugh) to 5 (brilliant).  Half stars (e.g. 3.5) are allowed.]

The average of this distribution is also 3.3 stars.

[collapse]

Question 6: If you'd like to make any additional comments about the topics covered by this survey, feel free to do so here

In addition to comments elaborating on solving experiences and comments relating specifically to my blog, there were also many comments falling into these three categories:

1.  Expressions of thanks to the entire BD community

  • “…now I shall read the blog and the comments though as they are always entertaining”
  • “I often use Big Dave’s site when I’ve put an answer in that I know is right but can’t work out why it’s right. I always find the hints and breakdown of the clues really helpful. Thank you all for adding to the enjoyment of solving the puzzle.”
  • ” Just thanks to you all from a grateful lurker”
  • ” I love the blog because you learn so much, not just about crossword solving.  I do not comment on the blog because I spend more time on crosswords than I actually have to spare.”
  • “… What a relief that this site is available – it’s a great learning resource.”
  • “… Thanks to everyone who contributes to this blog – I rely on you every day :)”
  • “I’ve never commented before but would like to thank all those who do contribute to Big Dave’s blog. Only I know how many hours of frustration and sleepless nights you’ve all prevented! Please keep up the good work – it really is appreciated by those of us who engage in the daily tussle with the brilliant compilers and those who give up their time to help the secret lurkers!”
  • “… Even when the puzzle has been completed without assistance the blog always raises a smile. … Your blog has helped greatly in this respect.  Many thanks to all who post the clues and to all those who post on the blog.”
  • “Always thoroughly enjoy these puzzles & thoroughly enjoy the blog, helps when I need it & enjoy reading the posts!”
  • “Wanted to say how much I appreciate and enjoy all the contributions to the Big Dave website.”
  • “Finding the Big Dave site and using it every day has allowed me to learn cryptics, providing a daily challenge and much enjoyment. A most unexpected delight in retirement. An additional bonus is the calibre of the bloggers. Thank you to all, especially you Mr K.”
  • “Keep going Big Dave. You are the last resort but we need you”
  • “I hugely appreciate the hints and tips in the blog, just didn’t need them today…”
  • “I love using the Big Dave’s website. It has improved my solving abilities no end and I learn something new every time I use the site (about 4 times per week). There are a lot of clever people out there who compile the solutions! It is much appreciated by the less clever ones, but who enjoy trying!”
  • “It’s great to have BD for a leg up at times. Thanks for all your input!”
  • “I’ve learnt a lot from this blog over the years but still come back to it if all else fails. Well done to all concerned for this very helpful blog.”
  • ” always grateful for this site!”

2.  Questions about the blog
Responses in italics are mine.

  • “I suspect, like me, there are many interested lurkers who don’t consider themselves expert enough to comment. Those that post are the most confident in their ability – and those with most time to do so”
    You make a good point about commenting taking time, but the only qualification required to comment is to have attempted the puzzle.  It would be great to see more comments from solvers who don’t consider themselves experts.  They are sure to bring both balance and an interesting perspective to the discussion, and I know from experience that non-experts will get only encouragement and support from the rest of the commentariat.  Remember too that the blog is anonymous.  All that anybody knows about a commenter is what they choose to share.
     
  • “How is the difficulty rating arrived at? Is it purely a time decision?”
    The difficulty rating is an informal indicator that shouldn’t be taken too seriously.  Most bloggers have their own rating system which may or may not be based on solving time.  In my case, now that I know that the readership feels that DT 28566 should be rated 3* for difficulty, I will try to use 2* for puzzles that I find are somewhat easier, 4* for puzzles that are somewhat harder, and reserve 1* and 5* for puzzles that are much easier and much harder respectively.  We don’t expect to see too many of those since the puzzles have been moderated by an editor at the Telegraph before publication.
     
  • “Know I would have struggled to solve today’s puzzle without the afore mentioned aids!  I would love to know if some bloggers would consider the aids used a bit of a cheat?  Thanks for all the effort you put in.”
    My thinking on this common question is that a solver should do whatever makes solving the puzzle rewarding for them and not worry about what anybody else thinks.  For some solvers that will be spending as long as it takes to complete the solve unaided, for others it will be turning to aids at some point so they have time to do other things.  In my opinion the fastest way to get better at solving cryptics is to solve as many puzzles as you can, in which case it is silly to spend hours stuck on one clue when you could have got unstuck with help from a crossword dictionary or the blog or somewhere else and then used that time to solve more puzzles.  Note too that the responses to Q1 and Q2 show that 90% of the blog readership would have struggled to solve DT 28566 without aids.  And since nobody is born knowing how to solve cryptic crosswords, every single solver out there has used some form of assistance to reach their current level of proficiency.
     
  • “Excellent blog. Thanks to all concerned. I know from experience, however, that criticism of the setter is not tolerated and any such remarks are expunged. I would be interested to know why this is.”
    I believe one reason that there’s a wonderful sense of community on the site is BD’s policy that personal attacks are not allowed on the blog and will be deleted.  On the other hand, thoughtful criticism is, of course, always welcome.  So “Setter X’s puzzles are always rubbish” will get deleted, while “I thought this puzzle from Setter X was weak because it contained too many anagrams” is fine and will probably stimulate some discussion.  Comments consisting only of unsupported negative assertions like “12a is the worst clue I have ever seen” are also likely to be deleted.
     
  • “How can you spot the setter and why won’t the telegraph people tell us?”
    I don’t know about the why, but the answer to the frequently-asked question “how do you know the setter?” is found in the site’s list of Frequently Asked Questions.  See #28.  There’s a lot of good stuff in the FAQ.  It’s worth reading.
     
  • “I have already looked at survey results and am interested in the age of most solvers. Why do you think the 60+ age group is the largest? is it that this group has more free time to solve a crossword? Is it that this group is less likely to use tech devices offering many options for passing time? Are cryptic crosswords going out of date?”
    I’m not old enough for my own circumstances to be useful here, so I’ll turn this one over to the community.  What do our 60+ solvers think?  Have you been able to get younger relatives interested in cryptic crosswords?

3.  Requests to retire “Read and Write”

  • “…Love the blog and comments although always a bit of a downer when you struggle for days and then someone else says it was a read and write!! Thank you for the blog.”
  • “Very pleased the site exists but so often disagree with the comments, especially the showoffs …”
  • “Just stop people making comments like “no blog needed; this was R&W”. Nothing worse, especially with the Toughie, to read that after a handful of answers in ninety minutes!! All answers are very easy IF you know them!! Keep up the good work”
  • “I like the discouragement of disclosing solving times but find terms like “R&W” even more distasteful …”
    [collapse]

35 Replies to “Solving Experience Survey Results”

  1. I missed this survey. I didn’t see it because I don’t always comment on the cryptic. With the 5-hour time difference it’s sometimes very late over there before I have the time to sit down and focus and by then there’s no-one around. Right now it’s 5 am and I’m taking advantage of being up since stupid o’clock!

    Back when I was a back-page lurker, the commenters seemed like a very tight group. Everyone seemed to know each other well. It was quite intimidating and took me some time to get up the courage to join in. I was so glad when I did. Everyone was so welcoming. I do encourage our lurker friends to take the plunge!

    On the Toughie… there used to be a gentleman who was a regular. No matter how challenging the puzzle, he invariably commented that “This was a nice wee toughie.” It drove me crazy when I had struggled mightily and had perhaps a handful of answers! I’ve improved with time, and the very helpful blog. It’s good to see others hanging in there.

    I’m in the 60+ category (though I will never tell how many pluses apply!) I do work full time, fortunately from a home office. I’d be interested to know how many of my fellow commenters also have full-time jobs. I get the impression that many are retired so have more time available during the day. Also the percentage of male versus female respondents.

    Pet peeve? Seeing the setters being referred to in comments by their given name rather than their chosen pseudonym.

    1. The gentleman concerned cancelled his Telegraph subscription after “a huge disagreement with the two main Scottish correspondents”, but I am still in touch with him via Facebook. Gazza will remember BigBoab as we both “met” him on AnswerBank and he joined us on here from Day 1.

    2. Working versus retired is an excellent suggestion. Thanks for that, Chris.

      I didn’t include gender in the first one because it didn’t seem that relevant, but i could certainly include it in the next survey if there’s interest out there in knowing that distribution.

  2. I am in two minds about the R&W issue. As the blog records our own particular experience with a given crossword, it is right to record how easy or difficult we find it, if not, I think the blogs would become somewhat turgid without our own personal view of the puzzle. The problem is how that is expressed.
    I was a beginner three years ago, and I never had a problem with bloggers expressing that they they had found a particular puzzle easy, after all, if they have been doing crosswords for 30+ years, so they should!! What I did not like was being patronised. There were two that I can recall, the famed “Junior Telegraph” quip, and the comment about wishing to make the back-pager “much harder”, such comments are out of order and the bloggers was rightly castigated. BD polices the site, sensitive of these types of comment, and our thanks are due for that.
    Does R&W fall into that category?? I’m not so sure, it certainly does not upset me, and if we appreciate that solvers are of a widely different standard then I don’t see a problem with it. Metaphors and phrases that imply speed, I am not so happy with.

    1. Sorry, I forgot to thank Mr.K. for another great survey, and continued thanks to BD for everything.

    2. I do see your point, Hoofit, thought similar myself.

      If I’m not to mention when a puzzle in my opinion is a R&W , solving time, finding a puzzle easy or suggesting it’s a good one for the beginners (though I do dislike that phrase), then what can I say? Giving it one star implies one or all of the above anyway.

      As you say, the blog needs the variety of opinion to keep it alive and interesting.

      I will refrain from using R&W, but is saying ‘nothing tricky’ or ‘over too soon’ really any different? I’m equally happy to post that a Toughie has sat on my desk all week unfinished, so why not R&W when that’s what it is for me?

    3. I see two issues here. One is that there’s a big difference between a commenter saying “Today I was so on wavelength that this was a read and write for me” and “I think this puzzle is a read and write”. The second is that R&W clearly means different things to different people. Yesterday R&W was defined as read-write-read-write-read-write-done, but I think in the minds of many lurkers it means something more like the video below showing Mark “Mr Magoo” Goodliffe solving in succession cryptics in the Independent, Guardian, and Times, parsing as he goes and doing it all in under 21 minutes. You’re right about the recurring speed metaphors, too.

      1. I think I’ll have a lie down. To watch someone read, solve write down and even comment on the clues of that first cryptic crossword, all in under 5 minutes has left me feeling slightly faint . . .
        I’ll take your word for the remaining two puzzles.
        I will plod happily on at my own very sedate pace – but am even more glad BD doesn’t allow solving times now. :-)

        1. Mr Goodliffe is a world-class solver, so nobody should feel bad because of that performance. But perhaps it’s useful for those implying speed in their comments to see what a fast solve really looks like.

  3. Another set of interesting results, many thanks Mr K.

    After the incredible response to the first survey, I was very surprised to see a much lower number this time, I had imagined that it might be slightly lower, say between 1,000 to 1,200, but 352 seems quite disappointing. Is “survey fatigue” setting in, I wonder, or could there be another reason? Any thoughts?

    P.S. I admit to being one of the worst culprits when it comes to Expat Chris’s pet peeve, since I will invariably thank Messrs. Squires, Mutch, Terrell, Greer etc. rather than use their pseudonyms. I do hope she’ll forgive me, but their identities are not exactly a secret, and the DT never uses the pseudonyms anyway where backpagers are concerned.

    1. I think the low response was because it was specific to a particular puzzle, also, I don’t think it was publicised very well, I did not know about it until about three days later.

      1. I agree. It provided a snapshot of the users on one particular day. If we had wanted to try to estimate the total number of users, we should have run it, and advertised it, for much longer.

        There are plenty of days when, having completed it with no great heartaches, I won’t come to the blog as I feel I have nothing to learn and nothing to offer.

        1. Yes. For the data about solving times and aids to be meaningful it had to apply to just one puzzle, which is why I didn’t post it on the front page.

          I feel that the response rate is actually very good. In the first 24 hours when the initial survey was only on my blog it got 600 responses. Another 900 were added over the next six days when it was a sticky announcement at the top of the home page. Since the second survey took more considerably time to complete than the first one and it required the reader to have solved the puzzle, I thought that getting 350 responses instead of the 600 from the first one was still impressive participation.

    2. Nothing to forgive, Silvanus! I was not aware that the print version published given names. I don’t have access to British newspapers over here except on line.

      Hard to comment this morning. The page is jumping all over the place!

      1. Hi Chris. For reasons I’ve never understood the DT publishes setter’s pseudonyms for the Toughie but, as mentioned above by Silvanus, they leave the back-pager anonymous. I never see the on-line versions but I believe the same applies for them too.

        There seems to be general awareness of some of the back-page setters either because it is known to be their regular slot (see BD’s FAQ #28) and/or because their style is so obvious (e.g.: RayT, Rufus). Some setters pop in to the blog and comment (RayT regularly; Giovanni and Mister Ron [aka Samuel] occasionally) which is something I think most solvers appreciate.

        BD’s FAQ #28 also uncovers the real names behind the aliases.

  4. I missed this survey altogether…..but would have filled it in if I had seen it.
    Lots of interesting information , as there was with the first survey.

    I am not keen on the ‘showing off’ element that sometimes creeps in to the blog…..especially when puzzles are described as ‘entry level’ or suchlike. Commenting favourably on your own performance is one thing, presuming to comment on other people’s is quite another matter.

    Thanks to Mr K for the huge amount of work he has put in on these surveys….and all the other stats he has given us.
    And, of course, thanks to Big Dave for this marvellous resource.

    1. Yes. There’s a world of difference between:

      “I did well on this and I want to share the joy”
      and
      “This was easy [therefore anyone who didn’t sail through must be stupid]”.

      We’ve discovered that people see comments using R&W as belonging in the latter category (even though I’m sure they’ve often been meant in the spirit of the former). It seems clear that the term often comes across as patronising and/or boasting — therefore it’s off-putting and might well be best retired.

  5. I missed this survey but would have said that I try to finish it by one o’clock in the morning, so in about a hour. Over that time and I give up and pick it up in the morning.

    As for assistance I use Anagram-solver.com, Onelook for wordsearch, and the-crossword-solver.com, when things really get sticky I even resort to Answerbank but consider that to be a defeat.

  6. Another fascinating survey, many thanks Mr K. I agree with all the complimentary comments about the site. Well done all and many thanks.

    1. Glad you enjoyed it. I particularly liked the fourth complimentary comment. It encapsulates a lot of what this site is about.

  7. Thank you for the response ‘re my question of using aids Mr K. An excellent answer I fully agree with. I hope other people will read this and continue to enjoy the daily challenge whatever methods we use. It’s about enjoying it after all. Excellent survey yet again.

    Thanks again for what you and all the others do on the blog.

  8. Yes, indeed, another very interesting survey which regrettably I missed or I would have completed it. Thank you, Mr K, for your valuable input to this site. I like to do the crossword in dribs and drabs, as the concentration gives out after a while, and I try not to use hints, which stretches it out a bit. Where necessary, I use a trial submit or letter hint and, if I run out of these, then I run to Big Dave’s site. I check the viability of an unfamiliar word on Wiktionary and write the long anagrams down. The solving skills have increased so much since finding BD’s site. I’d have to get a bit better and quicker to join in the conversation though. Thanks to all the contributors.

  9. Once again, all very interesting. Thank you Mr Kitty for gathering and presenting the info.

    A couple of comments:
    1) On the subject of aids, on this occasion I wouldn’t say that my using a dictionary was a matter of improving my time – it was a matter of being able to complete at all. I could have stared at the puzzle in questsion for a month, but I still wouldn’t have had the word ‘wolfram’ anywhere even in the deepest depths of my mind to pull forward, and ‘two animals’ was a bit too vague for me to work it out without the dictionary. That said, I do have a threshold where I’ll start to get bored of staring at the same last few clues, whereupon looking at a hint or two will save a good crossword experience from turning to a bad one.

    On the R&W issue, I’ve only been doing the cryptic for three or four months and have been visiting BD for about three (I like to read both the hints and the comments, whether I need them or not), and I don’t have an issue with this phrase. I’d like to think that the people who sail through a crossword that I toil over just have brains that are wired differently to mine, which I don’t take as being a slight on my mind.

    1. Hi, Gayle, and thanks. Most solvers fell into the same category as you did, namely using aids to get it finished. I gave the two extreme cases just to make the point that time spent on the puzzle doesn’t tell the whole story.

  10. Very interesting as always Mr K.

    This particular puzzle I completed without any solving aids but I do, on occasions, make use of anagram solvers and more infrequently, word finders.

    When I first started solving I found it mildly annoying if I’d been trying to anagram letters that simply didn’t yield the answer – by either clever misdirection from the setter or my own failure to realise which letters needed rearranging.

    I suppose that with more solving experience one gets, the less need there is for external aids, but I would personally prefer to complete a grid sooner rather than later as my impatience gets the better of me!

  11. In an attempt to make it easier to follow, I’ve reformatted the presentation of the results above using spoiler boxes.

Comments are closed.