This seems like a recurring pattern in learning/rationality for me
I discover X —> Few months later I find that someone has written extensively about X.
Only if I could minimise this and reduce that gap.
You could ask in a LW shortform. "I discovered X, can you recommend me something good written on X?"
The only inertia—these are my Cached Thoughts based on past online experiences— with that is social incentives, frequent low quality posts may lead to people ignore my comments, more or less classifying me as low status, which is not good for times when feedback is even more important than everyday basic observations. Lesswrong has higher standard than let's say discord or my personal notes in terms of conduct, I can be an edge-lord all I want on discord, but the replies I get on there are lower quality. For a change I could get more comfortable with taking social risks here—contrary to my general tendencies— and be an aspiring @Caledonian2 , but that seemed to have went poorly for him from what I can tell.
I'd advise that whenever you come up with what seems like an original idea or discovery, immediately do at bare minimum a quick Google search about it, or if you have the time, a reasonably thorough literature search in whatever field(s) it's related to. It is really, really easy to come up with something you think is new when it's actually not so much. While the space of possible ideas is vast, the low hanging fruit are very likely to have already been picked by someone somewhere, so especially be wary of a seemingly simple idea that seems super elegant and obvious. It probably is, and odds are someone on the Internet has made at least a blog post about it or there's an obscure paper on ArXiv discussing it.
Also, be aware that often people will use different terminology to describe the same thing, so part of that search for existing work should involve enumerating different ways of describing it. I know it's tedious to go through this process, but it helps to not be reinventing the wheel all the time.
Generally, a really unique, novel idea that actually works requires a lot of effort and domain knowledge to come up with, and probably needs experiments to really test and validate it. A lot of the ideas that aren't amenable to testing will sound nice but be unverifiable, and many ideas that can be tested will sound great on paper but actually not work as expected in the real world.
Also, be aware that often people will use different terminology to describe the same thing, so part of that search for existing work should involve enumerating different ways of describing it. I know it's tedious to go through this process, but it helps to not be reinventing the wheel all the time.
This is the difficult part and basically the main taut constraint I will have to loose. It's often difficult for me to search about insightful things, bare minimum google searches often do hit up result, but it's not what I am looking for, certainly not in the particular context. I have over time learnt to get better hits on lesswrong, substack or using chatgpt. I am not sure whether arxiv search is any good (haven't tried it extensively). Once I get to know about the common or related terminology[1], it's often a breeze, but that can sometimes take up days or worse, weeks. (although this time has decreased significantly)
Only if there was a summary or a type of resource, which basically went over the major terms, somewhat like a dictionary. Which would hit all vaguely adjacent topics, perhaps I should grep an introductory textbook I'm not sure.
Nowadays , I have defaulted to the prior of "If I—an amateur— am able to think about it, someone has written extensively on it" , and this has been working surprisingly well to get me into searching mode, instead of reinventing the wheel.
I guess one could make the case for traditional schooling and courses here, they make this process fairly straightforward by introducing a bunch of terms superficially.
I would expect modern LLMs to be fairly proficient at helping you identify existing concepts. Here's Claude's suggested strategy:
Stage 1: Concept Mapping
Start with a prompt that explores the concept from multiple angles: I've been thinking about [describe your observation/idea in your own words, with specific examples].
Can you help me identify:
Please provide multiple terms/phrases for each, as different communities may use different vocabulary.
Stage 2: Terminology Expansion
Once you have initial terms, use this follow-up: For the concept of [your description], you mentioned it might relate to [terms from Stage 1].
Can you:
Stage 3: Literature Discovery
With your expanded vocabulary, use: I'm trying to find existing work on [concept]. I now know it might be called [list of terms].
Please suggest:
Is there any pattern to how you come to find the extensive writing about X? Is it incidental as you research other things or is it a direct result of discovering X that now you have the concepts that lead you to reading about it?
Not that I'm aware of sometimes it's pure coincidence whilst searching for unrelated things, other times I remember some old context out of nowhere, or if I am looking into adjacent topics, which leads me to stumble upon it. Nowadays AI can also help, but I'm currently bad at prompt engineering.
I'm a bad prompt engineer myself and I get quite envious when people announce these amazing results they are getting from the LLM which feel like pulling teeth from a chicken to me.
The reason I ask is because if it is more a matter of incidental as you research other things, I was wondering if there's in a sense a way of reverse engineering that process of recognizing it after the fact?
I don't think I have observed any patterns, it could be due to low past written record of such process or a bound set by working-memory/intelligence. From next time I will take note when such incidents happen.