Launch HN: Undermind (YC S24) – AI agent for discovering scientific papers
Josh and Tom are developing Undermind, a search engine for complex scientific research, using large language models to enhance search accuracy and comprehensiveness, inviting user feedback for improvements.
Josh and Tom, physicists and founders of Undermind, are developing a search engine specifically designed for complex scientific research. Their motivation stems from personal frustrations during their graduate studies, where they often struggled to efficiently find relevant research papers, leading to wasted time and missed opportunities. They aim to address this issue by creating a system that mimics effective human research strategies, utilizing a pipeline that incorporates large language models (LLMs) to provide tailored search results.
The search process begins with a conversation between the user and the LLM to clarify complex research goals. The system then conducts a thorough search for approximately three minutes, employing a tree search method that follows citations and adapts based on findings. This approach prioritizes accuracy and comprehensiveness, ensuring that users receive specific recommendations and are aware of all relevant existing research.
Undermind's automated pipeline tracks the discovery process, allowing for statistical modeling of search saturation, which helps determine when all useful leads have been exhausted. The founders are currently focusing on reading abstracts and citations, with plans to include full texts in the future. They have made a demo video available and are inviting users to try the search engine without a signup requirement for a limited time. Feedback from users is encouraged to improve the platform further.
- Many users find the search engine effective, discovering relevant research they previously missed.
- Suggestions for enhancements include better citation formats, the ability to save results, and improved search algorithms to prioritize important papers.
- Users express interest in pricing models, with requests for student tiers and pay-per-query options.
- Some users note limitations in the search results, such as missing key references and the need for more comprehensive coverage of gray literature.
- Overall, there is excitement about the potential of Undermind to streamline research processes.
62 commentsI'd love to see limitations like this quantified and clearly flagged. Otherwise there's a danger that people may the assume results are definitive, and this could have the opposite outcome to that intended (much time spent working on something only to disocver it's been done already).
Claude 3.5 is reluctant to provide references—-although it will if coaxed by prompting.
Undermind solves this particular problem. A great complement for my research question —- the evidence that brain volume is reduced as a function of age in healthy cognitively normal humans. In mice we see a steady slow increase that averages out to a gain of 5% between the human equivalents of 20 to 65 years of age. This increase is almost linear as a function of log of age.
Here is the question that was refined with Undermind’s help:
>I want to find studies on adult humans (ages 20-100+) that have used true longitudinal repeated measures designs to study variations in brain volume over several years, focusing on individuals who are relatively healthy and cognitively functional.
I received 100 ranked and lightly annotated set of 100 citations in this format:
>[1] Characterization of Brain Volume Changes in Aging Individuals With Normal Cognition Using Serial Magnetic Resonance Imaging S. Fujita, ..., and O. Abe JAMA Network Open 2023 - 21 citations - Show abstract - Cite - PDF 99.6% topic match Provides longitudinal data on brain volume changes in aging individuals with normal cognition. Analyzes annual MRI data from 653 adults over 10 years to observe brain volume trajectories. Excludes populations with neurodegenerative diseases; employs true longitudinal design with robust MRI techniques.
If you can get this to work for patent searches across multiple languages, you'd really have a killer product. Patent searches, via an attorney, cost thousands of dollars each and are nevertheless frequently imperfect. (I had a patent denied because somewhere, in a throwaway paragraph buried in the 100-page description of an entirely different invention, something similar was briefly mentioned and never referred to again.)
I'd gladly pay $100/month for "Deep Patent Search," and more than that if it's really good.
Is exactly how you gain expertise in a field and/or find those subtle gaps in knowledge that are the seeds of real breakthrough
One thing I'd improve is how it asks followup questions.
For example, I asked "What are the best types of videogames that improve cognition?"
The followup response was
```
Are you specifically looking for studies that focus on cognitive improvements as measured by standardized tests, or are you more interested in everyday functional improvements observed in real-world settings?
Could you be more precise about which cognitive domains you're interested in? For example, are you focusing on memory, problem-solving, attention, or something else? Additionally, are you looking for papers that compare different genres of videogames (e.g., action, puzzle, strategy) or studies targeting specific age groups or populations (e.g., children, older adults)?
Lastly, are you interested in experimental studies where the cognitive benefits were directly measured pre- and post-intervention, or are observational studies that report correlations between videogame use and cognitive performance also of interest to you? Understanding this will help determine the type of articles we should prioritize in the search.
```
It would be great if it turned those into multiple choice. For example:
```
Could you be more precise about which cognitive domains you're interested in?
[] memory
[] problem-solving
[] attention
[] something else (please specify)
```
Would save a ton of time having to reply/ reread everything.
Arbitrary numbers are really the least information. At least use last names and years, so I can have some idea which paper you are talking about without scrolling back and forth.
I tried a search on my previous research area (https://www.undermind.ai/query_app/display_one_search/5408b4...) and it missed some key theoretical papers. At the same time, it picked up the three or four papers I’d expected it to find plus a PhD thesis I expected it to find. The results at the top of the list though are very recent and one of them is on something totally different to what I asked it for (“Skyrmion bubbles” != “Skyrmions”). The 7th result is an absolutely core paper, would be the one I’d give to a new PhD student going into this area and the one I’d have expected it to push up to the top of the list.
There's a 50/50 false positive rate, but I can deal with that. It means looking at 10 papers to find 5 useful ones instead of looking at 1000 papers to also find 5 useful ones.
I'm impressed.
One thing that I would like to suggest (other than saving to PDF, as discussed elsewhere in the thread) is to give the possibility, not just to "expand" the search, but also to "refine" the search.
If it was possible for me, after reading through the result page, to go back to your conversational UI and say, "OK this was my original intent, and here's what's wrong with the results I've got" for your system to provide a "refined" version of my query, that would be next-level.
Keep up the good work. Congrats on a successful launch!
One anecdote that I heard from the team developing it: turned out that researchers more readily sourced material from their social networks, notably twitter at the time. Meta's search functionality didn't receive enough traffic and eventually was shut down.
Perhaps LLMs will make the search capability more compelling. I guess we'll see.
this was the search <https://www.undermind.ai/query_app/display_one_search/cba773...> if you need a reference too it, ie bugs or performance monitoring...
You should have seen what it used to be like a few decades ago :)
But the fact I wanted to save a result is a good sign. Nice work!
One suggestion: The back-and-forth chat in the beginning could be improved with a more extensive interaction. So, the final prompt could be more fine-grained into a specific area/context/anything one would aim for.
This is obnoxious. Please remove this unnecessary roadblock.
In my opinion elicit has better looking UI and much more features and further along
It doesn’t recognize my university.
Are you breakdown the question into subtopics, doing a broad search and then doing some sort of dim reduction -> topical clustering to get it in the format?
I did 2 searches.
First I asked about a very specific niche thing. I gave me results but none I wanted. It looked like I missed a crucial piece of information.
So I did the second search. I started with the final request it written for the previous search and added the information I though I missed. It gave me virtually the same results with a little sprinkle of what I was actually after.
A few observations:
1. I'm not sure but it seems like it relies too much on citation count. Or maybe citations in papers make it think that the paper is absolutely a must read. I specifically said I'm not interested in what's in that paper and I still got those results.
2. I don't see much dissertations/theses in the result. I know for sure that there a good results for my request in a few dissertations. None of them are in the results.
That said, while I didn't get exactly what I want I've found a few interesting papers even if they're tangential to the actual request.
let’s say i am interested in coffee and i’d like to get new research papers on it. would this work?
Are there any plans on releasing any sort of API integration? I work in Technology Transfer consultancy for research institutes in Europe, and often we have to do manual evaluation of publications for novelty check and similar developments. Since most of the projects we work on were developed by leading researchers in academic institutions, it is important for us to quickly assess if a certain topic has been studied already.
Currently, one of my company's internal projects is a LLM-powered software to automate much of the manual search, together with other features related to the industry.
I think it be very beneficial for us to implement academic papers search function, but for that an API system would be required.
Great work nonetheless, good luck on the journey
With the current attitudes to AI, the name feels a little tone deaf being so easily mistaken for AI undermining people.
I gave it a version of my question, it asked me reasonable follow-ups, and we refined the search to:
> I want to find randomized controlled trials published by December 2023, investigating interventions to reduce consumption of meat and animal products with control groups receiving no treatment, measuring direct consumption (self-reported outcomes are acceptable), with at least 25 subjects in treatment and control groups (or at least 10 clusters for cluster-assigned studies), and with outcomes measured at least one day after treatment begins.
I just got the results back: https://www.undermind.ai/query_app/display_one_search/e5d964....
It certainly didn't find everything in my dataset, but:
* the first result is in the dataset.
* The second one is a study I excluded for something buried deep in the text.
* The third is in our dataset.
* The fourth is excluded for something the machine should have caught (32 subjects in total), but perhaps I needed to clarify 25 subjects in treatment and control each.
* The fifth result is a protocol for the study in result 3, so a more sophisticated search would have identified that these were related.
* The sixth study was entirely new to me, and though it didn't qualify because of the way the control group received some aspect of treatment, it's still something that my existing search processes missed, so right away I see real value.
So, overall, I am impressed, and I can easily imagine my lab paying for this. It would have to advance substantially before it was my only search method for a meta-analysis -- it seems to have missed a lot of the gray literature, particularly those studies published on animal advocacy websites -- but that's a much higher bar than I need for it to be part of my research toolkit.