Discussion about this post

User's avatar
Dov Jacobson's avatar

I fear sycophancy more than, say, hallucination. Enthusiastic endorsement is great for endorphins, but when I am pointed in the wrong direction, I need to be told - unwaveringly.

Fortunately, I am married.

MCJ's avatar

This is an excellent, thoughtful guide, and I especially appreciate the section on Deep Research and the importance of connecting the AI to your data.

However, I'm surprised there was no mention of a critical issue related to using advanced AI for deep, specialized research—especially when dealing with less-common or proprietary texts.

The point is this: if a user starts asking complex questions about a particular, lesser-known document or text that the model hasn't been explicitly trained on, the quality of the "Deep Research" or "Thinking" response will likely be slop unless the user can constrain the LLM by directly uploading a copy of the relevant text. Without that constraint, the model's instinct to "fill in the blanks" with a web search still often leads to confident, but ultimately inaccurate, answers. As a college professor, I'm seeing students fall into this trap way too often, with disastrous outcomes when they replicate the fake information for an assignment. (Relatedly, if you seed an LLM with your own notes, they will often erroneously put your own words in quotes, as if they were from the primary text, leading to similar reliability issues.)

In my experience, tools that specialize in this document-grounded Q&A—like NotebookLM—remain the superior choice for this specific use case, precisely because they are designed to limit the model's scope to the uploaded text and are least likely to go off-script by searching the web to fill in lacunae. It feels like an important nuance when discussing "Getting better answers."

77 more comments...

No posts

Ready for more?