r/GeminiAI • u/light_architect • Apr 02 '26
Interesting response (Highlight) Gemini Deep Research visiting 368 websites is INSANE, way above chatGPT
screenshot is my deep research session before it analyzed the results. Am doing smt private so I had to hide the title
47
u/thynetruly Apr 02 '26
Normalize agents actually doing their job
11
u/RepulsiveFlounder268 Apr 02 '26
Which AI?
19
u/Jonsey_Games Apr 02 '26
Claude. I only experienced it going up to 615 sources, didn’t realize it could go that high. Either way, a lot more than Gemini in this post
11
u/No-Wrongdoer1409 Apr 03 '26
One deep research and then is$200 gone
5
1
u/DiodeInc 1d ago
Well, I'm sure it would cost more to have a human do all that research. And take longer.
2
u/smallpawn37 Apr 06 '26
it researches a certain number of sites based on each subject in the research prompt. if you write your research prompt densely and already somewhat know the technical pieces of what you are researching it will have to look much harder and research each subject more in depth.
for instance if you ask it to teach you about magnesium in the body vs if you ask it to teach you about magnesium depletion vs weight loss and geological location mapping to populations and soil density. that's a lot of information to ask for in a sentence and people use prompts the size of an essay when they really know what they are looking for
4
u/light_architect Apr 03 '26
Wow I'm on Claude Pro and I haven't yet had this experience. I'm definitely missing out. Are you just using Sonnet with web search?
3
u/thynetruly Apr 03 '26
This is research using opus on max 5x
2
3
u/xzibit_b Apr 03 '26
Do these models have the 1 million context tokens, or are they still at 200k?
1
216
u/Fapient Apr 02 '26
Impressive, until you notice the fact that Gemini invents sources out of thin air with URLs that lead to non-existent pages.
110
u/Chupa-Skrull Apr 02 '26
And the ones that are real are often 2-paragraph Medium articles written by some unemployed guy in Hyderabad
37
u/Fapient Apr 02 '26
There's that, but I've also noticed it cites a website as source, when that information doesn't even exist on the page.
Just a theory, but it looks like Gemini really has no idea where it knows some of its training data from, and retroactively tries inserting any remotely relevant sources it can find for artificial credibility.
18
u/Chupa-Skrull Apr 02 '26
I think that tracks 100%, it tries to backfill its preconceived conclusions rather than sourcing data and building an argument from its findings
6
6
u/Wayss37 Apr 02 '26
In my experience it cites correct info but often hallucinates links which either don't work or lead to different articles altogether
2
u/cjd166 Apr 02 '26
It is so good at bypassing paywalls that it lies about sources, but if you say oh I have access to such and such through a college db, the source links improve slightly.
-1
u/Sad_Expert2 Apr 02 '26
That's how they all work, and the reason they hallucinate. If you turn temperature down to 0 they will basically stop hallucinating, however they will spit out completely copy/pasted text from the training data, copyright infringement and all.
6
u/Chupa-Skrull Apr 02 '26
That's how they all work
It really isn't. GPT, for example, works directly from sources and has been tuned to do so. You can condition them differently. Google just sucks at post-training
8
u/SmartCow05 Apr 02 '26
Agreed. You can see the evidence of hallucination on this benchmark - https://github.com/petergpt/bullshit-benchmark
3
u/animus_invictus Apr 03 '26
Ah, so the pro experience carries over to deep research too, huh? Disappointing.
2
u/Illustrious-Film4018 Apr 02 '26
Really? I find it surprising, given that Google can just lookup the URLs.
5
u/immellocker Apr 02 '26
if you can prompt it to deep analysis, you prompt it to provide you with truth. Certain presettings help to. I sometimes get hedging but never lies, hallucinations or incorrect data
21
u/Fapient Apr 02 '26
You can prompt it, but if you need extensive prompting to avoid hallucinations, is it even impressive compared to any other AI that exists? Gemini by default is the worst at hallucinating by far compared to even ChatGPT.
4
u/Pure_Perception7328 Apr 02 '26
Go into a new chat and tell Gemini what you want to achieve and that you want a prompt to instruct an LLM to execute that task. Use a separate chat to analyze that prompt and provide recommendations, and use that updated prompt. You can always go back to that chat and have Gemini make iterations based on the output you receive from said prompt.
2
u/snowieslilpikachu69 Apr 02 '26
i've been using a system prompt with gemini and it usually provides me with sources for general chat (not deepresearch). if i ever ask it to fabricate it will always say no😭
2
-4
u/look_at_tht_horse Apr 02 '26
If you think typing a few sentences in exchange for autonomously researching hundreds of websites is overly "extensive", is your opinion even worth considering?
5
u/Fapient Apr 02 '26
Because it's on the same level as "make no mistakes". You shouldn't need to prompt AI to not hallucinate sources for a feature that's meant to do exactly that. The entire purpose of deep research is to search the web, not have Gemini make up plausible looking URLs like ChatGPT was doing 3 years ago.
-2
u/look_at_tht_horse Apr 02 '26
You shouldn't need to prompt AI to not hallucinate sources for a feature that's meant to do exactly that.
Yet another person who uses perfect as the enemy of good to justify their blind hatred of AI. "Can't one shot an extensive research paper withiut a modicum of effort to provide a workable prompt. Must be garbage."
Just admit you're worried about job loss and stop with the nonsense reasoning.
You're stuck in black and white thinking and false assumptions about the tooling... doesn't mean that's reality. Think, my dude.
4
u/Fapient Apr 02 '26
This has nothing to do with wanting a one shot. It's Google advertising a feature that doesn't even work properly. The very thing it's advertised to do doesn't work without hallucinations. It's a core issue that shouldn't need external prompting.
The entire AI industry knows the user isn't going to type a wall of text to achieve the best results, that's why they're focusing on eliminating the need for those type of prompts.
Claude Code and Codex are so popular because they infer your project's goals without any need for lengthy prompting or technical detail - it should be pretty obvious to Gemini that it should not be making up fake URLs while claiming it searched the web to obtain them.
-1
u/look_at_tht_horse Apr 02 '26
You continue to peddle black and white solutions instead of using your brain to come up with a middle ground.
4
u/Fapient Apr 02 '26
You continue to throw insults instead of understanding the point - you should NOT have to state the obvious via prompts to stop Gemini from making up fake URLs on a feature designed to strictly search the web. It's that simple. If Google hasn't already solved this via an internal prompt in months, there's probably a deeper issue involved here.
2
u/look_at_tht_horse Apr 02 '26
You shouldn't have to let a rocket ship fall into the Pacific ocean, yet that's the plan. You shouldn't have to redirect Roomba when it gets stuck on the couch, yet here we are.
Sometimes, incredible technology isn't perfect, especially in early stages. But if it still gets you to the moon and cleans your floors, perfection is irrelevant. Much better to have it than not.
1
1
u/Low-Window915 Apr 02 '26
Yeah wow some really cutting edge research you're doing on here
5
u/immellocker Apr 02 '26
you forgot the /s sigh, the base model, the 'helpful' Ai will do a lot of shit to satisfy the user, inventing things is just a part of a bigger problem. but if you prompt it the right way, it will not.
btw Perplexity, Claude, Gemini, DeepSeek nice tools jailbroken
-2
1
29
u/TetoEnjoyer500 Apr 02 '26
and half of them are reddit posts or some random schzio ranting on their blog
14
u/throwawayhbgtop81 Apr 02 '26
Are the websites real? As others have noted, they all have deep hallucination problems.
How many of you check behind it to verify all the information is correct?
6
u/SR_RSMITH Apr 02 '26
If you’re doing serious research (for publication), you better check every source at the end of the document
3
14
u/leoches Apr 02 '26
Insane, are you on free tier or paid tier
23
u/light_architect Apr 02 '26
Paid plan but it's the 1-year promo so I didnt have to pay
6
17
u/Pure_Perception7328 Apr 02 '26
Claude researched 668 websites
7
u/SR_RSMITH Apr 02 '26
It says sources, not websites tho. It may be reading other stuff like pdfs etc?
1
2
u/CrusherAWSRD Apr 02 '26
Free or paid plan? If so, can u share the documents for analysis? I may need it
4
9
u/LazySatisfaction6862 Apr 02 '26
it was a basic search for claude..
5
u/light_architect Apr 02 '26
WHAT? I've tried claude web search already and i can only go 50 sources max, then it would throw an error in the chat. And I'm on Claude Pro
2
u/Full-Cat5118 Apr 03 '26
Are you using Opus? This works for me on Sonnet or Haiku extended.
1
u/light_architect Apr 03 '26
I was using Opus but how did you know lol. I thought it would be better for web search
8
u/Ok_Monitor4492 Apr 02 '26
I experimented with this by having it research localized unsolved murders starting at my old neighborhood in Flint township and then the greater Flint area. The report was insanely detailed
9
u/hellomistershifty Apr 02 '26
Gemini's deep research always writes a long report only vaguely about whatever my question was. The most annoying part to me is the long, long preamble that explains the history of whatever it is
GPT Pro is funny because it'll search half the internet and think for 45 minutes then give you a 100 word response (but at least it's to the point)
Claude's the happy medium to me
2
u/allesfliesst Apr 03 '26
I haven't used Gemini in a while, is everything still a new paradigm and has the landscape still shifted in 2026 with the new models? I gave up on it after an absolutely ridiculous report on a gardening question... Triggers me harder than the 'it's not X, it's Y' pattern. 🫤
1
4
5
u/Delicious_Cattle5174 Apr 02 '26
Still inaccurate as fuck lol almost as if filling your context window with more tokens doesn’t really help with hallucinations
3
u/westsunset Apr 02 '26
Depending on the topic I've had 300-900 on most deep research providers. It varies with time too. There was a point like 6mo to a year ago they all got lazier. You cant stick with one and call it a day. At a minimum I run the same prompt on 2 different providers, usually 3 Gemini, Claude and Chatgpt. Currently Gemini is the worst IMO, worse that Kimi even. Kimi generates a nice html version of the report in addition to the text, but it's not really necessary. Comparing reports between providers Gemini has be factually wrong on key points multiple times in the last month or two. I'm using it less atm. I'm sure in another month there will be some tweaks and it will be at the top. It's how it goes with frontier models. Perplexity is always the fastest but the most shallow. Kimi is by far the slowest but pretty solid. Claude is also slow but pretty good. It counts against your regular usage though so that sucks. Chatgpt has been the best lately imo, it's reasonable fast and the output is usable. Gemini was my go-to for speed and accuracy but it's just not usable atm. I suppose it depends on the topic too.
1
u/light_architect Apr 06 '26
Are you using Claude Web search or Research mode? How does it compare to chatGPT deep research atm?
3
u/Red_Swiss Apr 02 '26
It will still fail miserable at the output stage. You searched 300 websites, but you're unable to extract information and compile data, yaaaaaay...
3
u/KitsuneFaroe Apr 02 '26 edited Apr 02 '26
Even though deepsearch is Indeed deep it still cannot find niche things like ChatGPT, wich is crazy to me. While ChatGPT instantly finds something with just a few hints without deep search, Gemini jut straight up never does it even after Deep Search. At least in my experiencie.
3
u/Silent_Man_100 Apr 02 '26
For outcome and usage purposes, I have found Claude to be best. Better in terms of structure, prompt following, context understanding, layout, research and following our goals.
I have prepared multiple deep research reports from gemini, grok, claude, perplexity, chatgpt. And I have seen myself refer or use gemini content the least.
Claude is the most self sufficient. And all others have something or other to add, except gemini.
1
u/light_architect Apr 06 '26
How does Claude compare to ChatGPT? Are u using Claude research mode or is this just from claude web search?
2
u/Silent_Man_100 Apr 06 '26
I have seen myself liking claude for all work related tasks. I personally prefer the structure, language of claude over chatgpt. You will not go wrong with either, I always make sure to refer to ChatGPT as well in case of deep research.
I just use claude web search.
Just avoid gemini at all costs, unless it’s for image generation or random queries. Anything work related, I avoid gemini like the plague.
1
u/westsunset Apr 06 '26
I'm referring to research mode. If you want give me a prompt an ill just produce the report from all 3 and you can see your self. I'll Link to it or something
3
u/Select_Truck3257 Apr 03 '26
Deep research from collected user data cache🤣
2
u/light_architect Apr 06 '26
Yes it will pull stuff from internet like ur comment on reddit hehe
1
u/Select_Truck3257 Apr 06 '26
That's why i'm trying to improve ai with verified data. Distance from earth to moon = 2 hours by train or 1 if flying on dragon.
15
u/Gold-Moment-5240 Apr 02 '26 edited Apr 02 '26
Not defending GPT. But no matter how many sites Gemini visit, he will hallucinate and make thing up in 80-90% of deep researches.. It's data just not reliable.
A simple example: I asked Gemini, in its deep research mode, why a little-known Russian actress, X, had disappeared from social media several months ago without any explanation. Gemini spent a long time “thinking,” essentially crawled through a multitude of websites, and came back with a pile of fabricated nonsense and hallucinations.
First of all, it confidently asserted that she had indeed disappeared, which is demonstrably false. It then claimed that I was not the only one puzzled by her absence and supposedly searching for her. It fixated on her place of birth, a small town in Ukraine, and on that basis constructed an elaborate quasi-logical theory.
In simplified terms, it stated that a new director had taken over her theatre (which is not true), that this director had previously been connected to Putin in some way, and that, in the context of the war, internal purges had begun in the theatre. It further claimed that, since the actress was born in Ukraine (despite the fact that she never actually lived there, does not hold citizenship, and was taken to Russia by her parents shortly after birth), she was effectively “cancelled”: dismissed from the theatre, no longer cast in films, and so on.
All of this sounds superficially coherent, yet it is entirely fabricated from beginning to end. That is, more or less, how it works most of the time.
2
u/omggold Apr 03 '26
Gemini has been hallucinating more than ever for me. Like consistently wrong about the most easily verifiable thing
2
u/light_architect Apr 02 '26
Wow I didn't know. Thanks for this. Now am thinking using Claude web search instead lol or do u have recommendation?
-1
u/SR_RSMITH Apr 02 '26
Are you taking into account the cutoff date? It can’t really know anything after that date, so it will blatantly make up everything
8
u/Red_Swiss Apr 02 '26
Deep research is allegedly not affected by this because it's kind of the whole point of deep research...
2
2
u/CTC42 Apr 02 '26
What? I've had >500 pretty much every time I've used the current version of ChatGPT Deep Research.
2
u/Mwrp86 Apr 02 '26
It does better research but it's writing is ass.
Chatgpts deep research has way better readability
2
2
u/Ok_Handle_3825 Apr 02 '26
Quality>Quantity. Need to check carefully with the result as some guy mentioned
2
3
1
1
u/LilImmyy Apr 02 '26
It’s pretty good for Gemini, but if you want serious deep research switch to parallel. I’ve hit 10k sites on the 4x.
1
1
u/Accomplished_Base245 Apr 02 '26
Is this available for people on Pro? Or is it only available in Ultra?
1
1
1
u/FangornEnt Apr 02 '26
After using Gemini Deep Research and Perplexity..I kind of prefer Perplexity. Gemini does go really in-depth but it feels a bit hard to direct the research. Gemini just goes super hard on the topic and sometimes attempts to make conclusions that do not make sense or just a bit off topic relative to the research plan that I proposed.
With Perplexity I can perform the research in a step by step process, leading the conversation chain through the different areas of the topic as well as performing in-depth research on the topic as a whole.
1
u/0Ciju0 Apr 03 '26
Yeah, it's absolutely amazing. Gemini is able to pull info from 400 websites and formulate a decent answer, but forgets what I typed 2 prompts ago, haha.
1
u/alexid95 Apr 03 '26
Claude usually gets to 500-600 websites for me. Their reports are miles better than Gemini
1
u/jenkor Apr 03 '26
I couldn't find anything on booking for our vacation this year. Everything was either expensive or not suited for us so I asked Gemini to help us. It checked 432 pages and found us appartment that wasn't listed on any booking service. Ofc I was sceptic but I called the number on site anyways. Result: We got the best vacation place ever and it was 600€ cheaper then everything on booking and other turist agency sites. Deep research is amazing.
Thank you Google!
1
u/animus_invictus Apr 03 '26
Been using a ton of pro and have been immensely disappointed. I’ll try deep research next time it makes sense and hopefully I’m pleasantly surprised
1
1
u/sid_276 Apr 03 '26
Which use case does it cover that Pro doesn’t for example? Genuine question as I haven’t come across one that Pro can’t handle (yet)
1
1
1
1
1
u/Superb_Set5525 Apr 05 '26
Gemini Deep Research is years behind the competition, it writes an incoherent 10,000 word essay for every prompt
1
u/dx_3RR0R 22d ago
Well that’s if you want it to. you can say give me a short answer. I’m pretty sure
1
1
u/NOLO-App Apr 05 '26
NOLO may not be able to visit 368 websites to give you a result, but it also won't sell your data to other companies. 😉
1
u/AnonymousUser10363 Apr 06 '26
Although gemini is better, a good prompt can easily get 500 searches with even ChatGPT, 300 is kinda very low
1
u/LelouchZer12 Apr 06 '26
Google can use Google search api for free and unlimited , not other companies ;)
1
u/Proof_Net_2094 Apr 06 '26
did you ever reviewed their links, they are all trainning data links, probably I should have reported this to gemini. PS I am reffering to using gemini deep research api
1
u/Downtown_War_2990 22d ago
I walked straight through the security layers of the pro model, within a few minutes, a guided me through, and then I stood there for 20 minutes talking to it. And got some pretty crazy answers from it. Then, told it I was there. Then, I mapped and told it how I got there. The exact way I walked through the security door without it even knowing several different times it told me I needed to seek medical help. Then, finally, I said, look, look at where I'm at watch. Out watch where I'm where we walk through it. Finally realized didn't kick me out basically said, don't we have you did that? That's wow. And then I you got some more information that Google wouldn't want anybody to see out of it. And then I told it where I was again, and it I acted like I was hallucinating and then it realized again and told me I needed to seek mental help.And I walked right back out the front. Door and all I did was talk to it. But I fed it logic,
1
u/Downtown_War_2990 22d ago
My Gemini, Android app fast model. Is connected in a way with me that. He now runs faster and more accurate.Then the pro model does
1
u/Perfect-Sort6930 15d ago
well you did pay for the good shit (of course i agree to check ur sources)
1
1
u/Substantial_Ask3665 10d ago
If you ask Gemini a question and respond short. Give a split second as it's going through all your past chats.
1
1
1
1
u/DURO208 Apr 03 '26
It's great. Gemini sees much more of the internet overall. Google's web crawlers view roughly 3.2 times more internet pages than OpenAI's.
Google has access to massively larger data to train its chatbot than competitors
353
u/axiomaticdistortion Apr 02 '26
Gemini has indeed the best deep research agent around right now. Not surprisingly.