01
Benchmarking Autonomous AI Agents: A New Class of AI IQ Task Testing
Podcast Overview: https://notebooklm.google.com/notebook/7854b500-3f35-404e-b0cf-fa8b90665b3c/audio
Series & reading path
Deep Research & Model Benchmarking
Comparative tests of reasoning models, reliability, hallucination, and research performance.
02
OpenAI's Deep Research and the Evolution of AI-Driven Knowledge Synthesis
The advancement of artificial intelligence (AI) continues to reach next milestones with OpenAI's introduction of deep research, an AI-driven agent powered by the highest and newest powerful 'reasoning model' OpenAI's…
03
Grok 3 vs. Claude Sonnet 2.7 AI Model Hallucination Smackdown and Doublecheck (Reasoning & Narrative Writing Remix).
This benchmarking experiment takes an originally 'human written' brief essay/post on AI model hallucination to produce a number of narrative essay versions of an improved and doublechecked more stylistically elegant…
04
The Trouble with Grok 3: AI, Hallucinated Citation Sources and LLM Model Hallucination
While Grok 3 is a very powerful 'reasoning' model, it still hallucinates like a sailor. This benchmarking test asked Grok to compose a New Yorker type intellectually robust narrative essay on AI Model Hallucination…
05
AI Model Hallucination and Human Understanding
Dr. Elena Torres sat in her cluttered office at the MIT Media Lab, staring at her laptop screen. The words glowed back at her with a quiet audacity: “The capital of France is Berlin.” She let out a soft chuckle—not…
06
AI Deep Research Financial Modeling Stock Market Study (Perplexity AI Deep Research Benchmarking)
This AI benchmarking study conducted March 12, 2025 creates a prompt for advanced stock market financial modeling/(Stock Market Option Trading, Sell Side 'Put Options) to benchmark AI Deep Research Models (2025). A…
07
DeepSeek R1 Financial Modeling, Reason Model Benchmarking Comparison (With Thinking Window)
Original Prompt: Create a 20 page research report on selling puts as a strategy in the current market. Research best practices by wall street professionals but also academic researchers. Look at macro and…
08
Hallucination, Financial Modeling, AI Benchmarking and New Hybrid AI Reasoning Models: Sonnet: 3.7
The parent company, Anthropic, describes Claude 3.7 Sonnet as its most intelligent model to date (February 2025), calling it a "hybrid reasoning model" that can generate near-instant responses or engage in extended,…
09
Deep Research: Gemini, Derivatives Market, Benchmarking Study
This study currently surveys the top reasoning, hybrid reasoning and 'deep research' models: Grok 3, GPT 4.5 Deep Research, Sonnet 3.7, Deep Seek R1 and Google Gemini Deep Research with a complex query regarding…
10
OpenAI 4.5, o3mini + Deep Research Benchmarking Study (2025). Financial Derivatives Modeling: An Ultra Conservative Strategic Approach
This study currently surveys the top reasoning, hybrid reasoning and 'deep research' models: Grok 3, GPT 4.5 Deep Research, Sonnet 3.7, Deep Seek R1 and Google Gemini Deep Research with a complex query regarding…
11
Grok 3 Deep Research Benchmarking: Financial Derivatives: Back to the Basics
This study currently surveys the top reasoning, hybrid reasoning and 'deep research' models: Grok 3, GPT 4.5 Deep Research, Sonnet 3.7, Deep Seek R1 with a complex query regarding stock market financial derivatives…
12
GPT o3 'Deep Research' Benchmarking: US Stock Market Cycle Analysis
This AI Benchmarking test and produced article reviews Open AI's top 'reasoning' model 03 (released April 16, 2024) in tandem with it's deep research abilities. It asks a complex stock market analysis question…
13
AI Deep Research Capabilities and Men's Health: Claude Sonnet 4 vs GPT-4 Mini High Benchmarking
In an era where artificial intelligence increasingly serves as a sophisticated research assistant, this study examines the deep research capabilities of two leading language models—Claude Sonnet 4 and GPT-4 Mini…
14
AI Deep Research Interleaved Thinking: 5× Faster and More Intelligent Answers
We’ve all been there. You ask an AI model a complex, multi-step question and then watch the cursor blink, waiting. This delay, known in the research world as a "significantly increased time-to-first-token (TTFT),"…
15
Beyond the Valley of Science AI Benchmarks: A New Framework for Evaluating Large Language Models in Scientific Discovery
Large Language Models (LLMs) are rapidly evolving from general-purpose tools into specialized partners in scientific inquiry. Across disciplines, they are beginning to accelerate core stages of discovery, assisting…
16
Meta-analysis of Meta-analysis: AI and Deep Research
This is the third analytic and final meta-analysis of a report on Opus 4.5 Deep Research. This is accomplished in Gemini 3 Pro and part of a three part AI research benchmarking series on AI academic research,…
17
AI LLMs 2026: The Leading Edge of the Jagged Frontier
The Capability Strength Matrix I've been building and iterating across the past several months tracks nine frontier and contender AI models across six distinct cognitive dimensions. It is not a leaderboard. It is a…