Latest articles

RSS
  1. Technology

    Repeated local-LLM tests separate average success from consistency

    This personal experiment covers 100 English tasks, not overall model quality or production reliability.

    • AI
    • Evaluation
  2. Technology

    Gemini 4 Carbon posts lower task costs in the GDPx evaluation

    Rankings change with the task mix and safety restrictions, so compare models on the work you actually do.

    • AI
    • Evaluation
  3. Technology

    StreamVibe evaluates when a video assistant should offer help

    The research studies per-second silence-or-response decisions; it is not a finished consumer assistant release.

    • Video AI
    • Evaluation
  4. Technology

    Memdoor's role-separated coding workflow passes 49 of 50 repeated tasks

    The ten-problem sample and possible contamination limit model comparisons despite a concrete multi-agent design.

    • AI Agents
    • Evaluation
  5. Technology

    Samaya reports financial-research gains from its smaller Sterling agent

    The results reflect joint model-and-harness tuning, not investment returns or general superiority.

    • AI agents
    • Evaluation
  6. Technology

    Game experiments reveal the coordination costs of AI swarms

    Outcomes depended on models, tool interfaces and execution environments.

    • Agents
    • MultiAgent
    • Evaluation
  7. Technology

    Data-agent evaluation needs more than correct SQL

    Real analysis combines data discovery, domain meaning and auditable evaluation.

    • DataEngineering
    • Agents
    • Evaluation
  8. Technology

    LLM research-novelty judgments shift sharply with prompt design

    Ai2 released evaluation data and code while highlighting unstable judgments

    • AI
    • Evaluation
  9. Technology

    Korean RAG test finds no detectable improvement from Nemotron Bolt E4B

    Thinking mode raises both accuracy and latency in a 299-question comparison

    • AI
    • RAG
    • Evaluation
  10. Technology

    A developer asks how AI memory tools perform beyond benchmarks

    A request for production experience, not a published product ranking

    • AI
    • Memory
    • Evaluation
  11. AI

    Coding-harness study finds clearer cost gaps than success gains

    Rerunning an unchanged setup helps distinguish small performance claims from measurement noise.

    • Evaluation
    • Coding agents
  12. AI Engineering

    Informal test groups 32 of 38 human-opinion answers from 13 AI models together

    Convergence in a small prompting exercise proves neither the claim about humans nor an intrinsic model belief.

    • AI
    • Evaluation
  13. AI Engineering

    Qwen multiple-choice experiment shows high token probabilities can still be wrong

    Restricting outputs to predefined answers does not ensure accuracy; probabilities need calibration against actual correctness.

    • AI
    • Evaluation
  14. AI Engineering

    Review of 857 Chinese AI models finds published safety results for 31

    Missing public evidence is not proof of missing internal tests; adoption decisions should check model-specific evaluations and disclosure dates.

    • AI
    • Evaluation

You’ve reached the end.

Looking for something specific? Search this section
Search

What are you curious about?

Find stories by title, topic, source or anything in the article.