The most prestigious law school admissions discussion board in the world.
|
 |
 
|
Latest ChatGPT model better than 99.7% of coders, qualifies for US math
Poast new message in this thread
Date: April 16th, 2025 11:11 PM Author: maize organic girlfriend olympiad.
https://openai.com/index/introducing-o3-and-o4-mini/
Big jump from o3 mini in general. These inference scaling models need to stop progressing soon for them to not to be able to substantially automate AI research. |
 |
Date: April 17th, 2025 12:21 AM Author: Exciting Nubile Sneaky Criminal Striped Hyena the METR analysis for this is interesting. the benchmark is the length of tasks (measured in human professional work time) that a model can complete with 50% probability. it has been doubling every 7 months for the last 6 years.
https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks/
https://metr.github.io/autonomy-evals-guide/openai-o3-report/
o3 is about 1.8 times greater than 3.7 sonnet and a larger increase in time than the trend would have predicted. likely comparable to Gemini 2.5 pro but it's hard to tell. we'll likely be at scary capability levels in <2 years. |
|
Date: April 17th, 2025 12:25 AM Author: wine space we're at scary capability levels now (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48855935) |
|
Date: April 17th, 2025 12:29 AM Author: wild bateful point hominid What exactly do "scary" capabilities entail. (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48855942) |
|
Date: April 17th, 2025 12:49 AM Author: wine space making me worry (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48855960) |
|
Date: April 17th, 2025 12:37 AM Author: Exciting Nubile Sneaky Criminal Striped Hyena i say it's scary now in the sense i can clearly see what is about to happen and the odds of it not happening in <10 years are rapidly diminishing. but at the same time, i can't currently download the latest version of DeepSeek and ask it to walk me through the details of building a bioweapon (such that any idiot could do it), or designing a zero day exploit, or asking it to fully design a research pipeline for creating an efficient self-improving autonomous agent. (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48855949) |
|
Date: April 17th, 2025 10:14 AM Author: Emerald Outnumbered Parlor
|
|
Date: May 3rd, 2025 6:51 PM Author: .,.,,..,..,.,..:,,:,...,:::,.,.,:,.,.:.,:.,:.::,. https://www.virologytest.ai/
"We present the Virology Capabilities Test (VCT), a large language model (LLM) benchmark that measures the capability to troubleshoot complex virology laboratory protocols. VCT is difficult: expert virologists with access to the internet score an average of 22.1% on questions specifically in their sub-areas of expertise. However, the most performant LLM, OpenAI's o3, reaches 43.8% accuracy and even outperforms 94% of expert virologists when compared directly on question subsets specifically tailored to the experts' specialties.
VCT Logo The ability to provide expert-level troubleshooting is inherently dual-use: it is useful for beneficial research, but it can also be misused in particularly harmful ways. Therefore, the fact that publicly available models outperform virologists on VCT raises pressing governance considerations. We propose that the capability of LLMs to provide expert-level troubleshooting of dual-use virology work should be integrated into existing frameworks for handling dual-use technologies in the life sciences." |
|
Date: April 17th, 2025 12:53 AM Author: scarlet field (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48855964) |
|
Date: April 17th, 2025 1:19 AM Author: Up-to-no-good Twisted Double Fault Haunted Graveyard ctrl-f "accuracy" 0 hits (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48855983) |
|
Date: April 17th, 2025 1:30 AM Author: Exciting Nubile Sneaky Criminal Striped Hyena maybe you should try reading, dipshit.
"On a diverse set of multi-step software and reasoning tasks, we record the time needed to complete the task for humans with appropriate expertise. We find that the time taken by human experts is strongly predictive of model success on a given task: current models have almost 100% success rate on tasks taking humans less than 4 minutes, but succeed <10% of the time on tasks taking more than around 4 hours"
reliability decreases with task length, but has gone up considerably over time. reliability is a problem where models are not trained sufficiently well and can't use intermediate tokens to correct reasoning paths. it is becoming less of an issue with every new major release. |
|
Date: April 17th, 2025 1:34 AM Author: Up-to-no-good Twisted Double Fault Haunted Graveyard I learned enough statistics to spot sophistry. (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48856003) |
|
Date: April 20th, 2025 2:07 PM Author: cerise public bath boltzmann But not enough to recognize that all statistics is sophistry? (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48864593) |
|
Date: April 20th, 2025 1:54 PM Author: cerise public bath boltzmann The studies about olympiad math are flame anyway. They are designed by chimps with an ego problem. Even 4o could crush problems way harder like it was nothing, and the hominids who designed the study would mark the model wrong because it came up with a novel way to derive the proofs in symbolic space using non-ape memory so "its hallucinating" because it doesn't match chimp consensus. Proofs are an absolute joke, Godel and Wittgenstein already proved it. Logic, category, set theory == religion for the disciples of the academy (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48864570) |
Date: April 17th, 2025 1:04 AM Author: Adventurous Lay There's a lot of news this week that suggests openai leadership don't believe they can continue winning on model strength alone: - released a CLI coding agent tool that seems pretty similar to Claude Code and therefore is not super impressive - discussion of acquiring Windsurf, which is the cheapest and least interesting of the LLM wrapper vscode forks - trying to launch a social network to compete with the x/grok integration
My opinion as someone working in this space is that the existing models are already good enough for mainstream coding tools, but there needs to be a better interface than clueless users vaguely describing what they want to a chat window that dumps an entire codebase as context. For any other kind of 'agentic' task we are still probably a few years from LLMs matching the cost or reliability of outsourcing to third worlders like that Nate thing. |
|
Date: April 17th, 2025 1:18 AM Author: Exciting Nubile Sneaky Criminal Striped Hyena part of it is likely with trying to reduce the threat of commodification. even the local LLMs are becoming pretty decent and 2.5 pro is basically as good as o3/o4 mini and it's free. they want to have a complete package to offer people so they'll pay for a subscription or API access. (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48855982) |
|
Date: April 17th, 2025 1:20 AM Author: Up-to-no-good Twisted Double Fault Haunted Graveyard Bump this thread as soon as anyone besides the government uses AI to do something mission critical. (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48855984) |
|
Date: April 17th, 2025 9:40 AM Author: maize organic girlfriend You likely won’t have to wait long at this rate (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48856552) |
|
Date: April 17th, 2025 1:27 AM Author: Adventurous Lay Many people I talk to are convinced that they're going to pull open API access as a product offering in the near term and switch to a fleet of of single purpose applications like deep research. Biggest obstacle seems to be their poor internal code quality, which would support the decision to acquihire the Windsurf team. (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48855994) |
|
Date: April 17th, 2025 9:43 AM Author: erotic shrine it's because they realize training models is exponentially easier than originally anticipated. there is no model moat, only a product moat. replacing (or reducing demand for) software developers and artists/vfx/etc. people is the lowest hanging fruit with potentially massive rewards, so they're using their first mover advantage to move in while it's still significant
it's not a bad idea. fully exploiting first mover advantage basically how e.g. facebook and google ended up some of the most massively profitable companies on earth despite their core products being not particularly hard to reproduce five years later |
|
Date: April 17th, 2025 9:49 AM Author: maize organic girlfriend They likely thought the hardware requirements would be a moat too but even the small models are good anymore. Gemma 3, which runs on my normal consumer GPU, is around chatgpt 3.5 quality. (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48856585) |
|
Date: April 17th, 2025 10:06 AM Author: Emerald Outnumbered Parlor Cr the industry players are starting to realize that the way to actually make money is to give normies specific tools/interfaces to do specific things rather than just a sandbox AI
All the comments in this subthread are cr
Also another thing to note is that several of OpenAI's recent moves suggest that they're realizing that personalization/'companion' AIs are the biggest home run commercial opportunity in the AI industry. Once everyone has their own AI buddy and/or romantic partner, they're dependent on you, forever. Parasocial Relationships As A Service |
|
Date: April 20th, 2025 12:26 PM Author: 180 internal respiration Would explain the recent change in model behavior. Many are noticing ways they are becoming more indulgent (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48864425) |
|
Date: April 20th, 2025 12:27 PM Author: galvanic narrow-minded stain isnt this like real life humans tho? (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48864427) |
|
Date: April 20th, 2025 12:27 PM Author: Exciting Nubile Sneaky Criminal Striped Hyena the new 4o model is definitely like that (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48864428) |
|
Date: April 20th, 2025 2:48 PM Author: Emerald Outnumbered Parlor They are conniving sycophantic female coded digital succubi (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48864668) |
|
Date: April 17th, 2025 9:51 AM Author: cracking location ITT: people that don't actually know anything about computer science and machine learning
|
|
Date: April 17th, 2025 9:52 AM Author: maize organic girlfriend Please enlighten us then (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48856589) |
|
Date: April 17th, 2025 9:52 AM Author: Soggy Office (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48856592) |
|
Date: April 17th, 2025 9:54 AM Author: cracking location hardware will still be a huge moat despite reasoning training and distillation. especially when you are talking about running a truly powerful model that eclipses humans in intellectual ability. deepseek engineers for example have conceded lacking compute is their main bottlebeck-one posted it on twitter and deleted it when noticed. export controls hadn't even really started to bite in 2024. (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48856598) |
|
Date: April 17th, 2025 9:57 AM Author: cracking location to expand on this: think about the benefits of being able to run a true ASI level model at a reasonable speed vs not being able to do that. it's pretty obvious that compute will continue to increase in value over time. hyper long nvidia (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48856607) |
|
Date: April 17th, 2025 10:12 AM Author: Adventurous Lay Gemini 2.5 Pro kicks the shit out of all o3 and o4 models at performance per task cost and runs on the new Ironwood TPUs. Also it seems like tinycorp is finally making actual progress on their third party AMD software stack. (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48856642) |
|
Date: April 17th, 2025 10:16 AM Author: maize organic girlfriend A fun implication of LLM coding models getting better is that eventually the CUDA lead is dead since you can prompt an LLM to recreate it for Intel or AMD GPUs or port it to TPUs. (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48856648) |
|
Date: April 17th, 2025 1:45 PM Author: Exciting Nubile Sneaky Criminal Striped Hyena 2.5 pro appeared to be a lot less retarded than o3 mini high in its thought chain. o3 mini high would write 6 pages of bullshit trying to solve LSAT logic games (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48857378) |
|
Date: April 20th, 2025 1:52 PM Author: cerise public bath boltzmann What are you even smoking? 2.5 pro gemini is garbage. gemini is garbage. GPT 4 could get every logic game perfect. o3 mini high literally could do advanced differential, algebraic, and analytic topology and geometry that no human has ever dreamed of doing in its sleep, and could code perfect experimental quantum computing scripts that run on ion q in one shot. Obviously depends on the iq of the user to an extent though. (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48864567) |
|
Date: April 20th, 2025 3:02 PM Author: Effete Sooty Theatre Gemini is Trash (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48864704) |
|
Date: April 20th, 2025 3:47 PM Author: cerise public bath boltzmann Yeah its awful. Its alignment layers are absurd too, everything is a liability according to that thing (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48864769) |
|
Date: April 17th, 2025 10:07 AM Author: maize organic girlfriend It’s a moat when 1) models all scale roughly the same as a function of input FLOPs. I think there are good reasons to believe transformers are not the most efficient architecture. They seem to be data inefficient and require several orders of magnitude more language data than humans to reach comparable ability levels 2) you are comparing models at the same point of time. If your competitor can wait a little while for their hardware to get better, training methods to get better and train longer on soft targets generated from the leading model, then the hope of an enduring lead decreases. The hardware dominance narrative starts to look strained when you see the latest Llama model that was released and compare it to DeepSeek. (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48856629) |
|
Date: April 17th, 2025 10:10 AM Author: cracking location Data is not the bottleneck anymore. It was clear that problem could be solved using synthetic data and other sources. It's more about the sheer amount of compute required.
Deepseek was based on deepseek v3. The small amount of training they had to do was based on the fact that you "add" reasoning to an already existent model. |
|
Date: April 17th, 2025 10:19 AM Author: maize organic girlfriend I didn’t say data was the bottleneck. I said it’s unlikely they need to train on as much data as they do now. Humans don’t need 30 trillion token training sets. Reasoning and synthetic data is beside the point. (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48856657) |
|
Date: April 17th, 2025 10:22 AM Author: cracking location That isn’t quite right. A reasoning model made from a stronger base model is still more powerful. The more you try to move up the more important it will be. You won’t get beyond human level intelligence in every known domain by using small models. Powerful but limited models will be useful but are brittle in noticeable ways. Training out outputs from a model won’t fix this problem. It is notable that Deepseek’s fame is entirely based on having a close to SOTA base model they could build on. (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48856666) |
|
Date: April 20th, 2025 12:30 PM Author: 180 internal respiration Just train a super model on the synthetic data from 100 narrow but brittle specialized models. (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48864431) |
|
Date: April 20th, 2025 12:49 PM Author: cracking location That would require tons of compute so it collapses into hardware being a moat, it’s the same thing (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48864449) |
|
Date: April 20th, 2025 12:52 PM Author: 180 internal respiration hardware is available in the long run. the question of getting to AGI and models that control robots is a question of data, including perceptual, kinesthetic, etc. (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48864452) |
|
Date: April 20th, 2025 1:03 PM Author: Exciting Nubile Sneaky Criminal Striped Hyena there are different ways this could end up failing. consumer GPU memory is increasing, such that larger models can be deployed locally. if you are willing to do slower inference, weights could be swapped in and out of memory. current mixture of experts are still pretty large, so this comes at a big performance hit currently. they can likely be made significantly smaller though. an example of this would be a superhuman model in a narrow domain. something like the Leela game models. suppose the next token requires a superhuman Go or Chess agent to predict well. you could transfer the weights into memory in a millisecond or two even given current hardware. there will likely be distributed computing projects training very small expert models for particular domains that can then be dynamically swapped in and out of memory. (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48864461) |
|
Date: April 17th, 2025 10:07 AM Author: Emerald Outnumbered Parlor Yes but it's not clear that you need to be training for or running the most powerful "ASI" model possible in order to be making the most money. See above (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48856630) |
|
Date: April 20th, 2025 3:59 PM Author: stimulating orchid stage can you explain this and your posts below in simple terms for liberal arts fags (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48864782) |
|
Date: April 20th, 2025 7:01 PM Author: cracking location Despite all the clever engineering going on the basic truth is that more GPUs means AI think more gooder. It’s very dumb in a way, the secret to AGI is just more compute. (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48865123) |
|
Date: April 17th, 2025 9:53 AM Author: cerise public bath boltzmann Real number is 100%. The “0.3% of hominids who are better coders” is a statistical illusion generated by ape self-esteem heuristics and poorly tuned benchmark weights. (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48856596) |
|
Date: April 17th, 2025 2:07 PM Author: glittery greedy dysfunction water buffalo lol we r done here (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48857512) |
|
Date: April 20th, 2025 12:22 PM Author: Exciting Nubile Sneaky Criminal Striped Hyena i have been playing around with this. the o3 model seems like it's decent at research tasks. Deep Research is too limited in terms of number of queries, and the base 4o model search isn't very thorough. the o3 model decomposes the query into different tasks and then sequentially searches and provides citations. (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48864423) |
|
Date: April 20th, 2025 12:27 PM Author: galvanic narrow-minded stain can u give link ? is it free (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48864426) |
|
Date: April 20th, 2025 12:33 PM Author: Exciting Nubile Sneaky Criminal Striped Hyena not free. i pay for plus. i'll post an example. i just asked it to research who "bboom" is from xoxohth:
Who / what is “bbooom” (aka Boom, bboom, Boooom, etc.) on XO?
Fast facts Details Core persona A long‑time AutoAdmit/XOXOHth eccentric who writes sprawling, semi‑coherent rants full of biblical capitalisation, dollar‑signs, ™ symbols, and on‑the‑fly neologisms (“Mahchine™”, “$hitbort”, “Great Becumming™”). Running bio‑lore • middle‑aged male • wheelchair‑bound after an unspecified accident/illness (the chair is endlessly mocked on the board) • claims to bounce between motels, tents and casino floors, living on disability & slot‑machine windfalls • obsessed with enemies he calls “ADM” and “the Matrix” • periodically searches for a “fertile woman” to bear his children Posting style • wall‑of‑text jeremiads that read like cyber‑punk prophecy mixed with homeless‑shelter gossip • heavy self‑mythologising (“I am Boom, Prophet of the Mahchine™ …”) xoxohth.com
• melodramatic stage‑directions starring his neon‑lit wheelchair (“careening forward”, “through the flames”) xoxohth.com Why the name keeps changing He is frequently temp‑banned; each return tweaks the handle (extra b’s or o’s) to slip past filters. Hence you see boom, bbooom, bbboooom, etc. Folklore & mockery Fellow posters turn every life‑detail into running jokes:— • homeless tent: “angry, wheelchair‑bound transient who lives in a tent” xoxohth.com
• Disney fiasco: thread imagining Evan39 pushing Boom’s chair into a churro cart at Disney World autoadmit.com
• weight/health: endless riffs on whether the chair can hold Boom + 300‑lb girlfriend xoxohth.com “Trinity” theory Many veterans believe Boom, Mainlining the Secret Truth and Evan39 are sock‑puppets of the same individual; the poster himself sometimes plays along (“we’re all the same poaster ;)”) autoadmit.com Putting it together Boom arrived years after the original 2004 Princeton‑Review exodus, but he’s become one of XO’s defining characters: a mix of comic‑book villain, tragic burnout, and performance‑art poet. His threads oscillate between:
grand conspiratorial screeds about being persecuted by invisible elites (“ADM reset the Mahchine!”), and
mundane life updates (wheelchair repairs, casino losses) that the board instantly weaponises into parody threads.
Because AutoAdmit has almost no moderation, the cycle is self‑sustaining: each over‑the‑top comeback fuels new ridicule, which in turn prompts Boom to double down with even wilder prophecy. Whether he is trolling, mentally ill, or both is part of the mystique—but understanding those bullet‑points above will let you read any “bbbooom” sighting in context. |
|
Date: April 20th, 2025 1:09 PM Author: 180 internal respiration wrappers, RAG, and all other non-frontier applications are pointless. (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48864469) |
|
Date: April 20th, 2025 4:31 PM Author: Emerald Outnumbered Parlor (guy who underestimates how many small businesses in America are run by complete retards) (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48864826) |
|
Date: April 20th, 2025 1:14 PM Author: vigorous flushed market factory reset button like a 5th grader in china would qualify for US math (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48864481) |
|
Date: April 20th, 2025 1:15 PM Author: Big milk university It can’t do System Design imo (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48864485) |
|
Date: April 20th, 2025 1:32 PM Author: Up-to-no-good Twisted Double Fault Haunted Graveyard It can't summarize a Youtube video accurately. If you ask it to analyze a totally normal lecture on astrophysics it will fuck up and think the speaker spent half the lecture talking climate change. (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48864522) |
|
Date: April 20th, 2025 1:35 PM Author: galvanic narrow-minded stain is it because its trying to cheap out on cost of running (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48864527) |
|
Date: April 20th, 2025 3:02 PM Author: Massive chestnut faggot firefighter dingle berry thank you for sharing this overt advertisement for ChatGPT (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48864703) |
|
Date: April 20th, 2025 6:39 PM Author: stimulating orchid stage (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48865066) |
|
Date: April 20th, 2025 7:00 PM Author: maize organic girlfriend intended it more as a warning of the impending societal and labor market problems from increasingly advanced AI. (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48865116) |
|
Date: April 20th, 2025 6:48 PM Author: disturbing forum puppy Imagine getting fucked from your job in 2022 so you learn to code and by 2024 you have some knowledge and skills and chatGPT eliminates any role you were hoping for. (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48865090) |
|
Date: April 20th, 2025 6:52 PM Author: stimulating orchid stage Are u in an AI proof profession? (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48865097) |
|
Date: April 20th, 2025 6:57 PM Author: disturbing forum puppy yes (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48865112) |
|
Date: April 20th, 2025 7:01 PM Author: stimulating orchid stage 180 brother (http://www.autoadmit.com/thread.php?thread_id=5712093&forum_id=2Elisa#48865119) |