Finally actually using Opus 5.5 for the first time after being Codex primarily for so long... I get it now. I knew it (I study benchmarks too much) but now I GET it.

Kinda rude of you all to give me hope right at the beginning there 😂

I just realized that Haiku 5.5 and 6.1 sol are the same distance apart in artificial analysis index scores as 6.1 sol and Opus 5.5 I finally got my first quad code sub a few days ago and I haven't even used it (got pulled away and had configuration issues with my remote setup). I think it's time I actually use the model that I've studied the benchmarks for so long

This model is cool. And it pairs really well with decision models. Between the two you get quite a lot of useful things

Idk why but a bunch of people are hating on this release saying that it's useless to have a super fast model that's Luna level intelligence. On the benchmarks this model is just a much faster Luna that's a little more expensive in reality and a little smarter at any given effort level but doesn't have xhigh or Max effort available. In my personal testing... Their having some launch day latency issues. Sometimes it'll take several seconds to get the first reasoning token. I'm hoping it improves but atm it's not actually better than Luna in speed because it's speed is unreliable.

This one image is extremely useful for people who care about the personality and style of ai models they use. So far, really impressed with Artificial Analysis's work here. It's a bit bold but it's also good enough to be useful and genuinely needed

Love this idea

Yeah this part is crazy. Traditionally minimax has been really good at not attempting to answer if it doesn't let the answer but it seems they flipped that this time and gave it all the knowledge and very little hesitation. As long as it's thorough about it's research to try and find answers I think this combination is actually more useful than the M3 combo

I'm starting to think I just need to get an annual $100 subscription to Codex, Claude, and a $60 Grok/Cursor so I get a bunch of whatever is best for roughly $175/month (because they all have deals for annual) But then Muse is looking tempting although not necessarily worth the sub. I already have Gemini for free and I don't use it so that's unfortunate

I feel like people are having too much trouble with voice to text and not enough trouble with tts. I kinda wanna fix it but idk... It's pretty hard to get anyone to even try stuff once these days

I think I'm tired of building cool stuff that nobody has time to look at... I'm sure I'm not the only one. How many awesome builders are gonna just give up because nobody has the time to take them all seriously?? Do we need some kind of contest or something that we all get to participate in to see how much the people actually bring vs the model?? (btw, I build cool stuff for my day job but it's all internal for the company I work at and I'm really enthused about the other stuff I do too. So it's less that I'm giving up and more than I'm looking for a better way... For all of us)

Why does it feel like Deepseek V4.1 Flash is almost like GPT-6-SOL... A bit cheaper, a bit more taste, a little more refined. But also... Similar somehow

📑📈🖊️ Benchmark Education Time! We should talk about hallucination rates (and we do) but we first should understand what is and isn't measured by a bench. AA's Omniscience bench is one of the most misunderstood benches because people don't realize what is actually being measured or what should be being measured. 1. It measures how likely a model is to lie when it CANNOT FIND THE ANSWER which means that models that have more knowledge embedded in them naturally are less likely to be in a situation where they would lie. (The index version of Omniscience addresses this by adding points for knowledge and subtracting points for hallucinations.) 2. It measures ANSWERS. It doesn't measure malformed tool calls (e.g. things it does know how to do but occasionally messes up). So it isn't directly measuring error-free output. 3. It DOESN'T measure RESEARCH DEPTH. Some models will either know something or not and leave it at that. Others will move heaven and earth to uncover the answer. This bench explicitly removes the model's research tooling so it measures the model in situations where it can't do things such as web search or explore a codebase which is a very rare model setup. Two examples of opposite extremes: Model A was measured by the AA index as very low hallucination and very high knowledge. It's score is the top in the index. However, when reviewers use the model they discover that in coding situations it will claim completely false things about their app and writes code using an older version of their frontend framework. This is a real situation than can and does happen because while model A has a 10% chance of making things up when it doesn't know the right answer it doesn't do it's due diligence to find the answers so 90% of the that 10% chance is active. That means 9% of it's code choices and responses are based on completely made up information. It guessed what framework version it should use instead of checking. It guessed what backend API format the app uses instead of checking. And that's a lot of making stuff up. Model B scored fairly low in the Artificial Analysis index, it had mid level knowledge but score a 90% hallucination rate. However, reviewers found that it almost always gave useful and accurate perspectives and recommendations about their codebase or research projects. This is a real situation as well. This model, while it would make up the answer when it simply had no way of finding it, always turns over every stone to make sure it finds the answer. It checked what framework version is being used and looked up documentation about that version because it didn't have much training data about the latest features. It then noticed it would need to touch an API and took the time to check the formatting and setup for the API. If you haven't guessed already: Model A is representative of Gemini models. Argon is better about hallucination but still doesn't do the necessary research. Model B is representative of Most GPT models. Sol has a very high hallucination rate yet it refuses to stop researching until it has a high confidence that it has found everything there is to find. Model B is clearly more useful as a type.

This image is insane. Obviously DeepSWE is a little compromised at this point and this chart comes from Deepseek itself but... I've never seen a chart look like this

1. Codex had specific (and bad) styling guidelines in its system prompt for quite a while. Probably were decent when they were added back in GPT 5 days but it didn't get updated for far too long. 2. Chat GPT are partially trained using chat histories from previous Codex sessions, meaning strong tendancies get reinforced unless there is intentional intervention. 3. OpenAI models are distinctly "tools" and have much less embedded "taste" or "personality". They do great with specific instructions but don't bring their own taste into the situation much. As models have gotten smarter this method has become more combersome but it's probably built into their whole training pipeline and would require massive refactoring to clean out at this point. 4. They clearly haven't made it a top priority, maybe a side emphasis but clearly not critical level

Serious question that is unfortunately cliche as well: How can AI cover adoption/marketing? I'm genuinely curious what people use for themselves (not the thing they made for other people)