No Stupid Questions

It’s insanely costly, insanely hard, and an insanely hard market for a new competitor to enter. Even something as “simple” as dev support is a gigantic hurdle.

Matt Stoller had a nice writeup recently in his monopoly newsletter BIG about how we got into the current mess. TL;DR: basically financialization (prioritizing stock price over innovation, like at Boeing) and a lack of antitrust enforcement as a previously competitive market got monopolized (see chart below)

Capitalism stifles innovation and suppresses competition.

Duopolies are very prevalent in tech, think AMD/Intel, AMD/nvidia, Windows/MacOS, iOS/Android, etc.

As to why? idk. Big companies buy up small ones until one left so they don't get sued for being a monopoly? Maybe, but I don't think that applies to all those cases.

It's not even really "two companies". Nvidia has 92% of the entire market. And the reason for that is mostly CUDA and its ecosystem which has become widespread among developers.

It's even better than that:

They all come from Taiwan Semiconductor (TSMC).

There used to be more of a split between many fabs. Then it was TSMC/Global Foundries/Samsung Foundry. Then it was TSMC/Samsung Foundry. Now AFAIK all GPUs are TSMC, with Nvidia's RTX 3000 series (excluding the A100) being the last Samsung chip. Even Intel fabs Arc there, as far as I know.

Hopefully Intel won't kill Arc, as they are planning to move it back to their fabs.

The short concise answer is mostly cost. Nvidia, AMD, and Intel are all spending multiple billions of dollars per year in R&D alone. It's just not a space where someone can invent something in their garage and disrupt the whole industry (like, even if someone were to come out of left field with a revolutionary chip design, they'd need to convince investors that they'd be a better bet than literal trillion dollar companies).

Market consolidation. Easier to compete when you have 1 or 2 competitors rather then 5.

Wait till I tell you about the handful of companies that own EVERYTHING else you buy.

You've watched Google,Facebook, and apple do it the last 20 years. If a good idea is spotted early enough, they buy the whole company before they can make it to market and grow to become a threat. It happens in any emerging tech and you're watching it happen now in the LLM space. Companies burn cash, waiting for their competitors to make a mistake or run out of money. Then they buy out the struggling company, absorbing any tech they might have, maybe some branding, but more importantly- their customers. Now they can jack up prices once market forces are eliminated.
If not for the threat of anti-trust laws, you would see single company rule in every single sector. That is the end goal of a company- a monopoly that crushes potential competition and squeezes consumers.
Railroads, telephone, petroleum, internet, airlines, all ended up as regional monopolies.

Intel never really tried to be a real competitor until a few years ago. 3dfx had market dominance in 90s but then basically committed suicide. There were a few other smaller manufacturers in the late 90s and early 2000s but they never really had significant market share and couldn't keep up with the investment required to be competitive.

Huawei have just started selling a gpu to the public that they made a few years back. It has a lot of VRAM, but it is old slow RAM, and doesn't have the software infrastructure (drivers etc) nvidia has. So currently it isn't a great option, but if you look at phones, or electric cars, there is every chance they become competitive in a relatively short time period. Time will tell.

I'm happy we still have two.

Short answer: hard to start a new company that can compete. Over the decades all the other companies have done poorly and gone bankrupt or been bought out.

still very distant in future but I believe in the power of arm. Dedicated gpu's are beasts now but I am rooting for arm to win this race

You are not mistaken.

In the early days of the PC, there were lot's of GPU options, as in literally dozens. So the first part of the question is why did they almost all disappear? The answer to that is that it became a much more complicated market with Windows, with way higher demands on the software side, and many hardware vendors suck at making software. So over time the best combo of hardware and driver beat out other high end manufacturers so we ended up with just 2, and the on-board / on-chip GPU made every low end 3rd party GPU next to irrelevant, with very little possibility of making a profit.

The low end chips were no-longer needed, as they can now be had cheaper and more efficiently as part of the CPU for both AMD and Intel. And since these are the only 2 CPU options for X86, Now that VIA has discontinued their X86 line acquired from Cyrix, there is no low end entry point in the PC market for a new maker of GPU.
The natural evolution is to start from a lower end, and if successful work up. This is not possible in the PC market, and makes entry to the market near impossible, except with enormous investments that may never pay off, especially since PC is a dwindling market.

As you mention Intel is dipping their toes, but despite doing a pretty good and big effort, and investing a lot to develop a better GPU, and actually delivering a good product at a reasonable price, that should be absolutely competitive on paper, their marketshare is absolutely minuscule, because Nvidia and AMD together dominate and already fill the needs for the mid to higher end market, and have brand recognition for graphics.

It's not that there aren't technologies that possibly could compete if scaled for PC, because those are actually pretty numerous on phones and tablets. But you can't port these cheaply to PC, because there is no market segment for them to slide in to easily.
It would require major investments to make them actually hardware performance competitive at higher scales, and investments in making good drivers. Intel had a big head start in these aspects, already making on-chip graphics that had drivers already. And still they are struggling, despite delivering a good product, and people have been screaming for a third option,because of high GPU prices.

This may not be the entire explanation, but I think it's a very significant part of it.
The better question IMO is why Intel never became more popular, considering how much people have raved that more competition in the GPU market is required.

And the explanation for that is:

but let’s be real, AMD and nVidia are the only options

Except Intel actually presented a good alternative, but was never seriously considered by most people for whatever reason.

Personally I didn't consider Intel, because I remember earlier attempts by Intel, where Intel quickly left the market again. And I didn't want a GPU where I'm left without driver support a year after I purchased it.
So in my case it was lack of trust in Intel to stay the course. But every other maker would have that exact same issue.
There have been a few attempts in the past from other makers, but they all had performance or driver issues or both, and left the market quickly . Intel delivered a stellar product by comparison. And if Intel drops out of GPU again, I think there's a pretty big risk it may be our last chance for a third significant mid-high end GPU maker on PC.

TLDR:

1. All the old competitors couldn't cut it on either the hardware or software side, and so they died out.
2. It's an insanely expensive market to enter and to stay in, with high risk of never making a profit.

Oh, and there are other graphics makers that could theoretically work on linux, like Imagination's PowerVR, and some Chinese startups. Qualcomm's already trying to push into laptops with Adreno (which has roots in AMD/ATI, hence 'Adreno' is an anagram for 'Radeon')

The problem is making a desktop-sized GPU has a massive capital cost (over $1,000,000,000, maybe even tens of billions these days) just to 'tape out' a single chip, much less a line, and AMD/Nvidia are just so far ahead in terms of architecture. It's basically uneconomical to catch up without a massive geopolitical motivation like there is in China.

I've noticed how poorly gpus are classified, and how it seems every intersting peice of AI software just has a list of gpus it can work on. So I can see customers just locking into one brand so they have less to memorize.