The GPT Era Is Already Ending
The GPT Era Is Already Ending
If this is the way to superintelligence, it remains a bizarre one. “This is back to a million monkeys typing for a million years generating the works of Shakespeare,” Emily Bender told me. But OpenAI’s technology effectively crunches those years down to seconds. A company blog boasts that an o1 model scored better than most humans on a recent coding test that allowed participants to submit 50 possible solutions to each problem—but only when o1 was allowed 10,000 submissions instead. No human could come up with that many possibilities in a reasonable length of time, which is exactly the point. To OpenAI, unlimited time and resources are an advantage that its hardware-grounded models have over biology. Not even two weeks after the launch of the o1 preview, the start-up presented plans to build data centers that would each require the power generated by approximately five large nuclear reactors, enough for almost 3 million homes.
![[Opinion Piece] The GPT Era Is Already Ending](https://lemmy.world/pictrs/image/0e6a9702-75ac-4199-83af-2d51127c89a4.jpeg?format=webp&thumbnail=196)
You're viewing a single thread.
a million monkeys typing for a million years generating the works of Shakespeare
FFS, it's one monkey and infinite years. This is the second time I've seen someone make this mistake in an AI article in the past month or so.
FFS, it’s one monkey and infinite years.
it is definitely not that long. we already had a monkey generating works of shakespeare. its name was shakespeare and it did not take longer than ~60 million years
A million isn't even close.
There's about a few million characters in shakespeares works. That means the chance of typing it randomly is very conservatively 1 in 261000000if a monkey types a million characters a week the amount of "attempts" a million monkeys makes in a million years is somewhere in the order of 52000000*1000000*1000000 = 5.2 × 1019
The difference is hillriously big. Like, if we multiply both the monkey amount and the number of years by the number of atoms in the knowable universe it still isn't even getting close.
I always thought it was a small team, not millions. But yeah, one monkey with infinite time makes sense.
The whole point is that one of the terms has to be infinite. But it also works with infinite number of monkeys, one will almost surely start typing Hamlet right away.
The interesting part is that has already happened, since an ape already typed Hamlet, we call him Shakespeare. But at the same time, monkeys aren't random letter generators, they are very intentional and conscious beings and not truly random at all.
one will almost surely start typing Hamlet right away
This is guaranteed with infinite monkeys. In fact, they will begin typing every single document to have ever existed, along with every document that will exist, right from the start. Infinity is very, very large.
they will begin typing every single document to have ever existed, along with every document that will exist
Excellent. Now I just need to figure out which one of them is doing my taxes.
This is guaranteed with infinite monkeys.
no, it is not. the chance of it happening will be really close to 100%, not 100% though. there is still small chance that all of the apes will start writing collected philosophical work of donald trump 😂
There’s 100% chance that all of Shakespeare’s and all of Trump’s writings will be started immediately with infinite monkeys. All of every writing past, present, and future will be immediately started (also, in every language assuming they have access to infinite keyboards of other spelling systems). There are infinite monkeys, if one gets it wrong there infinite chances to get it right. One monkey will even write your entire biography, including events that have yet to happen, with perfect accuracy. Another will have written a full transcript of your internal monologue. Literally every single possible combination of letters/words will be written by infinite monkeys.
No, not how it works
Care to elaborate?
sure. 100% means something will happen every single time in the observed set. if something does not happen every single time, then it is not 100%.
this will not happen every single time. among all possible results, there will be results where none of the monkeys start any kind of shakespeare. there will be instances where every single work they start will be just the paper full of letter "a". or something else than shakespeare. as you add monkeys (approach the infinity) the smaller such chancegets, until it gets extremely unlikely, but it is not going to be zero.
imagine you are throwing a 6 sided dice hundred times and i ask you - is it possible there will be no 6 among those one hundred throws?
anyone who passed some basic math understand it is indeed unlikely, but it is not impossible. if you keep throwing long enough, there will be cases with zero 6s in it.
probability of that happening is (5/6)^100, which is 1,2 x 10^-8, eg it will happen roughly 1,2 times in ten million cases. not likely, but not impossible.
in 1000 dice throws, the chance drops to (5/6)^1000, roughly 6,6 x 10^(-80), or 6,6 in 100 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 cases.
fun fact: the above number (1 with 80 zeros) is called One Hundred Quinvigintillion (had to google that indeed).
if you further increase the number of throws in the series, the chance of not having single 6 will be getting even smaller, but never zero.
or in other word, if you raise 5/6 to any positive number, the resulting number is always positive number.
https://www.wolframalpha.com/input?i=%285%2F6%29%5En
for further study, the relevant concept here is limit of the function
we say that limit of the above function is zero, which means it will approach the zero really close (infinitely close), but will never reach it.
Thanks for elaborating. I knew limits were going to show up.
You make a good point, although I would like to point out that one hundred quinvigintillion is basically right next to the number 1 on the number line that goes to infinity. The chance of the monkeys not writing Shakespeare is infinitesimally small. You winning every possible lottery every day for the rest of your life is infinitely more probable than the monkeys not writing Shakespeare.
It's not close to 100%, it is by formal definition 100%. It's a calculus thing, when there's a y value that depends on an x value. And y approaches 1 when x approaches infinity, then y = 1 when x = infinite.
it is by formal definition 100%.
it is not
And y approaches 1 when x approaches infinity, then y = 1 when x = infinite.
you weren't paying attention in your calculus.
y is never 1, because x is never infinite. if you could reach the infinity, it wouldn't be infinity.
for any n within the function's domain: abs(value of y in n minus limit of y) is number bigger than zero. that is the definition of the limit. brush up on your definitions 😆
Except, that's in the real world of physics. In this mathematical/philosophical hypothetical metaphysical scenario, x is infinite. Thus the probability is 1. It doesn't just approach infinite, it is infinite.
Except, that’s in the real world of physics. In this (...) scenario, x is infinite.
oh boy, no. if anything, it would be the other way around. in real world calculations, you can sometime approximate and still get reasonably precise result, or boundary, depending on your needs. not so in math.
hence the jokes like "for mathematician, pi as a pi. for physicist, pi is roughly 3,14, for civil engineer, pi is roughly 3."
Thus the probability is 1.
it is not.
It doesn’t just approach infinite, it is infinite.
x is not infinite. x is a variable, that is to be substituted by specific number. infinity is not a number, it is a concept that express the fact that you explore how the function behaves when you are substituting bigger and bigger numbers. but none of these numbers are "infinity". it is always specific number and the result never reaches the limit of the function. in this case, it is never 1, no matter how big number you substitute.
But it also works with infinite number of monkeys, one will almost surely start typing Hamlet right away.
Wouldn't it even be not just one, but an infinite number of them that would start typing out Hamlet right away ?
In typical statistical mathematician fashion, it's ambiguously “almost surely at least one”. Infinite is very large.
That's the thing though, infinity isn't "large" - that is the wrong way to think about it, large implies a size or bounds - infinity is boundless. An infinity can contain an infinite number of other infinities within itself.
Mathematically, if the monkeys are generating truly random sequences of letters, then an infinite number (and not just "at least one") of them will by definition immediately start typing out Hamlet, and the probability of that is 100% (
not "almost surely"edit: I was wrong on this part, 100% here does actually mean "almost surely", see below). At the same time, every possible finite combination of letters will begin to be typed out as well, including every possible work of literature ever written, past, present or future, and each of those will begin to be typed out each by an infinite number of other monkeys, with 100% probability.Almost surely, I'm quoting mathematicians. Because an infinite anything also includes events that exist but with probability zero. So, sure, the probability is 100% (more accurately, it tends to 1 as the number of monkeys approach infinite) but that doesn't mean it will occur. Just like 0% doesn't mean it won't, because, well, infinity.
Calculus is a bitch.
Ok, this is interesting, so thanks for pointing me to it. I think it's still safe to say "almost surely an infinite number of monkeys" as opposed to "almost surely at least one", since the probability of both cases is still 100% (can their probability even be quantitatively compared ? is one 100% more likely than another 100% in this case ?)
The idea that something with probability of 0 can happen in an infinite set is still a bit of a mindfuck - although I understand why this is necessary (e.g. picking a random marble from an infinite set of marbles where 1 is blue and all others red for example - the probability of picking the blue marble is 0, but it is obviously still possible)
Indeed, the formal definition actually doesn't specify how many monkeys will write what given an infinite number of monkeys, it's unknowable (that's just how probability is). We just know that it will almost surely happen, but that doesn't mean it will happen an infinite amount of occurrences.
The infinite amount of time version is just as vague, one monkey will almost surely type a specific thing, eventually, given infinite time to type it. This is because when you throw infinites at probability, all probabilities tend to 1. Given an infinite amount of time, all things that can happen, will almost surely happen, eventually.
But your citation gives both statements:
"In fact, the monkey would almost surely type every possible finite text an infinite number of times."
and
"The theorem can be generalized to state that any sequence of events that has a non-zero probability of happening will almost certainly occur an infinite number of times, given an infinite amount of time or a universe that is infinite in size."
So when you say the number of times is "unknowable" the actual answer is "almost surely an infinite number of times" no ? Since the probability of that can be calculated as 100%. The mindfuck part is that it is still possible that no monkey at all will type a particular text, even though the probability of that is 0.
The probability that only 2 monkeys will type the text is also still 0, same as 3 monkeys, 4 monkeys, etc. - in fact the probability of any specific finite number of monkeys only typing out the text is still 0 - only the probability of an infinite number of monkeys typing it out is 100% (the probabilities of all possible outcomes, even when infinite, have to sum up to 1 after all)
We just know that it will almost surely happen, but that doesn't mean it will happen an infinite amount of occurrences.
Basically, if we know "it will almost surely happen" then we also know just as surely (p=1) that it will also happen an infinite number of times (but it might also never happen, although with p=0)
