I read one of the papers. About the specific question you have: given a string of bits s, they're making the choice to associate the empirical distribution to s, as if s was generated by an iid Bernoulli process. So if s has 10 zero bits and 30 one bits, its associated empirical distribution is Ber(3/4). This is the distribution which they're calculating the entropy of. I have no idea on what basis they are making this choice.
The rest of the paper didn't make sense to me - they are somehow assigning a number N of "information states" which can change over time as the memory cells fail. I honestly have no idea what it's supposed to mean and kinda suspect the whole thing is rubbish.
Edit: after reading the author's quotes from the associated hype article I'm 100% sure it's rubbish. It's also really funny that they didn't manage to catch the COVID-19 research hype train so they've pivoted to the simulation hypothesis.
Mr. Costantino said the design was not at fault and that the towering mast, which stood 237 feet tall, had not created “any kind of problem.”
“The ship was an unsinkable ship,” he said. “I say it, I repeat it.”
- Designer of sunken ship
For some reason the previous week's thread doesn't show up on the feed for me (and didn't all week)... nvm, i somehow managed to block froztbyte by accident, no idea how
I don't think it's very surprising. The various CS departments are extremely happy to ride the wave of easy funding and spend a lot of time boosting AI, just like how a few years ago all the cryptographers were getting into blockchains. For instance they added an entire new "AI" major, while eliminating the electrical engineering major on the grounds that "computation" is more important than electrical engineering.
No, but the moon does.
the moon could get mad - fact.
the computational cost of operating over a matrix is always going to be convex relative to its size
This makes no sense - "convex" doesn't mean fast-growing. For instance a constant function is convex.
My university sends me checks occasionally, like when they overcharged the premium on my dental insurance. No idea why they can't just do an electronic transfer like for my stipend.
Harry Potter and the Surprisingly Good Take
If you want a serious discussion of interpretations of quantum mechanics, here is a transcript of a lecture "Quantum Mechanics in Your Face" which has the best explanation I've ever seen. I'd recommend the first 6 of Peter Shor's Quantum Computation notes (don't worry they're each very short) for just enough background to understand the transcript.
I honestly think anyone who writes "quantum" in an article should be required to take a linear algebra exam to avoid being instantly sacked
Possibly the worst misunderstanding of quantum mechanics I've ever seen. I have no idea how anyone managed to convince themselves that the laws of physics are somehow different for conscious observers.
don't mention skull sizes for 5 minutes challenge
Wasn't Heidegger a Nazi, and his works famously avoid any mention of the Holocaust?
im doing my part
Yudkowskian Probability Theory
what a throwback
Dan Luu's "A discussion of discussions on AI bias", about techbros trying to gaslight the rest of the world into thinking ML models don't have problems
okay but if they hadn't done that we wouldn't have gotten this work of art
Why when we look into the stars do we not see a sign of life anywhere else? Has life not emerged yet or has it wiped itself out? With what? Nukes? AI? Synthetic viruses made with AI? Who knows…
entertaining this awful sci-fi schtick for a moment - if every civilization is wiped out by "superintelligent AI", how come you can't look through a telescope and see signs of artificial life? in this fantasy world shouldn't planets taken over by paperclip factories be even more conspicuous?