GPT-4 generate real, working, code in many cases including non trivial ones.
It still requires an engineer to proofread and just generally to prompt the system correctly.
Yet, this is like having Magneto in the real world but people thinking it is some kind of tricks and still going through demolition firms... who then hire Magneto.
But as an engineer in the field, I won't complain.
It hallucinates as much as the previous version, but now it does it even more convincingly. Your Magneto secretly hires five drunken guys with a barrel of dynamite, tells them different adresses and instructs them to go nuts. They're drunk af so most of the time they do some random shit, sometimes something resembling what you want them to do, and then your Magneto turns to you and tells you that job is done, but you trust that he is a real superhero so you don't double check. He does the same even if you ask him to move your car or take care of your dog.
People genuinely think gpt is some sort of god machine pulling true and factual information out of the aether when it's literally just fancy phone keyboard text prediction.
I just asked it (3.5) to list counties by what side of the road they drive on and by population
It got Bangladesh, India and Indonesia wrong and put Pakistan on both lists
I do think it could be the future of search but it's obviously got a way to go with regards to error checking if it wants to be
GPT-4 is usually much, much better.
Also, you have to keep in mind that asking it this way relies on its information storage mechanism inside its neural net, which is really not optimal. For many things, it is better to try get it to generate a program that does the task rather than extract information from it.
Unfortunately they removed for now its ability to access web page, but at that moment, asking it to check on wikipedia which side of the road you drive in each country would have worked much better.
just fancy phone keyboard text prediction.
..as if saying that somehow makes what chatGPT does trivial.
This response, which I wouldn't expect from anyone with true understanding of neural nets and machine learning, reminds me of the attempt in the 70s to make a computer control a robot arm to catch a ball. How hard could it be, given that computers at that time were already able to solve staggeringly complex equations? The answer was, of course, "fucking hard".
You're never going to get coherent text from autocomplete and nor can it understand any arbitrary English phrase.
ChatGPT does both those things. You can pose it any question you like in your own words and it will respond with a meaningful and often accurate response. What it can accomplish is truly remarkable, and I don't get why anybody but the most boomer luddite feels this need to rubbish it.
…as if saying that somehow makes what chatGPT does trivial.
That is moving the goalpost. @RickyRigatoni is quite correct that the structure of an autoregressive LLM like (Chat)GPT is, well, autoregressive, i.e. to predict the next word. It is not a statement about triviality until you shifted the goalpost.
What was genuinely lost in the conversation was how the loss function of a LLM is not the truthfulness. The loss function is for the most part, as you noted below, “coherence,” or that it could have been a plausible completion of the text. Only with RLHF there is some weak guidance on truthfulness, which is far meager than the training loss for pure plausibility.
You’re never going to get coherent text from autocomplete and nor can it understand any arbitrary English phrase.
Because those are small models. GPT-3 was already trained on the equivalent text volume that would required > 100 years reading by a human, which is a good size to generate the statistical model, but ridiculous for any sign of “intelligence” or “knowing” what is correct.
Also, “coherence” is not the goal of normal autocomplete for input, which is scored by producing each next word ranked by frequency, and not playing “the long game” in reaching coherence (e.g. involving a few rare words to get the text flow going). Though both are autoregressive, the training losses are absolutely not the same.
And if you had not veered off-topic with your 1970s reference from text generation, you might know that the Turing test was demonstratively passable even without neural networks back then, let alone plausible text generation:
