TikTok’s parent launched a web scraper that’s gobbling up the world’s online data 25-times faster than OpenAI
TikTok’s parent launched a web scraper that’s gobbling up the world’s online data 25-times faster than OpenAI
The crawler, dubbed Bytespider, is scraping the internet at 3,000 times the rate of other genAI tools like Anthropic.
You're viewing a single thread.
It's illegal when a regular person steals something, but it's innovation and courage, when a huge corporation steals something. Interesting how that works
Honestly it’s fucking angering. So much regulation and geo-restrictions and licensing schemes… but it’s cool that there are data brokers, and shit like this. On top of it all Chrome screwing us with manifest v3 and killing ad blocking on chrome. It’s already in canary build.
WHAT THE FUCK IS WRONG WITH THIS SPECIES?!
WHAT THE FUCK IS WRONG WITH THIS SPECIES?!
Yes.
WHAT THE FUCK IS WRONG WITH THIS SPECIES?!
Capitalism.
But muh stocks and bonds
The line must go UP.
I get it that everyone wants ad blockers in their browser, but it doesn’t solve the problem of resources loading outside the browser.
I think DNS or IP filtering is much more effective. I only bring it up because everyone uses apps all the time and I’m constantly seeing apps trying to connect to tracking domains.
Google are actually doing really awesome work with manifest v3. A pimp needs to smack their b1tches around every once in a while to remind them who's boss.
I’m glad there are at least some people enjoying this, you know being a bitch to big corps.
how’s that Google dick tasting?
I respect the troll, but this isn't 2011 anymore
Sorry, I stepped out of my cryochamber just a couple days ago. My energy suit is still powering up.
Do you have a crowbar
They're not stealing your data, they're pirating it.
They’re not pirating it. They’re collecting it.
They’re not collecting it. They’re archiving it.
Oh, like the way back machine?
No, that's stealing /s
Is the problem that wayback machine isn't profiting from it?
It's not so much the lack of profits, more like the lack of kickbacks.
Aaron Schwartz killed himself over punishments for less
Worse punishments. For far less.
Should have waited until he was in the billionaire class before breaking the law.
Not that there's anything right about anything right now, but a web crawler crawling the web hardly seems newsworthy. It's not like everyone else's crawlers haven't been feeding data into giant AI mulchers for years now.
This is just "you know that thing everyone else does? Now the Chinese do it too! Boooo!"
Do you even know what robots.txt is?
the scraper bots surely don't
And nobody in this scenario has done anything illegal.
If you have to defer to the law as justification for doing something purely selfish... then people will judge you to be an asshole.
Edit: Not you personally.
No they will judge you as being above the law (original commenter) and they will be wrong, which doesnt matter, as long as we feel continuity with our synthesized narrative.
Because truth doesnt matter. Our narrative just needs to be as loud as the opposition and then we can confuse people just like those in power... and then the impressionable people trying to understand whats going on or whats morally right will believe one side or the other and truth will not need to be discussed, because its not as catchy anyways.
Then people wont need to be trusted to form their own worldview based on facts, they can neatly choose between a few curated viewpoints, and holding views from multiple viewpoints will isolate them from relevance when they are shunned for not memeing their ideologies like everyone else.
Any regular person can scrape and use public data for AI use, it's not illegal for companies or individuals and it shouldn't be.
proceeds to download the entirety of one piece to train an AI
Except companies have sued people for web scraping.
https://www.informationweek.com/it-leadership/linkedin-sues-after-scraping-of-user-data
Even web scraping for research purposes, not for competitive purposes.
https://www.engadget.com/2016-05-17-publicly-released-okcupid-profiles-taken-down-dmca-claim.html
The first one was because it was data which wasn't public that was scrapped. The second was because the data contained personal information and I'm guessing also because it wasn't public.
As long as you don't need a user account to access the data and it doesn't contain personal information, it is fair game.
It will be the way we have all come to hate AI. Patriot act 2.0
What's being stolen
Data, network bandwidth, and CPU/Processing time from essentially every website in the world, and when you're paying for cloud power to run your website the cost of webscrapers running a train on your digital asshole adds up QUICK.
It's why normal human being people get sued to shit for webscraping data from certain companies who care. But companies don't get sued because go fuck yourself. Kill bytedance.
Kill bytedance
What are their network CIDR blocks? Only half joking…