Technology @lemmy.world misk @sopuli.xyz 1d ago

Announcing the Data.gov Archive

lil.law.harvard.edu Announcing the Data.gov Archive | Library Innovation Lab

15 comments

16TB is within the capacity of individuals. Given I don’t trust even Harvard to be safe with the impending onslaught, anyone with the means of downloading a copy for themselves should do so.
- Cold storage is $2 per TB/mo. Could easily be crowdfunded. $410/yr to keep it around.
I'll officially be seeding more of this type of stuff (annas, cdc, etc) than all of my "Linux isos". Feels good.
- Did they create a torrent for this, or just straight downloads?
  
  It appears to be hosted via S3 compatible object store. Their docs provide examples to browse/download via awscli or boto3 (as well as rclone).
  
  https://source.coop/repositories/harvard-lil/gov-data/description https://source.coop/repositories/harvard-lil/gov-data/access
At least private institutions are stepping up. Where's the rest of the universities?
- I work for a Canadian university library and I assure you that data archival efforts are proceeding apace. I can't really say more without possibly endangering American colleagues.
- Dependent on federal money.
  
  ... Thus, not saying a thing if they do actually cache it.
Remember folks, to avoid Linux ISO honeypots you must enable Super-Seeding.
- Is there a list of super-seeding complaint clients? I can't seem to find one. Hell, I can't even check to see if my client supports it.
  
  Most p2p clients support it, you'll need to refer to their documentation to enable.
  
  For the most part all you need to do to enable it is right click a completed download then select "super seeding" mode. It's at least that straightforward for QBT.
🙏
Thank goodness. I was hoping for something like this.

15 comments