Lot's of anime are sad but most of them end on a positive or hopeful note. That's why I consider anime that end on a bleak note to be sadder. Stuff like Grave of the Fireflies or the recent Look Back.
Trigun. Vash is just getting shit on for almost 2 centuries and all he's doing is trying to help humans, but his brother is just out to make him miserable.
Something that had sad moments and I think is often overlooked is Gene of AI. It is more of a philosophical anthology series though rather than a show that builds up characters to hit you with emotional baggage.
It pretty much takes place in a limbo/purgatory. Children who die without finding peace or have unresolved emotions end up there. The story slowly reveals this and explores each characters' reasons for ending up there - often confronting their death and untangling the things which kept them from moving on. Time itself is kind of nebulous there; two of the main characters, one who starts as a sort of antagonist and the protagonist, are kind of tied together by fate with the protag being a heart donor to the antagonist. The protagonist died before and his heart was given to her, but somehow she ended up there before him even though she lived a full life (if I'm remembering right) l. It's honestly a super beautiful story, and I think one of my favorite parts is the ending song/mv which plays at the end up each episode- it features every character, and as they move on during the series their spot in the ED is just blank with the final episode ending without any of them.
Is this like Higehiro, where it starts out really suggestive but turns into a kind of decent story? Because I'm getting nothing but red flags from the first episode.
Distant-future apocalypse with child soldiers and a scenery-chewing madman (not of the fun type) in charge. The adjective that comes to mind for me is "grim", but "sad" works too—no one escapes the story unscathed. I have the original DVDs around here somewhere.
Lots of good contenders in this thread. I'll just add
Made in Abyss
for the scene where...
Nanachi tries to gather the strength to end Mitty's suffering.
I have never cried so hard during any other piece of media, ever. My GF had to hold and comfort me because I just couldn't stop sobbing. That scene just... struck a nerve, I guess.