My battery died (can't get above 10.5V, loosing voltage while sitting etc) and I need to get a new one. What should I look out for? There are a lot of different types of them like AGM, Lithium Iron, Flooded Acid etc
I'm not knowledgeable to get into the specifics of the differences, but for your older Integra with a gasoline engine, a regular lead-acid battery will suffice. You don't need lean angles (like AGM for motorcycles or off road applications) and you don't need other special properties that vehicles with Start-Stop would require. Basically all you need to look for is the specified amperage/capacity and form format. Don't spend a whole lot; the right battery for your car will probably be under 100 Dollars and last somewhere between 5 and 10-12 years, depending on which price bracket and quality level you opt for.
I'd just like to add to this and recommend interstate batteries. They are a little bit more than house brand, but are much higher quality. For example: Every duralast brand battery I've bought started leaking after ~ 3 years. The last interstate I bought lasted for ~ 8 years/90k miles and was like $40 more than the cheapest battery