Is it possible to have subtitles come on only in parts of a show/movie that aren't in the base language? Like you'd expect if you were watching a movie in a theater or a show on TV.
Usually a DVD or whatever will have two different subtitle tracks. One for translating all the dialogue and another for what you are talking about. Tools like Handbrake have an option for “foreign audio scan” which will identify that track and mark it as “forced”.
If your video files don’t have that track I would search for one on the sub sites and mux it in using MKVTOOLNIX or something. If you do that, set the forced flag yourself and Jellyfin should play it automagically.