Is there an example program that illustrates how a buffer overflow attack actually executes arbitrary code?

Thanks for your reply. This extends "smashing the stack for fun and profit" in an important way.

Interesting point.

This makes it seem like the whole concern about memory safety has become almost redundant, the chances of exploitation are just so remote, it must take incomprehensible work to discover a functional exploit that would be useful to attackers in modern software

So, I'm not going to discourage people from doing stuff that they're interested in, but if you're interested in heading down doing low-level security stuff like this as a career path, I'll leave you with this piece of advice, which I think is probably more-valuable than the actual technical information I provided here:

A very great amount of security knowledge, and especially low-level security knowledge, has a short shelf life. That is, you spend time to understand something, and it's only useful for a limited period of time.

In software engineering, if you spend time to learn an algorithm, it will probably be useful pretty much forever; math doesn't change. Sometimes the specific problems that an algorithm is especially useful for go away, and sometimes algorithms are superseded by generally-superior algorithms, but algorithms have a very long shelf life.

Knowledge of software engineering at, say, a programming language level don't last as long. There's only so much demand for COBOL programmers today, though you can use things learned in one environment in another, to a fair degree. But as long as you choose a programming language that is going to be around for a long time, the knowledge you spend time acquiring can continue to be valuable for decades, probably your entire career.

Knowledge of a particular program has a shorter life. There are a very few programs that will be around for an extremely long period of time, like emacs, but it's hard to know in advance which these will be (though my experience has been that open-source software tends to do better here). For example, I have gone through various version control systems -- CVS, VCS, SVN, Bitkeeper, mercurial, git, and a handful of others. The time I spent learning the specifics of most of those is no longer very useful.

Your professional value depends on your skillset, what you bring to the table. If you spend a lot of time learning a skill that will be applicable for your entire working life, then it will continue to add to the value that you bring to the table for your entire working life. If you spend a lot of time learning a skill that will not be of great use in five or ten years, then the time you invested won't be providing a return to you after that point.

That does not mean that everything in the computer security world has a short shelf life. Things like how public/private key systems work or understanding what a man-in-the-middle attack is remain applicable for the long haul.

But a lot of security knowledge involves understanding flaws in very specific systems, and those flaws will go away or become less relevant over time, and often dealing with low-level security, implementation characteristics of specific systems is an example of such a thing.

The world does need low-level computer security experts.

But I would suggest to anyone who is interested in a career in computer security to, when studying things, to keep in mind the likely longevity of what they are teaching themselves and ask themselves whether they believe that that knowledge will likely be relevant at the time that they expect to retire. Everyone needs to learn some short-shelf-life material. But if one specializes in only short-shelf-life things, then they will need to be committing time to re-learn new short-shelf-life material down the line as their current knowledge loses value. I'd try to keep a mix, where a substantial portion of what I'm learning will have a long shelf life, and the short-shelf-life stuff I learn is learned with the understanding that I'm going to need to replace it at some point.

I've spent time hand-assembling 680x0 and x86 code, have written exploits dependent upon quirks in particular compilers and for long-dead binary environments. A lot of that isn't terribly useful knowledge in 2024. That's okay -- I've got other things I know that are useful. But if you go down this road, I would be careful to also allocate time to things that you can say with a higher degree of confidence will be relevant twenty, thirty, and forty years down the line.

A commenter already provided a fairly comprehensive description of low-level computer security positions. But I also want to note that a firm foundation in low-level implementations is also useful for designing embedded software and firmware.

As in, writing or deploying against custom BIOS/UEFI images, or for real-time devices where timing is of the essence. Most anyone dealing with an RTOS or kernel drivers or protocol buses will necessarily require an understanding of both the hardware architecture plus the programming language available to them. And if that appeals to you, you might consider looking into embedded software development.

The field spans anything from writing the control loop for washing machines, to managing data exchange between multiple video co-processors onboard a flying drone to identify and avoid collisions, to negotiating the protocol to set up a 400 Gbps optical transceiver to shoot a laser down 40 km of fibre.

If something "thinks" but doesn't have a monitor and keyboard, it's likely to have one or more processors running embedded software. Look around the room you're in and see what this field has enabled.

That's what they all say 😉

Jokes aside: have fun.

Since you are interested in practical examples, I would recommend you watch and maybe even follow along with Ben Eater’s 6502 breadboard computer series on YouTube (piped link). The kit is cheap and works great but more importantly it introduces so many core concepts about how computers actually work from a raw metal and machine code standpoint while touching on so many different aspects about computers that still apply today.