Stack inspection is a technique for determining the level of trust of currently running code at runtime. It was first introduced in the Java Virtual Machine.
This technique was particularly relevant when it seemed like Java applets were going to take over the web. Java applets were discontinued in 2016, and web page apps now run on JavaScript and WebAssembly.
The goal of stack inspection is clearly described in the 1998 Oakland paper by Wallach and Felten:
To implement a Java application that runs untrusted
code within itself (such as the HotJava Web browser), the Java system libraries need a way to distinguish between calls originating from untrusted code, which should be restricted, and calls originating from the application itself, which should be allowed to proceed (subject to any access controls applied by the underlying operating system). To solve this problem, the Java runtime system exports an interface to allow security-checking code to examine the runtime stack for frames executing untrusted code, and allows security decisions to be made at runtime based on the state of the stack.
[Wallach and Felten 1998] also describe the technique itself in a simplified manner:
To explain how stack inspection works, we will first consider a simplified model of stack inspection. In this model, the only principals are “system” and “untrusted”. Likewise, the only privilege available is “full.” This model resembles the stack inspection system used internally in
Netscape Navigator 3.0 [...].In this model, every stack frame is labeled with a principal (“system” if the frame is executing code that is part of the virtual machine or its built-in libraries, and “untrusted” otherwise), and contains a privilege flag which may be set by a system class which chooses to “enable its privileges,” explicitly stating that it wants to do something dangerous. An untrusted class cannot set its privilege flag. When a stack frame exits, its privilege flag (if any) automatically disappears.
All procedures about to perform a dangerous operation such as accessing the file system or network first apply a stack inspection algorithm to decide whether access is allowed. The stack inspection algorithm searches the frames on the caller’s stack in sequence, from newest to oldest. The search terminates, allowing access, upon finding a stack frame with a privilege flag. The search also terminates, forbidding access and throwing an exception, upon finding an untrusted stack frame (which could never have gotten a privilege flag).
D. S. Wallach and E. W. Felten, "Understanding Java stack inspection," Proceedings. 1998 IEEE Symposium on Security and Privacy (Cat. No.98CB36186), 1998, pp. 52-63, doi: 10.1109/SECPRI.1998.674823. [pdf]
The journal version of this paper was published at ACM TSEM in 2000. The backronym SAFKASI stands for "The Security Architecture Formerly Known as Stack Inspection":
Dan S. Wallach, Andrew W. Appel, and Edward W. Felten. 2000. SAFKASI: a security mechanism for language-based systems. ACM Trans. Softw. Eng. Methodol. 9, 4 (Oct. 2000), 341–378. https://doi.org/10.1145/363516.363520 [pdf]
Cédric Fournet and Andrew D. Gordon. 2002. Stack inspection: theory and variants. In Proceedings of the 29th ACM SIGPLAN-SIGACT symposium on Principles of programming languages (POPL '02). Association for Computing Machinery, New York, NY, USA, 307–318. https://doi.org/10.1145/503272.503301
There is a journal version of this paper too, again at TSEM:
Cédric Fournet and Andrew D. Gordon. 2003. Stack inspection: Theory and variants. ACM Trans. Program. Lang. Syst. 25, 3 (May 2003), 360–399. https://doi.org/10.1145/641909.641912