JITs

JITs, or just-in-time compilers, are components in many language runtimes that translate frequently executed code into native machine code at runtime. They aim to accelerate performance by moving hot code paths from an interpreter or VM’s intermediate representation to optimized native instructions, often while the program is running.

A typical JIT workflow begins with interpretation or lightweight execution to collect profiling data about which methods or execution traces are hot. The JIT then compiles those hot regions into native code and stores the result in a code cache. Subsequent calls to the compiled regions execute the machine code directly. JITs often perform speculative optimizations based on observed runtime types or values, such as inlining, escape analysis, or constant folding. If runtime conditions invalidate these assumptions, the system may deoptimate, reverting to interpretation or less-optimized code. Many JITs use tiered compilation, starting with quick, conservative optimizations and progressively applying more aggressive ones as execution reaches higher confidence levels.

JITs can be categorized by what they compile. Method-based JITs compile entire methods, while tracing JITs identify and compile hot execution traces that may cross method boundaries. Dynamic or adaptive JITs adjust their optimizations during program execution. Popular examples include the Java Virtual Machine’s HotSpot with multiple compiler tiers, the .NET Common Language Runtime’s JIT, and JavaScript engines like V8, which employ tiered and specialized optimizing compilers. PyPy also uses a tracing JIT for Python.

The use of JITs can improve long-running performance and energy efficiency, but introduces startup overhead, memory usage, and structural complexity. Effectiveness depends on workload characteristics and the balance between interpretation, compilation time, and optimization gains.