Vectorization

Vectorization is the process of reformulating computations to operate on vectors, or arrays, of data rather than on individual scalar values. In computing, this enables execution with a single instruction that affects multiple data points (SIMD). Vectorized code can run much faster on modern CPUs and GPUs due to data‑level parallelism and improved memory throughput.

In software development, vectorization typically means applying operations to whole arrays at once, rather than writing explicit loops. Languages and libraries such as NumPy (Python), MATLAB, and R provide vectorized operators and broadcasting. The compiler or runtime may translate these into SIMD instructions or rely on optimized library kernels.

Hardware support includes vector units like SSE and AVX on CPUs and CUDA or OpenCL on GPUs. These units execute the same operation on multiple data elements in parallel, improving performance for tasks such as numerical simulations, image processing, and machine learning when data is laid out contiguously in memory.

Vectorization in graphics also refers to converting raster images to vector graphics. This image vectorization or tracing produces scalable shapes described by mathematical curves rather than pixel-based bitmaps.

In data science and machine learning, vectorization appears as batch processing of feature matrices or token sequences, enabling efficient use of linear algebra libraries and accelerating model training and inference. It is also a general design principle that favors operations on arrays over elementwise loops.

Limitations include that not all algorithms vectorize well; some data dependencies prevent parallelization. Vectorized code may incur memory overhead or produce numerical differences due to floating-point arithmetic. See also SIMD, vector processor, and scalable vector graphics.