pyautogui

PyAutoGUI is a Python library for GUI automation. It allows scripts to control the mouse and keyboard, take screenshots, and perform simple image-based GUI interactions on Windows, macOS, and Linux. It is open source and widely used for automation tasks, testing, and productivity tools.

Core capabilities include moving and clicking the mouse, typing keystrokes, dragging and dropping, scrolling, and pressing hotkeys. It exposes high-level functions such as moveTo, moveRel, click, doubleClick, rightClick, dragTo, dragRel, scroll, and typeWrite, as well as utilities to pause between actions and to set a global failsafe by moving the mouse to the screen corner.

In addition to direct input, PyAutoGUI can locate UI elements on the screen by image matching using locateOnScreen, locateCenterOnScreen, and locateAllOnScreen. This relies on image templates and, if available, OpenCV for a confidence parameter to tolerate minor differences.

Usage patterns include simple scripts for repetitive tasks, automated testing, or accessibility tools. Limitations include sensitivity to screen changes, window positions, and color variations; it may be brittle for dynamic UIs. The library is cross-platform but behavior can differ by platform and environment; on some systems you may need extra permissions or dependencies for screenshots and input control.

Installation typically uses pip: pip install pyautogui. Optional components such as opencv-python enable image recognition with a confidence threshold; Pillow is required for image handling; additional platform-specific dependencies may be needed for screenshot functionality on Linux or macOS. Basic examples can run in any Python 3 environment.