programmitest

Programmitest is a cross-language programming assessment framework designed to help educators, employers, and coding platforms evaluate coding ability and problem-solving skills. It provides a standardized problem format, automated scoring, and detailed feedback across multiple languages, with an emphasis on reproducibility, transparency, and extensibility.

Originating as an open-source project in the mid-2010s, Programmitest arose from the need for a common baseline for programming assignments. It evolved into a modular system with a core execution engine, problem library, and language adapters, with security improvements and analytics added in later releases.

Key features include a JSON or YAML problem description, multi-language support (Python, Java, C++, JavaScript, Go and others), and a sandboxed runner enforcing time and memory limits. It offers unit-test based grading, rubric-based scoring, plagiarism checks, and automated reporting suitable for classrooms and interviews.

Architecture comprises a problem store, an API, a sandboxed executor, and a results database. Submissions run in isolated containers, with inputs, outputs, and resource usage logged. Authors can publish problems via an editor or YAML/JSON, and integrations with LMS or CI systems enable streamlined workflows.

Adoption spans universities, coding bootcamps, and technical hiring platforms seeking consistent assessment. The project is characterized by openness and interoperability, though effective use often requires careful policy configuration and validation against course objectives to avoid bias or gaming.

See also: automated code grading, online judge, programming education technology.