Sketchbased

Sketchbased is a field of study and a shorthand for methods that take hand-drawn sketches as input to retrieve, generate, or manipulate visual content. It focuses on bridging the gap between freehand sketches and photograph-like images or 3D shapes, enabling intuitive search and design.

Common tasks include sketch-based image retrieval (SBIR), where a sketch query is matched to a database of images; sketch-based 3D model retrieval, which finds shapes that correspond to a sketch; and sketch-guided synthesis and editing, where a sketch guides the creation or modification of images or models.

Approaches typically learn cross-domain representations that map sketches and target data into a shared latent space. Early work relied on hand-crafted features; contemporary methods use deep neural networks, often with siamese or triplet losses, to learn robust embeddings. Representations may be raster sketches or stroke-based, vector, or sequence models. Data augmentation and stroke-aware normalization help reduce variability.

Common datasets include TU-Berlin Sketch, Quick,Draw!, Sketchy Database for SBIR, and ShapeNet or ModelNet for 3D retrieval. Evaluation commonly uses mean average precision, precision at k, and top-k accuracy, sometimes with cross-domain adaptation.

Challenges include high intra-class variation in sketches, abstraction levels, inconsistent stroke orders, and the domain gap between sketches and real images or 3D models. Issues of scalability, user heterogeneity, and dataset bias persist.

Applications span visual search in e-commerce, digital art tools, rapid prototyping, and educational software. The field also overlaps with multimodal generation and 3D reconstruction.

Future directions emphasize improved cross-domain alignment, few-shot learning, sketch-based diffusion or generative models, and interactive editors that better translate sketch semantics into high-quality content.