Studio / Area

Data & Preparation

Turn incoming information into usable, documented material.

What information enters the system, and what must happen before it can be used?

In scope

Ingestion, parsing, extraction, cleanup, validation, labeling, and dataset assembly

Boundary

Workflow scheduling and runtime orchestration; identity of a trained model

Start with the outcome

Functions

Function

Extract document content

Recover text or metadata from documents for downstream use.

Function

Document parsing

Detect and extract content or metadata from supported file formats.

Function

Interactive data cleaning

Review and transform inconsistent values through an inspectable sequence of edits.

Function

RDF shape validation

Check an RDF data graph against explicit graph-shaped constraints.

Function

Schema-based validation

Apply explicit structural rules to identify records that do not conform.

Function

Ajv

JavaScript validator that compiles JSON Schemas into validation functions.

Function

Apache Tika

Software for detecting and extracting text and metadata from many file types.

Function

OpenRefine

Tool for cleaning, transforming, and reconciling tabular data.

Function

JSON Schema Draft 2020-12

Specification for describing and validating the structure of JSON data.

Function

SHACL

W3C language for validating RDF graphs against stated shapes.

Function

Normalize source data

Make inconsistent values and structures more consistent before reuse.

Function

Validate structured data

Check whether data conforms to explicit structural constraints.

Function

Annotate data

Apply human judgments or labels to source material for downstream use.

Function

Human-in-the-loop annotation

Collect labels from people and optionally use model suggestions to assist them.

Function

Label Studio

Data-labeling application for human annotation across several data types.

Function

Recognize text in images

Turn visible text in images into machine-readable characters.

Function

Optical character recognition

Analyze image pixels to recognize visible text characters.

Function

Tesseract OCR

Open-source engine for recognizing printed text in images.

Function

Expectation-based validation

Run explicit assertions about data properties against selected datasets.

Function

GX Core

Python library for defining data expectations and running validation workflows.

Function

Catalog datasets

Describe and publish datasets so people can find and assess their resources.

Function

Dataset metadata cataloging

Maintain dataset descriptions and resource pointers as searchable, governed entries.

Function

CKAN

Open-source platform for publishing and finding datasets with metadata, resources, and organization controls.

Function

Docling

Open-source document conversion toolkit with PDF layout and table-structure processing and structured exports.

Function

Discover data assets

Find datasets, dashboards, and pipelines through searchable descriptive metadata.

Function

Metadata asset search

Search indexed metadata across multiple asset types and systems.

Function

DataHub Core

Self-hosted open-source metadata platform for discovering data assets.

Function

Move source data into a destination

Copy selected records between source and destination systems for downstream use.

Function

Connector-based data replication

Use source and destination connectors to copy selected streams across systems.

Function

Airbyte Core (self-managed)

Free open-source Airbyte data-replication platform for deployment in your own infrastructure.

Function

Find records for the same entity

Find duplicate records within one dataset or records across datasets that may describe the same person, organization, or other entity.

Function

Probabilistic record linkage

An entity-resolution method that scores multiple record attributes to estimate which records refer to the same entity.

Function

Splink (Python package)

Open-source Python package for probabilistic linkage and deduplication of records without unique identifiers.

Function

Remove or mask sensitive text

Redact or mask selected personal information in text before it is reused or shared.

Function

Detected-span text treatment

Apply chosen redaction, masking, or replacement operators to identified sensitive spans in text.

Representative paths

Methods and options

approach

Governance connection

Sensitive information handling and quality criteria connect to governance.

Related records

In this areaExtract document content
In this areaDocument parsing
In this areaInteractive data cleaning
In this areaRDF shape validation
In this areaSchema-based validation
In this areaAjv
In this areaApache Tika
In this areaOpenRefine
In this areaJSON Schema Draft 2020-12
In this areaSHACL
In this areaNormalize source data
In this areaValidate structured data
In this areaAnnotate data
In this areaHuman-in-the-loop annotation
In this areaLabel Studio
In this areaRecognize text in images
In this areaOptical character recognition
In this areaTesseract OCR
In this areaExpectation-based validation
In this areaGX Core
In this areaCatalog datasets
In this areaDataset metadata cataloging
In this areaCKAN
In this areaDocling
In this areaDiscover data assets
In this areaMetadata asset search
In this areaDataHub Core
In this areaMove source data into a destination
In this areaConnector-based data replication
In this areaAirbyte Core (self-managed)
In this areaFind records for the same entity
In this areaProbabilistic record linkage
In this areaSplink (Python package)
In this areaRemove or mask sensitive text
In this areaDetected-span text treatment

Sources

AI Studio methodologyCameron Smith · checked 2026-09-29

Last reviewed 2026-09-29