Croissant

DataSource is the class describing the data that can be extracted from files to populate a RecordSet. This class should be used when the data coming from the source needs to be transformed or formatted to be included in the ML dataset; otherwise a simple Reference can be used instead to point to the source.

DataSource is a subclass of schema.org/Intangible.

Properties

Usage

DataSource is used within Field definitions to specify where the data for the field comes from and how it should be processed. The source can be a FileObject, FileSet, or another RecordSet, and the data can be extracted and transformed using the extract, transform, and format properties.

Example

{
  "source": {
    "fileSet": { "@id": "image-files" },
    "extract": {
      "fileProperty": "filename"
    },
    "transform": {
      "regex": "([^\\/]*)\\.jpg"
    }
  }
}

This example extracts filenames from a set of image files and applies a regular expression transformation to extract just the base filename without the path and extension.

The data type of values expected for a Field in a RecordSet. This class is inspired by the Datatype class in CSVW. In addition to simple atomic types, types can be semantic types, such as schema.org classes, as well types defined in other vocabularies.

Key Features

• A field may have more than a single assigned dataType, in which case at least one must be an atomic data type (e.g.: sc:Text), while other types can provide more semantic information, possibly in the context of ML.
• Can be specified at two levels: on individual Fields and on entire RecordSets.

Atomic Data Types

ML-Specific Data Types

Using Data Types from Other Vocabularies

Croissant datasets can use data types from other vocabularies, such as Wikidata. These may be supported by the tools consuming the data, but don't need to. For example:

Examples

Simple Field Type

{
  "@id": "images/color_sample",
  "@type": "cr:Field",
  "dataType": "sc:ImageObject"
}

Multiple Data Types

{
  "@id": "cities/url",
  "@type": "cr:Field",
  "dataType": ["https://schema.org/URL", "https://www.wikidata.org/wiki/Q515"]
}

This example shows a field that is expected to be a URL, whose semantic type is City, so values will be URLs referring to cities.

Sometimes, not all the data from the source is needed, but only a subset. The Extract class can be used to specify how to do that, depending on the type of the data.

Extraction Methods

FileProperty Values

• fullpath: The full path to the file within the Croissant extraction or download folders. Example: data/train/metadata.csv
• filename: The name of the file. In data/train/metadata.csv, the file name is metadata.csv
• content: The byte content of the file
• lines: The byte content of each line in the file
• lineNumbers: The number of each line in the file (starting from 0)

Examples

Extracting File Content

{
  "extract": {
    "fileProperty": "content"
  }
}

Extracting CSV Column

{
  "extract": {
    "column": "userId"
  }
}

Extracting with JSONPath

{
  "extract": {
    "jsonPath": "$.metadata.title"
  }
}

Extracting Filename

{
  "extract": {
    "fileProperty": "filename"
  }
}

This class is typically used within a DataSource to specify exactly what part of the source data should be extracted for a particular field.

A Field is part of a RecordSet. It may represent a column of a table, or a nested data structure or even a nested RecordSet in the case of hierarchical data.

Field is a subclass of schema.org/Intangible.

Properties

Key Features

• Each field has a name (unique identifier within the RecordSet)
• Supports foreign key relationships through the references property
• Supports hierarchical nesting with subField and parentField
• Can specify multiple data types for semantic enrichment

Examples

Simple Field

{
  "@type": "cr:Field",
  "@id": "ratings/user_id",
  "dataType": "sc:Integer",
  "source": {
    "fileObject": { "@id": "ratings-table" },
    "extract": {
      "column": "userId"
    }
  }
}

Field with Reference (Foreign Key)

{
  "@type": "cr:Field",
  "@id": "ratings/movie_id",
  "dataType": "sc:Integer",
  "source": {
    "fileObject": { "@id": "ratings-table" },
    "extract": {
      "column": "movieId"
    }
  },
  "references": {
    "@id": "movies/movie_id"
  }
}

Nested Field with SubFields

{
  "@type": "cr:Field",
  "@id": "gps_coordinates",
  "description": "GPS coordinates where the image was taken.",
  "dataType": "sc:GeoCoordinates",
  "subField": [
    {
      "@type": "cr:Field",
      "@id": "gps_coordinates/latitude",
      "dataType": "sc:Float",
      "source": {
        "fileObject": { "@id": "metadata" },
        "extract": { "column": "latitude" }
      }
    },
    {
      "@type": "cr:Field",
      "@id": "gps_coordinates/longitude",
      "dataType": "sc:Float",
      "source": {
        "fileObject": { "@id": "metadata" },
        "extract": { "column": "longitude" }
      }
    }
  ]
}

This example shows how fields can be hierarchically structured to represent complex data types like geographical coordinates.

FileObject is the Croissant class used to represent individual files that are part of a dataset.

FileObject is a general purpose class that inherits from Schema.org CreativeWork, and can be used to represent instances of more specific types of content like DigitalDocument and MediaObject.

Most of the important properties needed to describe a FileObject are defined in the classes it inherits from:

In addition, FileObject defines the following property:

Let's look at a few examples of FileObject definitions.

First, a single CSV file:

{
  "@type": "cr:FileObject",
  "@id": "pass_metadata.csv",
  "contentUrl": "https://zenodo.org/record/6615455/files/pass_metadata.csv",
  "encodingFormat": "text/csv",
  "sha256": "0b033707ea49365a5ffdd14615825511"
}

Next: An archive and some files extracted from it (represented via the containedIn property):

{
  "@type": "cr:FileObject",
  "@id": "ml-25m.zip",
  "contentUrl": "https://files.grouplens.org/datasets/movielens/ml-25m.zip",
  "encodingFormat": "application/zip",
  "sha256": "6b51fb2759a8657d3bfcbfc42b592ada"
},
{
  "@type": "cr:FileObject",
  "@id": "ratings-table",
  "contentUrl": "ratings.csv",
  "containedIn": { "@id": "ml-25m.zip" },
  "encodingFormat": "text/csv"
},
{
  "@type": "cr:FileObject",
  "@id": "movies-table",
  "contentUrl": "movies.csv",
  "containedIn": { "@id": "ml-25m.zip" },
  "encodingFormat": "text/csv"
}

In many datasets, data comes in the form of collections of homogeneous files, such as images, videos or text files, where each file needs to be treated as an individual item, e.g., as a training example. FileSet is a class that describes such collections of files.

A FileSet is a set of files located in a container, which can be an archive FileObject or a "manifest" file. A FileSet may also specify inclusion / exclusion filters using file patterns.

FileSet extends schema.org/Intangible.

Properties

Pattern Processing

The includes and excludes properties use glob patterns, a common mechanism to specify a set of files along a path, like ".jpg" for all jpg images, or "/foo/pic.jpg" for all jpg images under the "foo" directory whose filename starts with "pic".

To get the set of FileObjects included in the FileSet: 1. The includes pattern(s) are evaluated first 2. If multiple includes are specified, the union of their results is taken 3. Then all the files corresponding to the excludes patterns are removed from that set 4. Patterns are evaluated from the root of the containedIn contents (e.g., the top level directory extracted from an archive)

Examples

Simple Image Archive

{
  "@type": "cr:FileObject",
  "@id": "train2014.zip",
  "contentSize": "13510573713 B",
  "contentUrl": "http://images.cocodataset.org/zips/train2014.zip",
  "encodingFormat": "application/zip",
  "sha256": "sha256"
},
{
  "@type": "cr:FileSet",
  "@id": "image-files",
  "containedIn": { "@id": "train2014.zip" },
  "encodingFormat": "image/jpeg",
  "includes": "*.jpg"
}

Complex Archive with Multiple FileSets

{
  "@type": "cr:FileObject",
  "@id": "flores200_dataset.tar.gz",
  "description": "Flores 200 is hosted on a webserver.",
  "contentSize": "25585843 B",
  "contentUrl": "https://tinyurl.com/flores200dataset",
  "encodingFormat": "application/x-gzip",
  "sha256": "c764ffdeee4894b3002337c5b1e70ecf6f514c00"
},
{
  "@type": "cr:FileSet",
  "@id": "files-dev",
  "description": "dev files are inside the tar.",
  "containedIn": { "@id": "flores200_dataset.tar.gz" },
  "encodingFormat": "application/json",
  "includes": "flores200_dataset/dev/*.dev"
},
{
  "@type": "cr:FileSet",
  "@id": "files-devtest",
  "description": "devtest files are inside the tar.",
  "containedIn": { "@id": "flores200_dataset.tar.gz" },
  "encodingFormat": "application/json",
  "includes": "flores200_dataset/devtest/*.devtest"
}

This example shows how multiple FileSets can be extracted from a single archive, each with different inclusion patterns to select different subsets of files.

A format string used to parse the values coming from a DataSource. For example, a date may be represented as the string "2022/11/10", and interpreted into the correct date via the format "yyyy/MM/dd". Formats correspond to a target data type.

Supported Format Types

Note: This list is not exhaustive, and not all Croissant implementations will support all formats.

Examples

Date Format Parsing

{
  "source": {
    "fileObject": { "@id": "metadata" },
    "extract": { "column": "datetaken" },
    "format": "%Y-%m-%d %H:%M:%S.%f"
  }
}

Bounding Box Format

{
  "@type": "cr:Field",
  "@id": "images/annotations/bbox",
  "description": "The bounding box around annotated object[s].",
  "dataType": "cr:BoundingBox",
  "source": {
    "fileSet": { "@id": "instancesperson_keypoints_annotations" },
    "extract": { "column": "bbox" },
    "format": "CENTER_XYWH"
  }
}

Usage

Format specifications are typically used within DataSource definitions to ensure that string representations of structured data (like dates, numbers, or coordinates) are correctly parsed into their intended data types. This is particularly important for ML datasets where precise data interpretation is crucial for model training and evaluation.

A RecordSet describes a set of structured records obtained from one or more data sources (typically a file or set of files) and the structure of these records, expressed as a set of fields (e.g., the columns of a table). A RecordSet can represent flat or nested data.

Purpose

RecordSet provides a common structure description that can be used across different modalities, in terms of records that may contain multiple fields. It handles:

• Unstructured content (like text and images) as single-field records
• Tabular data as one record per row in the table, with fields for each column
• Tree-structured data with nested and repeated fields

RecordSet is a subclass of schema.org/Intangible.

Properties

Additional Features

• Embedding: Supports embedding small enumerations directly via the data property
• Typing: Supports typing with dataType for entire RecordSets
• Joins: Supports joins through field references (foreign keys)
• Hierarchical: Supports hierarchical structures with nested records

Examples

Simple Tabular RecordSet

{
  "@type": "cr:RecordSet",
  "@id": "ratings",
  "key": [{ "@id": "ratings/user_id" }, { "@id": "ratings/movie_id" }],
  "field": [
    {
      "@type": "cr:Field",
      "@id": "ratings/user_id",
      "dataType": "sc:Integer",
      "source": {
        "fileObject": { "@id": "ratings-table" },
        "extract": { "column": "userId" }
      }
    },
    {
      "@type": "cr:Field",
      "@id": "ratings/rating",
      "description": "The score of the rating on a five-star scale.",
      "dataType": "sc:Float",
      "source": {
        "fileObject": { "@id": "ratings-table" },
        "extract": { "column": "rating" }
      }
    }
  ]
}

Enumeration with Embedded Data

{
  "@type": "cr:RecordSet",
  "@id": "gender_enum",
  "description": "Maps gender ids (0, 1) to labeled values.",
  "key": { "@id": "gender_enum/id" },
  "field": [
    { "@id": "gender_enum/id", "@type": "cr:Field", "dataType": "sc:Integer" },
    { "@id": "gender_enum/label", "@type": "cr:Field", "dataType": "sc:String" }
  ],
  "data": [
    { "gender_enum/id": 0, "gender_enum/label": "Male" },
    { "gender_enum/id": 1, "gender_enum/label": "Female" }
  ]
}

Geographic Data with Type Mapping

{
  "@id": "cities",
  "@type": "cr:RecordSet",
  "dataType": "sc:GeoCoordinates",
  "field": [
    {
      "@id": "cities/latitude",
      "@type": "cr:Field"
    },
    {
      "@id": "cities/longitude", 
      "@type": "cr:Field"
    }
  ]
}

This example shows how RecordSets can be typed with semantic types like sc:GeoCoordinates, and fields can be implicitly mapped to properties of that type (latitude and longitude).

Croissant supports a few simple transformations that can be applied on the source data. Transformations are used to modify extracted data before it's included in the final dataset.

Supported Transformations

• delimiter: Split a string into an array using the supplied character
• regex: A regular expression to parse the data
• jsonQuery: A JSON query to evaluate on the (JSON) data source

Examples

Regular Expression Transformation

{
  "fileSet": {
    "@id": "files"
  },
  "extract": {
    "fileProperty": "filename"
  },
  "transform": {
    "regex": "^(train|val|test)2014/.*\\.jpg$"
  }
}

This example extracts filenames and applies a regex to parse training/validation/test split information.

Filename Parsing

{
  "source": {
    "fileSet": { "@id": "image-files" },
    "extract": {
      "fileProperty": "filename"
    },
    "transform": {
      "regex": "([^\\/]*)\\.jpg"
    }
  }
}

This extracts the base filename (without path and extension) from image files.

Delimiter Transformation

{
  "transform": {
    "delimiter": ","
  }
}

This would split a comma-separated string into an array of values.

JSON Query Transformation

{
  "transform": {
    "jsonQuery": "$.metadata.authors[*].name"
  }
}

This would extract all author names from a JSON structure using a JSON query.

Usage

Transformations are typically used within DataSource definitions, applied after data extraction but before final formatting. They provide a way to clean, parse, or restructure data to make it suitable for machine learning workflows without requiring external preprocessing steps.