Backend Type

Instance Data Directory

VAE Cache Directory

Text Embed Cache

Image Embed Cache

Bucket Name

Object Prefix

Region

Endpoint URL

Access Key ID

Secret Access Key

CSV File

Caption Column

Cache Directory

Dataset Name

Config

Split

Image Column

Caption Column

Revision

Enable streaming

Stream data instead of downloading

Auth Token

Audio Caption Fields

Comma-separated columns joined for caption

Lyrics Column

Column for lyrics (used by lyric encoder)

Dataset Source

Metadata Source

Pass the metadata repo id; Webshart follows shard subfolders like data/.

Cache Directory

Shard Cache GB

Parallel Downloads

VAE Cache Directory

Local path for storing VAE latent cache

Write Batch Size

Preserve backend cache

Keep data backend cache between runs

Encode missing text embeds on demand

Skip full text-cache pre-computation and encode cache misses during use.

Disable text cache writes

Read existing entries, encode misses on demand, and do not store new embeddings.

Storage Backend

Local Paths

AWS S3 Configuration

CSV Configuration

Hugging Face Dataset

Webshart Dataset

Cache Directory

Embed Options